[nvidia_stable-11.9] NVIDIA: SAUCE: qemu: Use pci_bus to identify multi-smmuv3 model

[NathanChenNVIDIA]

The previous qemu virt-machine smmuv3 model and the new arm-smmuv3
model cannot both be specified at the same time. When launching
a Libvirt VM with a single arm-smmuv3 model denoted by the
attribute, qemuBuildMachineCommandLine() still appends
the iommu=smmuv3 -machine parameter in addition to the arm-smmuv3
model passed to -device, preventing the VM from booting.

Use the presence of a non-negative pci_bus to
identify the multi-smmuv3 IOMMU model, instead of the
niommus attribute. This allows for specifying a single
arm-smmuv3 on the qemu command line, instead of both
the virt-machine smmuv3 and arm-smmuv3 being specified
at the same time.

Signed-off-by: Nathan Chen nathanc@nvidia.com

[nvmochs]

@NathanChenNVIDIA Two comments

• Can you provide an example snippets of the libvirt XML and corresponding QEMU command that is generated for the broken and resolved cases?
• Can you provide more context about the non-negative vs. negative pci_bus? Is it the case that for a single arm-smmuv3, the pci_bus is not initialized and therefore is negative?

[NathanChenNVIDIA]

@nvmochs

@NathanChenNVIDIA Two comments

• Can you provide an example snippets of the libvirt XML and corresponding QEMU command that is generated for the broken and resolved cases?

The libvirt XML does not change, it would be something like the following for both cases:

<domain>
  ...
  <devices>
  ...
    <controller type='pci' index='1' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='252'>
        <node>0</node>
      </target>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </controller>
  ...
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
  ...
    <hostdev mode='subsystem' type='pci' managed='no'>
      <driver iommufd='yes'/>
      <source>
        <address domain='0x0009' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <acpi nodeset='1-8'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </hostdev>
    **<iommu model='smmuv3'>
      <driver pciBus='1' accel='on' ats='on' ril='off' pasid='on' oas='48'/>
    </iommu>**
  </devices>
</domain>

The broken qemu command line case contains both smmuv3 models:

-machine virt-10.1,**iommu=smmuv3**
-device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"smmuv3.0","accel":true,"cmdqv":true,"ats":true,"ril":false,"pasid":true,"oas":48}' 

The fixed qemu command line only contains the arm-smmuv3 model:
-device '{"driver":"arm-smmuv3","primary-bus":"pci.1","id":"smmuv3.0","accel":true,"ats":true,"ril":false,"pasid":true,"oas":48}' \

• Can you provide more context about the non-negative vs. negative pci_bus? Is it the case that for a single arm-smmuv3, the pci_bus is not initialized and therefore is negative?

pci_bus is initialized to -1 on creation of a virDomainIOMMUDef:

https://github.com/NVIDIA/libvirt/blob/8aa224dab16940c212431e7dafe18cb99420e60c/src/conf/domain_conf.c#L2818

And if a value is specified for the pciBus attribute in the domain definition, it gets picked up here:
https://github.com/NVIDIA/libvirt/blob/8aa224dab16940c212431e7dafe18cb99420e60c/src/conf/domain_conf.c#L14496

[nvmochs]

Thanks for clarifying, no issues from me.

Acked-by: Matthew R. Ochs <mochs@nvidia.com>

[ianm-nv]

LGTM
Acked-by: Ian May <ianm@nvidia.com>

[MitchellAugustin]

LGTM, thanks!
Acked-by: Mitchell Augustin <mitchell.augustin@canonical.com>