Changelog v1->v2: * Rebased on top of vfio_device cdev v2 series. * Update the kdoc and commit message of iommu_group_replace_domain(). * Dropped revert-to-core-domain part in iommu_group_replace_domain(). * Dropped !ops->dma_unmap check in vfio_iommufd_emulated_attach_ioas(). * Added missing rc value in vfio_iommufd_emulated_attach_ioas() from the iommufd_access_set_ioas() call. * Added a new patch in vfio_main to deny vfio_pin/unpin_pages() calls if vdev->ops->dma_unmap is not implemented. * Added a __iommmufd_device_detach helper and let the replace routine do a partial detach(). * Added restriction on auto_domains to use the replace feature. * Added the patch "iommufd/device: Make hwpt_list list_add/del symmetric" from the has_group removal series.
Hi all,
The existing IOMMU APIs provide a pair of functions: iommu_attach_group() for callers to attach a device from the default_domain (NULL if not being supported) to a given iommu domain, and iommu_detach_group() for callers to detach a device from a given domain to the default_domain. Internally, the detach_dev op is deprecated for the newer drivers with default_domain. This means that those drivers likely can switch an attaching domain to another one, without stagging the device at a blocking or default domain, for use cases such as: 1) vPASID mode, when a guest wants to replace a single pasid (PASID=0) table with a larger table (PASID=N) 2) Nesting mode, when switching the attaching device from an S2 domain to an S1 domain, or when switching between relevant S1 domains.
This series introduces a new iommu_group_replace_domain() for that. And add corresponding support throughout the uAPI. So user space can do such a REPLACE ioctl reusing the existing VFIO_DEVICE_ATTACH_IOMMUFD_PT. This means that user space needs to be aware whether the device is attached or not: an unattached device calling VFIO_DEVICE_ATTACH_IOMMUFD_PT means a regular ATTACH; an attached device calling VFIO_DEVICE_ATTACH_IOMMUFD_PT on the other hand means a REPLACE.
QEMU with this feature should have the vIOMMU maintain a cache of the guest io page table addresses and assign a unique IOAS to each unique guest page table.
As the guest writes the page table address to the HW registers qemu should then use the 'replace domain' operation on VFIO to assign the VFIO device to the correct de-duplicated page table.
The algorithm where QEMU uses one VFIO container per-device and removes all the mappings to change the assignment should ideally not be used with iommufd.
To apply this series, please rebase on top of the following patches: 1) [PATCH v2 00/14] Add vfio_device cdev for iommufd support https://lore.kernel.org/kvm/20230206090532.95598-1-yi.l.liu@intel.com/
Or you can also find this series on Github: https://github.com/nicolinc/iommufd/commits/iommu_group_replace_domain-v2
Thank you Nicolin Chen
Nicolin Chen (9): iommu: Introduce a new iommu_group_replace_domain() API iommufd: Create access in vfio_iommufd_emulated_bind() iommufd/selftest: Add IOMMU_TEST_OP_ACCESS_SET_IOAS coverage iommufd: Add replace support in iommufd_access_set_ioas() iommufd/selftest: Add coverage for access->ioas replacement iommufd/device: Make hwpt_list list_add/del symmetric iommufd/device: Use iommu_group_replace_domain() vfio: Support IO page table replacement vfio: Do not allow !ops->dma_unmap in vfio_pin/unpin_pages()
Yi Liu (1): iommu: Move dev_iommu_ops() to private header
drivers/iommu/iommu-priv.h | 22 ++ drivers/iommu/iommu.c | 30 +++ drivers/iommu/iommufd/device.c | 221 +++++++++++++----- drivers/iommu/iommufd/iommufd_private.h | 4 + drivers/iommu/iommufd/iommufd_test.h | 4 + drivers/iommu/iommufd/selftest.c | 25 +- drivers/vfio/iommufd.c | 30 ++- drivers/vfio/vfio_main.c | 4 + include/linux/iommu.h | 11 - include/linux/iommufd.h | 3 +- include/uapi/linux/vfio.h | 6 + tools/testing/selftests/iommu/iommufd.c | 29 ++- tools/testing/selftests/iommu/iommufd_utils.h | 22 +- 13 files changed, 321 insertions(+), 90 deletions(-) create mode 100644 drivers/iommu/iommu-priv.h