On 5/15/23 10:00 PM, Jason Gunthorpe wrote:
The devices list was used as a simple way to avoid having per-group information. Now that this seems to be unavoidable, just commit to per-group information fully and remove the devices list from the HWPT.
The iommufd_group stores the currently assigned HWPT for the entire group and we can manage the per-device attach/detach with a list in the iommufd_group.
I am preparing the patches to route I/O page faults to user space through iommufd. The iommufd page fault handler knows the hwpt and the device pointer, but it needs to convert the device pointer into its iommufd object id and pass the id to user space.
It's fine that we remove the hwpt->devices here, but perhaps I need to add the context pointer in ioas later,
struct iommufd_ioas { struct io_pagetable iopt; struct mutex mutex; struct list_head hwpt_list; + struct iommufd_ctx *ictx; };
and, use below helper to look up the device id.
+u32 iommufd_get_device_id(struct iommufd_ctx *ictx, struct device *dev) +{ + struct iommu_group *group = iommu_group_get(dev); + u32 dev_id = IOMMUFD_INVALID_OBJ_ID; + struct iommufd_group *igroup; + struct iommufd_device *cur; + unsigned int id; + + if (!group) + return IOMMUFD_INVALID_OBJ_ID; + + id = iommu_group_id(group); + xa_lock(&ictx->groups); + igroup = xa_load(&ictx->groups, id); + if (!iommufd_group_try_get(igroup, group)) { + xa_unlock(&ictx->groups); + iommu_group_put(group); + return IOMMUFD_INVALID_OBJ_ID; + } + xa_unlock(&ictx->groups); + + mutex_lock(&igroup->lock); + list_for_each_entry(cur, &igroup->device_list, group_item) { + if (cur->dev == dev) { + dev_id = cur->obj.id; + break; + } + } + mutex_unlock(&igroup->lock); + + iommufd_put_group(igroup); + iommu_group_put(group); + + return dev_id; +}
and, use it like this in the fault handler:
dev_id = iommufd_get_device_id(hwpt->ioas->ictx, dev); + if (dev_id == IOMMUFD_INVALID_OBJ_ID) + return IOMMU_PAGE_RESP_FAILURE;
Will this look good to you?
For destruction the flow is organized to make the following patches easier, the actual call to iommufd_object_destroy_user() is done at the top of the call chain without holding any locks. The HWPT to be destroyed is returned out from the locked region to make this possible. Later patches create locking that requires this.
Reviewed-by: Lu Baolubaolu.lu@linux.intel.com Reviewed-by: Kevin Tiankevin.tian@intel.com Signed-off-by: Jason Gunthorpejgg@nvidia.com
drivers/iommu/iommufd/device.c | 100 +++++++++++------------- drivers/iommu/iommufd/hw_pagetable.c | 22 +----- drivers/iommu/iommufd/iommufd_private.h | 13 ++- 3 files changed, 54 insertions(+), 81 deletions(-)
Best regards, baolu