On Mon, Dec 11, 2023 at 05:57:38PM -0400, Jason Gunthorpe wrote:
On Mon, Dec 11, 2023 at 01:27:49PM -0800, Nicolin Chen wrote:
On Fri, Dec 08, 2023 at 09:47:26PM -0400, Jason Gunthorpe wrote:
What is in a Nested domain: ARM: A CD table pointer Nesting domains are created for every unique CD table top pointer.
I think we basically implemented in a way of syncing STE, i,e, vSTE.Config must be "S1 Translate" besides a CD table pointer, and a nested domain is freed when vSTE.Config=BYPASS even if a CD table pointer is present, right?
Yes, but you can also de-duplicate the nested domains based on the CD table pointer. It is not as critical for ARM as others, but may still be worth doing.
Ah right! Should do that.
If we do implement an IOMMUFD_DEV_ASSIGN_VIRTUAL_ID, do we need an IOMMUFD_DEV_RESIGN_VIRTUAL_ID? (or better word than resign).
I don't think so.. The vRID is basically fixed, if it needs to be changed then the device can be destroyed (or assign can just change it)
OK. Will insert the cleanup piece to the unbind()/destroy().
Could the structure just look like this? struct iommu_dev_assign_virtual_id { __u32 size; __u32 dev_id; __u32 id_type; __u32 id; };
It needs to take in the viommu_id also, and I'd make the id 64 bits just for good luck.
What is viommu_id required for in this context? I thought we already know which SMMU instance to issue commands via dev_id?
IOMMUFD_DEV_INVALIDATE should be introduced with the same design as HWPT invalidate. This would be used for AMD/ARM's ATC invalidation (and just force the stream ID, userspace must direct the vRID to the correct dev_id).
SMMU's CD invalidations could fall into this category too.
Do we need a different iommu API for this ioctl? We currently have the cache_invalidate_user as a domain op. The new device op will be an iommu op?
And should we rename the "cache_invalidate_user"? Would VT-d still uses it for device cache?
I previously drafted something to test it out with iommufd. Basically it needs the pairing of vRID/pRID in attach_dev() and another ioctl to mmap/config user queue(s): +struct iommu_hwpt_cache_config_tegra241_vcmdq {
__u32 vcmdq_id; // queue id__u32 vcmdq_log2size; // queue size__aligned_u64 vcmdq_base; // queue guest PA+};
vRID/pRID pairing should come from IOMMUFD_DEV_ASSIGN_VIRTUAL_ID. When a HWPT is allocated it would be connected to the viommu_id and then it would all be bundled together in the HW somehow
Since we were talking about sharing stage-2 domain, the HWPT to the stage-2 domain will be shared among the vIOMMU/pIOMMU instances too? I think I am not quite getting the viommu_id part yet. From the other side of this thread, viommu object is created per vIOMMU instance and each one actually tied to a pIOMMU by nature?
Thanks Nicolin