On Mon, Oct 16, 2023 at 08:48:03AM +0000, Tian, Kevin wrote:
From: Nicolin Chen nicolinc@nvidia.com Sent: Saturday, October 14, 2023 8:45 AM
On Tue, Sep 26, 2023 at 01:16:35AM -0700, Tian, Kevin wrote:
From: Liu, Yi L yi.l.liu@intel.com Sent: Thursday, September 21, 2023 3:51 PM
From: Nicolin Chen nicolinc@nvidia.com
Now enforce_cache_coherency and msi_cookie are kernel-managed hwpt things. So, they should be only setup on kernel-managed domains. If the
attaching
domain is a user-managed domain, redirect the hwpt to hwpt->parent to
do
it correctly.
No redirection. The parent should already have the configuration done when it's created. It shouldn't be triggered in the nesting path.
iommufd_hw_pagetable_enforce_cc() is not only called in alloc(), but also in hwpt_attach/replace() if cc is not enforced by the alloc() because the idev that initiates the hwpt_alloc() might not have idev->enforce_cache_coherency. Only when another idev that has idev->enforce_cache_coherency attaches to the shared hwpt, the cc configuration would be done.
is this a bug already? If the 1st device doesn't have enforce_cc in its iommu, setting the snp bit in the hwpt would lead to reserved bit violation.
I suspect there are technically some gaps in the intel driver, yes..
another problem is that intel_iommu_enforce_cache_coherency() doesn't update existing entries. It only sets a domain flag to affect future mappings. so it means the 2nd idev is also broken.
This is such a gap, intel driver should not permit that.
The simplest option is to follow vfio type1 i.e. don't mix devices with different enforce_cc in one domain.
This is why I wanted to get rid of this bad mechanism going forward.
Manually created hwpt should have a manual specification of cc and then we don't have so many problems.
It means userspace needs to compute if they want to use CC or not, but userspace already needs to figure this out since without autodomains it must create two hwpts manually anyhow.
Jason