Intel IOMMU driver implements IOTLB flush queue with domain selective or PASID selective invalidations. In this case there's no need to track IOVA page range and sync IOTLBs, which may cause significant performance hit.

This patch adds a check to avoid IOVA gather page and IOTLB sync for the lazy path.

The performance difference on Sapphire Rapids 100Gb NIC is improved by the following (as measured by iperf send):

w/o this fix~48 Gbits/s. with this fix ~54 Gbits/s

Cc: stable@vger.kernel.org Fixes: 2a2b8eaa5b25 ("iommu: Handle freelists when using deferred flushing in iommu drivers") Reviewed-by: Robin Murphy robin.murphy@arm.com Reviewed-by: Kevin Tian kevin.tian@intel.com Tested-by: Sanjay Kumar sanjay.k.kumar@intel.com Signed-off-by: Sanjay Kumar sanjay.k.kumar@intel.com Signed-off-by: Jacob Pan jacob.jun.pan@linux.intel.com --- v3: reword comments, add reviewed by tag. (Kevin) v2: use helper function iommu_iotlb_gather_queued() instead of open coding --- drivers/iommu/intel/iommu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 59df7e42fd53..2ee270e4d484 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4346,7 +4346,12 @@ static size_t intel_iommu_unmap(struct iommu_domain *domain, if (dmar_domain->max_addr == iova + size) dmar_domain->max_addr = iova;

- iommu_iotlb_gather_add_page(domain, gather, iova, size); + /* + * We do not use page-selective IOTLB invalidation in flush queue, + * so there is no need to track page and sync iotlb. + */ + if (!iommu_iotlb_gather_queued(gather)) + iommu_iotlb_gather_add_page(domain, gather, iova, size);

return size; }