From: Leon Romanovsky leonro@nvidia.com
P2PDMA documentation describes ACS controls as path-wide, although Request and Completion controls apply to different transaction directions and only affect peer-versus-upstream decisions at the path divergence.
Document the fixed client and provider roles, the divergence port checked for each TLP direction, and the conservative handling of unreadable ACS state. Clarify which controls disable_acs_redir changes.
Reviewed-by: Logan Gunthorpe logang@deltatee.com Tested-by: Tushar Dave tdave@nvidia.com Signed-off-by: Leon Romanovsky leonro@nvidia.com --- Documentation/admin-guide/kernel-parameters.txt | 15 +++++++++------ Documentation/driver-api/pci/p2pdma.rst | 13 +++++++++++++ 2 files changed, 22 insertions(+), 6 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 68647ff4bdd2..bc83e07dd5fc 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5291,12 +5291,15 @@ Kernel parameters disable_acs_redir=<pci_dev>[; ...] Specify one or more PCI devices (in the format specified above) separated by semicolons. - Each device specified will have the PCI ACS - redirect capabilities forced off which will - allow P2P traffic between devices through - bridges without forcing it upstream. Note: - this removes isolation between devices and - may put more devices in an IOMMU group. + Each device specified will have the PCI ACS P2P + Request Redirect, Completion Redirect, and Egress + Control features forced off. This may allow P2P + traffic through bridges that would otherwise be + redirected upstream. This may allow P2P traffic + through bridges that would otherwise be redirected + upstream and thus this removes isolation between + devices and may cause affected devices to share + an IOMMU group. config_acs= Format: <ACS flags>@<pci_dev>[; ...] diff --git a/Documentation/driver-api/pci/p2pdma.rst b/Documentation/driver-api/pci/p2pdma.rst index 80f8fec9b0e9..42b18610bf7d 100644 --- a/Documentation/driver-api/pci/p2pdma.rst +++ b/Documentation/driver-api/pci/p2pdma.rst @@ -15,6 +15,19 @@ then based on the ACS settings the transaction can route entirely within the PCIe hierarchy and never reach the root port. The kernel will evaluate the PCIe topology and always permit P2P in these well-defined cases.
+The client remains the PCIe requester when it reads or writes provider memory. +Where the paths diverge, the kernel therefore evaluates P2P Request Redirect +and Egress Control on the client-side port, and P2P Completion Redirect on the +provider-side port for completions from a read. An enabled Egress Control is +conservatively treated as a Request redirect. + +Below the divergence, the route toward the other branch is already upstream, +so those P2P redirect controls do not affect it. Redirect controls for the +reverse transaction directions do not affect the mapping. P2P DMA is routed +through the host bridge when either applicable port redirects. If an ACS +Control register cannot be read, P2P DMA is rejected because the kernel cannot +establish a usable route. + This evaluation assumes clients issue strictly ordered Requests carrying an Untranslated address. Its result is not defined when clients use Relaxed Ordering or issue ATS-translated Requests because those TLP attributes can