From: Leon Romanovsky leonro@nvidia.com
The P2PDMA documentation stated that the mapping result is not defined for Relaxed Ordering or ATS-translated Requests. It now is.
Replace that paragraph with what the three TLP-sensitive ACS controls do, the table of outcomes per class, and the interface a client uses to name its traffic. Record that a caller claiming Relaxed Ordering Completions is asserting provider behaviour the PCIe specification leaves optional.
Tested-by: Tushar Dave tdave@nvidia.com Signed-off-by: Leon Romanovsky leonro@nvidia.com --- Documentation/driver-api/pci/p2pdma.rst | 64 +++++++++++++++++++++++++++++---- 1 file changed, 58 insertions(+), 6 deletions(-)
diff --git a/Documentation/driver-api/pci/p2pdma.rst b/Documentation/driver-api/pci/p2pdma.rst index 42b18610bf7d..7a7f29f455b0 100644 --- a/Documentation/driver-api/pci/p2pdma.rst +++ b/Documentation/driver-api/pci/p2pdma.rst @@ -28,12 +28,64 @@ through the host bridge when either applicable port redirects. If an ACS Control register cannot be read, P2P DMA is rejected because the kernel cannot establish a usable route.
-This evaluation assumes clients issue strictly ordered Requests carrying an -Untranslated address. Its result is not defined when clients use Relaxed -Ordering or issue ATS-translated Requests because those TLP attributes can -select different routes through the fabric. Unless ACS Translation Blocking -is enabled, a Port with ACS Direct Translated P2P enabled routes a -Translated Request directly to the peer regardless of the redirect controls. +Three of those controls act on TLP attributes that the client chooses rather +than on the topology, so the same path routes differently for different +traffic. ACS Translation Blocking rejects any Request whose Address Type is +not Untranslated, and takes precedence over every other P2P control. ACS +Direct Translated P2P routes a Translated Request to the peer regardless of +Request Redirect and Egress Control. ACS Completion Redirect leaves alone +Completions that carry the Relaxed Ordering attribute. + +A client therefore describes its traffic with ``enum pci_p2pdma_tlp_flags`` +and asks ``pci_p2pdma_map_type_tlp()``. ``pci_p2pdma_map_type()`` answers for +the default: strictly ordered Requests carrying an Untranslated address. + +The two directions are decided independently. Translation Blocking (TB), +Direct Translated P2P (DT), Request Redirect (RR) and Egress Control (EC) on +the client-side port decide the Request: + +===== ===== ======= ============ ========== +TB DT RR/EC TLP class Request +===== ===== ======= ============ ========== +set x x translated blocked +clear set x translated direct +clear clear clear translated direct +clear clear set translated redirected +x x clear untranslated direct +x x set untranslated redirected +===== ===== ======= ============ ========== + +Completion Redirect (CR) on the provider-side port decides the Completions: + +===== ========= ========== +CR TLP class Completion +===== ========= ========== +x relaxed direct +clear strict direct +set strict redirected +===== ========= ========== + +Combining the two tables gives the mapping type. A class is bus addressable +when its Request and its Completions both route directly, and anything else +goes through the host bridge. A blocked Request has neither route, because +Translation Blocking rejects the Address Type itself rather than the +destination. With no ACS control set anywhere, every class routes directly. + +Note that DT only matters where RR or EC would otherwise redirect: it +overrides them for a Translated address rather than granting a direct route +that was not already there. + +Translation Blocking is not a routing control, so it is evaluated on every +port the Request passes rather than at the divergence alone. A direct route +turns around at the divergence and only passes the ports below it, while the +host bridge route keeps climbing and passes that port and everything above it +as well. Neither route falls back to the other, because the Address Type is +rejected wherever the Request is addressed. + +The Completer chooses whether a Completion carries Relaxed Ordering, and the +PCIe specification does not require it to copy that attribute from the +Request. A caller passing ``PCI_P2PDMA_TLP_RELAXED_CPL`` is asserting that +its provider does.
However, if the P2P transaction reaches the host bridge then it might have to hairpin back out the same root port, be routed inside the CPU SOC to another