From: Leon Romanovsky leonro@nvidia.com
The P2PDMA documentation leaves Relaxed Ordering and ATS-translated Requests undefined, although routing now depends on their attributes.
Describe the three TLP-sensitive ACS controls and the outcomes for each class. Clarify that Direct Translated P2P and Relaxed Ordering cannot create an internal route between sibling functions when hardware lacks that route.
Signed-off-by: Leon Romanovsky leonro@nvidia.com --- Documentation/driver-api/pci/p2pdma.rst | 68 ++++++++++++++++++++++++++++++--- 1 file changed, 62 insertions(+), 6 deletions(-)
diff --git a/Documentation/driver-api/pci/p2pdma.rst b/Documentation/driver-api/pci/p2pdma.rst index 91e6367151f1d..ff7f619616e8e 100644 --- a/Documentation/driver-api/pci/p2pdma.rst +++ b/Documentation/driver-api/pci/p2pdma.rst @@ -50,12 +50,68 @@ the host route while leaving direct routes available. The host bridge must also support P2P. Clearing redirect controls cannot create an internal route that the device does not support.
-This evaluation assumes clients issue strictly ordered Requests carrying an -Untranslated address. Its result is not defined when clients use Relaxed -Ordering or issue ATS-translated Requests because those TLP attributes can -select different routes through the fabric. Unless ACS Translation Blocking -is enabled, a Port with ACS Direct Translated P2P enabled routes a -Translated Request directly to the peer regardless of the redirect controls. +Three of those controls act on TLP attributes that the Requester chooses rather +than on the topology, so the same path routes differently for different +traffic. ACS Translation Blocking rejects any upstream Request whose Address +Type is not Untranslated, and takes precedence over every other P2P control. +ACS Direct Translated P2P routes a Translated Request to the peer regardless of +Request Redirect and Egress Control. ACS Completion Redirect leaves alone +Completions that carry the Relaxed Ordering attribute, which a Completion +copies from the Request it answers. + +P2PDMA therefore decides each class of traffic separately, and +``pci_p2pdma_map_type()`` answers for the one a DMA mapping is built for: +strictly ordered Requests carrying an Untranslated address. + +The two directions are decided independently. Translation Blocking (TB), +Direct Translated P2P (DT), Request Redirect (RR) and Egress Control (EC) on +the Requester-side port decide the Request: + +===== ===== ======= ============ ========== +TB DT RR/EC TLP class Request +===== ===== ======= ============ ========== +set x x translated blocked +clear set x translated direct +clear clear clear translated direct +clear clear set translated redirected +x x clear untranslated direct +x x set untranslated redirected +===== ===== ======= ============ ========== + +Completion Redirect (CR) on the Completer-side port decides the Completions: + +===== ========= ========== +CR TLP class Completion +===== ========= ========== +x relaxed direct +clear strict direct +set strict redirected +===== ========= ========== + +Combining the two tables gives the mapping type. A class is bus addressable +when its Request and its Completions both route directly, and anything else +goes through the host bridge. A blocked Request has neither route, because +Translation Blocking rejects the Address Type itself rather than the +destination. With no ACS control set anywhere, every class routes directly +provided that any required internal route between sibling functions exists. +Neither Direct Translated P2P nor Relaxed Ordering can create such a route. + +Some ports without an ACS capability, such as certain Zhaoxin Switch +Downstream Ports, redirect peer-to-peer traffic anyway, and a device-specific +ACS quirk tells the kernel so. P2PDMA treats such a port as having Request and +Completion Redirect set, but with no exception for Relaxed Ordering, so it +redirects every Completion. + +Note that DT only matters where RR or EC would otherwise redirect: it +overrides them for a Translated address rather than granting a direct route +that was not already there. + +Translation Blocking is not a routing control, so it is evaluated on every +port the Request passes on its way upstream rather than at the divergence +alone. A direct route turns around at the divergence and only passes the ports +below it, while the host bridge route keeps climbing and passes that port and +everything above it as well. Neither route falls back to the other, because +the Address Type is rejected wherever the Request is addressed.
However, if the P2P transaction reaches the host bridge then it might have to hairpin back out the same root port, be routed inside the CPU SOC to another