From: Leon Romanovsky leonro@nvidia.com
mlx5_umem_needs_ats() enables ATS for any dma-buf whose caller asked for Relaxed Ordering, on the assumption that a switch in the path has CR, RR and DT all set. It also enables it for a buffer already mapped with the peer's bus addresses, which are not translatable at all.
P2PDMA has read the ACS controls, so ask it through dma_buf_p2pdma_map_type(): enable ATS only where the path is not routed directly as it stands, but would be for a Translated Request whose Completions carry Relaxed Ordering. Exporters that name no provider keep the old assumption, since their ACS settings remain hidden.
Tested-by: Tushar Dave tdave@nvidia.com Signed-off-by: Leon Romanovsky leonro@nvidia.com --- drivers/infiniband/hw/mlx5/mlx5_ib.h | 36 ++------------------------- drivers/infiniband/hw/mlx5/mr.c | 47 ++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+), 34 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index e9ddf2e97a76..ab32742b2180 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -1646,40 +1646,8 @@ static inline bool rt_supported(int ts_cap) ts_cap == MLX5_TIMESTAMP_FORMAT_CAP_FREE_RUNNING_AND_REAL_TIME; }
-/* - * PCI Peer to Peer is a trainwreck. If no switch is present then things - * sometimes work, depending on the pci_distance_p2p logic for excluding broken - * root complexes. However if a switch is present in the path, then things get - * really ugly depending on how the switch is setup. This table assumes that the - * root complex is strict and is validating that all req/reps are matches - * perfectly - so any scenario where it sees only half the transaction is a - * failure. - * - * CR/RR/DT ATS RO P2P - * 00X X X OK - * 010 X X fails (request is routed to root but root never sees comp) - * 011 0 X fails (request is routed to root but root never sees comp) - * 011 1 X OK - * 10X X 1 OK - * 101 X 0 fails (completion is routed to root but root didn't see req) - * 110 X 0 SLOW - * 111 0 0 SLOW - * 111 1 0 fails (completion is routed to root but root didn't see req) - * 111 1 1 OK - * - * Unfortunately we cannot reliably know if a switch is present or what the - * CR/RR/DT ACS settings are, as in a VM that is all hidden. Assume that - * CR/RR/DT is 111 if the ATS cap is enabled and follow the last three rows. - * - * For now assume if the umem is a dma_buf then it is P2P. - */ -static inline bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev, - struct ib_umem *umem, int access_flags) -{ - if (!MLX5_CAP_GEN(dev->mdev, ats) || !umem->is_dmabuf) - return false; - return access_flags & IB_ACCESS_RELAXED_ORDERING; -} +bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev, struct ib_umem *umem, + int access_flags);
int set_roce_addr(struct mlx5_ib_dev *dev, u32 port_num, unsigned int index, const union ib_gid *gid, diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c index 00e13028762a..ee8982125c8c 100644 --- a/drivers/infiniband/hw/mlx5/mr.c +++ b/drivers/infiniband/hw/mlx5/mr.c @@ -38,6 +38,7 @@ #include <linux/export.h> #include <linux/delay.h> #include <linux/dma-buf.h> +#include <linux/dma-buf-mapping.h> #include <linux/dma-resv.h> #include <rdma/frmr_pools.h> #include <rdma/ib_umem_odp.h> @@ -47,6 +48,52 @@ #include "data_direct.h" #include "dmah.h"
+MODULE_IMPORT_NS("DMA_BUF"); + +bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev, struct ib_umem *umem, + int access_flags) +{ + struct dma_buf_attachment *attach; + bool ats; + + if (!MLX5_CAP_GEN(dev->mdev, ats) || !umem->is_dmabuf) + return false; + + /* + * The Completer decides whether its Completions carry Relaxed + * Ordering, and only a Request that asked for it can expect them to. + */ + if (!(access_flags & IB_ACCESS_RELAXED_ORDERING)) + return false; + + attach = to_ib_umem_dmabuf(umem)->attach; + + dma_resv_lock(attach->dmabuf->resv, NULL); + switch (dma_buf_p2pdma_map_type(attach, 0)) { + case PCI_P2PDMA_MAP_NONE: + /* Nothing is known about the route, so fall back to the bet. */ + ats = true; + break; + case PCI_P2PDMA_MAP_BUS_ADDR: + /* + * The path is routed directly already and is programmed with + * the peer's bus addresses. Those are not translatable, so + * ATS would be wrong as well as pointless. + */ + ats = false; + break; + default: + ats = dma_buf_p2pdma_map_type(attach, + PCI_P2PDMA_TLP_TRANSLATED | + PCI_P2PDMA_TLP_RELAXED_CPL) == + PCI_P2PDMA_MAP_BUS_ADDR; + break; + } + dma_resv_unlock(attach->dmabuf->resv); + + return ats; +} + static int mkey_max_umr_order(struct mlx5_ib_dev *dev) { if (MLX5_CAP_GEN(dev->mdev, umr_extended_translation_offset))