On Thu, Oct 16, 2025 at 11:30:06PM -0700, Christoph Hellwig wrote:
On Mon, Oct 13, 2025 at 06:26:03PM +0300, Leon Romanovsky wrote:
The DMA API now has a new flow, and has gained phys_addr_t support, so it no longer needs struct pages to perform P2P mapping.
That's news to me. All the pci_p2pdma_map_state machinery is still based on pgmaps and thus pages.
We had this discussion already three months ago:
https://lore.kernel.org/all/20250729131502.GJ36037@nvidia.com/
These couple patches make the core pci_p2pdma_map_state machinery work on struct p2pdma_provider, and pgmap is just one way to get a p2pdma_provider *
The struct page paths through pgmap go page->pgmap->mem to get p2pdma_provider.
The non-struct page paths just have a p2pdma_provider * without a pgmap. In this series VFIO uses
+ *provider = pcim_p2pdma_provider(pdev, bar);
To get the provider for a specific BAR.
Lifecycle management can be delegated to the user, DMABUF for instance has a suitable invalidation protocol that does not require struct page.
How?
I think I've answered this three times now - for DMABUF the DMABUF invalidation scheme is used to control the lifetime and no DMA mapping outlives the provider, and the provider doesn't outlive the driver.
Hotplug works fine. VFIO gets the driver removal callback, it invalidates all the DMABUFs, refuses to re-validate them, destroys the P2P provider, and ends its driver. There is no lifetime issue.
Obviously you cannot use the new p2provider mechanism without some kind of protection against use after hot unplug, but it doesn't have to be struct page based.
For VFIO the invalidation scheme is linked to dma_buf_move_notify(), for instance the hotunplug case goes:
static const struct vfio_device_ops vfio_pci_ops = { .close_device = vfio_pci_core_close_device,
vfio_pci_dma_buf_cleanup(vdev);
dma_buf_move_notify(priv->dmabuf);
And then if we follow that into an importer like RDMA:
static struct dma_buf_attach_ops mlx5_ib_dmabuf_attach_ops = { .move_notify = mlx5_ib_dmabuf_invalidate_cb,
mlx5r_umr_update_mr_pas(mr, MLX5_IB_UPD_XLT_ZAP); ib_umem_dmabuf_unmap_pages(umem_dmabuf); dma_buf_unmap_attachment(umem_dmabuf->attach, umem_dmabuf->sgt, DMA_BIDIRECTIONAL); vfio_pci_dma_buf_unmap()
XLT_ZAP tells the HW to stop doing DMA and the unmap_pages -> unmap_attachment -> vfio_pci_dma_buf_unmap() flow will tear down the DMA API mapping and remove it from the IOMMU. All of this happens before device_driver remove completes.
There is no lifecycle issue here and we don't need pgmap to solve a livecycle problem or to help find the p2pdma_provider.
Jason