Hi all,
The goal of this series is to enable userspace driver designs that use VFIO to export DMABUFs representing subsets of PCI device BARs, and "vend" those buffers from a primary process to other subordinate processes by fd. These processes then mmap() the buffers and their access to the device is isolated to the exported ranges. This is an improvement on sharing the VFIO device fd to subordinate processes, which would allow unfettered access.
This is achieved by enabling mmap() of vfio-pci DMABUFs, passed by fd to subordinate processes. Second, a new revocation mechanism is added to allow the primary process to forcibly revoke access to previously-shared BAR spans, even if the subordinate processes haven't cleanly exited.
(The related topic of safe delegation of iommufd control to the subordinate processes is not addressed here, and is follow-up work.)
The background/rationale is covered in more detail in the RFC cover letters.
Feedback from the RFCs requested that, instead of creating DMABUF-specific vm_ops and .fault paths, to go the whole way and migrate the existing VFIO PCI BAR mmap() to be backed by a DMABUF too, resulting in a common vm_ops and fault handler for mmap()s of both the VFIO device and explicitly-exported DMABUFs. This will help future iommufd emulation of VFIO Type1 peer-to-peer, making it easier to get a DMABUF for a VFIO BAR as a DMA target.
mmap() conversion to use DMABUF underneath has been done for vfio-pci, but not sub-drivers:
nvgrace-gpu's mmap() override path is unchanged; I kept this out of scope for now not least because I don't have a thorough test setup for this system. I would prefer to help the nvgrace-gpu maintainers enable BAR mmap() DMABUFs themselves.
Notes on patches ================
PCI/P2PDMA: Split pool-related cleanup out of pci_p2pdma_release() PCI/P2PDMA: Add CONFIG_PCI_P2PDMA_CORE
Later in the series, vfio-pci's mmap() is going to depend on pcim_p2pdma_provider() which depended on CONFIG_PCI_P2PDMA, which in turn depended on ZONE_DEVICE. That isn't available on 32-bit and some archs, because they lack MEMORY_HOTPLUG and friends. VFIO does _not_ require actual P2P to be present for basic mmap() functionality, only for the optional CONFIG_DMA_SHARED_BUFFER feature.
These split out p2pdma_core.c under CONFIG_PCI_P2PDMA_CORE (which currently contains pcim_p2pdma_provider()), and an optional CONFIG_PCI_P2PDMA which depends on ZONE_DEVICE etc. providing P2P functionality in the existing p2pdma.c. The first splits out pool cleanup from the release path, and the second does the refactor/code move to the new file.
vfio/pci: Add a helper to look up PFNs for DMABUFs vfio/pci: Add a helper to create a DMABUF for a BAR-map VMA
The first adds a DMABUF VMA fault handler helper to determine arbitrary-sized PFNs from ranges in DMABUF. The second refactors DMABUF export for use by the existing export feature, and adds a helper that creates a DMABUF corresponding to a VFIO BAR mmap() request.
vfio/pci: Convert BAR mmap() to use a DMABUF
The vfio-pci core mmap() creates a DMABUF with the helper above, and the vm_ops fault handler uses the other helper to resolve the fault. Because this depends on DMABUF structs/code, CONFIG_VFIO_PCI_CORE needs to depend on CONFIG_DMA_SHARED_BUFFER. The CONFIG_VFIO_PCI_DMABUF still conditionally enables the export support code.
NOTE: The user mmap()s a device fd, but the resulting VMA's vm_file becomes that of the DMABUF. The DMABUF takes ownership of the device file and put()s it on release, which maintains the existing behaviour of a VMA keeping the VFIO device open.
BAR zapping then happens via the existing vfio_pci_dma_buf_move() path, which now needs to unmap PTEs in the DMABUF's address_space.
vfio/pci: Provide a user-facing name for BAR mappings
There was a request for decent debug naming in /proc/<pid>/maps etc. comparable to the existing VFIO names: since the VMAs are DMABUFs, they have a "dmabuf:" prefix and can't be 100% identical to before. This is a user-visible change, but this patch at least now gives us extra info on the BDF & BAR being mapped.
vfio/pci: Clean up BAR zap and revocation
In general (see NOTE!) the vfio_pci_zap_bars() is now obsolete, since it unmaps PTEs in the VFIO device address_space which is now unused. This consolidates all calls (e.g. around reset) with the neighbouring vfio_pci_dma_buf_move()s into new functions, to revoke/unrevoke (making the steps clearer).
NOTE: Because drivers can use their own vm_ops and override .mmap, the core must conservatively assume an overridden .mmap might still add PTEs to the VFIO device address_space and therefore still does the zap. A new flag, zap_bars_on_revoke, enables the zap when .mmap is overridden. A driver that does not need the zap can clear this to opt-out, e.g. if the driver calls down to the common mmap (and so uses DMABUFs). hisi-acc-vfio-pci does just this, and thus sets the opt-out flag.
vfio/pci: Support mmap() of a VFIO DMABUF
Adds mmap() for a DMABUF fd exported from vfio-pci.
It was a goal to keep the VFIO device fd lifetime behaviour unchanged with respect to the DMABUFs. An application can close all device fds, and this will revoke/clean up all DMABUFs; then, no mappings or other access can be performed. When enabling mmap() of the DMABUFs, this means access through the VMA is also revoked. This complicates the fault handler because whilst the DMABUF exists, it has no guarantee that the corresponding VFIO device is still alive. Adds synchronisation ensuring the vdev is available before vdev->memory_lock is touched; this holds the device registration so that even if the buffer has been cleaned up, vdev hasn't been freed and so the lock can be safely taken.
vfio/pci: Permanently revoke a DMABUF on request
This is mostly a rename of `revoked` to an enum, `status`, and adding a third state for a buffer: usable, revoked temporary, revoked permanent. A new VFIO feature is added, VFIO_DEVICE_FEATURE_DMA_BUF_REVOKE, which takes a DMABUF (exported from the same device) and permanently revokes it. Thus a userspace driver can guarantee any downstream consumers of a shared fd are prevented from accessing a BAR range, and that range can be reused. NOTE: This might block userspace, waiting on importers to detach.
The code doing revocation in vfio_pci_dma_buf_move() is moved, to a common function for use by ..._move() and this new feature.
NOTE: See changelog, by request v4 added a condition to the existing code to elide the unnecessary invalidation/sync on the un-revoke path.)
NOTE: Previous versions contained an additional feature patch, "vfio/pci: Add mmap() attributes to DMABUF feature". This has been dropped in v5 because:
- The mechanism simply set vma->vm_page_prot. This would be sufficient for arm64 and other architectures.
- However, (locally-run claude-opus-4-8) Sashiko flagged that, on x86, additional memtype handling is required to set up the PAT. Without this, the memtype is returned back to UC- by pfnmap_setup_cachemode() upon PTE creation.
Most other sources of userspace WC mappings create PTEs eagerly with e.g. io_remap_pfn_range() which memtype_reserve() WC for the range. Getting them with lazy-fault used by vfio-pci is more complicated (e.g. perhaps registering WC for BARs with PAT/MTRRs, and deciding how to deal with aliasing...). Since this feature is not critical for this series to be useful, I've decided for now to drop it in favour of a simpler series now and revisiting this separ*ately.
Testing =======
(The [RFC ONLY] userspace test program, for QEMU edu-plus, can be found in the GitHub branch below. It at least illustrates how the export, map, revoke, and close semantics interoperate.)
This code has been tested in mapping DMABUFs of single/multiple ranges from multiple BARs, aliasing mmap()s, aliasing ranges across DMABUFs, vm_pgoff > 0, revocation, shutdown/cleanup scenarios, and hugepage mappings. No regressions observed on the VFIO selftests, or on our internal vfio-pci applications. VFIO on i386 has been build-tested.
Dear Reviewers, ===============
I was grateful for the reviews and Reviewed-Bys on previous versions. Thanks; I've added some Reviewed-Bys/Acks. I have NOT included your tags where the patch has materially changed after your review (or where requested changes ended up more than super-trivial). I hope that's okay.
End ===
This is based on v7.2-rc3.
These commits are on GitHub for easier browsing, along with "[RFC ONLY] selftests: vfio: Add standalone vfio_dmabuf_mmap_test":
https://github.com/metamev/linux/compare/v7.2-rc3...dev/mev/vfio-dmabuf-mmap...
Thanks for reading,
Matt
================================================================================ Changelog:
v5:
- Rebased on 7.2-rc3
- Dropped the memattr/WC feature (see explanation above).
- "vfio/pci: Convert BAR mmap() to use a DMABUF": Fixed a potentially-nasty bug (which (locally-run) Sashiko found!) whereby the unmap_mapping_range() performed in cleanup was passed a range up from offset zero for the DMABUF size. Initially this was how all DMABUFs were created and an appropriate zap, but a new version kept the VFIO region index encoded in the offset -- for BAR > 0 the unmap span would then mismatch. Instead, pass size 0 to mean an "all" range. Because the goal is to shoot down everything relating to one DMABUF and the address_space can only contain things relating to that DMABUF, this is equivalent and has the bonus of never failing to match mappings...
Praan, Kevin, I kept your R-Bs on this fix.
- The revoke patch converts vfio_pci_dma_buf_cleanup()'s priv->vdev = NULL to a WRITE_ONCE, corresponding to the revoke function's READ_ONCE (performed to test that the VFIO and DMABUF are related).
- Clarified the VFIO_DEVICE_FEATURE_DMA_BUF_REVOKE UAPI comments, documenting previously-missing error cases and their reasons.
v4: https://lore.kernel.org/all/20260701171245.90111-1-matt@ozlabs.org/ v3: https://lore.kernel.org/all/20260610154327.37758-1-matt@ozlabs.org/ v2: https://lore.kernel.org/all/20260527102319.100128-1-mattev@meta.com/ v1: https://lore.kernel.org/kvm/20260416131815.2729131-1-mattev@meta.com/ RFCv2: https://lore.kernel.org/kvm/20260312184613.3710705-1-mattev@meta.com/ RFCv1: https://lore.kernel.org/all/20260226202211.929005-1-mattev@meta.com/ Tech topic: https://lore.kernel.org/linux-iommu/20250918214425.2677057-1-amastro@fb.com/
Matt Evans (9): PCI/P2PDMA: Split pool-related cleanup out of pci_p2pdma_release() PCI/P2PDMA: Add CONFIG_PCI_P2PDMA_CORE vfio/pci: Add a helper to look up PFNs for DMABUFs vfio/pci: Add a helper to create a DMABUF for a BAR-map VMA vfio/pci: Convert BAR mmap() to use a DMABUF vfio/pci: Provide a user-facing name for BAR mappings vfio/pci: Clean up BAR zap and revocation vfio/pci: Support mmap() of a VFIO DMABUF vfio/pci: Permanently revoke a DMABUF on request
MAINTAINERS | 2 +- drivers/pci/Kconfig | 5 + drivers/pci/Makefile | 1 + drivers/pci/p2pdma.c | 113 +--- drivers/pci/p2pdma.h | 29 + drivers/pci/p2pdma_core.c | 122 +++++ drivers/vfio/pci/Kconfig | 5 +- drivers/vfio/pci/Makefile | 3 +- .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 8 + drivers/vfio/pci/vfio_pci_config.c | 30 +- drivers/vfio/pci/vfio_pci_core.c | 210 +++++-- drivers/vfio/pci/vfio_pci_dmabuf.c | 515 +++++++++++++++--- drivers/vfio/pci/vfio_pci_priv.h | 53 +- include/linux/pci-p2pdma.h | 24 +- include/linux/pci.h | 2 +- include/linux/vfio_pci_core.h | 1 + include/uapi/linux/vfio.h | 25 + 17 files changed, 875 insertions(+), 273 deletions(-) create mode 100644 drivers/pci/p2pdma.h create mode 100644 drivers/pci/p2pdma_core.c