Linaro-mm-sig

linaro-mm-sig@lists.linaro.org

5 participants
3604 discussions

Re: [RFC PATCH v1 1/2] dma-buf: keep DMABUF_DEBUG off by default

by Christian König

On 9/23/26 09:42, Jianfeng Liu wrote: > [Sie erhalten nicht häufig E-Mails von liujianfeng1994(a)gmail.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ] > > Commit 143755bdabaa9 ("dma-buf: Make DMABUF_DEBUG default to y on > DEBUG_KERNEL kernels") fixed a dangling reference in the DMABUF_DEBUG > default, which had the side effect of enabling the option (and with it > the page-stripping sg_table wrapper handed to importers) on every > kernel with DEBUG_KERNEL=y - i.e. virtually every distro kernel. > > drm/msm is broken by this: it maps imported dma-bufs into the GPU's > own pagetables with iommu_map_sgtable(), which needs the struct page > of the attachment sg_table, and it fills the GEM object's page array > through drm_prime_sg_to_page_array(). With the debug wrapper in > place both silently produce garbage (the wrapper zeroes sg->length, Interesting point, we should probably change that. > so the page iterator yields nothing and an uninitialized array is > kept). The VM_BIND map job then fails asynchronously after userspace > has already enqueued GPU work referencing the mapping, which shows up > as an arm-smmu translation fault from UCHE, e.g.: > > gpu fault: ttbr0=000000088a889000 iova=000000010741c000 dir=READ > type=TRANSLATION source=UCHE > > This breaks hardware video decode (clapper, chromium) on Adreno > systems; bisected on a Snapdragon laptop as v7.3-rc3 good, > v7.3-rc4 bad, culprit 143755bdabaa9. > > Revert the default until importers that legitimately need to build > phys-based mappings have been converted. Well that won't work like this, pointing those things out is exactly what DMABUF_DEBUG is made for. What MSM is doing here is not allowed at all and can break badly. We gave drivers 5 years to get that fixed and I'm now pushing for completely deprecating that hack. See patch 84335675f2223cbd25d0de7d38ecc7d40b95bd4a: Author: Simona Vetter <simona.vetter(a)ffwll.ch> Date: Fri Jan 15 17:47:39 2021 +0100 dma-buf: Add debug option When MSM needs the struct page then it must import a shmemfd and not a DMA-buf. What we could do is to either fix MSM or mark it as broken. Regards, Christian. > > Fixes: 143755bdabaa9 ("dma-buf: Make DMABUF_DEBUG default to y on DEBUG_KERNEL kernels") > Signed-off-by: Jianfeng Liu <liujianfeng1994(a)gmail.com> > --- > > drivers/dma-buf/Kconfig | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/drivers/dma-buf/Kconfig b/drivers/dma-buf/Kconfig > index e4f078a326a41..b3c581ef4c987 100644 > --- a/drivers/dma-buf/Kconfig > +++ b/drivers/dma-buf/Kconfig > @@ -43,7 +43,14 @@ config UDMABUF > config DMABUF_DEBUG > bool "DMA-BUF debug checks" > depends on DMA_SHARED_BUFFER > - default y if DEBUG_KERNEL > + # NOTE: keep this default n. The page-stripping sg_table wrapper that > + # this option installs for importers breaks drivers that build a > + # second-stage IOMMU mapping (phys -> iova) from the attachment sg_table > + # and therefore still need the struct page, e.g. drm/msm with its > + # per-process GPU pagetables. Until those importers are fixed, making > + # this default y breaks hardware video decode and GPU workloads out of > + # the box on affected systems. > + default n > help > This option enables additional checks for DMA-BUF importers and > exporters. Specifically it validates that importers do not peek at the > -- > 2.47.3 >

1 week, 2 days

Re: [PATCH v2] dma-fence: deliver set_deadline callback even if fence is already signaled

by Christian König

On 9/22/26 18:57, Jeffrey Boody wrote: > The set_deadline callback is currently skipped if the fence has already > been signaled. This prevents GPU drivers from performing power > management adjustments when the deadline hint arrives after fence > completion. > > In triple-buffered rendering, a staged frame may be completed well > ahead of the vblank deadline. When a display driver delivers the > deadline hint, the fence has already been signaled and the callback is > silently dropped. This leaves the GPU driver unable to evaluate the > headroom between the fence signal time and the vblank deadline, and > therefore unable to reduce GPU frequency when the target headroom is > exceeded. > > Remove the dma_fence_is_signaled() guard from dma_fence_set_deadline() > so that the callback is invoked unconditionally when ops->set_deadline > is present. Implementations of set_deadline must already tolerate > concurrent and repeated calls; handling a post-signal invocation > requires no additional locking. The fence signaler can compare the fence > signal time against the supplied deadline to determine whether frequency > scaling is warranted. Sorry but I have to clearly reject that patch. No callback whatsoever is allowed to be used after the fence has signaled or otherwise we break module unloading for the originator of the fence. So that approach you want to have here simply doesn't work at all. Regards, Christian. > > Signed-off-by: Jeffrey Boody <jeffrey.boody(a)oss.qualcomm.com> > --- > Signed-off-by: Jeff Boody <jeffrey.boody(a)oss.qualcomm.com> > --- > drivers/dma-buf/dma-fence.c | 22 ++++++++++++++++++++-- > drivers/gpu/drm/msm/msm_fence.c | 3 +++ > include/linux/dma-fence.h | 11 ++++++++++- > 3 files changed, 33 insertions(+), 3 deletions(-) > > diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c > index bd58688b81a7..ebc7c5ca6f69 100644 > --- a/drivers/dma-buf/dma-fence.c > +++ b/drivers/dma-buf/dma-fence.c > @@ -999,8 +999,19 @@ EXPORT_SYMBOL(dma_fence_wait_any_timeout); > * Multiple deadlines may be set on a given fence, even in parallel. See the > * documentation for &dma_fence_ops.set_deadline. > * > + * The deadline hint may also be delivered *after* the fence has already been > + * signaled. This is intentional and supports the case where a fence signaler > + * aware of a periodic deadline (e.g. vblank) and the fence's signal time can > + * evaluate the headroom between the two. In triple-buffered rendering, for > + * example, a staged frame that is completed well ahead of the vblank deadline > + * represents excess headroom; delivering the deadline hint post-signal allows > + * the fence signaler to consider reducing frequency for subsequent workloads, > + * rather than holding an unnecessarily high frequency. Implementations > + * of &dma_fence_ops.set_deadline must therefore tolerate invocation on > + * already-signaled fences. > + * > * The deadline hint is just that, a hint. The driver that created the fence > - * may react by increasing frequency, making different scheduling choices, etc. > + * may react by changing frequency, making different scheduling choices, etc. > * Or doing nothing at all. > */ > > @@ -1016,6 +1027,13 @@ EXPORT_SYMBOL(dma_fence_wait_any_timeout); > * to aid in power management decisions, such as boosting GPU frequency > * if a periodic vblank deadline is approaching but the fence is not > * yet signaled.. > + * > + * This function may also be called after the fence has already been > + * signaled. In that case the fence signaler can compare the fence's signal > + * time against the deadline to determine the available headroom. If the > + * fence was signaled significantly ahead of the deadline, the fence > + * signaler may choose to reduce frequency for subsequent workloads to > + * avoid unnecessarily high power consumption. > */ > void dma_fence_set_deadline(struct dma_fence *fence, ktime_t deadline) > { > @@ -1023,7 +1041,7 @@ void dma_fence_set_deadline(struct dma_fence *fence, ktime_t deadline) > > rcu_read_lock(); > ops = rcu_dereference(fence->ops); > - if (ops && ops->set_deadline && !dma_fence_is_signaled(fence)) > + if (ops && ops->set_deadline) > ops->set_deadline(fence, deadline); > rcu_read_unlock(); > } > diff --git a/drivers/gpu/drm/msm/msm_fence.c b/drivers/gpu/drm/msm/msm_fence.c > index 3dca8e09c192..3c5de96d4092 100644 > --- a/drivers/gpu/drm/msm/msm_fence.c > +++ b/drivers/gpu/drm/msm/msm_fence.c > @@ -136,6 +136,9 @@ static void msm_fence_set_deadline(struct dma_fence *fence, ktime_t deadline) > unsigned long flags; > ktime_t now; > > + if (dma_fence_is_signaled(fence)) > + return; > + > spin_lock_irqsave(&fctx->spinlock, flags); > now = ktime_get(); > > diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h > index ffa99b930843..839ef2e5dad9 100644 > --- a/include/linux/dma-fence.h > +++ b/include/linux/dma-fence.h > @@ -264,7 +264,16 @@ struct dma_fence_ops { > * an upcoming deadline, such as vblank, by which point the waiter > * would prefer the fence to be signaled by. This is intended to > * give feedback to the fence signaler to aid in power management > - * decisions, such as boosting GPU frequency. > + * decisions, such as boosting GPU frequency if the deadline has > + * not yet been met, or reducing GPU frequency if the fence was > + * signaled significantly ahead of the deadline. > + * > + * This callback may be invoked even after the fence has been > + * signaled. In this case, the signaler may use the deadline and > + * the fence's signal time to evaluate whether the GPU frequency > + * should be adjusted for future workloads. Implementations must > + * therefore be prepared to handle calls on already-signaled fences > + * without error. > * > * This is called without &dma_fence.lock held, it can be called > * multiple times and from any context. Locking is up to the callee > > --- > base-commit: f0100363d8c374bd8e9ea7c9ba02744f0b802ca4 > change-id: 20260917-dma-fence-set-deadline-a778746e9abd > > Best regards, > -- > Jeff Boody <jeffrey.boody(a)oss.qualcomm.com> >

1 week, 2 days

Re: [PATCH v6 06/13] nvme-pci: implement dma-buf backed requests

by Christoph Hellwig

On Tue, Sep 22, 2026 at 02:37:08PM +0100, Pavel Begunkov wrote: >> Sashiko had a few comments, which I think are correct - if we use >> dma_map_sg to map the data, we need to use the sync_sg APIs to >> transfer ownership. That only matters on non-coherent architectures >> with MMU, but we need to get it right. > > I've seen that and fixed everything locally that should be fixed, > apart from the sync. I wonder what we can do about that? I can > somehow replace it with the sg variant for now, but sync'ing the > entire possibly multi-GB mapping for, let's say, a 512B I/O, sounds > not wise. If we'd want to fix this for real we'd need to add an offset to the sync by sg methods. Or move away from scatterlists in dma-buf, given that they are a horrible API. All of these are bigger projects, though. So for now I think you'd want to do the sync all, and have the people who run high-performance io_uring code on non-coherent platforms suffer. > And I can think of another place that does mix sync_single > with sgs. Which one?

1 week, 2 days

Re: [PATCH v6 04/13] block: introduce dma map backed bio type

by Christoph Hellwig

On Tue, Sep 22, 2026 at 02:54:25PM +0100, Pavel Begunkov wrote: >> >> Can you please expland on the splitting considerations a bit? Preferably >> both in a comment for the details, and in the commit log for how we >> arrived at them and why they are fine for now? > > I'll add something to the commit log, but not sure which specific > details in comments you mean. Do you want me to expand the comment > above limiting the number of segments? Maybe I should add there that > it's stricter than necessary and might cause more splitting than > necessary? This one: Yes, explain why we apply a relatively arbitrary limit not fully exploiting the hardware capabilities, and why we thing that is okay.

1 week, 2 days

Re: [PATCH v6 06/13] nvme-pci: implement dma-buf backed requests

by Christoph Hellwig

On Mon, Sep 21, 2026 at 02:38:50PM +0100, Pavel Begunkov wrote: > Enable BIO_DMABUF_MAP backed requests. On registration we map the This is now REQ_DMABUF. Sashiko had a few comments, which I think are correct - if we use dma_map_sg to map the data, we need to use the sync_sg APIs to transfer ownership. That only matters on non-coherent architectures with MMU, but we need to get it right. And the reset deadlock also looks plausible.

1 week, 2 days

Re: [PATCH v6 04/13] block: introduce dma map backed bio type

by Christoph Hellwig

On Mon, Sep 21, 2026 at 02:38:48PM +0100, Pavel Begunkov wrote: > Premapped buffers don't require a generic bio_vec since these have > already been dma mapped. Repurpose the bi_io_vec space to strore dmabuf s/strore/store/ Can you please expland on the splitting considerations a bit? Preferably both in a comment for the details, and in the commit log for how we arrived at them and why they are fine for now? Otherwise looks good.

1 week, 2 days

[PATCH v6 0/9] vfio/pci: Add mmap() for DMABUFs

by Matt Evans

Hi all, The goal of this series is to enable userspace driver designs that use VFIO to export DMABUFs representing subsets of PCI device BARs, and "vend" those buffers from a primary process to other subordinate processes by fd. These processes then mmap() the buffers and their access to the device is isolated to the exported ranges. This is an improvement on sharing the VFIO device fd to subordinate processes, which would allow unfettered access. This is achieved by enabling mmap() of vfio-pci DMABUFs, passed by fd to subordinate processes. Second, a new revocation mechanism is added to allow the primary process to forcibly revoke access to previously-shared BAR spans, even if the subordinate processes haven't cleanly exited. (The related topic of safe delegation of iommufd control to the subordinate processes is not addressed here, and is follow-up work.) The background/rationale is covered in more detail in the RFC cover letters. Reviews requested that we migrate the existing VFIO PCI BAR mmap() to be backed by a DMABUF too, resulting in a common vm_ops and fault handler for mmap()s of both the VFIO device and explicitly-exported DMABUFs. This will help future iommufd emulation of VFIO Type1 peer-to-peer, making it easier to get a DMABUF for a VFIO BAR as a DMA target. mmap() conversion to use DMABUF underneath has been done for vfio-pci, but not sub-drivers: nvgrace-gpu's mmap() override path is unchanged; I kept this out of scope for now not least because I don't have a thorough test setup for this system. I would prefer to help the nvgrace-gpu maintainers enable BAR mmap() DMABUFs themselves. Notes on patches ================ vfio/pci: Remove DMABUF export dependency on vdev->memory_lock In v5 of this series [1] we discover that doing an export from the VFIO mmap() path (fundamental!) adds a dependency between mmap_lock and vdev->memory_lock(W) because export was using memory_lock(W) to protect the vdev->dmabufs list and state within, and a deadlock scenario leapt out to bite. (Details in [1].) The suggestion was to add a dedicated mutex/rwsem specifically for the DMABUFs/list, which is cleaner than overloading memory_lock(W). But whilst export could now downgrade to holding memory_lock just for _read_ to test __vfio_pci_memory_enabled(), a very similar deadlock can still arise due to a memory_lock(W) elsewhere depending on a prior memory_lock(R) to be released, and attempting to take memory_lock(R) will queue behind the (W) for fairness (effectively an R->R dependency). I'd overlooked that rwsem cannot guarantee multiple readers. To be able to export while holding mmap_lock, export cannot hold memory_lock at all. Instead of testing __vfio_pci_memory_enabled() (which tracks PCI_COMMAND.MSE and PM state), this patch tracks device-global DMABUF revocation state in a new vdev->bars_revoked flag updated by vfio_pci_dma_buf_move(). If an mmap() is somehow performed during a period when DMABUFs are all revoked, then the DMABUF is created revoked. Move() already bookends reset, PM transitions etc., so subsequent revocation state changes work as-is. This flag is also protected by the dmabuf_lock and thus memory_lock is not required to export. vfio/pci: Un-revoke DMABUFs in LOW_POWER_ENTRY_WITH_WAKEUP resume On LOW_POWER entry, DMABUFs have a move(revoke=true), but the runtime resume path didn't un-revoke. This adds a corresponding move(revoke=false), which will later turn into vfio_pci_unrevoke_bars(). NOTE: the unrevoke is reordered _before_ the eventfd_signal() in vfio_pci_core_runtime_resume(). This removes a window in which a waking waiter could have observed the DMABUF state as still revoked (or pm_runtime_engaged = true). (The UAPI docs state the event means the resume's complete, so observing otherwise seemed unintended.) Also, when DMABUFs are later mmap()ed, a waiter waking and taking a fault could have seen the unrevoked state and SIGBUS just before taking the memory_lock. (If the handler gets to acquire the lock, though, the resume sequence is complete, and the handler observes pm_runtime_engaged = false.) This fix is in this series because the issue will impact CPU access to the VMA as well (once they use DMABUFs), and so it's a strict dependency of later commits. dma-buf: Export dma_buf_set_name() Makes dma_buf_set_name() available for use (by helper patch), taking a kernel-allocated string. The pre-existing local helper becomes a wrapper copying a __user string for the set-name ioctl. vfio/pci: Add a helper to look up PFNs for DMABUFs Adds a DMABUF VMA fault handler helper to determine arbitrary-sized PFNs from ranges in DMABUF. vfio/pci: Add a helper to create a DMABUF for a BAR-map VMA Refactors DMABUF export for use by the existing export feature, and adds a helper that creates a DMABUF corresponding to a VFIO BAR mmap() request. There was a request for decent debug naming in /proc/<pid>/maps etc. comparable to the existing VFIO names: since the VMAs are DMABUFs, they have a "dmabuf:" prefix and can't be 100% identical to before. This is a user-visible change, but this patch at least now gives us extra info on the BDF & BAR being mmap()ed. The name is installed with the new dma_buf_set_name() above. An alternative would be to add a creation-time name field to struct dma_buf_export_info, but a setter function is more useful: the name can be changed at will after init. vfio/pci: Convert BAR mmap() to use a DMABUF The vfio-pci core mmap() creates a DMABUF with the helper above, and the vm_ops fault handler uses the other helper to resolve the fault. Because this depends on DMABUF structs/code, CONFIG_VFIO_PCI_CORE needs to depend on CONFIG_DMA_SHARED_BUFFER. The CONFIG_VFIO_PCI_DMABUF still conditionally enables the export support code. NOTE: The user mmap()s a device fd, but the resulting VMA's vm_file becomes that of the DMABUF. The DMABUF takes ownership of the device file and put()s it on release, which maintains the existing behaviour of a VMA keeping the VFIO device open. BAR zapping then happens via the existing vfio_pci_dma_buf_move() path, which now needs to unmap PTEs in the DMABUF's address_space. NOTE: As local LLM reviews did, Sashiko might falsely worry about the DMABUF fd being obtained through /proc/pid/map_files and remapped, but the file is an anon inode and doesn't support the open op (-ENXIO) so AFAICT this is currently impossible. NOTE: A side-effect of this is that is_mergeable_vma() will be false between adjacent mappings of VFIO BARs; merging would require rebuilding a larger DMABUF representing the union, not just plugging the VMAs together. The DMABUF backing the BAR is an implicit/internal export, even when the CONFIG_VFIO_PCI_DMABUF feature is not included because CONFIG_PCI_P2PDMA is not available. Without P2PDMA, it's acceptable for the DMABUF to not have a P2PDMA provider. In this configuration, the VFIO DMABUF .attach prohibits any import, which avoids getting as far as a WARN in dma_buf_map_attachment(), which would fail without P2PDMA anyway. vfio/pci: Clean up BAR zap and revocation In general (see NOTE!) the vfio_pci_zap_bars() is now obsolete, since it unmaps PTEs in the VFIO device address_space which is now unused. This consolidates all calls (e.g. around reset) with the neighbouring vfio_pci_dma_buf_move()s into new functions, to revoke/unrevoke (making the steps clearer). NOTE: Because drivers can use their own vm_ops and override .mmap, the core must conservatively assume an overridden .mmap might still add PTEs to the VFIO device address_space and therefore still does the zap. A new flag, zap_bars_on_revoke, enables the zap when .mmap is overridden. A driver that does not need the zap can clear this to opt-out, e.g. if the driver calls down to the common mmap (and so uses DMABUFs). hisi-acc-vfio-pci does just this, and thus sets the opt-out flag. vfio/pci: Support mmap() of a VFIO DMABUF Adds mmap() for a DMABUF fd exported from vfio-pci. It was a goal to keep the VFIO device fd lifetime behaviour unchanged with respect to the DMABUFs. An application can close all device fds, and this will revoke/clean up all DMABUFs; then, no mappings or other access can be performed. When enabling mmap() of the DMABUFs, this means access through the VMA is also revoked. This complicates the fault handler because whilst the DMABUF exists, it has no guarantee that the corresponding VFIO device is still alive. Adds synchronisation ensuring the vdev is available before the locks in vdev are touched; this holds the device registration so that even if the buffer has been cleaned up, vdev hasn't been freed and so the locks can be safely taken. vfio/pci: Permanently revoke a DMABUF on request This is mostly a rename of `revoked` to an enum, `status`, and adding a third state for a buffer: usable, revoked temporary, revoked permanent. A new VFIO feature is added, VFIO_DEVICE_FEATURE_DMA_BUF_REVOKE, which takes a DMABUF (exported from the same device) and permanently revokes it. Thus a userspace driver can guarantee any downstream consumers of a shared fd are prevented from accessing a BAR range, and that range can be reused. NOTE: This might block userspace, waiting on importers to detach. The code doing revocation in vfio_pci_dma_buf_move() is moved, to a common function for use by ..._move() and this new feature. Testing ======= (The [RFC ONLY] userspace test program, which drives a QEMU bochs-display function, can be found in the GitHub branch below. It at least illustrates how the export, map, revoke, and close semantics interoperate. WIP on a follow-up with a proper vfio-selftests style test based on this -- this won't be part of this series.) This code has been tested in mapping DMABUFs of single/multiple ranges from multiple BARs, aliasing mmap()s, aliasing ranges across DMABUFs, vm_pgoff > 0, revocation, shutdown/cleanup scenarios, and hugepage mappings. No regressions observed on the VFIO selftests, or on our internal vfio-pci applications. VFIO on i386 has been build-tested. Thanks to Alex Mastro for building a (WIP) testcase for the mmap_lock->memory_lock issue (which is now OK...). Dear Reviewers, =============== There was a lot of finessing v5->v6, all patches had fixes/changes/cleanups/rewordings made, and so I have dropped previous R-B tags. Along the way several related issues came up that warrant more eyes, and I'd be grateful for your input: 1. The mmap fault handler takes a bunch of locks non-interruptibly, and potentially depends on a lot of DMABUF-related activities completing. I had a go at converting them to interruptible/killable forms, but pulling on the thread revealed there seems to be a wider issue if move/revoke doesn't complete in a timely fashion (due to buggy importers). Where I got to was that just updating the fault handler won't fix the user experience of an unkillable task, and move()/revocation will need thought too. I don't intend to fix this here but wanted to start discussion so we can address it in a follow up. There's now a dependency between mmap_lock in the fault handler and the DMABUF resv (which might take a while to resolve). 2. vfio_basic_config_write() has an error path if vfio_default_config_write() fails that releases memory_lock but doesn't un-revoke BARs in the case of PCI_COMMAND.MSE being cleared. When can the write fail, in practice, perhaps surprise removal? The effect on this series would be: a write of MSE=0 revokes BARs, vconfig[PCI_COMMAND]'s MSE becomes 0, but if the physical write fails then the physical MSE remains 1 and BAR VMAs stay revoked. This seemed a mess; fixing isn't as simple as un-revoking on the error path since vfio_default_config_write() has already trampled vconfig so that'd need unwinding. It felt like a catastrophic scenario where BARs staying revoked isn't a bad outcome, but want to hear your experience of the likelihood of this issue. 3. The exchange of a VFIO fd mmap()'s vma->vm_file with an implicitly created DMABUF's file has implications on LSM. For example, an mmap will be checked against the policy for a VFIO fd, but a subsequent mprotect() relates to the DMABUF file's policy (which is anon/unique to the mapping). This is pretty confusing. 4. If VFIO fd is opened O_RDONLY, it currently can't be mmap()ed (because PROT_WRITE is rejected in do_mmap(), and PROT_READ alone drops the VM_SHARED so VFIO's mmap rejects it). But it seems we can export a DMABUF from it, and then pass the resulting fd around for P2P writes. I don't know if this is intentional, relied on, or a known limitation so haven't included a change here, but: a) We could reject export unless the device fd's f_mode has O_RDWR, to reflect the abilities of P2P b) Or, instead of just failing if !O_RDWR, we limit the get_dma_buf.open_flags to the VFIO device fd's f_mode, such as: VFIO device fd perms: Export flags: Result: O_RDWR O_RDWR, O_RDONLY OK O_RDONLY O_RDONLY OK O_RDONLY O_RDWR -EPERM O_WRONLY * -EPERM * O_WRONLY -EPERM (Skipping WRONLY because a PROT_WRITE-only mmap() won't work, though it probably should be included for P2P.) If we can do at least (a) that seems good, because the knock-on effect in this series is that we can export a DMABUF RW from an O_RDONLY device fd and then succeed to mmap() the DMABUF with RW. (That said, even if one has an O_RDONLY device fd, the device state can still be changed/reset. But it feels cleaner to least prevent export for a O_RDONLY device fd, and match the device fd mmap() behaviour.) Maybe (b) is step 2: Allowing finer-grained RD/WR could be useful if there's a future goal to tie DMABUF permissions to, say, iommufd IOMMU_READ/IOMMU_WRITE permissions. I don't htink this is in place today, apologies if I'm missed something. END === This is based on v7.2. These commits are on GitHub for easier browsing, along with "[RFC ONLY] selftests: vfio: Add standalone vfio_dmabuf_mmap_test": https://github.com/metamev/linux/compare/v7.2...dev/mev/vfio-dmabuf-mmap-v6 Thanks for reading [this astonishingly-long cover letter], Matt [1] https://lore.kernel.org/kvm/9a615f22-c0d4-46ae-9654-db11e94e5fec@ozlabs.org/ ================================================================================ Changelog: v6: - Dropped patches: PCI/P2PDMA: Split pool-related cleanup out of pci_p2pdma_release() PCI/P2PDMA: Add CONFIG_PCI_P2PDMA_CORE - New patch, "vfio/pci: Remove DMABUF export dependency on vdev->memory_lock": memory_lock was overloaded to protect the vdev->dmabufs list, the entries' revoked status, etc. Towards the goal of not needing memory_lock in export (so not creating a mmap_lock -> memory_lock dependency due to exporting from mmap()), add a new vdev->dmabuf_lock which protects the list and (writing) the revocation status. (See details in cover letter.) - New patch, "vfio/pci: Un-revoke DMABUFs in LOW_POWER_ENTRY_WITH_WAKEUP resume": Bugfix, as the resume path didn't un-revoke. Previously affected "only" P2P, but since DMABUFs-everywhere it would affect BAR mmap()s too. - New patch, "dma-buf: Export dma_buf_set_name()": Change in dma-buf.c's dma_buf_set_name() to take a kernel-allocated string. The original IOCTL-related path wraps this to strdup the user string first. - "vfio/pci: Add a helper to look up PFNs for DMABUFs": Commit message reworded explaining vma_pgoff_adjust, return -ERANGE for (non-transient) pagesize failure instead of -EAGAIN. Add dmabuf_lock annotation and assert that the DMABUF isn't revoked (else new -ENODEV error), using a new vdev parameter (caller is expected to safely extract it from priv). Significant bugfix in the case of a VMA offset exceeding the 1TB stride between VFIO_PCI_OFFSET_MASK-sized regions, which would have silently wrapped due to being masked to 1TB. This is fixed by including the VFIO region index in the high bits of vma_pgoff_adjust for the traditional mmap() path, so the subtraction from vma->vm_pgoff in ...find_pfn() removes the region index without masking. For the DMABUF mmap() path, vma_pgoff_adjust = 0 and (thanks to no masking) large offsets can be used. - "vfio/pci: Add a helper to create a DMABUF for a BAR-map VMA": Now permits a NULL p2pdma provider to be installed in the vfio_pci_dma_buf. The regular BAR mmap() path doesn't need a provider for CPU access; the DMABUF backing the VMA exists primarily for the CPU, and removing this check decouples vfio-pci from depending on CONFIG_PCI_P2PDMA (_without_ going through the previous hoops of splitting up P2PDMA into a _CORE subset to provide the provider). Only DMABUF import/map needs a provider, which is available when P2PDMA is available; the previous series' first two patches (to always provide a provider) are now unnecessary and have been dropped. Squashed "vfio/pci: Provide a user-facing name for BAR mappings" into this patch, because it's now just very small due to the new dma_buf_set_name(). Comment that, because of careful construction of the name length, the setting of the name can't fail; but in case assumptions change, a check/kfree prevents it leaking. (A new error path doing full unwind of the DMABUF creation is overkill.) - "vfio/pci: Convert BAR mmap() to use a DMABUF": Instead of the previous dropped P2PDMA split commits, allows pcim_p2pdma_provider() to return NULL if !CONFIG_VFIO_PCI_DMABUF (meaning !CONFIG_PCI_P2PDMA). The provider isn't used unless the DMABUF is imported; prevent import to make this fail early (rather than at map), by creating an always-fail vfio_pci_dma_buf_attach() stub. (Somewhat belt and braces, but makes clear you need P2PDMA to import a BAR VMA's DMABUF!) - "vfio/pci: Clean up BAR zap and revocation": The zap_bars_on_revoke flag is moved back to the bitfield, and is set from vfio_pci_core_init_dev() (instead of registration time). Reworked the hisi driver change to update this earlier. - "vfio/pci: Support mmap() of a VFIO DMABUF": Previous versions introduced a build issue (which was fixed by the next patch) due to a READ_ONCE of the priv->revoked bitfield member; removed. It is added by the next patch in the series (s/revoked/status/). Added an explicit rejection for mmap of a DMABUF with vma_pgoff_adjust > 0. Such buffers can only be created implicitly by the VFIO BAR mmap path, and an fd can't currently be recreated from a VMA (e.g. fished out of /proc/pid/map_files) because the anon inode ops don't support open. But check added if a future mechanism arises and for clarity. Rejecting an mmap of a DMABUF fd having vma_pgoff_adjust > 0 means vfio_pci_dma_buf_find_pfn() doesn't need to cope with the case of vma->vm_pgoff < priv->vma_pgoff_adjust and underflow. - "vfio/pci: Permanently revoke a DMABUF on request": Clarified docs for returned error values; re-added READ_ONCE(priv->status) for unlocked reads in both vfio_pci_dma_buf_attach() and vfio_pci_dma_buf_mmap(). v5: https://lore.kernel.org/all/20260715174737.15287-1-matt@ozlabs.org/ v4: https://lore.kernel.org/all/20260701171245.90111-1-matt@ozlabs.org/ v3: https://lore.kernel.org/all/20260610154327.37758-1-matt@ozlabs.org/ v2: https://lore.kernel.org/all/20260527102319.100128-1-mattev@meta.com/ v1: https://lore.kernel.org/kvm/20260416131815.2729131-1-mattev@meta.com/ RFCv2: https://lore.kernel.org/kvm/20260312184613.3710705-1-mattev@meta.com/ RFCv1: https://lore.kernel.org/all/20260226202211.929005-1-mattev@meta.com/ Tech topic: https://lore.kernel.org/linux-iommu/20250918214425.2677057-1-amastro@fb.com/ Matt Evans (9): vfio/pci: Remove DMABUF export dependency on vdev->memory_lock vfio/pci: Un-revoke DMABUFs in LOW_POWER_ENTRY_WITH_WAKEUP resume dma-buf: Export dma_buf_set_name() vfio/pci: Add a helper to look up PFNs for DMABUFs vfio/pci: Add a helper to create a DMABUF for a BAR-map VMA vfio/pci: Convert BAR mmap() to use a DMABUF vfio/pci: Clean up BAR zap and revocation vfio/pci: Support mmap() of a VFIO DMABUF vfio/pci: Permanently revoke a DMABUF on request drivers/dma-buf/dma-buf.c | 58 +- drivers/vfio/pci/Kconfig | 4 +- drivers/vfio/pci/Makefile | 3 +- .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 14 +- drivers/vfio/pci/vfio_pci_config.c | 30 +- drivers/vfio/pci/vfio_pci_core.c | 222 +++++-- drivers/vfio/pci/vfio_pci_dmabuf.c | 607 +++++++++++++++--- drivers/vfio/pci/vfio_pci_priv.h | 54 +- include/linux/dma-buf.h | 2 + include/linux/vfio_pci_core.h | 3 + include/uapi/linux/vfio.h | 24 + 11 files changed, 851 insertions(+), 170 deletions(-) -- 2.50.1 (Apple Git-155)

1 week, 2 days

Re: [PATCH v7 15/19] PCI/P2PDMA: Document TLP-class routing

by Leon Romanovsky

On Mon, Sep 21, 2026 at 04:16:56PM -0600, Logan Gunthorpe wrote: > > > On 2026-09-20 05:41, Leon Romanovsky wrote: > > +A path is bus addressable only where both directions route directly, so with > > +nothing enabled every class is. > > I don't understand the sentence. I tried to say that we need both Requests and Completions route directly. Claude suggested something like that: --------------------------------------------------------------------------- Combining the two tables gives the mapping type: a class is bus addressable when its Request and its Completions both route directly, a blocked Request is not supported because Translation Blocking rejects the Address Type wherever the Request is addressed, and anything else goes through the host bridge. With no ACS control set anywhere, every class routes directly. --------------------------------------------------------------------------- Thanks > > > Logan

1 week, 3 days

Re: [PATCH v2] dma-buf/udmabuf: reinstate a reasonable default size limit

by Christian König

On 9/19/26 02:42, Jiakai Xu wrote: > Commit 44e9eb5a7621 ("dma-buf/udmabuf: Disable the size limit by > default") raised the default size_limit_mb from 64 to INT_MAX. With > 4K pages, creating a buffer of more than INT_MAX / 8 pages (about 1 TB > of pinned memory) now makes the two per-page pointer arrays in > init_udmabuf() exceed what kvmalloc() can handle: kvmalloc() warns and > fails when the requested size exceeds INT_MAX bytes (WARN_ON_ONCE in > __kvmalloc_node_noprof), which is trivially triggerable from > unprivileged users through the UDMABUF_CREATE ioctls: > > WARNING: CPU: 0 PID: 412 at mm/slub.c:7021 __kvmalloc_node_noprof > Call Trace: > udmabuf_create > udmabuf_ioctl_create_list > > Restore a reasonable default limit of 256 MB, which is the value > proposed in the RFC discussion that led to commit 44e9eb5a7621 That is way to low, we want to allocate multiple gigabytes using that API. Please make that (((INT_MAX / sizeof(struct folio *)) - 1) >> 20) << PAGE_SHIFT. That should allow to allocate a bit below 1TiB again without triggering the issue. Regards, Christian. > ("dma-buf/udmabuf: Disable the size limit by default") and comfortably > covers the 8K video use cases cited there, while keeping the > kvmalloc() request far below its threshold. The limit remains > runtime-tunable via the size_limit_mb module parameter for setups > that need more. > > Fixes: 44e9eb5a7621 ("dma-buf/udmabuf: Disable the size limit by default") > Assisted-by: OpenCode:DeepSeek-V4-Flash > Signed-off-by: Jiakai Xu <xujiakai24(a)mails.ucas.ac.cn> > --- > V1 -> V2: > - Instead of adding a pgcnt check in udmabuf_create(), restore a > reasonable default size limit (256 MB), as suggested by Christian > König. > - Drop the second Fixes tag; the regression is solely due to > 44e9eb5a7621. > - Drop the "Cc: stable" tag; 44e9eb5a7621 is only in v7.3-rc1 and has > not shipped in any released kernel yet. > > V1: https://lore.kernel.org/all/20260918013434.1849135-1-xujiakai24@mails.ucas.… > --- > drivers/dma-buf/udmabuf.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c > index df6dd00462423..4d90c2cb14225 100644 > --- a/drivers/dma-buf/udmabuf.c > +++ b/drivers/dma-buf/udmabuf.c > @@ -20,9 +20,9 @@ static int list_limit = 1024; > module_param(list_limit, int, 0644); > MODULE_PARM_DESC(list_limit, "udmabuf_create_list->count limit. Default is 1024."); > > -static int size_limit_mb = INT_MAX; > +static int size_limit_mb = 256; > module_param(size_limit_mb, int, 0644); > -MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in megabytes. Default is INT_MAX."); > +MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in megabytes. Default is 256."); > > struct udmabuf { > pgoff_t pagecount; > -- > 2.34.1 > > -- > Below is the crash report: > ------------[ cut here ]------------ > WARNING: mm/slub.c:7013 at __kvmalloc_node_noprof+0x516/0x5d0 home/zzzrrll/tmp/kf_src/linux-7.3-rc2/mm/slub.c:7013, CPU#0: syz.5.27938/167363 > Modules linked in: > CPU: 0 UID: 0 PID: 167363 Comm: syz.5.27938 Tainted: G W L 7.3.0-rc2 #1 PREEMPT(full) > Tainted: [W]=WARN, [L]=SOFTLOCKUP > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014 > RIP: 0010:__kvmalloc_node_noprof+0x516/0x5d0 home/zzzrrll/tmp/kf_src/linux-7.3-rc2/mm/slub.c:7013 > Code: 05 bf fa 2d 08 49 8b 0f c6 04 08 00 65 48 ff 0d 90 fa 2d 08 0f 85 4c fd ff ff e8 05 43 8c ff e9 42 fd ff ff e8 9b 86 0e 04 90 <0f> 0b 90 31 db e9 d2 fe ff ff f3 0f 1e fa 65 8b 05 8d fa 2d 08 89 > RSP: 0018:ffffc90002bc3cf8 EFLAGS: 00010246 > RAX: 0000000000000100 RBX: 0000000000000000 RCX: 0000008000000000 > RDX: 0000008000000000 RSI: 0000000000000000 RDI: ffffffff835e3226 > RBP: 00000000ffffffff R08: 00000000000028c0 R09: 00000000ffffffff > R10: 0000000000000006 R11: 0000000000000002 R12: 0000008000000000 > R13: 00000000000028c0 R14: 0000001000000000 R15: 0000008000000000 > FS: 00007fc89ae45640(0000) GS:ffff8880f3de4000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 000000110c2ff305 CR3: 000000002b3d4000 CR4: 0000000000752ef0 > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000006000 > DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > PKRU: 80000000 > Call Trace: > <TASK> > init_udmabuf home/zzzrrll/tmp/kf_src/linux-7.3-rc2/drivers/dma-buf/udmabuf.c:193 [inline] > udmabuf_create+0x1b6/0x750 home/zzzrrll/tmp/kf_src/linux-7.3-rc2/drivers/dma-buf/udmabuf.c:386 > udmabuf_ioctl_create home/zzzrrll/tmp/kf_src/linux-7.3-rc2/drivers/dma-buf/udmabuf.c:465 [inline] > udmabuf_ioctl+0x132/0x1d0 home/zzzrrll/tmp/kf_src/linux-7.3-rc2/drivers/dma-buf/udmabuf.c:496 > vfs_ioctl home/zzzrrll/tmp/kf_src/linux-7.3-rc2/fs/ioctl.c:51 [inline] > __do_sys_ioctl home/zzzrrll/tmp/kf_src/linux-7.3-rc2/fs/ioctl.c:597 [inline] > __se_sys_ioctl+0xbc/0x130 home/zzzrrll/tmp/kf_src/linux-7.3-rc2/fs/ioctl.c:583 > do_syscall_x64 home/zzzrrll/tmp/kf_src/linux-7.3-rc2/arch/x86/entry/syscall_64.c:61 [inline] > do_syscall_64+0x12b/0x350 home/zzzrrll/tmp/kf_src/linux-7.3-rc2/arch/x86/entry/syscall_64.c:84 > entry_SYSCALL_64_after_hwframe+0x77/0x7f > RIP: 0033:0x590d6d > Code: 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48 > RSP: 002b:00007fc89ae44fd8 EFLAGS: 00000216 ORIG_RAX: 0000000000000010 > RAX: ffffffffffffffda RBX: 0000000000611e7f RCX: 0000000000590d6d > RDX: 00002000000023c0 RSI: 0000000040187542 RDI: 000000000000000a > RBP: 00007fc89ae45010 R08: 0000000000000000 R09: 0000000000000000 > R10: 0000000000000000 R11: 0000000000000216 R12: 00007fc89ae45640 > R13: 000000000000000b R14: 0000000000528d40 R15: 00007fc89ae25000 > </TASK> > ---[ end trace 0000000000000000 ]--- > --- >

1 week, 3 days

Re: [PATCH] dma-buf/udmabuf: reinstate a reasonable default size limit

by Christian König

On 9/22/26 03:23, Jiakai Xu wrote: > Commit 44e9eb5a7621 ("dma-buf/udmabuf: Disable the size limit by > default") raised the default size_limit_mb from 64 to INT_MAX. > > With 4 KiB pages on a 64-bit system, creating a buffer larger than > about 1 TiB makes the two per-page pointer arrays in init_udmabuf() > exceed what kvmalloc() can handle. __kvmalloc_node_noprof() warns and > fails when the requested size exceeds INT_MAX bytes, which is > trivially triggerable by unprivileged users through the UDMABUF_CREATE > ioctls: > > WARNING: CPU: 0 PID: 412 at mm/slub.c:7021 __kvmalloc_node_noprof > Call Trace: > udmabuf_create > udmabuf_ioctl_create_list > > Instead of using a fixed megabyte value, derive the default limit from > the kvmalloc() threshold: > > (((INT_MAX / sizeof(struct folio *)) - 1) >> 20) << PAGE_SHIFT > > On a 64-bit system with 4 KiB pages, this evaluates to 1044480 MB > (1020 GiB), allowing buffers a little below 1 TiB while keeping the > per-page pointer array allocations below INT_MAX bytes. > > Fixes: 44e9eb5a7621 ("dma-buf/udmabuf: Disable the size limit by default") > Suggested-by: Christian König <christian.koenig(a)amd.com> > Assisted-by: OpenCode:DeepSeek-V4-Flash > Signed-off-by: Jiakai Xu <xujiakai24(a)mails.ucas.ac.cn> Reviewed-by: Christian König <christian.koenig(a)amd.com> Going to push that to drm-misc-fixes later today. Regards, Christian. > --- > V2 -> V3: > - Replace the fixed 256 MB default with a limit derived from the > kvmalloc() INT_MAX threshold, as suggested by Christian König. On > 64-bit systems with 4 KiB pages, this allows buffers just below > 1 TiB. > > V1 -> V2: > - Instead of adding a pgcnt check in udmabuf_create(), restore a > reasonable default size limit (256 MB), as suggested by Christian > König. > - Drop the second Fixes tag; the regression is solely due to > 44e9eb5a7621. > - Drop the "Cc: stable" tag; 44e9eb5a7621 is only in v7.3-rc1 and has > not shipped in any released kernel yet. > > V1: https://lore.kernel.org/all/20260918013434.1849135-1-xujiakai24@mails.ucas.… > V2: https://lore.kernel.org/all/20260919015731.2077604-1-xujiakai24@mails.ucas.… > --- > drivers/dma-buf/udmabuf.c | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c > index df6dd00462423..7ec269809fb49 100644 > --- a/drivers/dma-buf/udmabuf.c > +++ b/drivers/dma-buf/udmabuf.c > @@ -20,9 +20,10 @@ static int list_limit = 1024; > module_param(list_limit, int, 0644); > MODULE_PARM_DESC(list_limit, "udmabuf_create_list->count limit. Default is 1024."); > > -static int size_limit_mb = INT_MAX; > +static int size_limit_mb = (((INT_MAX / sizeof(struct folio *)) - 1) >> 20) << PAGE_SHIFT; > module_param(size_limit_mb, int, 0644); > -MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in megabytes. Default is INT_MAX."); > +MODULE_PARM_DESC(size_limit_mb, > + "Max size of a dmabuf, in megabytes. Default is derived from the kvmalloc() limit."); > > struct udmabuf { > pgoff_t pagecount;

1 week, 3 days

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig