On 11/23/25 23:51, Pavel Begunkov wrote:
Picking up the work on supporting dmabuf in the read/write path.
IIRC that work was completely stopped because it violated core dma_fence and DMA-buf rules and after some private discussion was considered not doable in general.
Or am I mixing something up here? Since I don't see any dma_fence implementation at all that might actually be the case.
On the other hand we have direct I/O from DMA-buf working for quite a while, just not upstream and without io_uring support.
Regards, Christian.
There are two main changes. First, it doesn't pass a dma addresss directly by rather wraps it into an opaque structure, which is extended and understood by the target driver.
The second big change is support for dynamic attachments, which added a good part of complexity (see Patch 5). I kept the main machinery in nvme at first, but move_notify can ask to kill the dma mapping asynchronously, and any new IO would need to wait during submission, thus it was moved to blk-mq. That also introduced an extra callback layer b/w driver and blk-mq.
There are some rough corners, and I'm not perfectly happy about the complexity and layering. For v3 I'll try to move the waiting up in the stack to io_uring wrapped into library helpers.
For now, I'm interested what is the best way to test move_notify? And how dma_resv_reserve_fences() errors should be handled in move_notify?
The uapi didn't change, after registration it looks like a normal io_uring registered buffer and can be used as such. Only non-vectored fixed reads/writes are allowed. Pseudo code:
// registration reg_buf_idx = 0; io_uring_update_buffer(ring, reg_buf_idx, { dma_buf_fd, file_fd });
// request creation io_uring_prep_read_fixed(sqe, file_fd, buffer_offset, buffer_size, file_offset, reg_buf_idx);
And as previously, a good bunch of code was taken from Keith's series [1].
liburing based example:
git: https://github.com/isilence/liburing.git dmabuf-rw link: https://github.com/isilence/liburing/tree/dmabuf-rw
[1] https://lore.kernel.org/io-uring/20220805162444.3985535-1-kbusch@fb.com/
Pavel Begunkov (11): file: add callback for pre-mapping dmabuf iov_iter: introduce iter type for pre-registered dma block: move around bio flagging helpers block: introduce dma token backed bio type block: add infra to handle dmabuf tokens nvme-pci: add support for dmabuf reggistration nvme-pci: implement dma_token backed requests io_uring/rsrc: add imu flags io_uring/rsrc: extended reg buffer registration io_uring/rsrc: add dmabuf-backed buffer registeration io_uring/rsrc: implement dmabuf regbuf import
block/Makefile | 1 + block/bdev.c | 14 ++ block/bio.c | 21 +++ block/blk-merge.c | 23 +++ block/blk-mq-dma-token.c | 236 +++++++++++++++++++++++++++++++ block/blk-mq.c | 20 +++ block/blk.h | 3 +- block/fops.c | 3 + drivers/nvme/host/pci.c | 217 ++++++++++++++++++++++++++++ include/linux/bio.h | 49 ++++--- include/linux/blk-mq-dma-token.h | 60 ++++++++ include/linux/blk-mq.h | 21 +++ include/linux/blk_types.h | 8 +- include/linux/blkdev.h | 3 + include/linux/dma_token.h | 35 +++++ include/linux/fs.h | 4 + include/linux/uio.h | 10 ++ include/uapi/linux/io_uring.h | 13 +- io_uring/rsrc.c | 201 +++++++++++++++++++++++--- io_uring/rsrc.h | 23 ++- io_uring/rw.c | 7 +- lib/iov_iter.c | 30 +++- 22 files changed, 948 insertions(+), 54 deletions(-) create mode 100644 block/blk-mq-dma-token.c create mode 100644 include/linux/blk-mq-dma-token.h create mode 100644 include/linux/dma_token.h