Linaro-mm-sig July 2023

linaro-mm-sig@lists.linaro.org

29 participants
22 discussions

[RFC PATCH] dma-buf/dma-fence: Use a successful read_trylock() annotation for dma_fence_begin_signalling()

by Thomas Hellström

Condsider the following call sequence: /* Upper layer */ dma_fence_begin_signalling(); lock(tainted_shared_lock); /* Driver callback */ dma_fence_begin_signalling(); ... The driver might here use a utility that is annotated as intended for the dma-fence signalling critical path. Now if the upper layer isn't correctly annotated yet for whatever reason, resulting in /* Upper layer */ lock(tainted_shared_lock); /* Driver callback */ dma_fence_begin_signalling(); We will receive a false lockdep locking order violation notification from dma_fence_begin_signalling(). However entering a dma-fence signalling critical section itself doesn't block and could not cause a deadlock. So use a successful read_trylock() annotation instead for dma_fence_begin_signalling(). That will make sure that the locking order is correctly registered in the first case, and doesn't register any locking order in the second case. The alternative is of course to make sure that the "Upper layer" is always correctly annotated. But experience shows that's not easily achievable in all cases. Signed-off-by: Thomas Hellström <thomas.hellstrom(a)linux.intel.com> --- drivers/dma-buf/dma-fence.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index f177c56269bb..17f632768ef9 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -308,8 +308,8 @@ bool dma_fence_begin_signalling(void) if (in_atomic()) return true; - /* ... and non-recursive readlock */ - lock_acquire(&dma_fence_lockdep_map, 0, 0, 1, 1, NULL, _RET_IP_); + /* ... and non-recursive successful read_trylock */ + lock_acquire(&dma_fence_lockdep_map, 0, 1, 1, 1, NULL, _RET_IP_); return false; } @@ -340,7 +340,7 @@ void __dma_fence_might_wait(void) lock_map_acquire(&dma_fence_lockdep_map); lock_map_release(&dma_fence_lockdep_map); if (tmp) - lock_acquire(&dma_fence_lockdep_map, 0, 0, 1, 1, NULL, _THIS_IP_); + lock_acquire(&dma_fence_lockdep_map, 0, 1, 1, 1, NULL, _THIS_IP_); } #endif -- 2.39.2

2 weeks

[PATCH] drm/shmem-helper: Remove duplicate include

by Jiapeng Chong

./drivers/gpu/drm/drm_gem_shmem_helper.c: linux/module.h is included more than once. Reported-by: Abaci Robot <abaci(a)linux.alibaba.com> Link: https://bugzilla.openanolis.cn/show_bug.cgi?id=4567 Signed-off-by: Jiapeng Chong <jiapeng.chong(a)linux.alibaba.com> --- drivers/gpu/drm/drm_gem_shmem_helper.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index 4ea6507a77e5..f401df26facf 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -10,7 +10,6 @@ #include <linux/shmem_fs.h> #include <linux/slab.h> #include <linux/vmalloc.h> -#include <linux/module.h> #ifdef CONFIG_X86 #include <asm/set_memory.h> -- 2.20.1.7.g153144c

6 months, 3 weeks

[syzbot] [dri?] KMSAN: uninit-value in drm_mode_setcrtc

by syzbot

Hello, syzbot found the following issue on: HEAD commit: 2741f1b02117 string: use __builtin_memcpy() in strlcpy/str.. git tree: https://github.com/google/kmsan.git master console+strace: https://syzkaller.appspot.com/x/log.txt?x=17bb33d1280000 kernel config: https://syzkaller.appspot.com/x/.config?x=753079601b2300f9 dashboard link: https://syzkaller.appspot.com/bug?extid=4fad2e57beb6397ab2fc compiler: Debian clang version 15.0.7, GNU ld (GNU Binutils for Debian) 2.35.2 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16d669a5280000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14d8f095280000 Downloadable assets: disk image: https://storage.googleapis.com/syzbot-assets/ebd05512d8d7/disk-2741f1b0.raw… vmlinux: https://storage.googleapis.com/syzbot-assets/aa555b09582c/vmlinux-2741f1b0.… kernel image: https://storage.googleapis.com/syzbot-assets/5ea0934e02cc/bzImage-2741f1b0.… IMPORTANT: if you fix the issue, please add the following tag to the commit: Reported-by: syzbot+4fad2e57beb6397ab2fc(a)syzkaller.appspotmail.com ===================================================== BUG: KMSAN: uninit-value in drm_mode_setcrtc+0x1ad3/0x24a0 drivers/gpu/drm/drm_crtc.c:896 drm_mode_setcrtc+0x1ad3/0x24a0 drivers/gpu/drm/drm_crtc.c:896 drm_ioctl_kernel+0x5ae/0x730 drivers/gpu/drm/drm_ioctl.c:788 drm_ioctl+0xd12/0x1590 drivers/gpu/drm/drm_ioctl.c:891 vfs_ioctl fs/ioctl.c:51 [inline] __do_sys_ioctl fs/ioctl.c:870 [inline] __se_sys_ioctl+0x222/0x400 fs/ioctl.c:856 __x64_sys_ioctl+0x96/0xe0 fs/ioctl.c:856 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd Uninit was created at: slab_post_alloc_hook+0x12d/0xb60 mm/slab.h:716 slab_alloc_node mm/slub.c:3451 [inline] __kmem_cache_alloc_node+0x4ff/0x8b0 mm/slub.c:3490 __do_kmalloc_node mm/slab_common.c:965 [inline] __kmalloc+0x121/0x3c0 mm/slab_common.c:979 kmalloc_array include/linux/slab.h:596 [inline] drm_mode_setcrtc+0x1dba/0x24a0 drivers/gpu/drm/drm_crtc.c:846 drm_ioctl_kernel+0x5ae/0x730 drivers/gpu/drm/drm_ioctl.c:788 drm_ioctl+0xd12/0x1590 drivers/gpu/drm/drm_ioctl.c:891 vfs_ioctl fs/ioctl.c:51 [inline] __do_sys_ioctl fs/ioctl.c:870 [inline] __se_sys_ioctl+0x222/0x400 fs/ioctl.c:856 __x64_sys_ioctl+0x96/0xe0 fs/ioctl.c:856 do_syscall_x64 arch/x86/entry/common.c:50 [inline] do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 entry_SYSCALL_64_after_hwframe+0x63/0xcd CPU: 1 PID: 4955 Comm: syz-executor275 Not tainted 6.4.0-rc4-syzkaller-g2741f1b02117 #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/25/2023 ===================================================== --- This report is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at syzkaller(a)googlegroups.com. syzbot will keep track of this issue. See: https://goo.gl/tpsmEJ#status for how to communicate with syzbot. If the bug is already fixed, let syzbot know by replying with: #syz fix: exact-commit-title If you want syzbot to run the reproducer, reply with: #syz test: git://repo/address.git branch-or-commit-hash If you attach or paste a git patch, syzbot will apply it before testing. If you want to change bug's subsystems, reply with: #syz set subsystems: new-subsystem (See the list of subsystem names on the web dashboard) If the bug is a duplicate of another bug, reply with: #syz dup: exact-subject-of-another-report If you want to undo deduplication, reply with: #syz undup

10 months

[RFC] drm/scheduler: Unwrap job dependencies

by Rob Clark

From: Rob Clark <robdclark(a)chromium.org> Container fences have burner contexts, which makes the trick to store at most one fence per context somewhat useless if we don't unwrap array or chain fences. Signed-off-by: Rob Clark <robdclark(a)chromium.org> --- tbh, I'm not sure why we weren't doing this already, unless there is something I'm overlooking drivers/gpu/drm/scheduler/sched_main.c | 43 +++++++++++++++++--------- 1 file changed, 28 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index c2ee44d6224b..f59e5335afbb 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -41,20 +41,21 @@ * 4. Entities themselves maintain a queue of jobs that will be scheduled on * the hardware. * * The jobs in a entity are always scheduled in the order that they were pushed. */ #include <linux/kthread.h> #include <linux/wait.h> #include <linux/sched.h> #include <linux/completion.h> +#include <linux/dma-fence-unwrap.h> #include <linux/dma-resv.h> #include <uapi/linux/sched/types.h> #include <drm/drm_print.h> #include <drm/drm_gem.h> #include <drm/gpu_scheduler.h> #include <drm/spsc_queue.h> #define CREATE_TRACE_POINTS #include "gpu_scheduler_trace.h" @@ -665,41 +666,27 @@ void drm_sched_job_arm(struct drm_sched_job *job) sched = entity->rq->sched; job->sched = sched; job->s_priority = entity->rq - sched->sched_rq; job->id = atomic64_inc_return(&sched->job_id_count); drm_sched_fence_init(job->s_fence, job->entity); } EXPORT_SYMBOL(drm_sched_job_arm); -/** - * drm_sched_job_add_dependency - adds the fence as a job dependency - * @job: scheduler job to add the dependencies to - * @fence: the dma_fence to add to the list of dependencies. - * - * Note that @fence is consumed in both the success and error cases. - * - * Returns: - * 0 on success, or an error on failing to expand the array. - */ -int drm_sched_job_add_dependency(struct drm_sched_job *job, - struct dma_fence *fence) +static int _add_dependency(struct drm_sched_job *job, struct dma_fence *fence) { struct dma_fence *entry; unsigned long index; u32 id = 0; int ret; - if (!fence) - return 0; - /* Deduplicate if we already depend on a fence from the same context. * This lets the size of the array of deps scale with the number of * engines involved, rather than the number of BOs. */ xa_for_each(&job->dependencies, index, entry) { if (entry->context != fence->context) continue; if (dma_fence_is_later(fence, entry)) { dma_fence_put(entry); @@ -709,20 +696,46 @@ int drm_sched_job_add_dependency(struct drm_sched_job *job, } return 0; } ret = xa_alloc(&job->dependencies, &id, fence, xa_limit_32b, GFP_KERNEL); if (ret != 0) dma_fence_put(fence); return ret; } + +/** + * drm_sched_job_add_dependency - adds the fence as a job dependency + * @job: scheduler job to add the dependencies to + * @fence: the dma_fence to add to the list of dependencies. + * + * Note that @fence is consumed in both the success and error cases. + * + * Returns: + * 0 on success, or an error on failing to expand the array. + */ +int drm_sched_job_add_dependency(struct drm_sched_job *job, + struct dma_fence *fence) +{ + struct dma_fence_unwrap iter; + struct dma_fence *f; + int ret = 0; + + dma_fence_unwrap_for_each (f, &iter, fence) { + ret = _add_dependency(job, f); + if (ret) + break; + } + + return ret; +} EXPORT_SYMBOL(drm_sched_job_add_dependency); /** * drm_sched_job_add_resv_dependencies - add all fences from the resv to the job * @job: scheduler job to add the dependencies to * @resv: the dma_resv object to get the fences from * @usage: the dma_resv_usage to use to filter the fences * * This adds all fences matching the given usage from @resv to @job. * Must be called with the @resv lock held. -- 2.39.2

10 months

[PATCH v3] misc: sram: Add DMA-BUF Heap exporting of SRAM areas

by Andrew Davis

This new export type exposes to userspace the SRAM area as a DMA-BUF Heap, this allows for allocations of DMA-BUFs that can be consumed by various DMA-BUF supporting devices. Signed-off-by: Andrew Davis <afd(a)ti.com> --- Changes from v2: - Make sram_dma_heap_allocate static (kernel test robot) - Rebase on v6.5-rc1 drivers/misc/Kconfig | 7 + drivers/misc/Makefile | 1 + drivers/misc/sram-dma-heap.c | 245 +++++++++++++++++++++++++++++++++++ drivers/misc/sram.c | 6 + drivers/misc/sram.h | 16 +++ 5 files changed, 275 insertions(+) create mode 100644 drivers/misc/sram-dma-heap.c diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index 75e427f124b28..ee34dfb61605f 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -448,6 +448,13 @@ config SRAM config SRAM_EXEC bool +config SRAM_DMA_HEAP + bool "Export on-chip SRAM pools using DMA-Heaps" + depends on DMABUF_HEAPS && SRAM + help + This driver allows the export of on-chip SRAM marked as both pool + and exportable to userspace using the DMA-Heaps interface. + config DW_XDATA_PCIE depends on PCI tristate "Synopsys DesignWare xData PCIe driver" diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index f2a4d1ff65d46..5e7516bfaa8de 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -47,6 +47,7 @@ obj-$(CONFIG_VMWARE_VMCI) += vmw_vmci/ obj-$(CONFIG_LATTICE_ECP3_CONFIG) += lattice-ecp3-config.o obj-$(CONFIG_SRAM) += sram.o obj-$(CONFIG_SRAM_EXEC) += sram-exec.o +obj-$(CONFIG_SRAM_DMA_HEAP) += sram-dma-heap.o obj-$(CONFIG_GENWQE) += genwqe/ obj-$(CONFIG_ECHO) += echo/ obj-$(CONFIG_CXL_BASE) += cxl/ diff --git a/drivers/misc/sram-dma-heap.c b/drivers/misc/sram-dma-heap.c new file mode 100644 index 0000000000000..c054c04dff33e --- /dev/null +++ b/drivers/misc/sram-dma-heap.c @@ -0,0 +1,245 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * SRAM DMA-Heap userspace exporter + * + * Copyright (C) 2019-2022 Texas Instruments Incorporated - https://www.ti.com/ + * Andrew Davis <afd(a)ti.com> + */ + +#include <linux/dma-mapping.h> +#include <linux/err.h> +#include <linux/genalloc.h> +#include <linux/io.h> +#include <linux/mm.h> +#include <linux/scatterlist.h> +#include <linux/slab.h> +#include <linux/dma-buf.h> +#include <linux/dma-heap.h> + +#include "sram.h" + +struct sram_dma_heap { + struct dma_heap *heap; + struct gen_pool *pool; +}; + +struct sram_dma_heap_buffer { + struct gen_pool *pool; + struct list_head attachments; + struct mutex attachments_lock; + unsigned long len; + void *vaddr; + phys_addr_t paddr; +}; + +struct dma_heap_attachment { + struct device *dev; + struct sg_table *table; + struct list_head list; +}; + +static int dma_heap_attach(struct dma_buf *dmabuf, + struct dma_buf_attachment *attachment) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + struct dma_heap_attachment *a; + struct sg_table *table; + + a = kzalloc(sizeof(*a), GFP_KERNEL); + if (!a) + return -ENOMEM; + + table = kmalloc(sizeof(*table), GFP_KERNEL); + if (!table) { + kfree(a); + return -ENOMEM; + } + if (sg_alloc_table(table, 1, GFP_KERNEL)) { + kfree(table); + kfree(a); + return -ENOMEM; + } + sg_set_page(table->sgl, pfn_to_page(PFN_DOWN(buffer->paddr)), buffer->len, 0); + + a->table = table; + a->dev = attachment->dev; + INIT_LIST_HEAD(&a->list); + + attachment->priv = a; + + mutex_lock(&buffer->attachments_lock); + list_add(&a->list, &buffer->attachments); + mutex_unlock(&buffer->attachments_lock); + + return 0; +} + +static void dma_heap_detatch(struct dma_buf *dmabuf, + struct dma_buf_attachment *attachment) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + struct dma_heap_attachment *a = attachment->priv; + + mutex_lock(&buffer->attachments_lock); + list_del(&a->list); + mutex_unlock(&buffer->attachments_lock); + + sg_free_table(a->table); + kfree(a->table); + kfree(a); +} + +static struct sg_table *dma_heap_map_dma_buf(struct dma_buf_attachment *attachment, + enum dma_data_direction direction) +{ + struct dma_heap_attachment *a = attachment->priv; + struct sg_table *table = a->table; + + /* + * As this heap is backed by uncached SRAM memory we do not need to + * perform any sync operations on the buffer before allowing device + * domain access. For this reason we use SKIP_CPU_SYNC and also do + * not use or provide begin/end_cpu_access() dma-buf functions. + */ + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, + direction, DMA_ATTR_SKIP_CPU_SYNC)) + return ERR_PTR(-ENOMEM); + + return table; +} + +static void dma_heap_unmap_dma_buf(struct dma_buf_attachment *attachment, + struct sg_table *table, + enum dma_data_direction direction) +{ + dma_unmap_sg_attrs(attachment->dev, table->sgl, table->nents, + direction, DMA_ATTR_SKIP_CPU_SYNC); +} + +static void dma_heap_dma_buf_release(struct dma_buf *dmabuf) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + + gen_pool_free(buffer->pool, (unsigned long)buffer->vaddr, buffer->len); + kfree(buffer); +} + +static int dma_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + int ret; + + /* SRAM mappings are not cached */ + vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot); + + ret = vm_iomap_memory(vma, buffer->paddr, buffer->len); + if (ret) + pr_err("Could not map buffer to userspace\n"); + + return ret; +} + +static int dma_heap_vmap(struct dma_buf *dmabuf, struct iosys_map *map) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + + iosys_map_set_vaddr(map, buffer->vaddr); + + return 0; +} + +static const struct dma_buf_ops sram_dma_heap_buf_ops = { + .attach = dma_heap_attach, + .detach = dma_heap_detatch, + .map_dma_buf = dma_heap_map_dma_buf, + .unmap_dma_buf = dma_heap_unmap_dma_buf, + .release = dma_heap_dma_buf_release, + .mmap = dma_heap_mmap, + .vmap = dma_heap_vmap, +}; + +static struct dma_buf *sram_dma_heap_allocate(struct dma_heap *heap, + unsigned long len, + unsigned long fd_flags, + unsigned long heap_flags) +{ + struct sram_dma_heap *sram_dma_heap = dma_heap_get_drvdata(heap); + struct sram_dma_heap_buffer *buffer; + + DEFINE_DMA_BUF_EXPORT_INFO(exp_info); + struct dma_buf *dmabuf; + int ret = 0; + + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL); + if (!buffer) + return ERR_PTR(-ENOMEM); + buffer->pool = sram_dma_heap->pool; + INIT_LIST_HEAD(&buffer->attachments); + mutex_init(&buffer->attachments_lock); + buffer->len = len; + + buffer->vaddr = (void *)gen_pool_alloc(buffer->pool, buffer->len); + if (!buffer->vaddr) { + ret = -ENOMEM; + goto free_buffer; + } + + buffer->paddr = gen_pool_virt_to_phys(buffer->pool, (unsigned long)buffer->vaddr); + if (buffer->paddr == -1) { + ret = -ENOMEM; + goto free_pool; + } + + /* create the dmabuf */ + exp_info.exp_name = dma_heap_get_name(heap); + exp_info.ops = &sram_dma_heap_buf_ops; + exp_info.size = buffer->len; + exp_info.flags = fd_flags; + exp_info.priv = buffer; + dmabuf = dma_buf_export(&exp_info); + if (IS_ERR(dmabuf)) { + ret = PTR_ERR(dmabuf); + goto free_pool; + } + + return dmabuf; + +free_pool: + gen_pool_free(buffer->pool, (unsigned long)buffer->vaddr, buffer->len); +free_buffer: + kfree(buffer); + + return ERR_PTR(ret); +} + +static struct dma_heap_ops sram_dma_heap_ops = { + .allocate = sram_dma_heap_allocate, +}; + +int sram_add_dma_heap(struct sram_dev *sram, + struct sram_reserve *block, + phys_addr_t start, + struct sram_partition *part) +{ + struct sram_dma_heap *sram_dma_heap; + struct dma_heap_export_info exp_info; + + dev_info(sram->dev, "Exporting SRAM Heap '%s'\n", block->label); + + sram_dma_heap = kzalloc(sizeof(*sram_dma_heap), GFP_KERNEL); + if (!sram_dma_heap) + return -ENOMEM; + sram_dma_heap->pool = part->pool; + + exp_info.name = kasprintf(GFP_KERNEL, "sram_%s", block->label); + exp_info.ops = &sram_dma_heap_ops; + exp_info.priv = sram_dma_heap; + sram_dma_heap->heap = dma_heap_add(&exp_info); + if (IS_ERR(sram_dma_heap->heap)) { + int ret = PTR_ERR(sram_dma_heap->heap); + kfree(sram_dma_heap); + return ret; + } + + return 0; +} diff --git a/drivers/misc/sram.c b/drivers/misc/sram.c index 5757adf418b1d..6dd173a2fba8e 100644 --- a/drivers/misc/sram.c +++ b/drivers/misc/sram.c @@ -120,6 +120,12 @@ static int sram_add_partition(struct sram_dev *sram, struct sram_reserve *block, ret = sram_add_pool(sram, block, start, part); if (ret) return ret; + + if (block->export) { + ret = sram_add_dma_heap(sram, block, start, part); + if (ret) + return ret; + } } if (block->export) { ret = sram_add_export(sram, block, start, part); diff --git a/drivers/misc/sram.h b/drivers/misc/sram.h index 397205b8bf6ff..062bdd25fa068 100644 --- a/drivers/misc/sram.h +++ b/drivers/misc/sram.h @@ -60,4 +60,20 @@ static inline int sram_add_protect_exec(struct sram_partition *part) return -ENODEV; } #endif /* CONFIG_SRAM_EXEC */ + +#ifdef CONFIG_SRAM_DMA_HEAP +int sram_add_dma_heap(struct sram_dev *sram, + struct sram_reserve *block, + phys_addr_t start, + struct sram_partition *part); +#else +static inline int sram_add_dma_heap(struct sram_dev *sram, + struct sram_reserve *block, + phys_addr_t start, + struct sram_partition *part) +{ + return 0; +} +#endif /* CONFIG_SRAM_DMA_HEAP */ + #endif /* __SRAM_H */ -- 2.39.2

1 year, 1 month

Re: [PATCH v2] dma-contiguous: define proper name for global cma region

by Christoph Hellwig

Hi Pintu, On Sat, Jul 29, 2023 at 08:05:15AM +0530, Pintu Kumar wrote: > The current global cma region name defined as "reserved" > which is misleading, creates confusion and too generic. > > Also, the default cma allocation happens from global cma region, > so, if one has to figure out all allocations happening from > global cma region, this seems easier. > > Thus, change the name from "reserved" to "global-cma-region". I agree that reserved is not a very useful name. Unfortuately the name of the region leaks to userspace through cma_heap. So I think we need prep patches to hardcode "reserved" in add_default_cma_heap first, and then remove the cma_get_name first.

1 year, 1 month

[RFC PATCH 00/10] Device Memory TCP

by Mina Almasry

* TL;DR: Device memory TCP (devmem TCP) is a proposal for transferring data to and/or from device memory efficiently, without bouncing the data to a host memory buffer. * Problem: A large amount of data transfers have device memory as the source and/or destination. Accelerators drastically increased the volume of such transfers. Some examples include: - ML accelerators transferring large amounts of training data from storage into GPU/TPU memory. In some cases ML training setup time can be as long as 50% of TPU compute time, improving data transfer throughput & efficiency can help improving GPU/TPU utilization. - Distributed training, where ML accelerators, such as GPUs on different hosts, exchange data among them. - Distributed raw block storage applications transfer large amounts of data with remote SSDs, much of this data does not require host processing. Today, the majority of the Device-to-Device data transfers the network are implemented as the following low level operations: Device-to-Host copy, Host-to-Host network transfer, and Host-to-Device copy. The implementation is suboptimal, especially for bulk data transfers, and can put significant strains on system resources, such as host memory bandwidth, PCIe bandwidth, etc. One important reason behind the current state is the kernel’s lack of semantics to express device to network transfers. * Proposal: In this patch series we attempt to optimize this use case by implementing socket APIs that enable the user to: 1. send device memory across the network directly, and 2. receive incoming network packets directly into device memory. Packet _payloads_ go directly from the NIC to device memory for receive and from device memory to NIC for transmit. Packet _headers_ go to/from host memory and are processed by the TCP/IP stack normally. The NIC _must_ support header split to achieve this. Advantages: - Alleviate host memory bandwidth pressure, compared to existing network-transfer + device-copy semantics. - Alleviate PCIe BW pressure, by limiting data transfer to the lowest level of the PCIe tree, compared to traditional path which sends data through the root complex. With this proposal we're able to reach ~96.6% line rate speeds with data sent and received directly from/to device memory. * Patch overview: ** Part 1: struct paged device memory Currently the standard for device memory sharing is DMABUF, which doesn't generate struct pages. On the other hand, networking stack (skbs, drivers, and page pool) operate on pages. We have 2 options: 1. Generate struct pages for dmabuf device memory, or, 2. Modify the networking stack to understand a new memory type. This proposal implements option #1. We implement a small framework to generate struct pages for an sg_table returned from dma_buf_map_attachment(). The support added here should be generic and easily extended to other use cases interested in struct paged device memory. We use this framework to generate pages that can be used in the networking stack. ** Part 2: recvmsg() & sendmsg() APIs We define user APIs for the user to send and receive these dmabuf pages. ** part 3: support for unreadable skb frags Dmabuf pages are not accessible by the host; we implement changes throughput the networking stack to correctly handle skbs with unreadable frags. ** part 4: page pool support We piggy back on Jakub's page pool memory providers idea: https://github.com/kuba-moo/linux/tree/pp-providers It allows the page pool to define a memory provider that provides the page allocation and freeing. It helps abstract most of the device memory TCP changes from the driver. This is not strictly necessary, the driver can choose to allocate dmabuf pages and use them directly without going through the page pool (if acceptable to their maintainers). Not included with this RFC is the GVE devmem TCP support, just to simplify the review. Code available here if desired: https://github.com/mina/linux/tree/tcpdevmem This RFC is built on top of v6.4-rc7 with Jakub's pp-providers changes cherry-picked. * NIC dependencies: 1. (strict) Devmem TCP require the NIC to support header split, i.e. the capability to split incoming packets into a header + payload and to put each into a separate buffer. Devmem TCP works by using dmabuf pages for the packet payload, and host memory for the packet headers. 2. (optional) Devmem TCP works better with flow steering support & RSS support, i.e. the NIC's ability to steer flows into certain rx queues. This allows the sysadmin to enable devmem TCP on a subset of the rx queues, and steer devmem TCP traffic onto these queues and non devmem TCP elsewhere. The NIC I have access to with these properties is the GVE with DQO support running in Google Cloud, but any NIC that supports these features would suffice. I may be able to help reviewers bring up devmem TCP on their NICs. * Testing: The series includes a udmabuf kselftest that show a simple use case of devmem TCP and validates the entire data path end to end without a dependency on a specific dmabuf provider. Not included in this series is our devmem TCP benchmark, which transfers data to/from GPU dmabufs directly. With this implementation & benchmark we're able to reach ~96.6% line rate speeds with 4 GPU/NIC pairs running bi-direction traffic, with all the packet payloads going straight to the GPU memory (no host buffer bounce). ** Test Setup Kernel: v6.4-rc7, with this RFC and Jakub's memory provider API cherry-picked locally. Hardware: Google Cloud A3 VMs. NIC: GVE with header split & RSS & flow steering support. Benchmark: custom devmem TCP benchmark not yet open sourced. Mina Almasry (10): dma-buf: add support for paged attachment mappings dma-buf: add support for NET_RX pages dma-buf: add support for NET_TX pages net: add support for skbs with unreadable frags tcp: implement recvmsg() RX path for devmem TCP net: add SO_DEVMEM_DONTNEED setsockopt to release RX pages tcp: implement sendmsg() TX path for for devmem tcp selftests: add ncdevmem, netcat for devmem TCP memory-provider: updates core provider API for devmem TCP memory-provider: add dmabuf devmem provider drivers/dma-buf/dma-buf.c | 444 ++++++++++++++++ include/linux/dma-buf.h | 142 +++++ include/linux/netdevice.h | 1 + include/linux/skbuff.h | 34 +- include/linux/socket.h | 1 + include/net/page_pool.h | 21 + include/net/sock.h | 4 + include/net/tcp.h | 6 +- include/uapi/asm-generic/socket.h | 6 + include/uapi/linux/dma-buf.h | 12 + include/uapi/linux/uio.h | 10 + net/core/datagram.c | 3 + net/core/page_pool.c | 111 +++- net/core/skbuff.c | 81 ++- net/core/sock.c | 47 ++ net/ipv4/tcp.c | 262 +++++++++- net/ipv4/tcp_input.c | 13 +- net/ipv4/tcp_ipv4.c | 8 + net/ipv4/tcp_output.c | 5 +- net/packet/af_packet.c | 4 +- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/ncdevmem.c | 693 +++++++++++++++++++++++++ 23 files changed, 1868 insertions(+), 42 deletions(-) create mode 100644 tools/testing/selftests/net/ncdevmem.c -- 2.41.0.390.g38632f3daf-goog

1 year, 2 months

[PATCH v2] dma-buf: Fix the typo in DMA-BUF statistics doc

by Luc Ma

From: Luc Ma <luc(a)sietium.com> The kernel-doc for DMA-BUF statistics mentions /sys/kernel/dma-buf/buffers but the correct path is /sys/kernel/dmabuf/buffers instead. Signed-off-by: Luc Ma <luc(a)sietium.com> Reviewed-by: Javier Martinez Canillas <javierm(a)redhat.com> --- drivers/dma-buf/dma-buf-sysfs-stats.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dma-buf/dma-buf-sysfs-stats.c b/drivers/dma-buf/dma-buf-sysfs-stats.c index 6cfbbf0720bd..b5b62e40ccc1 100644 --- a/drivers/dma-buf/dma-buf-sysfs-stats.c +++ b/drivers/dma-buf/dma-buf-sysfs-stats.c @@ -33,7 +33,7 @@ * into their address space. This necessitated the creation of the DMA-BUF sysfs * statistics interface to provide per-buffer information on production systems. * - * The interface at ``/sys/kernel/dma-buf/buffers`` exposes information about + * The interface at ``/sys/kernel/dmabuf/buffers`` exposes information about * every DMA-BUF when ``CONFIG_DMABUF_SYSFS_STATS`` is enabled. * * The following stats are exposed by the interface: -- 2.25.1

1 year, 2 months

[PATCH v2] scsi: lpfc: Fix potential deadlock on &ndlp->lock

by Chengfeng Ye

As &ndlp->lock is acquired by timer lpfc_els_retry_delay() under softirq context, process context code acquiring the lock &ndlp->lock should disable irq or bh, otherwise deadlock could happen if the timer preempt the execution while the lock is held in process context on the same CPU. The two lock acquisition inside lpfc_cleanup_pending_mbox() does not disable irq or softirq. [Deadlock Scenario] lpfc_cmpl_els_fdisc() -> lpfc_cleanup_pending_mbox() -> spin_lock(&ndlp->lock); <irq> -> lpfc_els_retry_delay() -> lpfc_nlp_get() -> spin_lock_irqsave(&ndlp->lock, flags); (deadlock here) This flaw was found by an experimental static analysis tool I am developing for irq-related deadlock. The patch fix the potential deadlock by spin_lock_irq() to disable irq. Signed-off-by: Chengfeng Ye <dg573847474(a)gmail.com> --- drivers/scsi/lpfc/lpfc_sli.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 58d10f8f75a7..8555f6bb9742 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -21049,9 +21049,9 @@ lpfc_cleanup_pending_mbox(struct lpfc_vport *vport) mb->mbox_flag |= LPFC_MBX_IMED_UNREG; restart_loop = 1; spin_unlock_irq(&phba->hbalock); - spin_lock(&ndlp->lock); + spin_lock_irq(&ndlp->lock); ndlp->nlp_flag &= ~NLP_IGNR_REG_CMPL; - spin_unlock(&ndlp->lock); + spin_unlock_irq(&ndlp->lock); spin_lock_irq(&phba->hbalock); break; } @@ -21067,9 +21067,9 @@ lpfc_cleanup_pending_mbox(struct lpfc_vport *vport) ndlp = (struct lpfc_nodelist *)mb->ctx_ndlp; mb->ctx_ndlp = NULL; if (ndlp) { - spin_lock(&ndlp->lock); + spin_lock_irq(&ndlp->lock); ndlp->nlp_flag &= ~NLP_IGNR_REG_CMPL; - spin_unlock(&ndlp->lock); + spin_unlock_irq(&ndlp->lock); lpfc_nlp_put(ndlp); } } -- 2.17.1

1 year, 2 months

[PATCH] scsi: lpfc: Fix potential deadlock on &ndlp->lock

by Chengfeng Ye

As &ndlp->lock is acquired by timer lpfc_els_retry_delay() under softirq context, process context code acquiring the lock &phba->hbalock should disable irq or bh, otherwise deadlock could happen if the timer preempt the execution while the lock is held in process context on the same CPU. The two lock acquisition inside lpfc_cleanup_pending_mbox() does not disable irq or softirq. [Deadlock Scenario] lpfc_cmpl_els_fdisc() -> lpfc_cleanup_pending_mbox() -> spin_lock(&ndlp->lock); <irq> -> lpfc_els_retry_delay() -> lpfc_nlp_get() -> spin_lock_irqsave(&ndlp->lock, flags); (deadlock here) This flaw was found by an experimental static analysis tool I am developing for irq-related deadlock. The patch fix the potential deadlock by spin_lock_irq() to disable irq. Signed-off-by: Chengfeng Ye <dg573847474(a)gmail.com> --- drivers/scsi/lpfc/lpfc_sli.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 58d10f8f75a7..8555f6bb9742 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -21049,9 +21049,9 @@ lpfc_cleanup_pending_mbox(struct lpfc_vport *vport) mb->mbox_flag |= LPFC_MBX_IMED_UNREG; restart_loop = 1; spin_unlock_irq(&phba->hbalock); - spin_lock(&ndlp->lock); + spin_lock_irq(&ndlp->lock); ndlp->nlp_flag &= ~NLP_IGNR_REG_CMPL; - spin_unlock(&ndlp->lock); + spin_unlock_irq(&ndlp->lock); spin_lock_irq(&phba->hbalock); break; } @@ -21067,9 +21067,9 @@ lpfc_cleanup_pending_mbox(struct lpfc_vport *vport) ndlp = (struct lpfc_nodelist *)mb->ctx_ndlp; mb->ctx_ndlp = NULL; if (ndlp) { - spin_lock(&ndlp->lock); + spin_lock_irq(&ndlp->lock); ndlp->nlp_flag &= ~NLP_IGNR_REG_CMPL; - spin_unlock(&ndlp->lock); + spin_unlock_irq(&ndlp->lock); lpfc_nlp_put(ndlp); } } -- 2.17.1

1 year, 2 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig July 2023