This series adds AF_XDP zero coppy support to am65-cpsw driver.
Tests were performed on AM62x-sk with xdpsock application [1].
A clear improvement is seen in 64 byte packets on Transmit (txonly)
and receive (rxdrop).
1500 byte test seems to be limited by line rate (1G link) so no
improvement seen there in packet rate. A test on higher speed link
(or PHY-less setup) might be worthwile.
There is some issue during l2fwd with 64 byte packets and benchmark
results show 0. This issue needs to be debugged further.
A 512 byte l2fwd test result has been added to compare instead.
AF_XDP performance using 64 byte packets in Kpps.
Benchmark: XDP-SKB XDP-Native XDP-Native(ZeroCopy)
rxdrop 322 491 845
txonly 390 394 723
l2fwd 205 257 0
AF_XDP performance using 512 byte packets in Kpps.
l2fwd 140 167 231
AF_XDP performance using 1500 byte packets in Kpps.
Benchmark: XDP-SKB XDP-Native XDP-Native(ZeroCopy)
rxdrop 82 82 82
txonly 82 82 82
l2fwd 82 82 82
[1]: https://github.com/xdp-project/bpf-examples/tree/master/AF_XDP-example
Signed-off-by: Roger Quadros <rogerq(a)kernel.org>
---
Changes in v2:
- Prevent crash on systems with 1 of 2 ports disabled in device tree. check
for valid ndev before registering/unregistering XDP RXQ.
Reported-by: Meghana Malladi <m-malladi(a)ti.com>
- Retain page pool on XDP program exchangae so we don't have to re-alloacate
memory.
- Fix clearing of irq_disabled flag in am65_cpsw_nuss_rx_poll().
- Link to v1: https://lore.kernel.org/r/20250520-am65-cpsw-xdp-zc-v1-0-45558024f566@kerne…
---
Roger Quadros (7):
net: ethernet: ti: am65-cpsw: fix BPF Program change on multi-port CPSW
net: ethernet: ti: am65-cpsw: Retain page_pool on XDP program exchange
net: ethernet: ti: am65-cpsw: add XSK pool helpers
net: ethernet: ti: am65-cpsw: Add AF_XDP zero copy for RX
net: ethernet: ti: am65-cpsw: Add AF_XDP zero copy for TX
net: ethernet: ti: am65-cpsw: enable zero copy in XDP features
net: ethernet: ti: am65-cpsw: Fix clearing of irq_disabled flag in rx_poll
drivers/net/ethernet/ti/Makefile | 2 +-
drivers/net/ethernet/ti/am65-cpsw-nuss.c | 583 ++++++++++++++++++++++++++-----
drivers/net/ethernet/ti/am65-cpsw-nuss.h | 37 +-
drivers/net/ethernet/ti/am65-cpsw-xdp.c | 155 ++++++++
4 files changed, 692 insertions(+), 85 deletions(-)
---
base-commit: a0c3aefb08cd81864b17c23c25b388dba90b9dad
change-id: 20250225-am65-cpsw-xdp-zc-2af9e4be1356
Best regards,
--
Roger Quadros <rogerq(a)kernel.org>
Hi everybody,
we have documented here https://www.kernel.org/doc/html/latest/driver-api/dma-buf.html#dma-fence-cr… that dma_fence objects must signal in a reasonable amount of time, but at the same time note that drivers might have a different idea of what reasonable means.
Recently I realized that this is actually not a good idea. Background is that the wall clock timeout means that for example the OOM killer might actually wait for this timeout to be able to terminate a process and reclaim the memory used. And this is just an example of how general kernel features might depend on that.
Some drivers and fence implementations used 10 seconds and that raised complains by end users. So at least amdgpu recently switched to 2 second which triggered an internal discussion about it.
This patch set here now adds a define to the dma_fence header which gives 2 seconds as reasonable amount of time. SW-sync is modified to always taint the kernel (since it doesn't has a timeout), VGEM is switched over to the new define and the scheduler gets a warning and taints the kernel if a driver uses a timeout longer than that.
I have not much intention of actually committing the patches (maybe except the SW-sync one), but question is if 2 seconds are reasonable?
Regards,
Christian.
From: Barry Song <v-songbaohua(a)oppo.com>
In many cases, the pages passed to vmap() may include
high-order pages—for example, the systemheap often allocates
pages in descending order: order 8, then 4, then 0. Currently,
vmap() iterates over every page individually—even the pages
inside a high-order block are handled one by one. This patch
detects high-order pages and maps them as a single contiguous
block whenever possible.
Another possibility is to implement a new API, vmap_sg().
However, that change seems to be quite large in scope.
When vmapping a 128MB dma-buf using the systemheap,
this RFC appears to make system_heap_do_vmap() 16× faster:
W/ patch:
[ 51.363682] system_heap_do_vmap took 2474000 ns
[ 53.307044] system_heap_do_vmap took 2469008 ns
[ 55.061985] system_heap_do_vmap took 2519008 ns
[ 56.653810] system_heap_do_vmap took 2674000 ns
W/o patch:
[ 8.260880] system_heap_do_vmap took 39490000 ns
[ 32.513292] system_heap_do_vmap took 38784000 ns
[ 82.673374] system_heap_do_vmap took 40711008 ns
[ 84.579062] system_heap_do_vmap took 40236000 ns
Cc: Uladzislau Rezki <urezki(a)gmail.com>
Cc: Sumit Semwal <sumit.semwal(a)linaro.org>
Cc: John Stultz <jstultz(a)google.com>
Cc: Maxime Ripard <mripard(a)kernel.org>
Signed-off-by: Barry Song <v-songbaohua(a)oppo.com>
---
mm/vmalloc.c | 49 +++++++++++++++++++++++++++++++++++++++++++------
1 file changed, 43 insertions(+), 6 deletions(-)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 0832f944544c..af2e3e8c052a 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -642,6 +642,34 @@ static int vmap_small_pages_range_noflush(unsigned long addr, unsigned long end,
return err;
}
+static inline int get_vmap_batch_order(struct page **pages,
+ unsigned int stride,
+ int max_steps,
+ unsigned int idx)
+{
+ /*
+ * Currently, batching is only supported in vmap_pages_range
+ * when page_shift == PAGE_SHIFT.
+ */
+ if (stride != 1)
+ return 0;
+
+ struct page *base = pages[idx];
+ if (!PageHead(base))
+ return 0;
+
+ int order = compound_order(base);
+ int nr_pages = 1 << order;
+
+ if (max_steps < nr_pages)
+ return 0;
+
+ for (int i = 0; i < nr_pages; i++)
+ if (pages[idx + i] != base + i)
+ return 0;
+ return order;
+}
+
/*
* vmap_pages_range_noflush is similar to vmap_pages_range, but does not
* flush caches.
@@ -655,23 +683,32 @@ int __vmap_pages_range_noflush(unsigned long addr, unsigned long end,
pgprot_t prot, struct page **pages, unsigned int page_shift)
{
unsigned int i, nr = (end - addr) >> PAGE_SHIFT;
+ unsigned int stride;
WARN_ON(page_shift < PAGE_SHIFT);
+ /*
+ * Some users may allocate pages from high-order down to order 0.
+ * We roughly check if the first page is a compound page. If so,
+ * there is a chance to batch multiple pages together.
+ */
if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC) ||
- page_shift == PAGE_SHIFT)
+ (page_shift == PAGE_SHIFT && !PageCompound(pages[0])))
return vmap_small_pages_range_noflush(addr, end, prot, pages);
- for (i = 0; i < nr; i += 1U << (page_shift - PAGE_SHIFT)) {
- int err;
+ stride = 1U << (page_shift - PAGE_SHIFT);
+ for (i = 0; i < nr; ) {
+ int err, order;
- err = vmap_range_noflush(addr, addr + (1UL << page_shift),
+ order = get_vmap_batch_order(pages, stride, nr - i, i);
+ err = vmap_range_noflush(addr, addr + (1UL << (page_shift + order)),
page_to_phys(pages[i]), prot,
- page_shift);
+ page_shift + order);
if (err)
return err;
- addr += 1UL << page_shift;
+ addr += 1UL << (page_shift + order);
+ i += 1U << (order + page_shift - PAGE_SHIFT);
}
return 0;
--
2.39.3 (Apple Git-146)
clear_page() translates into memset(*p, 0, PAGE_SIZE) on some
architectures, but on the major architectures it will call
an optimized assembly snippet so use this instead of open
coding a memset().
Signed-off-by: Linus Walleij <linus.walleij(a)linaro.org>
---
drivers/dma-buf/heaps/cma_heap.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c
index 0df007111975..9eaff80050f2 100644
--- a/drivers/dma-buf/heaps/cma_heap.c
+++ b/drivers/dma-buf/heaps/cma_heap.c
@@ -315,7 +315,7 @@ static struct dma_buf *cma_heap_allocate(struct dma_heap *heap,
while (nr_clear_pages > 0) {
void *vaddr = kmap_local_page(page);
- memset(vaddr, 0, PAGE_SIZE);
+ clear_page(vaddr);
kunmap_local(vaddr);
/*
* Avoid wasting time zeroing memory if the process
---
base-commit: 3a8660878839faadb4f1a6dd72c3179c1df56787
change-id: 20251129-dma-buf-heap-clear-page-248bb236e4c4
Best regards,
--
Linus Walleij <linus.walleij(a)linaro.org>
Barely anyone uses dma_fence_signal()'s (and similar functions') return
code. Checking it is pretty much useless anyways, because what are you
going to do if a fence was already signal it? Unsignal it and signal it
again? ;p
Removing the return code simplifies the API and makes it easier for me
to sit on top with Rust DmaFence.
Philipp Stanner (6):
dma-buf/dma-fence: Add dma_fence_test_signaled_flag()
amd/amdkfd: Ignore return code of dma_fence_signal()
drm/gpu/xe: Ignore dma_fenc_signal() return code
dma-buf: Don't misuse dma_fence_signal()
drm/ttm: Remove return check of dma_fence_signal()
dma-buf/dma-fence: Remove return code of signaling-functions
drivers/dma-buf/dma-fence.c | 59 ++++++-------------
drivers/dma-buf/st-dma-fence.c | 7 +--
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 5 +-
.../gpu/drm/ttm/tests/ttm_bo_validate_test.c | 3 +-
drivers/gpu/drm/xe/xe_hw_fence.c | 5 +-
include/linux/dma-fence.h | 33 ++++++++---
6 files changed, 53 insertions(+), 59 deletions(-)
--
2.49.0