We observed an issue with NXP 5.15 LTS kernel that dma_alloc_coherent() may fail sometimes when there're multiple processes trying to allocate CMA memory.
This issue can be very easily reproduced on MX6Q SDB board with latest linux-next kernel by writing a test module creating 16 or 32 threads allocating random size of CMA memory in parallel at the background. Or simply enabling CONFIG_CMA_DEBUG, you can see endless of CMA alloc retries during booting: [ 1.452124] cma: cma_alloc(): memory range at (ptrval) is busy,retrying .... (thousands of reties)
The root cause of this issue is that since commit a4efc174b382 ("mm/cma.c: remove redundant cma_mutex lock"), CMA supports concurrent memory allocation. It's possible that the memory range process A try to alloc has already been isolated by the allocation of process B during memory migration.
The problem here is that the memory range isolated during one allocation by start_isolate_page_range() could be much bigger than the real size we want to alloc due to the range is aligned to MAX_ORDER_NR_PAGES.
Taking an ARMv7 platform with 1G memory as an example, when MAX_ORDER_NR_PAGES is big (e.g. 32M with max_order 14) and CMA memory is relatively small (e.g. 128M), there're only 4 MAX_ORDER slot, then it's very easy that all CMA memory may have already been isolated by other processes when one trying to allocate memory using dma_alloc_coherent(). Since current CMA code will only scan one time of whole available CMA memory, then dma_alloc_coherent() may easy fail due to contention with other processes.
This patchset introduces a retry mechanism to rescan CMA bitmap for -EBUSY error in case the target pageblock may has been temporarily isolated by others and released later.
It also improves the CMA allocation performance by trying the next MAX_ORDER_NR_PAGES range during reties rather than looping within the same isolated range in small steps which wasting CPU mips.
The following test is based on linux-next: next-20211213.
Without the fix, it's easily fail. # insmod cma_alloc.ko pnum=16 [ 274.322369] CMA alloc test enter: thread number: 16 [ 274.329948] cpu: 0, pid: 692, index 4 pages 144 [ 274.330143] cpu: 1, pid: 694, index 2 pages 44 [ 274.330359] cpu: 2, pid: 695, index 7 pages 757 [ 274.330760] cpu: 2, pid: 696, index 4 pages 144 [ 274.330974] cpu: 2, pid: 697, index 6 pages 512 [ 274.331223] cpu: 2, pid: 698, index 6 pages 512 [ 274.331499] cpu: 2, pid: 699, index 2 pages 44 [ 274.332228] cpu: 2, pid: 700, index 0 pages 7 [ 274.337421] cpu: 0, pid: 701, index 1 pages 38 [ 274.337618] cpu: 2, pid: 702, index 0 pages 7 [ 274.344669] cpu: 1, pid: 703, index 0 pages 7 [ 274.344807] cpu: 3, pid: 704, index 6 pages 512 [ 274.348269] cpu: 2, pid: 705, index 5 pages 148 [ 274.349490] cma: cma_alloc: reserved: alloc failed, req-size: 38 pages, ret: -16 [ 274.366292] cpu: 1, pid: 706, index 4 pages 144 [ 274.366562] cpu: 0, pid: 707, index 3 pages 128 [ 274.367356] cma: cma_alloc: reserved: alloc failed, req-size: 128 pages, ret: -16 [ 274.367370] cpu: 0, pid: 707, index 3 pages 128 failed [ 274.371148] cma: cma_alloc: reserved: alloc failed, req-size: 148 pages, ret: -16 [ 274.375348] cma: cma_alloc: reserved: alloc failed, req-size: 144 pages, ret: -16 [ 274.384256] cpu: 2, pid: 708, index 0 pages 7 ....
With the fix, 32 threads allocating in parallel can pass overnight stress test.
root@imx6qpdlsolox:~# insmod cma_alloc.ko pnum=32 [ 112.976809] cma_alloc: loading out-of-tree module taints kernel. [ 112.984128] CMA alloc test enter: thread number: 32 [ 112.989748] cpu: 2, pid: 707, index 6 pages 512 [ 112.994342] cpu: 1, pid: 708, index 6 pages 512 [ 112.995162] cpu: 0, pid: 709, index 3 pages 128 [ 112.995867] cpu: 2, pid: 710, index 0 pages 7 [ 112.995910] cpu: 3, pid: 711, index 2 pages 44 [ 112.996005] cpu: 3, pid: 712, index 7 pages 757 [ 112.996098] cpu: 3, pid: 713, index 7 pages 757 ... [41877.368163] cpu: 1, pid: 737, index 2 pages 44 [41877.369388] cpu: 1, pid: 736, index 3 pages 128 [41878.486516] cpu: 0, pid: 737, index 2 pages 44 [41878.486515] cpu: 2, pid: 739, index 4 pages 144 [41878.486622] cpu: 1, pid: 736, index 3 pages 128 [41878.486948] cpu: 2, pid: 735, index 7 pages 757 [41878.487279] cpu: 2, pid: 738, index 4 pages 144 [41879.526603] cpu: 1, pid: 739, index 3 pages 128 [41879.606491] cpu: 2, pid: 737, index 3 pages 128 [41879.606550] cpu: 0, pid: 736, index 0 pages 7 [41879.612271] cpu: 2, pid: 738, index 4 pages 144 ... v1: https://patchwork.kernel.org/project/linux-mm/cover/20211215080242.3034856-1...
v2: https://patchwork.kernel.org/project/linux-mm/cover/20220112131552.3329380-1...
Dong Aisheng (2): mm: cma: fix allocation may fail sometimes mm: cma: try next MAX_ORDER_NR_PAGES during retry
mm/cma.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-)