Lkft-triage July 2025

lkft-triage@lists.linaro.org

1 participants
1 discussions

next-20250627: IOMMU DMA warning during NVMe I/O completion after 06cae0e3f61c

by Ben Copeland

A regression in the NVMe PCI driver triggers IOMMU DMA warnings during I/O completion on ARM64 systems. The issue manifests as drivers/iommu/dma-iommu.c warnings during the DMA unmapping process. IOMMU DMA unmapping warnings occur during NVMe completion processing. The warnings appear consistently during normal NVMe I/O operations. - CONFIG_ARM64_64K_PAGES=y Test Environment: * Ampere Altra - AmpereOne Regression Analysis: - New regression? Yes - Reproducibility? Yes First seen on next-20250627 Good: next-20250626 Bad: next-20250627 Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Test log [ 32.699872] WARNING: drivers/iommu/dma-iommu.c:1232 at iommu_dma_unmap_page+0xc4/0xe8, CPU#13: swapper/13/0 [ 32.714204] Modules linked in: cdc_ether usbnet sm3_ce nvme sha3_ce nvme_core xhci_pci_renesas arm_cspmu_module arm_spe_pmu ipmi_devintf arm_cmn ipmi_msghandler cppc_cpufreq fuse drm backlight ip_tables x_tables [ 32.732967] CPU: 13 UID: 0 PID: 0 Comm: swapper/13 Tainted: G W 6.16.0-rc3-next-20250627 #1 PREEMPT [ 32.743562] Tainted: [W]=WARN [ 32.749381] Hardware name: Inspur NF5280R7/Mitchell MB, BIOS 04.04.00004001 2025-02-04 22:23:30 02/04/2025 [ 32.759020] pstate: 63400009 (nZCv daif +PAN -UAO +TCO +DIT -SSBS BTYPE=--) [ 32.768746] pc : iommu_dma_unmap_page+0xc4/0xe8 [ 32.776040] lr : iommu_dma_unmap_page+0x40/0xe8 [ 32.780559] sp : ffff8000801afde0 [ 32.783861] x29: ffff8000801afde0 x28: 0000000000000005 x27: fff00001d7d230f0 [ 32.790983] x26: 0000000000000000 x25: fff00001003da0c8 x24: 0000000000000002 [ 32.798106] x23: 0000000000000000 x22: 0000000000001000 x21: 00000000feed5000 [ 32.805229] x20: fff00001003da0c8 x19: fff00001d7d23000 x18: 0000000000080000 [ 32.812352] x17: 0000000000000040 x16: ffffae5c7594ea68 x15: 0000000000000000 [ 32.819474] x14: 000000000007ffff x13: 0000000000000001 x12: 000000000000002c [ 32.826597] x11: 00000000000fffff x10: ffffffffffffffff x9 : ffffae5c76358e60 [ 32.833719] x8 : ffff8000801afd68 x7 : ffffae5c76358a78 x6 : 00000000feed5001 [ 32.840842] x5 : 000000000000000d x4 : ffffae5c76358a78 x3 : 0000000000000000 [ 32.847965] x2 : 0000000000000000 x1 : fff00001027e54c0 x0 : 0000000000000000 [ 32.855088] Call trace: [ 32.857521] iommu_dma_unmap_page+0xc4/0xe8 (P) [ 32.862039] dma_unmap_page_attrs+0x238/0x2a0 [ 32.866385] nvme_unmap_data+0x1dc/0x278 [nvme] [ 32.870904] nvme_pci_complete_rq+0xb8/0x118 [nvme] [ 32.878632] blk_complete_reqs+0x5c/0x80 [ 32.885320] blk_done_softirq+0x28/0x40 [ 32.892006] handle_softirqs+0x16c/0x408 [ 32.896436] __do_softirq+0x1c/0x28 [ 32.899912] ____do_softirq+0x18/0x30 [ 32.903561] call_on_irq_stack+0x24/0x30 [ 32.907472] do_softirq_own_stack+0x24/0x38 [ 32.911642] __irq_exit_rcu+0xd4/0x118 [ 32.915378] irq_exit_rcu+0x18/0x48 [ 32.918854] el1_interrupt+0x38/0x68 [ 32.922418] el1h_64_irq_handler+0x18/0x28 [ 32.926502] el1h_64_irq+0x6c/0x70 [ 32.929891] cpuidle_enter_state+0xcc/0x4c0 (P) [ 32.934410] cpuidle_enter+0x40/0x60 [ 32.937972] do_idle+0x204/0x288 [ 32.941188] cpu_startup_entry+0x3c/0x50 [ 32.945098] secondary_start_kernel+0x140/0x168 [ 32.949617] __secondary_switched+0xc0/0xc8 [ 32.953788] ---[ end trace 0000000000000000 ]--- ## Bisection git bisect start # bad: [06cae0e3f61c4c1ef18726b817bbb88c29f81e57] nvme-pci: merge the simple PRP and SGL setup into a common helper git bisect bad 06cae0e3f61c4c1ef18726b817bbb88c29f81e57 # good: [d6c12c69ef4fa33e32ceda4a53991ace01401cd9] block: add scatterlist-less DMA mapping helpers git bisect good d6c12c69ef4fa33e32ceda4a53991ace01401cd9 # good: [07c81cbf438b769e0d673be3b5c021a424a4dc6f] nvme-pci: refactor nvme_pci_use_sgls git bisect good 07c81cbf438b769e0d673be3b5c021a424a4dc6f # first bad commit: [06cae0e3f61c4c1ef18726b817bbb88c29f81e57] nvme-pci: merge the simple PRP and SGL setup into a common helper ## Source * Kernel version: 6.16.0-rc3-next-20250627 * Git tree: https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git * Git sha: 2aeda9592360c200085898a258c4754bfe879921 * Git describe: 6.16.0-rc3-next-20250627 * Project details: https://qa-reports.linaro.org/lkft/linux-next-master-ampere/build/next-2025… * Architectures: arm64 * Toolchains: gcc-13 * Kconfigs: gcc-13-lkftconfig-64k_page_size ## Build arm64 * Test LAVA log: https://lkft-staging.validation.linaro.org/scheduler/job/180925#L7760 * Build link: https://storage.tuxsuite.com/public/ampere/ci/builds/2z6SDAcW8aM7PBWZ8oJ82Q… * Kernel config: https://storage.tuxsuite.com/public/ampere/ci/builds/2z6SDAcW8aM7PBWZ8oJ82Q… ## Steps to reproduce: # https://storage.tuxsuite.com/public/ampere/ci/builds/2z6SDAcW8aM7PBWZ8oJ82Q… Linaro LKFT https://lkft.linaro.org

4 hours, 29 minutes

2025

2024

2023

2022

2021

2020

2019

2018

2017

Lkft-triage July 2025