Following deadlock warning noticed on arm64 Qcom APQ 8016 SBC device with a given config (kselftest merge configs) while booting.
This is almost always reproducible with a given config. started happening from stable-rc v6.1.55.
Reported-by: Linux Kernel Functional Testing lkft@linaro.org
Boot log: ------- [ 28.082133] ====================================================== [ 28.082673] WARNING: possible circular locking dependency detected [ 28.088664] 6.1.55 #1 Not tainted [ 28.094813] ------------------------------------------------------ [ 28.098211] (udev-worker)/278 is trying to acquire lock: [ 28.104285] ffff80000b2b2d30 (rpm_smd_clk_lock){+.+.}-{3:3}, at: clk_smd_rpm_set_rate (drivers/clk/qcom/clk-smd-rpm.c:321) [ 28.109772] [ 28.109772] but task is already holding lock: [ 28.118525] ffff80000b49f3e0 (icc_bw_lock){+.+.}-{3:3}, at: icc_node_add (drivers/interconnect/core.c:973) [ 28.124175] [ 28.124175] which lock already depends on the new lock. [ 28.124175] [ 28.131729] [ 28.131729] the existing dependency chain (in reverse order) is: [ 28.140058] [ 28.140058] -> #2 (icc_bw_lock){+.+.}-{3:3}: [ 28.147518] icc_init (drivers/interconnect/core.c:1159 (discriminator 1)) [ 28.153235] do_one_initcall (init/main.c:1292) [ 28.156884] kernel_init_freeable (init/main.c:1364 (discriminator 3) init/main.c:1381 (discriminator 3) init/main.c:1400 (discriminator 3) init/main.c:1620 (discriminator 3)) [ 28.161227] kernel_init (init/main.c:1510) [ 28.165910] ret_from_fork (arch/arm64/kernel/entry.S:861) [ 28.170077] [ 28.170077] -> #1 (fs_reclaim){+.+.}-{0:0}: [ 28.174079] fs_reclaim_acquire (mm/page_alloc.c:4700 mm/page_alloc.c:4691) [ 28.179890] __kmem_cache_alloc_node (include/linux/sched/mm.h:272 mm/slab.h:710 mm/slub.c:3318 mm/slub.c:3437) [ 28.184492] __kmalloc (mm/slab_common.c:956 mm/slab_common.c:968) [ 28.189434] qcom_rpm_smd_write (include/linux/slab.h:558 drivers/soc/qcom/smd-rpm.c:116) [ 28.193084] clk_smd_rpm_prepare (drivers/clk/qcom/clk-smd-rpm.c:192 drivers/clk/qcom/clk-smd-rpm.c:253) [ 28.197945] clk_core_prepare (drivers/clk/clk.c:971) [ 28.202545] clk_prepare (drivers/clk/clk.c:1004 drivers/clk/clk.c:1027 drivers/clk/clk.c:1022) [ 28.206881] amba_get_enable_pclk (include/linux/clk.h:1072 drivers/amba/bus.c:70) [ 28.211052] amba_read_periphid (drivers/amba/bus.c:146) [ 28.215653] amba_match (drivers/amba/bus.c:222) [ 28.220336] __device_attach_driver (drivers/base/base.h:147 (discriminator 1) drivers/base/dd.c:913 (discriminator 1)) [ 28.223900] bus_for_each_drv (drivers/base/bus.c:427) [ 28.229106] __device_attach (drivers/base/dd.c:1017) [ 28.233359] device_initial_probe (drivers/base/dd.c:1065) [ 28.237699] bus_probe_device (drivers/base/bus.c:489) [ 28.242385] deferred_probe_work_func (drivers/base/dd.c:124) [ 28.246729] process_one_work (kernel/workqueue.c:2297) [ 28.251761] worker_thread (include/linux/list.h:292 (discriminator 2) kernel/workqueue.c:2352 (discriminator 2) kernel/workqueue.c:2444 (discriminator 2)) [ 28.256186] kthread (kernel/kthread.c:376) [ 28.260697] ret_from_fork (arch/arm64/kernel/entry.S:861) [ 28.264258] [ 28.264258] -> #0 (rpm_smd_clk_lock){+.+.}-{3:3}: [ 28.268351] __lock_acquire (kernel/locking/lockdep.c:3091 (discriminator 1) kernel/locking/lockdep.c:3209 (discriminator 1) kernel/locking/lockdep.c:3824 (discriminator 1) kernel/locking/lockdep.c:5048 (discriminator 1)) [ 28.274503] lock_acquire (arch/arm64/include/asm/percpu.h:40 kernel/locking/lockdep.c:467 kernel/locking/lockdep.c:5663 kernel/locking/lockdep.c:5626) [ 28.278929] __mutex_lock (kernel/locking/mutex.c:605 kernel/locking/mutex.c:747) [ 28.283095] mutex_lock_nested (kernel/locking/mutex.c:800) [ 28.287263] clk_smd_rpm_set_rate (drivers/clk/qcom/clk-smd-rpm.c:321) [ 28.291691] clk_change_rate (drivers/clk/clk.c:2278) [ 28.296464] clk_core_set_rate_nolock (drivers/clk/clk.c:2382 drivers/clk/clk.c:2340) [ 28.300895] clk_set_rate (drivers/clk/clk.c:2423 drivers/clk/clk.c:2410) [ 28.306012] qcom_icc_set+0x1d4/0x270 icc_smd_rpm [ 28.310186] icc_node_add (drivers/interconnect/core.c:994) [ 28.315561] qnoc_probe+0x240/0x420 icc_smd_rpm [ 28.319558] platform_probe (drivers/base/platform.c:1400) [ 28.324675] really_probe (drivers/base/dd.c:560 drivers/base/dd.c:639) [ 28.328841] __driver_probe_device (drivers/base/dd.c:785) [ 28.333011] driver_probe_device (drivers/base/dd.c:815) [ 28.337786] __driver_attach (drivers/base/dd.c:1202) [ 28.342646] bus_for_each_dev (drivers/base/bus.c:301) [ 28.346984] driver_attach (drivers/base/dd.c:1219) [ 28.351411] bus_add_driver (drivers/base/bus.c:618) [ 28.355404] driver_register (drivers/base/driver.c:246) [ 28.359832] __platform_driver_register (drivers/base/platform.c:868) [ 28.364177] msm8916_noc_driver_init+0x28/0x1000 qnoc_msm8916 [ 28.369217] do_one_initcall (init/main.c:1292) [ 28.375804] do_init_module (kernel/module/main.c:2464) [ 28.380144] load_module (kernel/module/main.c:2873) [ 28.384485] __do_sys_finit_module (kernel/module/main.c:2973 (discriminator 1)) [ 28.388742] __arm64_sys_finit_module (kernel/module/main.c:2940) [ 28.393342] invoke_syscall (arch/arm64/include/asm/current.h:19 arch/arm64/kernel/syscall.c:57) [ 28.398546] el0_svc_common.constprop.0 (arch/arm64/kernel/syscall.c:149) [ 28.402894] do_el0_svc (arch/arm64/kernel/syscall.c:207) [ 28.408179] el0_svc (arch/arm64/include/asm/daifflags.h:28 arch/arm64/kernel/entry-common.c:133 arch/arm64/kernel/entry-common.c:142 arch/arm64/kernel/entry-common.c:638) [ 28.411825] el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:656) [ 28.415651] el0t_64_sync (arch/arm64/kernel/entry.S:581) [ 28.420248] [ 28.420248] other info that might help us debug this: [ 28.420248] [ 28.424438] Chain exists of: [ 28.424438] rpm_smd_clk_lock --> fs_reclaim --> icc_bw_lock [ 28.424438] [ 28.432432] Possible unsafe locking scenario: [ 28.432432] [ 28.442481] CPU0 CPU1 [ 28.448374] ---- ---- [ 28.452888] lock(icc_bw_lock); Linux runner-vwmj3eza-project-40964107-concurrent-0 6.1.55 #1 SMP PREEMPT @1695871287 aarch6[ 28.457396] lock(fs_reclaim); 4 [ 28.475726] lock(rpm_smd_clk_lock); [ 28.475770] [ 28.475770] *** DEADLOCK *** [ 28.475770] [ 28.478630] 4 locks held by (udev-worker)/278: [ 28.484350] #0: ffff000004e968f8 (&dev->mutex){....}-{3:3}, at: __driver_attach (drivers/base/dd.c:1201) [ 28.488885] #1: ffff80000b49f2e8 (icc_lock){+.+.}-{3:3}, at: icc_node_add (drivers/interconnect/core.c:971) [ 28.497475] #2: ffff80000b49f3e0 (icc_bw_lock){+.+.}-{3:3}, at: icc_node_add (drivers/interconnect/core.c:973) [ 28.505202] #3: ffff80000b23eeb8 (prepare_lock){+.+.}-{3:3}, at: clk_prepare_lock (drivers/clk/clk.c:129 (discriminator 1)) [ 28.513363] [ 28.513363] stack backtrace: [ 28.521766] CPU: 3 PID: 278 Comm: (udev-worker) Not tainted 6.1.55 #1 [ 28.526028] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT) [ 28.532453] Call trace: [ 28.539202] dump_backtrace (arch/arm64/kernel/stacktrace.c:160 arch/arm64/kernel/stacktrace.c:143) [ 28.541379] show_stack (arch/arm64/kernel/stacktrace.c:167) [ 28.545194] dump_stack_lvl (lib/dump_stack.c:107) [ 28.548582] dump_stack (lib/dump_stack.c:114) [ 28.552225] print_circular_bug (kernel/locking/lockdep.c:2048) [ 28.555529] check_noncircular (kernel/locking/lockdep.c:2173) [ 28.559521] __lock_acquire (kernel/locking/lockdep.c:3091 (discriminator 1) kernel/locking/lockdep.c:3209 (discriminator 1) kernel/locking/lockdep.c:3824 (discriminator 1) kernel/locking/lockdep.c:5048 (discriminator 1)) [ 28.563687] lock_acquire (arch/arm64/include/asm/percpu.h:40 kernel/locking/lockdep.c:467 kernel/locking/lockdep.c:5663 kernel/locking/lockdep.c:5626) [ 28.567767] __mutex_lock (kernel/locking/mutex.c:605 kernel/locking/mutex.c:747) [ 28.571411] mutex_lock_nested (kernel/locking/mutex.c:800) [ 28.575059] clk_smd_rpm_set_rate (drivers/clk/qcom/clk-smd-rpm.c:321) [ 28.578968] clk_change_rate (drivers/clk/clk.c:2278) [ 28.583220] clk_core_set_rate_nolock (drivers/clk/clk.c:2382 drivers/clk/clk.c:2340) [ 28.587128] clk_set_rate (drivers/clk/clk.c:2423 drivers/clk/clk.c:2410) [ 28.591724] qcom_icc_set+0x1d4/0x270 icc_smd_rpm [ 28.595029] icc_node_add (drivers/interconnect/core.c:994) [ 28.599884] qnoc_probe+0x240/0x420 icc_smd_rpm [ 28.603709] platform_probe (drivers/base/platform.c:1400) [ 28.608305] really_probe (drivers/base/dd.c:560 drivers/base/dd.c:639) [ 28.611949] __driver_probe_device (drivers/base/dd.c:785) [ 28.615599] driver_probe_device (drivers/base/dd.c:815) [ 28.619851] __driver_attach (drivers/base/dd.c:1202) [ 28.623844] bus_for_each_dev (drivers/base/bus.c:301) [ 28.628010] driver_attach (drivers/base/dd.c:1219) [ 28.631568] bus_add_driver (drivers/base/bus.c:618) [ 28.635388] driver_register (drivers/base/driver.c:246) [ 28.638948] __platform_driver_register (drivers/base/platform.c:868) [ 28.642771] msm8916_noc_driver_init+0x28/0x1000 qnoc_msm8916 [ 28.647638] do_one_initcall (init/main.c:1292) [ 28.653356] do_init_module (kernel/module/main.c:2464) [ 28.657176] load_module (kernel/module/main.c:2873) [ 28.660995] __do_sys_finit_module (kernel/module/main.c:2973 (discriminator 1)) [ 28.664732] __arm64_sys_finit_module (kernel/module/main.c:2940) [ 28.669159] invoke_syscall (arch/arm64/include/asm/current.h:19 arch/arm64/kernel/syscall.c:57) [ 28.673496] el0_svc_common.constprop.0 (arch/arm64/kernel/syscall.c:149) [ 28.677321] do_el0_svc (arch/arm64/kernel/syscall.c:207) [ 28.682087] el0_svc (arch/arm64/include/asm/daifflags.h:28 arch/arm64/kernel/entry-common.c:133 arch/arm64/kernel/entry-common.c:142 arch/arm64/kernel/entry-common.c:638) [ 28.685558] el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:656) [ 28.688516] el0t_64_sync (arch/arm64/kernel/entry.S:581)
Links: - https://lkft.validation.linaro.org/scheduler/job/6819688#L3326 - https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-6.1.y/build/v6.1.57...
metadata: git_ref: linux-6.1.y git_repo: https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable-rc git_sha: d23900f974e0fb995b36ef47283a5aa74ca25f51 git_describe: v6.1.55 kernel_version: 6.1.55 kernel-config: https://storage.tuxsuite.com/public/linaro/lkft/builds/2W0YTi5jyhwfHW8YVU86F... artifact-location: https://storage.tuxsuite.com/public/linaro/lkft/builds/2W0YTi5jyhwfHW8YVU86F... toolchain: gcc-13
-- Linaro LKFT https://lkft.linaro.org