February 2025 - Linux-stable-mirror

[PATCH v2] sched/rt: Fix race in push_rt_task

by Harshit Agarwal

Overview ======== When a CPU chooses to call push_rt_task and picks a task to push to another CPU's runqueue then it will call find_lock_lowest_rq method which would take a double lock on both CPUs' runqueues. If one of the locks aren't readily available, it may lead to dropping the current runqueue lock and reacquiring both the locks at once. During this window it is possible that the task is already migrated and is running on some other CPU. These cases are already handled. However, if the task is migrated and has already been executed and another CPU is now trying to wake it up (ttwu) such that it is queued again on the runqeue (on_rq is 1) and also if the task was run by the same CPU, then the current checks will pass even though the task was migrated out and is no longer in the pushable tasks list. Crashes ======= This bug resulted in quite a few flavors of crashes triggering kernel panics with various crash signatures such as assert failures, page faults, null pointer dereferences, and queue corruption errors all coming from scheduler itself. Some of the crashes: -> kernel BUG at kernel/sched/rt.c:1616! BUG_ON(idx >= MAX_RT_PRIO) Call Trace: ? __die_body+0x1a/0x60 ? die+0x2a/0x50 ? do_trap+0x85/0x100 ? pick_next_task_rt+0x6e/0x1d0 ? do_error_trap+0x64/0xa0 ? pick_next_task_rt+0x6e/0x1d0 ? exc_invalid_op+0x4c/0x60 ? pick_next_task_rt+0x6e/0x1d0 ? asm_exc_invalid_op+0x12/0x20 ? pick_next_task_rt+0x6e/0x1d0 __schedule+0x5cb/0x790 ? update_ts_time_stats+0x55/0x70 schedule_idle+0x1e/0x40 do_idle+0x15e/0x200 cpu_startup_entry+0x19/0x20 start_secondary+0x117/0x160 secondary_startup_64_no_verify+0xb0/0xbb -> BUG: kernel NULL pointer dereference, address: 00000000000000c0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x183/0x350 ? __warn+0x8a/0xe0 ? exc_page_fault+0x3d6/0x520 ? asm_exc_page_fault+0x1e/0x30 ? pick_next_task_rt+0xb5/0x1d0 ? pick_next_task_rt+0x8c/0x1d0 __schedule+0x583/0x7e0 ? update_ts_time_stats+0x55/0x70 schedule_idle+0x1e/0x40 do_idle+0x15e/0x200 cpu_startup_entry+0x19/0x20 start_secondary+0x117/0x160 secondary_startup_64_no_verify+0xb0/0xbb -> BUG: unable to handle page fault for address: ffff9464daea5900 kernel BUG at kernel/sched/rt.c:1861! BUG_ON(rq->cpu != task_cpu(p)) -> kernel BUG at kernel/sched/rt.c:1055! BUG_ON(!rq->nr_running) Call Trace: ? __die_body+0x1a/0x60 ? die+0x2a/0x50 ? do_trap+0x85/0x100 ? dequeue_top_rt_rq+0xa2/0xb0 ? do_error_trap+0x64/0xa0 ? dequeue_top_rt_rq+0xa2/0xb0 ? exc_invalid_op+0x4c/0x60 ? dequeue_top_rt_rq+0xa2/0xb0 ? asm_exc_invalid_op+0x12/0x20 ? dequeue_top_rt_rq+0xa2/0xb0 dequeue_rt_entity+0x1f/0x70 dequeue_task_rt+0x2d/0x70 __schedule+0x1a8/0x7e0 ? blk_finish_plug+0x25/0x40 schedule+0x3c/0xb0 futex_wait_queue_me+0xb6/0x120 futex_wait+0xd9/0x240 do_futex+0x344/0xa90 ? get_mm_exe_file+0x30/0x60 ? audit_exe_compare+0x58/0x70 ? audit_filter_rules.constprop.26+0x65e/0x1220 __x64_sys_futex+0x148/0x1f0 do_syscall_64+0x30/0x80 entry_SYSCALL_64_after_hwframe+0x62/0xc7 -> BUG: unable to handle page fault for address: ffff8cf3608bc2c0 Call Trace: ? __die_body+0x1a/0x60 ? no_context+0x183/0x350 ? spurious_kernel_fault+0x171/0x1c0 ? exc_page_fault+0x3b6/0x520 ? plist_check_list+0x15/0x40 ? plist_check_list+0x2e/0x40 ? asm_exc_page_fault+0x1e/0x30 ? _cond_resched+0x15/0x30 ? futex_wait_queue_me+0xc8/0x120 ? futex_wait+0xd9/0x240 ? try_to_wake_up+0x1b8/0x490 ? futex_wake+0x78/0x160 ? do_futex+0xcd/0xa90 ? plist_check_list+0x15/0x40 ? plist_check_list+0x2e/0x40 ? plist_del+0x6a/0xd0 ? plist_check_list+0x15/0x40 ? plist_check_list+0x2e/0x40 ? dequeue_pushable_task+0x20/0x70 ? __schedule+0x382/0x7e0 ? asm_sysvec_reschedule_ipi+0xa/0x20 ? schedule+0x3c/0xb0 ? exit_to_user_mode_prepare+0x9e/0x150 ? irqentry_exit_to_user_mode+0x5/0x30 ? asm_sysvec_reschedule_ipi+0x12/0x20 Above are some of the common examples of the crashes that were observed due to this issue. Details ======= Let's look at the following scenario to understand this race. 1) CPU A enters push_rt_task a) CPU A has chosen next_task = task p. b) CPU A calls find_lock_lowest_rq(Task p, CPU Z’s rq). c) CPU A identifies CPU X as a destination CPU (X < Z). d) CPU A enters double_lock_balance(CPU Z’s rq, CPU X’s rq). e) Since X is lower than Z, CPU A unlocks CPU Z’s rq. Someone else has locked CPU X’s rq, and thus, CPU A must wait. 2) At CPU Z a) Previous task has completed execution and thus, CPU Z enters schedule, locks its own rq after CPU A releases it. b) CPU Z dequeues previous task and begins executing task p. c) CPU Z unlocks its rq. d) Task p yields the CPU (ex. by doing IO or waiting to acquire a lock) which triggers the schedule function on CPU Z. e) CPU Z enters schedule again, locks its own rq, and dequeues task p. f) As part of dequeue, it sets p.on_rq = 0 and unlocks its rq. 3) At CPU B a) CPU B enters try_to_wake_up with input task p. b) Since CPU Z dequeued task p, p.on_rq = 0, and CPU B updates B.state = WAKING. c) CPU B via select_task_rq determines CPU Y as the target CPU. 4) The race a) CPU A acquires CPU X’s lock and relocks CPU Z. b) CPU A reads task p.cpu = Z and incorrectly concludes task p is still on CPU Z. c) CPU A failed to notice task p had been dequeued from CPU Z while CPU A was waiting for locks in double_lock_balance. If CPU A knew that task p had been dequeued, it would return NULL forcing push_rt_task to give up the task p's migration. d) CPU B updates task p.cpu = Y and calls ttwu_queue. e) CPU B locks Ys rq. CPU B enqueues task p onto Y and sets task p.on_rq = 1. f) CPU B unlocks CPU Y, triggering memory synchronization. g) CPU A reads task p.on_rq = 1, cementing its assumption that task p has not migrated. h) CPU A decides to migrate p to CPU X. This leads to A dequeuing p from Y's queue and various crashes down the line. Solution ======== The solution here is fairly simple. After obtaining the lock (at 4a), the check is enhanced to make sure that the task is still at the head of the pushable tasks list. If not, then it is anyway not suitable for being pushed out. Testing ======= The fix is tested on a cluster of 3 nodes, where the panics due to this are hit every couple of days. A fix similar to this was deployed on such cluster and was stable for more than 30 days. Co-developed-by: Jon Kohler <jon(a)nutanix.com> Signed-off-by: Jon Kohler <jon(a)nutanix.com> Co-developed-by: Gauri Patwardhan <gauri.patwardhan(a)nutanix.com> Signed-off-by: Gauri Patwardhan <gauri.patwardhan(a)nutanix.com> Co-developed-by: Rahul Chunduru <rahul.chunduru(a)nutanix.com> Signed-off-by: Rahul Chunduru <rahul.chunduru(a)nutanix.com> Signed-off-by: Harshit Agarwal <harshit(a)nutanix.com> Tested-by: Will Ton <william.ton(a)nutanix.com> --- Changes in v2: - As per Steve's suggestion, removed some checks that are done after obtaining the lock that are no longer needed with the addition of new check. - Moved up is_migration_disabled check. - Link to v1: https://lore.kernel.org/lkml/20250211054646.23987-1-harshit@nutanix.com/ --- kernel/sched/rt.c | 54 +++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 4b8e33c615b1..4762dd3f50c5 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1885,6 +1885,27 @@ static int find_lowest_rq(struct task_struct *task) return -1; } +static struct task_struct *pick_next_pushable_task(struct rq *rq) +{ + struct task_struct *p; + + if (!has_pushable_tasks(rq)) + return NULL; + + p = plist_first_entry(&rq->rt.pushable_tasks, + struct task_struct, pushable_tasks); + + BUG_ON(rq->cpu != task_cpu(p)); + BUG_ON(task_current(rq, p)); + BUG_ON(task_current_donor(rq, p)); + BUG_ON(p->nr_cpus_allowed <= 1); + + BUG_ON(!task_on_rq_queued(p)); + BUG_ON(!rt_task(p)); + + return p; +} + /* Will lock the rq it finds */ static struct rq *find_lock_lowest_rq(struct task_struct *task, struct rq *rq) { @@ -1915,18 +1936,16 @@ static struct rq *find_lock_lowest_rq(struct task_struct *task, struct rq *rq) /* * We had to unlock the run queue. In * the mean time, task could have - * migrated already or had its affinity changed. - * Also make sure that it wasn't scheduled on its rq. + * migrated already or had its affinity changed, + * therefore check if the task is still at the + * head of the pushable tasks list. * It is possible the task was scheduled, set * "migrate_disabled" and then got preempted, so we must * check the task migration disable flag here too. */ - if (unlikely(task_rq(task) != rq || + if (unlikely(is_migration_disabled(task) || !cpumask_test_cpu(lowest_rq->cpu, &task->cpus_mask) || - task_on_cpu(rq, task) || - !rt_task(task) || - is_migration_disabled(task) || - !task_on_rq_queued(task))) { + task != pick_next_pushable_task(rq))) { double_unlock_balance(rq, lowest_rq); lowest_rq = NULL; @@ -1946,27 +1965,6 @@ static struct rq *find_lock_lowest_rq(struct task_struct *task, struct rq *rq) return lowest_rq; } -static struct task_struct *pick_next_pushable_task(struct rq *rq) -{ - struct task_struct *p; - - if (!has_pushable_tasks(rq)) - return NULL; - - p = plist_first_entry(&rq->rt.pushable_tasks, - struct task_struct, pushable_tasks); - - BUG_ON(rq->cpu != task_cpu(p)); - BUG_ON(task_current(rq, p)); - BUG_ON(task_current_donor(rq, p)); - BUG_ON(p->nr_cpus_allowed <= 1); - - BUG_ON(!task_on_rq_queued(p)); - BUG_ON(!rt_task(p)); - - return p; -} - /* * If the current CPU has more than one RT task, see if the non * running task can migrate over to a CPU that is running a task -- 2.22.3

6 months, 3 weeks

4
5
0 0

[PATCH v2] sched/fair: Fix integer underflow

by Pierre Gondois

(struct sg_lb_stats).idle_cpus is of type 'unsigned int'. (local->idle_cpus - busiest->idle_cpus) can underflow to UINT_MAX for instance, and max_t(long, 0, UINT_MAX) will output UINT_MAX. Use lsub_positive() instead of max_t(). Fixes: 16b0a7a1a0af ("sched/fair: Ensure tasks spreading in LLC during LB") cc: stable(a)vger.kernel.org Signed-off-by: Pierre Gondois <pierre.gondois(a)arm.com> Reviewed-by: Vincent Guittot <vincent.guittot(a)linaro.org> --- kernel/sched/fair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 9057584ec06d..6d9124499f52 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -10775,8 +10775,8 @@ static inline void calculate_imbalance(struct lb_env *env, struct sd_lb_stats *s * idle CPUs. */ env->migration_type = migrate_task; - env->imbalance = max_t(long, 0, - (local->idle_cpus - busiest->idle_cpus)); + env->imbalance = local->idle_cpus; + lsub_positive(&env->imbalance, busiest->idle_cpus); } #ifdef CONFIG_NUMA -- 2.25.1

6 months, 3 weeks

2
2
0 0

[PATCH v7 2/3] x86/tdx: Fix arch_safe_halt() execution for TDX VMs

by Vishal Annapurve

Direct HLT instruction execution causes #VEs for TDX VMs which is routed to hypervisor via TDCALL. If HLT is executed in STI-shadow, resulting #VE handler will enable interrupts before TDCALL is routed to hypervisor leading to missed wakeup events, as current TDX spec doesn't expose interruptibility state information to allow #VE handler to selectively enable interrupts. Commit bfe6ed0c6727 ("x86/tdx: Add HLT support for TDX guests") prevented the idle routines from executing HLT instruction in STI-shadow. But it missed the paravirt routine which can be reached via this path as an example: kvm_wait() => safe_halt() => raw_safe_halt() => arch_safe_halt() => irq.safe_halt() => pv_native_safe_halt() To reliably handle arch_safe_halt() for TDX VMs, introduce explicit dependency on CONFIG_PARAVIRT and override paravirt halt()/safe_halt() routines with TDX-safe versions that execute direct TDCALL and needed interrupt flag updates. Executing direct TDCALL brings in additional benefit of avoiding HLT related #VEs altogether. Cc: stable(a)vger.kernel.org Fixes: bfe6ed0c6727 ("x86/tdx: Add HLT support for TDX guests") Reviewed-by: Kirill A. Shutemov <kirill.shutemov(a)linux.intel.com> Signed-off-by: Vishal Annapurve <vannapurve(a)google.com> --- arch/x86/Kconfig | 1 + arch/x86/coco/tdx/tdx.c | 26 +++++++++++++++++++++++++- arch/x86/include/asm/tdx.h | 4 ++-- arch/x86/kernel/process.c | 2 +- 4 files changed, 29 insertions(+), 4 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index be2c311f5118..933c046e8966 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -902,6 +902,7 @@ config INTEL_TDX_GUEST depends on X86_64 && CPU_SUP_INTEL depends on X86_X2APIC depends on EFI_STUB + depends on PARAVIRT select ARCH_HAS_CC_PLATFORM select X86_MEM_ENCRYPT select X86_MCE diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index 32809a06dab4..6aad910d119d 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -14,6 +14,7 @@ #include <asm/ia32.h> #include <asm/insn.h> #include <asm/insn-eval.h> +#include <asm/paravirt_types.h> #include <asm/pgtable.h> #include <asm/set_memory.h> #include <asm/traps.h> @@ -398,7 +399,7 @@ static int handle_halt(struct ve_info *ve) return ve_instr_len(ve); } -void __cpuidle tdx_safe_halt(void) +void __cpuidle tdx_halt(void) { const bool irq_disabled = false; @@ -409,6 +410,16 @@ void __cpuidle tdx_safe_halt(void) WARN_ONCE(1, "HLT instruction emulation failed\n"); } +static void __cpuidle tdx_safe_halt(void) +{ + tdx_halt(); + /* + * "__cpuidle" section doesn't support instrumentation, so stick + * with raw_* variant that avoids tracing hooks. + */ + raw_local_irq_enable(); +} + static int read_msr(struct pt_regs *regs, struct ve_info *ve) { struct tdx_module_args args = { @@ -1109,6 +1120,19 @@ void __init tdx_early_init(void) x86_platform.guest.enc_kexec_begin = tdx_kexec_begin; x86_platform.guest.enc_kexec_finish = tdx_kexec_finish; + /* + * Avoid "sti;hlt" execution in TDX guests as HLT induces a #VE that + * will enable interrupts before HLT TDCALL invocation if executed + * in STI-shadow, possibly resulting in missed wakeup events. + * + * Modify all possible HLT execution paths to use TDX specific routines + * that directly execute TDCALL and toggle the interrupt state as + * needed after TDCALL completion. This also reduces HLT related #VEs + * in addition to having a reliable halt logic execution. + */ + pv_ops.irq.safe_halt = tdx_safe_halt; + pv_ops.irq.halt = tdx_halt; + /* * TDX intercepts the RDMSR to read the X2APIC ID in the parallel * bringup low level code. That raises #VE which cannot be handled diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index b4b16dafd55e..40f9a97371a9 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -58,7 +58,7 @@ void tdx_get_ve_info(struct ve_info *ve); bool tdx_handle_virt_exception(struct pt_regs *regs, struct ve_info *ve); -void tdx_safe_halt(void); +void tdx_halt(void); bool tdx_early_handle_ve(struct pt_regs *regs); @@ -72,7 +72,7 @@ void __init tdx_dump_td_ctls(u64 td_ctls); #else static inline void tdx_early_init(void) { }; -static inline void tdx_safe_halt(void) { }; +static inline void tdx_halt(void) { }; static inline bool tdx_early_handle_ve(struct pt_regs *regs) { return false; } diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 6da6769d7254..d11956a178df 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -934,7 +934,7 @@ void __init select_idle_routine(void) static_call_update(x86_idle, mwait_idle); } else if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST)) { pr_info("using TDX aware idle routine\n"); - static_call_update(x86_idle, tdx_safe_halt); + static_call_update(x86_idle, tdx_halt); } else { static_call_update(x86_idle, default_idle); } -- 2.48.1.711.g2feabab25a-goog

6 months, 3 weeks

2
1
0 0

[PATCH 6.1.y 6.6.y 0/3] mm/filemap: fix page cache corruption with large folios

by Kairui Song

From: Kairui Song <kasong(a)tencent.com> This series fixes the page cache corruption issue reported by Christian Theune [1]. The issue was reported affects kernels back to 5.19. Current maintained effected branches includes 6.1 and 6.6 and the fix was included in 6.10 already. This series can be applied for both 6.1 and 6.6. Patch 3/3 is the fixing patch. It was initially submitted and merge as an optimization but found to have fixed the corruption by handling race correctly. Patch 1/3 and 2/3 is required for 3/3. Patch 3/3 included some unit test code, making the LOC of the backport a bit higher, but should be OK to be kept, since they are just test code. Note there seems still some unresolved problem in Link [1] but that should be a different issue, and the commits being backported have been well tested, they fix the corruption issue just fine. Link: https://lore.kernel.org/linux-mm/A5A976CB-DB57-4513-A700-656580488AB6@flyin… [1] Kairui Song (3): mm/filemap: return early if failed to allocate memory for split lib/xarray: introduce a new helper xas_get_order mm/filemap: optimize filemap folio adding include/linux/xarray.h | 6 +++ lib/test_xarray.c | 93 ++++++++++++++++++++++++++++++++++++++++++ lib/xarray.c | 49 ++++++++++++++-------- mm/filemap.c | 50 ++++++++++++++++++----- 4 files changed, 169 insertions(+), 29 deletions(-) -- 2.46.1

6 months, 3 weeks

6
14
0 0

[PATCH 5.15] net: defer final 'struct net' free in netns dismantle

by Vasiliy Kovalev

From: Eric Dumazet <edumazet(a)google.com> commit 0f6ede9fbc747e2553612271bce108f7517e7a45 upstream. Ilya reported a slab-use-after-free in dst_destroy [1] Issue is in xfrm6_net_init() and xfrm4_net_init() : They copy xfrm[46]_dst_ops_template into net->xfrm.xfrm[46]_dst_ops. But net structure might be freed before all the dst callbacks are called. So when dst_destroy() calls later : if (dst->ops->destroy) dst->ops->destroy(dst); dst->ops points to the old net->xfrm.xfrm[46]_dst_ops, which has been freed. See a relevant issue fixed in : ac888d58869b ("net: do not delay dst_entries_add() in dst_release()") A fix is to queue the 'struct net' to be freed after one another cleanup_net() round (and existing rcu_barrier()) [1] BUG: KASAN: slab-use-after-free in dst_destroy (net/core/dst.c:112) Read of size 8 at addr ffff8882137ccab0 by task swapper/37/0 Dec 03 05:46:18 kernel: CPU: 37 UID: 0 PID: 0 Comm: swapper/37 Kdump: loaded Not tainted 6.12.0 #67 Hardware name: Red Hat KVM/RHEL, BIOS 1.16.1-1.el9 04/01/2014 Call Trace: <IRQ> dump_stack_lvl (lib/dump_stack.c:124) print_address_description.constprop.0 (mm/kasan/report.c:378) ? dst_destroy (net/core/dst.c:112) print_report (mm/kasan/report.c:489) ? dst_destroy (net/core/dst.c:112) ? kasan_addr_to_slab (mm/kasan/common.c:37) kasan_report (mm/kasan/report.c:603) ? dst_destroy (net/core/dst.c:112) ? rcu_do_batch (kernel/rcu/tree.c:2567) dst_destroy (net/core/dst.c:112) rcu_do_batch (kernel/rcu/tree.c:2567) ? __pfx_rcu_do_batch (kernel/rcu/tree.c:2491) ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4339 kernel/locking/lockdep.c:4406) rcu_core (kernel/rcu/tree.c:2825) handle_softirqs (kernel/softirq.c:554) __irq_exit_rcu (kernel/softirq.c:589 kernel/softirq.c:428 kernel/softirq.c:637) irq_exit_rcu (kernel/softirq.c:651) sysvec_apic_timer_interrupt (arch/x86/kernel/apic/apic.c:1049 arch/x86/kernel/apic/apic.c:1049) </IRQ> <TASK> asm_sysvec_apic_timer_interrupt (./arch/x86/include/asm/idtentry.h:702) RIP: 0010:default_idle (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:92 arch/x86/kernel/process.c:743) Code: 00 4d 29 c8 4c 01 c7 4c 29 c2 e9 6e ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 90 0f 00 2d c7 c9 27 00 fb f4 <fa> c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 90 RSP: 0018:ffff888100d2fe00 EFLAGS: 00000246 RAX: 00000000001870ed RBX: 1ffff110201a5fc2 RCX: ffffffffb61a3e46 RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffb3d4d123 RBP: 0000000000000000 R08: 0000000000000001 R09: ffffed11c7e1835d R10: ffff888e3f0c1aeb R11: 0000000000000000 R12: 0000000000000000 R13: ffff888100d20000 R14: dffffc0000000000 R15: 0000000000000000 ? ct_kernel_exit.constprop.0 (kernel/context_tracking.c:148) ? cpuidle_idle_call (kernel/sched/idle.c:186) default_idle_call (./include/linux/cpuidle.h:143 kernel/sched/idle.c:118) cpuidle_idle_call (kernel/sched/idle.c:186) ? __pfx_cpuidle_idle_call (kernel/sched/idle.c:168) ? lock_release (kernel/locking/lockdep.c:467 kernel/locking/lockdep.c:5848) ? lockdep_hardirqs_on_prepare (kernel/locking/lockdep.c:4347 kernel/locking/lockdep.c:4406) ? tsc_verify_tsc_adjust (arch/x86/kernel/tsc_sync.c:59) do_idle (kernel/sched/idle.c:326) cpu_startup_entry (kernel/sched/idle.c:423 (discriminator 1)) start_secondary (arch/x86/kernel/smpboot.c:202 arch/x86/kernel/smpboot.c:282) ? __pfx_start_secondary (arch/x86/kernel/smpboot.c:232) ? soft_restart_cpu (arch/x86/kernel/head_64.S:452) common_startup_64 (arch/x86/kernel/head_64.S:414) </TASK> Dec 03 05:46:18 kernel: Allocated by task 12184: kasan_save_stack (mm/kasan/common.c:48) kasan_save_track (./arch/x86/include/asm/current.h:49 mm/kasan/common.c:60 mm/kasan/common.c:69) __kasan_slab_alloc (mm/kasan/common.c:319 mm/kasan/common.c:345) kmem_cache_alloc_noprof (mm/slub.c:4085 mm/slub.c:4134 mm/slub.c:4141) copy_net_ns (net/core/net_namespace.c:421 net/core/net_namespace.c:480) create_new_namespaces (kernel/nsproxy.c:110) unshare_nsproxy_namespaces (kernel/nsproxy.c:228 (discriminator 4)) ksys_unshare (kernel/fork.c:3313) __x64_sys_unshare (kernel/fork.c:3382) do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Dec 03 05:46:18 kernel: Freed by task 11: kasan_save_stack (mm/kasan/common.c:48) kasan_save_track (./arch/x86/include/asm/current.h:49 mm/kasan/common.c:60 mm/kasan/common.c:69) kasan_save_free_info (mm/kasan/generic.c:582) __kasan_slab_free (mm/kasan/common.c:271) kmem_cache_free (mm/slub.c:4579 mm/slub.c:4681) cleanup_net (net/core/net_namespace.c:456 net/core/net_namespace.c:446 net/core/net_namespace.c:647) process_one_work (kernel/workqueue.c:3229) worker_thread (kernel/workqueue.c:3304 kernel/workqueue.c:3391) kthread (kernel/kthread.c:389) ret_from_fork (arch/x86/kernel/process.c:147) ret_from_fork_asm (arch/x86/entry/entry_64.S:257) Dec 03 05:46:18 kernel: Last potentially related work creation: kasan_save_stack (mm/kasan/common.c:48) __kasan_record_aux_stack (mm/kasan/generic.c:541) insert_work (./include/linux/instrumented.h:68 ./include/asm-generic/bitops/instrumented-non-atomic.h:141 kernel/workqueue.c:788 kernel/workqueue.c:795 kernel/workqueue.c:2186) __queue_work (kernel/workqueue.c:2340) queue_work_on (kernel/workqueue.c:2391) xfrm_policy_insert (net/xfrm/xfrm_policy.c:1610) xfrm_add_policy (net/xfrm/xfrm_user.c:2116) xfrm_user_rcv_msg (net/xfrm/xfrm_user.c:3321) netlink_rcv_skb (net/netlink/af_netlink.c:2536) xfrm_netlink_rcv (net/xfrm/xfrm_user.c:3344) netlink_unicast (net/netlink/af_netlink.c:1316 net/netlink/af_netlink.c:1342) netlink_sendmsg (net/netlink/af_netlink.c:1886) sock_write_iter (net/socket.c:729 net/socket.c:744 net/socket.c:1165) vfs_write (fs/read_write.c:590 fs/read_write.c:683) ksys_write (fs/read_write.c:736) do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Dec 03 05:46:18 kernel: Second to last potentially related work creation: kasan_save_stack (mm/kasan/common.c:48) __kasan_record_aux_stack (mm/kasan/generic.c:541) insert_work (./include/linux/instrumented.h:68 ./include/asm-generic/bitops/instrumented-non-atomic.h:141 kernel/workqueue.c:788 kernel/workqueue.c:795 kernel/workqueue.c:2186) __queue_work (kernel/workqueue.c:2340) queue_work_on (kernel/workqueue.c:2391) __xfrm_state_insert (./include/linux/workqueue.h:723 net/xfrm/xfrm_state.c:1150 net/xfrm/xfrm_state.c:1145 net/xfrm/xfrm_state.c:1513) xfrm_state_update (./include/linux/spinlock.h:396 net/xfrm/xfrm_state.c:1940) xfrm_add_sa (net/xfrm/xfrm_user.c:912) xfrm_user_rcv_msg (net/xfrm/xfrm_user.c:3321) netlink_rcv_skb (net/netlink/af_netlink.c:2536) xfrm_netlink_rcv (net/xfrm/xfrm_user.c:3344) netlink_unicast (net/netlink/af_netlink.c:1316 net/netlink/af_netlink.c:1342) netlink_sendmsg (net/netlink/af_netlink.c:1886) sock_write_iter (net/socket.c:729 net/socket.c:744 net/socket.c:1165) vfs_write (fs/read_write.c:590 fs/read_write.c:683) ksys_write (fs/read_write.c:736) do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130) Fixes: a8a572a6b5f2 ("xfrm: dst_entries_init() per-net dst_ops") Reported-by: Ilya Maximets <i.maximets(a)ovn.org> Closes: https://lore.kernel.org/netdev/CANn89iKKYDVpB=MtmfH7nyv2p=rJWSLedO5k7wSZgtY… Signed-off-by: Eric Dumazet <edumazet(a)google.com> Acked-by: Paolo Abeni <pabeni(a)redhat.com> Reviewed-by: Kuniyuki Iwashima <kuniyu(a)amazon.com> Link: https://patch.msgid.link/20241204125455.3871859-1-edumazet@google.com Signed-off-by: Jakub Kicinski <kuba(a)kernel.org> Signed-off-by: Vasiliy Kovalev <kovalev(a)altlinux.org> --- Backport to fix CVE-2024-56658 Link: https://www.cve.org/CVERecord/?id=CVE-2024-56658 --- include/net/net_namespace.h | 1 + net/core/net_namespace.c | 21 ++++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h index c47baa623ba586..a5d6e04c8e8b55 100644 --- a/include/net/net_namespace.h +++ b/include/net/net_namespace.h @@ -80,6 +80,7 @@ struct net { * or to unregister pernet ops * (pernet_ops_rwsem write locked). */ + struct llist_node defer_free_list; struct llist_node cleanup_list; /* namespaces on death row */ #ifdef CONFIG_KEYS diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c index 3addbce20f8ed0..0217dd2635cdb4 100644 --- a/net/core/net_namespace.c +++ b/net/core/net_namespace.c @@ -430,11 +430,28 @@ static struct net *net_alloc(void) goto out; } +static LLIST_HEAD(defer_free_list); + +static void net_complete_free(void) +{ + struct llist_node *kill_list; + struct net *net, *next; + + /* Get the list of namespaces to free from last round. */ + kill_list = llist_del_all(&defer_free_list); + + llist_for_each_entry_safe(net, next, kill_list, defer_free_list) + kmem_cache_free(net_cachep, net); + +} + static void net_free(struct net *net) { if (refcount_dec_and_test(&net->passive)) { kfree(rcu_access_pointer(net->gen)); - kmem_cache_free(net_cachep, net); + + /* Wait for an extra rcu_barrier() before final free. */ + llist_add(&net->defer_free_list, &defer_free_list); } } @@ -609,6 +626,8 @@ static void cleanup_net(struct work_struct *work) */ rcu_barrier(); + net_complete_free(); + /* Finally it is safe to free my network namespace structure */ list_for_each_entry_safe(net, tmp, &net_exit_list, exit_list) { list_del_init(&net->exit_list); -- 2.33.8

6 months, 3 weeks

4
3
0 0

[PATCH v2] mm: zswap: fix crypto_free_acomp() deadlock in zswap_cpu_comp_dead()

by Yosry Ahmed

Currently, zswap_cpu_comp_dead() calls crypto_free_acomp() while holding the per-CPU acomp_ctx mutex. crypto_free_acomp() then holds scomp_lock (through crypto_exit_scomp_ops_async()). On the other hand, crypto_alloc_acomp_node() holds the scomp_lock (through crypto_scomp_init_tfm()), and then allocates memory. If the allocation results in reclaim, we may attempt to hold the per-CPU acomp_ctx mutex. The above dependencies can cause an ABBA deadlock. For example in the following scenario: (1) Task A running on CPU #1: crypto_alloc_acomp_node() Holds scomp_lock Enters reclaim Reads per_cpu_ptr(pool->acomp_ctx, 1) (2) Task A is descheduled (3) CPU #1 goes offline zswap_cpu_comp_dead(CPU #1) Holds per_cpu_ptr(pool->acomp_ctx, 1)) Calls crypto_free_acomp() Waits for scomp_lock (4) Task A running on CPU #2: Waits for per_cpu_ptr(pool->acomp_ctx, 1) // Read on CPU #1 DEADLOCK Since there is no requirement to call crypto_free_acomp() with the per-CPU acomp_ctx mutex held in zswap_cpu_comp_dead(), move it after the mutex is unlocked. Also move the acomp_request_free() and kfree() calls for consistency and to avoid any potential sublte locking dependencies in the future. With this, only setting acomp_ctx fields to NULL occurs with the mutex held. This is similar to how zswap_cpu_comp_prepare() only initializes acomp_ctx fields with the mutex held, after performing all allocations before holding the mutex. Opportunistically, move the NULL check on acomp_ctx so that it takes place before the mutex dereference. Fixes: 12dcb0ef5406 ("mm: zswap: properly synchronize freeing resources during CPU hotunplug") Reported-by: syzbot+1a517ccfcbc6a7ab0f82(a)syzkaller.appspotmail.com Closes: https://lore.kernel.org/all/67bcea51.050a0220.bbfd1.0096.GAE@google.com/ Cc: <stable(a)vger.kernel.org> Co-developed-by: Herbert Xu <herbert(a)gondor.apana.org.au> Signed-off-by: Herbert Xu <herbert(a)gondor.apana.org.au> Signed-off-by: Yosry Ahmed <yosry.ahmed(a)linux.dev> Acked-by: Herbert Xu <herbert(a)gondor.apana.org.au> --- v1 -> v2: - Explained the problem more clearly in the commit message. - Moved all freeing calls outside the lock critical section. v1: https://lore.kernel.org/all/Z72FJnbA39zWh4zS@gondor.apana.org.au/ --- mm/zswap.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index ac9d299e7d0c1..adf745c66aa1d 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -881,18 +881,32 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) { struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); + struct acomp_req *req; + struct crypto_acomp *acomp; + u8 *buffer; + + if (IS_ERR_OR_NULL(acomp_ctx)) + return 0; mutex_lock(&acomp_ctx->mutex); - if (!IS_ERR_OR_NULL(acomp_ctx)) { - if (!IS_ERR_OR_NULL(acomp_ctx->req)) - acomp_request_free(acomp_ctx->req); - acomp_ctx->req = NULL; - if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) - crypto_free_acomp(acomp_ctx->acomp); - kfree(acomp_ctx->buffer); - } + req = acomp_ctx->req; + acomp = acomp_ctx->acomp; + buffer = acomp_ctx->buffer; + acomp_ctx->req = NULL; + acomp_ctx->acomp = NULL; + acomp_ctx->buffer = NULL; mutex_unlock(&acomp_ctx->mutex); + /* + * Do the actual freeing after releasing the mutex to avoid subtle + * locking dependencies causing deadlocks. + */ + if (!IS_ERR_OR_NULL(req)) + acomp_request_free(req); + if (!IS_ERR_OR_NULL(acomp)) + crypto_free_acomp(acomp); + kfree(buffer); + return 0; } -- 2.48.1.658.g4767266eb4-goog

6 months, 4 weeks

4
10
0 0

[PATCH 1/3] drm/i915: Fix scanline_offset for LNL+ and BMG+

by Ville Syrjala

From: Ville Syrjälä <ville.syrjala(a)linux.intel.com> Turns out LNL+ and BMG+ no longer have the weird extra scanline offset for HDMI outputs. Fix intel_crtc_scanline_offset() accordingly so that scanline evasion/etc. works correctly on HDMI outputs on these new platforms. Cc: stable(a)vger.kernel.org Signed-off-by: Ville Syrjälä <ville.syrjala(a)linux.intel.com> --- drivers/gpu/drm/i915/display/intel_vblank.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/display/intel_vblank.c b/drivers/gpu/drm/i915/display/intel_vblank.c index 4efd4f7d497a..7b240ce681a0 100644 --- a/drivers/gpu/drm/i915/display/intel_vblank.c +++ b/drivers/gpu/drm/i915/display/intel_vblank.c @@ -222,7 +222,9 @@ int intel_crtc_scanline_offset(const struct intel_crtc_state *crtc_state) * However if queried just before the start of vblank we'll get an * answer that's slightly in the future. */ - if (DISPLAY_VER(display) == 2) + if (DISPLAY_VER(display) >= 20 || display->platform.battlemage) + return 1; + else if (DISPLAY_VER(display) == 2) return -1; else if (HAS_DDI(display) && intel_crtc_has_type(crtc_state, INTEL_OUTPUT_HDMI)) return 2; -- 2.45.3

7 months

3
2
0 0

[PATCH v2] Revert "mmc: sdhci_am654: Add sdhci_am654_start_signal_voltage_switch"

by Josua Mayer

This reverts commit 941a7abd4666912b84ab209396fdb54b0dae685d. This commit uses presence of device-tree properties vmmc-supply and vqmmc-supply for deciding whether to enable a quirk affecting timing of clock and data. The intention was to address issues observed with eMMC and SD on AM62 platforms. This new quirk is however also enabled for AM64 breaking microSD access on the SolidRun HimmingBoard-T which is supported in-tree since v6.11, causing a regression. During boot microSD initialization now fails with the error below: [ 2.008520] mmc1: SDHCI controller on fa00000.mmc [fa00000.mmc] using ADMA 64-bit [ 2.115348] mmc1: error -110 whilst initialising SD card The heuristics for enabling the quirk are clearly not correct as they break at least one but potentially many existing boards. Revert the change and restore original behaviour until a more appropriate method of selecting the quirk is derived. Fixes: 941a7abd4666 ("mmc: sdhci_am654: Add sdhci_am654_start_signal_voltage_switch") Closes: https://lore.kernel.org/linux-mmc/a70fc9fc-186f-4165-a652-3de50733763a@soli… Cc: stable(a)vger.kernel.org Signed-off-by: Josua Mayer <josua(a)solid-run.com> Acked-by: Adrian Hunter <adrian.hunter(a)intel.com> --- Changes in v2: - Fixed "Fixes:" tag invalid commit description copied from history (Reported-by: Adrian Hunter <adrian.hunter(a)intel.com>) (Reported-by: Greg KH <gregkh(a)linuxfoundation.org>) - Link to v1: https://lore.kernel.org/r/20250127-am654-mmc-regression-v1-1-d831f9a13ae9@s… --- drivers/mmc/host/sdhci_am654.c | 30 ------------------------------ 1 file changed, 30 deletions(-) diff --git a/drivers/mmc/host/sdhci_am654.c b/drivers/mmc/host/sdhci_am654.c index b73f673db92bbc042392995e715815e15ace6005..f75c31815ab00d17b5757063521f56ba5643babe 100644 --- a/drivers/mmc/host/sdhci_am654.c +++ b/drivers/mmc/host/sdhci_am654.c @@ -155,7 +155,6 @@ struct sdhci_am654_data { u32 tuning_loop; #define SDHCI_AM654_QUIRK_FORCE_CDTEST BIT(0) -#define SDHCI_AM654_QUIRK_SUPPRESS_V1P8_ENA BIT(1) }; struct window { @@ -357,29 +356,6 @@ static void sdhci_j721e_4bit_set_clock(struct sdhci_host *host, sdhci_set_clock(host, clock); } -static int sdhci_am654_start_signal_voltage_switch(struct mmc_host *mmc, struct mmc_ios *ios) -{ - struct sdhci_host *host = mmc_priv(mmc); - struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host); - struct sdhci_am654_data *sdhci_am654 = sdhci_pltfm_priv(pltfm_host); - int ret; - - if ((sdhci_am654->quirks & SDHCI_AM654_QUIRK_SUPPRESS_V1P8_ENA) && - ios->signal_voltage == MMC_SIGNAL_VOLTAGE_180) { - if (!IS_ERR(mmc->supply.vqmmc)) { - ret = mmc_regulator_set_vqmmc(mmc, ios); - if (ret < 0) { - pr_err("%s: Switching to 1.8V signalling voltage failed,\n", - mmc_hostname(mmc)); - return -EIO; - } - } - return 0; - } - - return sdhci_start_signal_voltage_switch(mmc, ios); -} - static u8 sdhci_am654_write_power_on(struct sdhci_host *host, u8 val, int reg) { writeb(val, host->ioaddr + reg); @@ -868,11 +844,6 @@ static int sdhci_am654_get_of_property(struct platform_device *pdev, if (device_property_read_bool(dev, "ti,fails-without-test-cd")) sdhci_am654->quirks |= SDHCI_AM654_QUIRK_FORCE_CDTEST; - /* Suppress v1p8 ena for eMMC and SD with vqmmc supply */ - if (!!of_parse_phandle(dev->of_node, "vmmc-supply", 0) == - !!of_parse_phandle(dev->of_node, "vqmmc-supply", 0)) - sdhci_am654->quirks |= SDHCI_AM654_QUIRK_SUPPRESS_V1P8_ENA; - sdhci_get_of_property(pdev); return 0; @@ -969,7 +940,6 @@ static int sdhci_am654_probe(struct platform_device *pdev) goto err_pltfm_free; } - host->mmc_host_ops.start_signal_voltage_switch = sdhci_am654_start_signal_voltage_switch; host->mmc_host_ops.execute_tuning = sdhci_am654_execute_tuning; pm_runtime_get_noresume(dev); --- base-commit: ffd294d346d185b70e28b1a28abe367bbfe53c04 change-id: 20250127-am654-mmc-regression-ed289f8967c2 Best regards, -- Josua Mayer <josua(a)solid-run.com>

7 months

4
7
0 0

[PATCH RESEND] fs/netfs/read_collect: add to next->prev_donated

by Max Kellermann

If multiple subrequests donate data to the same "next" request (depending on the subrequest completion order), each of them would overwrite the `prev_donated` field, causing data corruption and a BUG() crash ("Can't donate prior to front"). Fixes: ee4cdf7ba857 ("netfs: Speed up buffered reading") Closes: https://lore.kernel.org/netfs/CAKPOu+_4mUwYgQtRTbXCmi+-k3PGvLysnPadkmHOyB7G… Cc: stable(a)vger.kernel.org Signed-off-by: Max Kellermann <max.kellermann(a)ionos.com> Signed-off-by: David Howells <dhowells(a)redhat.com> --- fs/netfs/read_collect.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c index 8878b46589ff..cafadfe8e858 100644 --- a/fs/netfs/read_collect.c +++ b/fs/netfs/read_collect.c @@ -284,7 +284,7 @@ static bool netfs_consume_read_data(struct netfs_io_subrequest *subreq, bool was netfs_trace_donate_to_deferred_next); } else { next = list_next_entry(subreq, rreq_link); - WRITE_ONCE(next->prev_donated, excess); + WRITE_ONCE(next->prev_donated, next->prev_donated + excess); trace_netfs_donate(rreq, subreq, next, excess, netfs_trace_donate_to_next); } -- 2.47.2

7 months

4
3
0 0

[BUG REPORT] cifs: Deadlock due to network reconnection during file writing

by Wang Zhaolong

In the code of the LTS branch that is being maintained (from linux-5.4 to linux-6.6), a deadlock occurs in the network reconnection scenario When multiple processes or threads write to the same file concurrently. Take the code of linux-5.10 as an example. The simplified deadlock process is as follows: ``` Process 1 Process 2 lock_page() - [1] wait_on_page_writeback() - [2] Waiting for writeback, blocked by [4] lock_page() - [3] Blocked by [1] end_page_writeback() - [4] Won't execute ``` Based on my research, I'm going to use two detailed scenarios to illustrate the issue. Scenarios 1: ``` P1 (dd) P2 (cifsd) P3 (cifsiod) cifs_writepages wdata_prepare_pages lock_page - [1] wait_on_page_writeback - [2] Waiting for writeback, blocked by [4] wait_on_page_bit cifs_demultiplex_thread cifs_read_from_socket cifs_readv_from_socket - If another process triggers reconnect at this point cifs_reconnect - mid->mid_state updated to MID_RETRY_NEEDED smb2_writev_callback mid_entry->callback() - mid_state leads to wdata->result = -EAGAIN wdata->result = -EAGAIN queue_work(cifsiod_wq, &wdata->work); cifs_writev_complete - worker function - wdata->result == -EAGAIN Condition satisfied cifs_writev_requeue lock_page - [3] Blocked by [1] end_page_writeback - [4] Won't execute unlock_page ``` Mainline refactoring commit d08089f649a0 ("cifs: Change the I/O paths to use an iterator rather than a page list") unlock folio while waiting for the writeback to complete. This patch is introduced in v6.3-rc1. Therefore, scenario 1 only affects LTS versions from linux-5.4 to linux-6.1. Call stack trace: ``` cat /proc/34/stack [<0>] __lock_page+0x147/0x3a0 [<0>] cifs_writev_requeue.cold+0x185/0x28e [<0>] process_one_work+0x1df/0x3b0 [<0>] worker_thread+0x4a/0x3c0 [<0>] kthread+0x125/0x160 [<0>] ret_from_fork+0x22/0x30 # cat /proc/465/stack [<0>] wait_on_page_bit+0x106/0x2e0 [<0>] wait_on_page_writeback+0x25/0xd0 [<0>] cifs_writepages+0x5ee/0xf60 [<0>] do_writepages+0x43/0xe0 [<0>] __filemap_fdatawrite_range+0xcd/0x110 [<0>] file_write_and_wait_range+0x40/0x90 [<0>] cifs_strict_fsync+0x35/0x470 [<0>] do_fsync+0x38/0x70 [<0>] __x64_sys_fsync+0x10/0x20 [<0>] do_syscall_64+0x33/0x40 [<0>] entry_SYSCALL_64_after_hwframe+0x67/0xd1 [ 369.826215] INFO: task kworker/1:1:34 blocked for more than 122 seconds. [ 369.828964] Not tainted 5.10.0+ #164 [ 369.830623] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.835104] task:kworker/1:1 state:D stack:13472 pid: 34 ppid: 2 flags:0x00004000 [ 369.838448] Workqueue: cifsiod cifs_writev_complete [ 369.840242] Call Trace: [ 369.841219] __schedule+0x401/0x8e0 [ 369.842568] schedule+0x49/0x130 [ 369.843785] io_schedule+0x12/0x40 [ 369.845079] __lock_page+0x147/0x3a0 [ 369.846444] ? add_to_page_cache_lru+0x180/0x180 [ 369.847963] cifs_writev_requeue.cold+0x185/0x28e [ 369.849193] process_one_work+0x1df/0x3b0 [ 369.850248] worker_thread+0x4a/0x3c0 [ 369.851216] ? process_one_work+0x3b0/0x3b0 [ 369.852308] kthread+0x125/0x160 [ 369.853167] ? kthread_park+0x90/0x90 [ 369.854142] ret_from_fork+0x22/0x30 [ 369.855054] INFO: task kworker/u8:3:96 blocked for more than 122 seconds. [ 369.856781] Not tainted 5.10.0+ #164 [ 369.857851] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.859419] task:kworker/u8:3 state:D stack:12744 pid: 96 ppid: 2 flags:0x00004000 [ 369.861041] Workqueue: writeback wb_workfn (flush-cifs-2) [ 369.862095] Call Trace: [ 369.862583] __schedule+0x401/0x8e0 [ 369.863280] schedule+0x49/0x130 [ 369.863912] io_schedule+0x12/0x40 [ 369.864604] __lock_page+0x147/0x3a0 [ 369.865322] ? add_to_page_cache_lru+0x180/0x180 [ 369.866246] cifs_writepages+0x620/0xf60 [ 369.867005] do_writepages+0x43/0xe0 [ 369.867737] ? __blk_mq_try_issue_directly+0x121/0x1c0 [ 369.868750] __writeback_single_inode+0x3d/0x320 [ 369.869589] writeback_sb_inodes+0x20d/0x480 [ 369.870367] __writeback_inodes_wb+0x4c/0xe0 [ 369.871148] wb_writeback+0x201/0x2f0 [ 369.871797] wb_workfn+0x38a/0x4e0 [ 369.872427] ? check_preempt_curr+0x47/0x70 [ 369.873191] ? ttwu_do_wakeup.isra.0+0x17/0x170 [ 369.873999] process_one_work+0x1df/0x3b0 [ 369.874741] worker_thread+0x4a/0x3c0 [ 369.875421] ? process_one_work+0x3b0/0x3b0 [ 369.876180] kthread+0x125/0x160 [ 369.876761] ? kthread_park+0x90/0x90 [ 369.877431] ret_from_fork+0x22/0x30 [ 369.878106] INFO: task a.out:465 blocked for more than 122 seconds. [ 369.879225] Not tainted 5.10.0+ #164 [ 369.879945] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.881316] task:a.out state:D stack:12752 pid: 465 ppid: 386 flags:0x00000002 [ 369.882791] Call Trace: [ 369.883263] __schedule+0x401/0x8e0 [ 369.883884] schedule+0x49/0x130 [ 369.884447] io_schedule+0x12/0x40 [ 369.885054] wait_on_page_bit+0x106/0x2e0 [ 369.885795] ? add_to_page_cache_lru+0x180/0x180 [ 369.886631] wait_on_page_writeback+0x25/0xd0 [ 369.887427] cifs_writepages+0x5ee/0xf60 [ 369.888151] do_writepages+0x43/0xe0 [ 369.888789] ? __generic_file_write_iter+0xfd/0x1d0 [ 369.889663] __filemap_fdatawrite_range+0xcd/0x110 [ 369.890523] file_write_and_wait_range+0x40/0x90 [ 369.891360] cifs_strict_fsync+0x35/0x470 [ 369.892094] do_fsync+0x38/0x70 [ 369.892657] __x64_sys_fsync+0x10/0x20 [ 369.893336] do_syscall_64+0x33/0x40 [ 369.893978] entry_SYSCALL_64_after_hwframe+0x67/0xd1 [ 369.894883] RIP: 0033:0x7f660e208950 [ 369.895538] RSP: 002b:00007fff52b27b78 EFLAGS: 00000202 ORIG_RAX: 000000000000004a [ 369.896882] RAX: ffffffffffffffda RBX: 00007fff52b28cb8 RCX: 00007f660e208950 [ 369.898139] RDX: 0000000000001000 RSI: 00007fff52b27b80 RDI: 0000000000000003 [ 369.899395] RBP: 00007fff52b28ba0 R08: 0000000000000410 R09: 0000000000000001 [ 369.900661] R10: 00007f660e11c400 R11: 0000000000000202 R12: 0000000000000000 [ 369.901925] R13: 00007fff52b28cc8 R14: 00007f660e328000 R15: 000055b5aeb6fdd8 [ 369.903202] INFO: task sync:468 blocked for more than 122 seconds. [ 369.904311] Not tainted 5.10.0+ #164 [ 369.905034] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.906457] task:sync state:D stack:13632 pid: 468 ppid: 386 flags:0x00004002 [ 369.907930] Call Trace: [ 369.908369] __schedule+0x401/0x8e0 [ 369.908984] schedule+0x49/0x130 [ 369.909582] io_schedule+0x12/0x40 [ 369.910208] wait_on_page_bit+0x106/0x2e0 [ 369.910918] ? add_to_page_cache_lru+0x180/0x180 [ 369.911758] wait_on_page_writeback+0x25/0xd0 [ 369.912560] __filemap_fdatawait_range+0x83/0x110 [ 369.913408] ? __add_pages+0x6f/0x1b0 [ 369.914089] filemap_fdatawait_keep_errors+0x1a/0x50 [ 369.914957] sync_inodes_sb+0x208/0x2a0 [ 369.915666] ? __x64_sys_tee+0xd0/0xd0 [ 369.916344] iterate_supers+0x90/0xe0 [ 369.916983] ksys_sync+0x40/0xb0 [ 369.917590] __do_sys_sync+0xa/0x20 [ 369.918240] do_syscall_64+0x33/0x40 [ 369.918884] entry_SYSCALL_64_after_hwframe+0x67/0xd1 [ 369.919800] RIP: 0033:0x7f746d820987 [ 369.920451] RSP: 002b:00007ffce853fd78 EFLAGS: 00000206 ORIG_RAX: 00000000000000a2 [ 369.921798] RAX: ffffffffffffffda RBX: 00007ffce853fed8 RCX: 00007f746d820987 [ 369.923063] RDX: 00007f746d8f4801 RSI: 00007ffce8541f71 RDI: 00007f746d8b05ad [ 369.924339] RBP: 0000000000000001 R08: 000000000000ffff R09: 0000000000000000 [ 369.925605] R10: 00007f746d7308a0 R11: 0000000000000206 R12: 000055b8487470fb [ 369.926866] R13: 0000000000000000 R14: 0000000000000000 R15: 000055b848749ce0 [ 369.928138] Kernel panic - not syncing: hung_task: blocked tasks [ 369.929191] CPU: 3 PID: 35 Comm: khungtaskd Not tainted 5.10.0+ #164 [ 369.952450] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.1-2.fc37 04/01/2014 [ 369.956984] Call Trace: [ 369.957973] dump_stack+0x57/0x6e [ 369.959273] panic+0x115/0x2f1 [ 369.960476] watchdog.cold+0xb5/0xb5 [ 369.961884] ? hungtask_pm_notify+0x40/0x40 [ 369.963310] kthread+0x125/0x160 [ 369.964354] ? kthread_park+0x90/0x90 [ 369.965551] ret_from_fork+0x22/0x30 [ 369.967673] Kernel Offset: 0xd600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) [ 369.971025] ---[ end Kernel panic - not syncing: hung_task: blocked tasks ]--- ``` Scenarios 2: Scenario 2 occurs in strict cache mode ``` P1 (dd) P2 (cifsd) P3 (cifsiod) cifs_strict_writev cifs_zap_mapping - If something breaks the oplock cifs_revalidate_mapping cifs_invalidate_mapping invalidate_inode_pages2 invalidate_inode_pages2_range lock_page - [1] wait_on_page_writeback - [2] Waiting for writeback, blocked by [4] wait_on_page_bit cifs_demultiplex_thread cifs_read_from_socket cifs_readv_from_socket - If another process triggers reconnect at this point cifs_reconnect - mid->mid_state updated to MID_RETRY_NEEDED smb2_writev_callback mid_entry->callback() - mid_state leads to wdata->result = -EAGAIN wdata->result = -EAGAIN queue_work(cifsiod_wq, &wdata->work); cifs_writev_complete - worker function - wdata->result == -EAGAIN Condition satisfied cifs_writev_requeue lock_page - [3] Blocked by [1] end_page_writeback - [4] Won't execute unlock_page ``` Mainline refactoring commit 3ee1a1fc3981 ("cifs: Cut over to using netfslib") directly terminates the file write instead of resending data when smb2_writev_callback() detects a write failure, thus avoiding this problem. This patch is introduced in v6.10-rc1. Therefore, scenario 2 affects LTS versions from linux-5.4 to linux-6.6. ``` cat /proc/522/stack [<0>] wait_on_page_bit+0x106/0x150 [<0>] invalidate_inode_pages2_range+0x2cc/0x580 [<0>] cifs_invalidate_mapping+0x2c/0x50 [cifs] [<0>] cifs_revalidate_mapping+0x4c/0x90 [cifs] [<0>] cifs_strict_writev+0x17a/0x250 [cifs] [<0>] __vfs_write+0x14f/0x1b0 [<0>] vfs_write+0xb6/0x1a0 [<0>] ksys_write+0x57/0xd0 [<0>] do_syscall_64+0x63/0x250 [<0>] entry_SYSCALL_64_after_hwframe+0x5c/0xc1 [<0>] 0xffffffffffffffff cat /proc/33/stack [<0>] __lock_page+0x10c/0x160 [<0>] cifs_writev_requeue.cold+0x17e/0x239 [cifs] [<0>] process_one_work+0x1a9/0x3f0 [<0>] worker_thread+0x50/0x3c0 [<0>] kthread+0x117/0x130 [<0>] ret_from_fork+0x35/0x40 [<0>] 0xffffffffffffffff ``` The root cause of the deadlock problem is that the page/folio is locked again in cifs_writev_requeue(). In order to safely fix it on the LTS branches, I would like to clarify the following questions:, 1. Whether resending is necessary. If retransmission is not required, simply terminating the write would avoids this problem. Is this an acceptable solution? 2. Is it necessary to lock the page/folio in cifs_writev_requeue()? Based on my code screening (possibly missing), there seems to be no process that modifies a page when it is marked as PG_writeback.Therefore, the page does not need to be locked during wait_on_page_writeback().

7 months

4
7
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror February 2025