If SMT is disabled or a partial SMT state is enabled, when a new kernel image is loaded for kexec, on reboot the following warning is observed:
kexec: Waking offline cpu 228. WARNING: CPU: 0 PID: 9062 at arch/powerpc/kexec/core_64.c:223 kexec_prepare_cpus+0x1b0/0x1bc [snip] NIP kexec_prepare_cpus+0x1b0/0x1bc LR kexec_prepare_cpus+0x1a0/0x1bc Call Trace: kexec_prepare_cpus+0x1a0/0x1bc (unreliable) default_machine_kexec+0x160/0x19c machine_kexec+0x80/0x88 kernel_kexec+0xd0/0x118 __do_sys_reboot+0x210/0x2c4 system_call_exception+0x124/0x320 system_call_vectored_common+0x15c/0x2ec
This occurs as add_cpu() fails due to cpu_bootable() returning false for CPUs that fail the cpu_smt_thread_allowed() check or non primary threads if SMT is disabled.
Fix the issue by enabling SMT and resetting the number of SMT threads to the number of threads per core, before attempting to wake up all present CPUs.
Fixes: 38253464bc82 ("cpu/SMT: Create topology_smt_thread_allowed()") Reported-by: Sachin P Bappalige sachinpb@linux.ibm.com Cc: stable@vger.kernel.org # v6.6+ Signed-off-by: Nysal Jan K.A. nysal@linux.ibm.com --- arch/powerpc/kexec/core_64.c | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c index 222aa326dace..ff6df43720c4 100644 --- a/arch/powerpc/kexec/core_64.c +++ b/arch/powerpc/kexec/core_64.c @@ -216,6 +216,11 @@ static void wake_offline_cpus(void) { int cpu = 0;
+ lock_device_hotplug(); + cpu_smt_num_threads = threads_per_core; + cpu_smt_control = CPU_SMT_ENABLED; + unlock_device_hotplug(); + for_each_present_cpu(cpu) { if (!cpu_online(cpu)) { printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
Hi Nysal,
kernel test robot noticed the following build errors:
[auto build test ERROR on powerpc/next] [also build test ERROR on powerpc/fixes linus/master v6.18-rc3 next-20251027] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Nysal-Jan-K-A/powerpc-kexec-E... base: https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next patch link: https://lore.kernel.org/r/20251025080512.85690-1-nysal%40linux.ibm.com patch subject: [PATCH] powerpc/kexec: Enable SMT before waking offline CPUs config: powerpc64-randconfig-001-20251028 (https://download.01.org/0day-ci/archive/20251028/202510280824.Fe2D1Sbw-lkp@i...) compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project e1ae12640102fd2b05bc567243580f90acb1135f) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251028/202510280824.Fe2D1Sbw-lkp@i...)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot lkp@intel.com | Closes: https://lore.kernel.org/oe-kbuild-all/202510280824.Fe2D1Sbw-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from <built-in>:3: In file included from include/linux/compiler_types.h:171: include/linux/compiler-clang.h:37:9: warning: '__SANITIZE_THREAD__' macro redefined [-Wmacro-redefined] 37 | #define __SANITIZE_THREAD__ | ^ <built-in>:353:9: note: previous definition is here 353 | #define __SANITIZE_THREAD__ 1 | ^
arch/powerpc/kexec/core_64.c:220:22: error: expression is not assignable
220 | cpu_smt_num_threads = threads_per_core; | ~~~~~~~~~~~~~~~~~~~ ^ arch/powerpc/kexec/core_64.c:221:18: error: expression is not assignable 221 | cpu_smt_control = CPU_SMT_ENABLED; | ~~~~~~~~~~~~~~~ ^ 1 warning and 2 errors generated.
vim +220 arch/powerpc/kexec/core_64.c
204 205 /* 206 * We need to make sure each present CPU is online. The next kernel will scan 207 * the device tree and assume primary threads are online and query secondary 208 * threads via RTAS to online them if required. If we don't online primary 209 * threads, they will be stuck. However, we also online secondary threads as we 210 * may be using 'cede offline'. In this case RTAS doesn't see the secondary 211 * threads as offline -- and again, these CPUs will be stuck. 212 * 213 * So, we online all CPUs that should be running, including secondary threads. 214 */ 215 static void wake_offline_cpus(void) 216 { 217 int cpu = 0; 218 219 lock_device_hotplug();
220 cpu_smt_num_threads = threads_per_core;
221 cpu_smt_control = CPU_SMT_ENABLED; 222 unlock_device_hotplug(); 223 224 for_each_present_cpu(cpu) { 225 if (!cpu_online(cpu)) { 226 printk(KERN_INFO "kexec: Waking offline cpu %d.\n", 227 cpu); 228 WARN_ON(add_cpu(cpu)); 229 } 230 } 231 } 232
* Nysal Jan K.A. nysal@linux.ibm.com [2025-10-25 13:35:09]:
diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c index 222aa326dace..ff6df43720c4 100644 --- a/arch/powerpc/kexec/core_64.c +++ b/arch/powerpc/kexec/core_64.c @@ -216,6 +216,11 @@ static void wake_offline_cpus(void) { int cpu = 0;
- lock_device_hotplug();
- cpu_smt_num_threads = threads_per_core;
- cpu_smt_control = CPU_SMT_ENABLED;
- unlock_device_hotplug();
Nit: can we add a comment mentioning why we do it and probably move to a small helper function of its own.
Otherwise looks good.
Reviewed-by: Srikar Dronamraju srikar@linux.ibm.com
for_each_present_cpu(cpu) { if (!cpu_online(cpu)) { printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
Hello Nysal,
On 25/10/25 13:35, Nysal Jan K.A. wrote:
If SMT is disabled or a partial SMT state is enabled, when a new kernel image is loaded for kexec, on reboot the following warning is observed:
kexec: Waking offline cpu 228. WARNING: CPU: 0 PID: 9062 at arch/powerpc/kexec/core_64.c:223 kexec_prepare_cpus+0x1b0/0x1bc [snip] NIP kexec_prepare_cpus+0x1b0/0x1bc LR kexec_prepare_cpus+0x1a0/0x1bc Call Trace: kexec_prepare_cpus+0x1a0/0x1bc (unreliable) default_machine_kexec+0x160/0x19c machine_kexec+0x80/0x88 kernel_kexec+0xd0/0x118 __do_sys_reboot+0x210/0x2c4 system_call_exception+0x124/0x320 system_call_vectored_common+0x15c/0x2ec
This occurs as add_cpu() fails due to cpu_bootable() returning false for CPUs that fail the cpu_smt_thread_allowed() check or non primary threads if SMT is disabled.
Fix the issue by enabling SMT and resetting the number of SMT threads to the number of threads per core, before attempting to wake up all present CPUs.
Fixes: 38253464bc82 ("cpu/SMT: Create topology_smt_thread_allowed()") Reported-by: Sachin P Bappalige sachinpb@linux.ibm.com Cc: stable@vger.kernel.org # v6.6+ Signed-off-by: Nysal Jan K.A. nysal@linux.ibm.com
arch/powerpc/kexec/core_64.c | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c index 222aa326dace..ff6df43720c4 100644 --- a/arch/powerpc/kexec/core_64.c +++ b/arch/powerpc/kexec/core_64.c @@ -216,6 +216,11 @@ static void wake_offline_cpus(void) { int cpu = 0;
- lock_device_hotplug();
- cpu_smt_num_threads = threads_per_core;
- cpu_smt_control = CPU_SMT_ENABLED;
Above variables are #define if CONFIG_SMP and CONFIG_HOTPLUG_SMT is not there.
I think the above code should go under #if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT).
Seems like the build failure reported below is also pointing the same issue: https://lore.kernel.org/all/202510280824.Fe2D1Sbw-lkp@intel.com/
- unlock_device_hotplug();
- for_each_present_cpu(cpu) { if (!cpu_online(cpu)) { printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
If SMT is disabled or a partial SMT state is enabled, when a new kernel image is loaded for kexec, on reboot the following warning is observed:
kexec: Waking offline cpu 228. WARNING: CPU: 0 PID: 9062 at arch/powerpc/kexec/core_64.c:223 kexec_prepare_cpus+0x1b0/0x1bc [snip] NIP kexec_prepare_cpus+0x1b0/0x1bc LR kexec_prepare_cpus+0x1a0/0x1bc Call Trace: kexec_prepare_cpus+0x1a0/0x1bc (unreliable) default_machine_kexec+0x160/0x19c machine_kexec+0x80/0x88 kernel_kexec+0xd0/0x118 __do_sys_reboot+0x210/0x2c4 system_call_exception+0x124/0x320 system_call_vectored_common+0x15c/0x2ec
This occurs as add_cpu() fails due to cpu_bootable() returning false for CPUs that fail the cpu_smt_thread_allowed() check or non primary threads if SMT is disabled.
Fix the issue by enabling SMT and resetting the number of SMT threads to the number of threads per core, before attempting to wake up all present CPUs.
Fixes: 38253464bc82 ("cpu/SMT: Create topology_smt_thread_allowed()") Reported-by: Sachin P Bappalige sachinpb@linux.ibm.com Cc: stable@vger.kernel.org # v6.6+ Reviewed-by: Srikar Dronamraju srikar@linux.ibm.com Signed-off-by: Nysal Jan K.A. nysal@linux.ibm.com --- arch/powerpc/kexec/core_64.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c index 222aa326dace..825ab8a88f18 100644 --- a/arch/powerpc/kexec/core_64.c +++ b/arch/powerpc/kexec/core_64.c @@ -202,6 +202,23 @@ static void kexec_prepare_cpus_wait(int wait_state) mb(); }
+ +/* + * The add_cpu() call in wake_offline_cpus() can fail as cpu_bootable() + * returns false for CPUs that fail the cpu_smt_thread_allowed() check + * or non primary threads if SMT is disabled. Re-enable SMT and set the + * number of SMT threads to threads per core. + */ +static void kexec_smt_reenable(void) +{ +#if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT) + lock_device_hotplug(); + cpu_smt_num_threads = threads_per_core; + cpu_smt_control = CPU_SMT_ENABLED; + unlock_device_hotplug(); +#endif +} + /* * We need to make sure each present CPU is online. The next kernel will scan * the device tree and assume primary threads are online and query secondary @@ -216,6 +233,8 @@ static void wake_offline_cpus(void) { int cpu = 0;
+ kexec_smt_reenable(); + for_each_present_cpu(cpu) { if (!cpu_online(cpu)) { printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
Hi Nysal.
On 10/28/25 4:25 PM, Nysal Jan K.A. wrote:
If SMT is disabled or a partial SMT state is enabled, when a new kernel image is loaded for kexec, on reboot the following warning is observed:
kexec: Waking offline cpu 228. WARNING: CPU: 0 PID: 9062 at arch/powerpc/kexec/core_64.c:223 kexec_prepare_cpus+0x1b0/0x1bc [snip] NIP kexec_prepare_cpus+0x1b0/0x1bc LR kexec_prepare_cpus+0x1a0/0x1bc Call Trace: kexec_prepare_cpus+0x1a0/0x1bc (unreliable) default_machine_kexec+0x160/0x19c machine_kexec+0x80/0x88 kernel_kexec+0xd0/0x118 __do_sys_reboot+0x210/0x2c4 system_call_exception+0x124/0x320 system_call_vectored_common+0x15c/0x2ec
This occurs as add_cpu() fails due to cpu_bootable() returning false for CPUs that fail the cpu_smt_thread_allowed() check or non primary threads if SMT is disabled.
Fix the issue by enabling SMT and resetting the number of SMT threads to the number of threads per core, before attempting to wake up all present CPUs.
Fixes: 38253464bc82 ("cpu/SMT: Create topology_smt_thread_allowed()") Reported-by: Sachin P Bappalige sachinpb@linux.ibm.com Cc: stable@vger.kernel.org # v6.6+ Reviewed-by: Srikar Dronamraju srikar@linux.ibm.com Signed-off-by: Nysal Jan K.A. nysal@linux.ibm.com
arch/powerpc/kexec/core_64.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
diff --git a/arch/powerpc/kexec/core_64.c b/arch/powerpc/kexec/core_64.c index 222aa326dace..825ab8a88f18 100644 --- a/arch/powerpc/kexec/core_64.c +++ b/arch/powerpc/kexec/core_64.c @@ -202,6 +202,23 @@ static void kexec_prepare_cpus_wait(int wait_state) mb(); }
+/*
- The add_cpu() call in wake_offline_cpus() can fail as cpu_bootable()
- returns false for CPUs that fail the cpu_smt_thread_allowed() check
- or non primary threads if SMT is disabled. Re-enable SMT and set the
- number of SMT threads to threads per core.
- */
+static void kexec_smt_reenable(void) +{ +#if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT)
- lock_device_hotplug();
I was looking at usage of lock_device_hotplug, looks like a good candidate for guard() use case. Could be done on its own patch/series.
- cpu_smt_num_threads = threads_per_core;
- cpu_smt_control = CPU_SMT_ENABLED;
- unlock_device_hotplug();
+#endif +}
Will this work too? It might be better since we anyway going to bring that CPU up by doing add_cpu afterwords.
cpu_smt_num_threads = threads_per_core; cpuhp_smt_enable()
- /*
- We need to make sure each present CPU is online. The next kernel will scan
- the device tree and assume primary threads are online and query secondary
@@ -216,6 +233,8 @@ static void wake_offline_cpus(void) { int cpu = 0;
- kexec_smt_reenable();
If we do above, just change the below logic to complain if any present CPU is offline.
for_each_present_cpu(cpu) { if (!cpu_online(cpu)) { printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
hi Shrikanth,
On Tue, Oct 28, 2025 at 10:56:05PM +0530, Shrikanth Hegde wrote:
Hi Nysal.
On 10/28/25 4:25 PM, Nysal Jan K.A. wrote:
[snip]
--- a/arch/powerpc/kexec/core_64.c +++ b/arch/powerpc/kexec/core_64.c @@ -202,6 +202,23 @@ static void kexec_prepare_cpus_wait(int wait_state) mb(); }
+/*
- The add_cpu() call in wake_offline_cpus() can fail as cpu_bootable()
- returns false for CPUs that fail the cpu_smt_thread_allowed() check
- or non primary threads if SMT is disabled. Re-enable SMT and set the
- number of SMT threads to threads per core.
- */
+static void kexec_smt_reenable(void) +{ +#if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT)
- lock_device_hotplug();
I was looking at usage of lock_device_hotplug, looks like a good candidate for guard() use case. Could be done on its own patch/series.
Agree, we can look at it as a separate patch.
- cpu_smt_num_threads = threads_per_core;
- cpu_smt_control = CPU_SMT_ENABLED;
- unlock_device_hotplug();
+#endif +}
Will this work too? It might be better since we anyway going to bring that CPU up by doing add_cpu afterwords.
cpu_smt_num_threads = threads_per_core; cpuhp_smt_enable()
There is some reasoning in 4d37cc2dc3df, which made the switch to use the core device API, against calling cpu_up() directly. The other issue is cpuhp_smt_enable() can skip bringing up a CPU in certain cases, for example when a core is offline.
- /*
- We need to make sure each present CPU is online. The next kernel will scan
- the device tree and assume primary threads are online and query secondary
@@ -216,6 +233,8 @@ static void wake_offline_cpus(void) { int cpu = 0;
- kexec_smt_reenable();
If we do above, just change the below logic to complain if any present CPU is offline.
for_each_present_cpu(cpu) { if (!cpu_online(cpu)) { printk(KERN_INFO "kexec: Waking offline cpu %d.\n",
Thanks for the review.
--Nysal
linux-stable-mirror@lists.linaro.org
-
kernel test robot -
Nysal Jan K.A. -
Shrikanth Hegde -
Sourabh Jain -
Srikar Dronamraju