#regzbot introduced: v6.1.70..v6.1.71
After kernel upgrade 6.1.70 to 6.1.71, the computer hangs during shutdown.
The problem is related to NFS service. Stopping NFS service hangs:
# /etc/init.d/nfs stop * Caching service dependencies ... [ ok ] * Stopping NFS mountd ... [ ok ] * Stopping NFS daemon ... [ ok ]
then it hangs
Shutdown does not hang when NFS service is removed.
# rc-update del nfs
I had this kernel error in the log:
Jan 10 17:32:25 [rpc.mountd] Caught signal 15, un-registering and exiting. Jan 10 17:32:25 [kernel] [ 2005.560991] ------------[ cut here ]------------ Jan 10 17:32:25 [kernel] [ 2005.560996] kernel BUG at net/sunrpc/svc.c:576! Jan 10 17:32:25 [kernel] [ 2005.561004] invalid opcode: 0000 [#1] PREEMPT SMP PTI Jan 10 17:32:25 [kernel] [ 2005.561012] CPU: 0 PID: 8079 Comm: nfsd Tainted: P O 6.1.71-gentoo #1 Jan 10 17:32:25 [kernel] [ 2005.561017] Hardware name: Gigabyte Technology Co., Ltd. X58A-UD3R/X58A-UD3R, BIOS FB 08/24/2010 Jan 10 17:32:25 [kernel] [ 2005.561020] RIP: 0010:svc_destroy+0x1f/0x56 Jan 10 17:32:25 [kernel] [ 2005.561033] Code: 5b 5d 41 5c 41 5d c3 cc cc cc cc 55 48 8d 6f ec 53 48 89 fb 48 83 c7 44 e8 b6 6c 5f ff 48 8b 53 1c 48 8d 43 1c 48 39 c2 74 02 <0f> 0b 48 8b 53 2c 48 8d 43 2c 48 39 c2 74 02 0f 0b 48 89 ef e8 6b Jan 10 17:32:25 [kernel] [ 2005.561038] RSP: 0018:ffffc90001edbee8 EFLAGS: 00010287 Jan 10 17:32:25 [kernel] [ 2005.561043] RAX: ffff88816c1a1c30 RBX: ffff88816c1a1c14 RCX: 0000000000000000 Jan 10 17:32:25 [kernel] [ 2005.561047] RDX: ffff88813dfcc018 RSI: 0000000000000286 RDI: ffff88890bc9b9c0 Jan 10 17:32:25 [kernel] [ 2005.561051] RBP: ffff88816c1a1c00 R08: ffff88810310b600 R09: 0000000000000000 Jan 10 17:32:25 [kernel] [ 2005.561055] R10: ffff8881bc3b4000 R11: ffff8881bc3b4000 R12: ffffffff82e061c0 Jan 10 17:32:25 [kernel] [ 2005.561058] R13: ffff8881bc0e8000 R14: ffff88810310b600 R15: ffffc90002323c80 Jan 10 17:32:25 [kernel] [ 2005.561062] FS: 0000000000000000(0000) GS:ffff88890bc00000(0000) knlGS:0000000000000000 Jan 10 17:32:25 [kernel] [ 2005.561067] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jan 10 17:32:25 [kernel] [ 2005.561071] CR2: 000055e971e376f8 CR3: 000000000260a000 CR4: 00000000000006f0 Jan 10 17:32:25 [kernel] [ 2005.561075] Call Trace: Jan 10 17:32:25 [kernel] [ 2005.561079] <TASK> Jan 10 17:32:25 [kernel] [ 2005.561082] ? __die_body+0x15/0x57 Jan 10 17:32:25 [kernel] [ 2005.561091] ? die+0x2b/0x44 Jan 10 17:32:25 [kernel] [ 2005.561097] ? do_trap+0x76/0xf9 Jan 10 17:32:25 [kernel] [ 2005.561102] ? svc_destroy+0x1f/0x56 Jan 10 17:32:25 [kernel] [ 2005.561108] ? svc_destroy+0x1f/0x56 Jan 10 17:32:25 [kernel] [ 2005.561114] ? do_error_trap+0x69/0x93 Jan 10 17:32:25 [kernel] [ 2005.561119] ? svc_destroy+0x1f/0x56 Jan 10 17:32:25 [kernel] [ 2005.561126] ? exc_invalid_op+0x49/0x5d Jan 10 17:32:25 [kernel] [ 2005.561133] ? svc_destroy+0x1f/0x56 Jan 10 17:32:25 [kernel] [ 2005.561139] ? asm_exc_invalid_op+0x16/0x20 Jan 10 17:32:25 [kernel] [ 2005.561148] ? svc_destroy+0x1f/0x56 Jan 10 17:32:25 [kernel] [ 2005.561155] ? svc_destroy+0x12/0x56 Jan 10 17:32:25 [kernel] [ 2005.561161] nfsd+0x13d/0x162 Jan 10 17:32:25 [kernel] [ 2005.561170] ? svc_put+0x2f/0x2f Jan 10 17:32:25 [kernel] [ 2005.561176] kthread+0xd0/0xd8 Jan 10 17:32:25 [kernel] [ 2005.561183] ? kthread_complete_and_exit+0x16/0x16 Jan 10 17:32:25 [kernel] [ 2005.561189] ret_from_fork+0x22/0x30 Jan 10 17:32:25 [kernel] [ 2005.561196] </TASK> Jan 10 17:32:25 [kernel] [ 2005.561198] Modules linked in: snd_seq_dummy snd_seq snd_seq_device nvidia_uvm(PO) bluetooth ecdh_generic ecc bridge stp llc ipv6 crc_ccitt ch341 usbserial nvidia_drm(PO) nvidia_modeset(PO) nvidia(PO) tda10048 tda8290 iTCO_wdt iTCO_vendor_support tda18271 it87 hwmon_vid dm_crypt coretemp dm_multipath dm_mod kvm_intel dax video snd_hda_codec_realtek snd_hda_codec_generic kvm drm_kms_helper irqbypass ledtrig_audio i2c_i801 pcspkr serio_raw drm i2c_smbus snd_hda_intel i2c_core snd_intel_dspcfg fb_sys_fops syscopyarea snd_hda_codec sysfillrect lpc_ich snd_hda_core sysimgblt mfd_core snd_hwdep uhci_hcd rtc_cmos wmi Jan 10 17:32:25 [kernel] [ 2005.561274] ---[ end trace 0000000000000000 ]--- Jan 10 17:32:25 [kernel] [ 2005.561277] RIP: 0010:svc_destroy+0x1f/0x56 Jan 10 17:32:25 [kernel] [ 2005.561284] Code: 5b 5d 41 5c 41 5d c3 cc cc cc cc 55 48 8d 6f ec 53 48 89 fb 48 83 c7 44 e8 b6 6c 5f ff 48 8b 53 1c 48 8d 43 1c 48 39 c2 74 02 <0f> 0b 48 8b 53 2c 48 8d 43 2c 48 39 c2 74 02 0f 0b 48 89 ef e8 6b
The problem is always reproducible. Steps to reproduce: 1. Update to kernel 6.1.71 2. Start nfs service 3. Try to shutdown 4. The computer hangs
Reverting the following 3 commits fixed the problem in kernel 6.1.71:
f9a01938e07910224d4a2fd00583725d686c3f38 bb4f791cb2de1140d0fbcedfe9e791ff364021d7 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
Reverting the following 3 commits fixed the problem in kernel 6.1.71:
f9a01938e07910224d4a2fd00583725d686c3f38 bb4f791cb2de1140d0fbcedfe9e791ff364021d7 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
When sending us git ids, please show the full context so we have a hint as to what they are. For this, it should be:
f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.") bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()") 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Do you also have these issues in the latest 6.6.y release? 6.7?
thanks,
greg k-h
Hi Greg
I'm sorry. This my first kernel report.
I didn't test 6.6.x and 6.7.x. I use only 6.1.x.
Best Regards
John G
On 11/1/24 20:30, Greg KH wrote:
On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
Reverting the following 3 commits fixed the problem in kernel 6.1.71:
f9a01938e07910224d4a2fd00583725d686c3f38 bb4f791cb2de1140d0fbcedfe9e791ff364021d7 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
When sending us git ids, please show the full context so we have a hint as to what they are. For this, it should be:
f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.") bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()") 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Do you also have these issues in the latest 6.6.y release? 6.7?
thanks,
greg k-h
Hi Greg
Here is the test results for the latest versions of 6.6.x and 6.7.x available in Gentoo portage
1- Stopping NFS service failed but, unlike 6.1.x, it did NOT hang.
# uname -r 6.6.11-gentoo # /etc/init.d/nfs stop * Stopping NFS mountd ... [ ok ] * Stopping NFS daemon ... * start-stop-daemon: 8 process(es) refused to stop [ !! ] * Unexporting NFS directories ... [ ok ] * ERROR: nfs failed to stop # /etc/init.d/nfs start * WARNING: nfs has already been started
# uname -r 6.7.0-gentoo # /etc/init.d/nfs stop * Stopping NFS mountd ... [ ok ] * Stopping NFS daemon ... * start-stop-daemon: 8 process(es) refused to stop [ !! ] * Unexporting NFS directories ... [ ok ] * ERROR: nfs failed to stop # /etc/init.d/nfs start * WARNING: nfs has already been started
2- Shutdown didn't hang in both of them
Best regards John G
On 11/1/24 21:21, Greg KH wrote:
On Thu, Jan 11, 2024 at 09:10:39PM +1100, email200202 wrote:
Hi Greg
I'm sorry. This my first kernel report.
I didn't test 6.6.x and 6.7.x. I use only 6.1.x.
Can you do so?
thanks,
greg k-h
On Jan 11, 2024, at 8:58 AM, email200202 email200202@yahoo.com wrote:
Hi Greg
Here is the test results for the latest versions of 6.6.x and 6.7.x available in Gentoo portage
1- Stopping NFS service failed but, unlike 6.1.x, it did NOT hang.
# uname -r 6.6.11-gentoo # /etc/init.d/nfs stop
- Stopping NFS mountd ... [ ok ]
- Stopping NFS daemon ...
- start-stop-daemon: 8 process(es) refused to stop [ !! ]
- Unexporting NFS directories ... [ ok ]
- ERROR: nfs failed to stop
# /etc/init.d/nfs start
- WARNING: nfs has already been started
# uname -r 6.7.0-gentoo # /etc/init.d/nfs stop
- Stopping NFS mountd ... [ ok ]
- Stopping NFS daemon ...
- start-stop-daemon: 8 process(es) refused to stop [ !! ]
- Unexporting NFS directories ... [ ok ]
- ERROR: nfs failed to stop
# /etc/init.d/nfs start
- WARNING: nfs has already been started
2- Shutdown didn't hang in both of them
Best regards John G
On 11/1/24 21:21, Greg KH wrote:
On Thu, Jan 11, 2024 at 09:10:39PM +1100, email200202 wrote:
Hi Greg
I'm sorry. This my first kernel report.
I didn't test 6.6.x and 6.7.x. I use only 6.1.x.
Can you do so?
thanks,
greg k-h
I haven't seen emails before this one. But here are the only three new NFSD commits in v6.1.71:
f9a01938e079 NFSD: fix possible oops when nfsd/pool_stats is closed. bb4f791cb2de nfsd: call nfsd_last_thread() before final nfsd_put() 03d68ffc48b9 nfsd: separate nfsd_last_thread() from nfsd_put()
And the latter two are directly related to NFSD shut down.
-- Chuck Lever
On Thu, 2024-01-11 at 10:30 +0100, Greg KH wrote:
On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
Reverting the following 3 commits fixed the problem in kernel 6.1.71:
f9a01938e07910224d4a2fd00583725d686c3f38 bb4f791cb2de1140d0fbcedfe9e791ff364021d7 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
When sending us git ids, please show the full context so we have a hint as to what they are. For this, it should be:
f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.") bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()") 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Do you also have these issues in the latest 6.6.y release? 6.7?
I wasn't able to reproduce the exact same bug as the reporter, but I did see a different panic due to the above patches. Some of those fixes aren't appropriate without pulling in earlier commits, but I think it's probably best to just drop these two:
bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()") 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Greg, does that sound OK?
Thanks,
On Thu, Jan 11, 2024 at 12:26:39PM -0500, Jeff Layton wrote:
On Thu, 2024-01-11 at 10:30 +0100, Greg KH wrote:
On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
Reverting the following 3 commits fixed the problem in kernel 6.1.71:
f9a01938e07910224d4a2fd00583725d686c3f38 bb4f791cb2de1140d0fbcedfe9e791ff364021d7 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
When sending us git ids, please show the full context so we have a hint as to what they are. For this, it should be:
f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.") bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()") 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Do you also have these issues in the latest 6.6.y release? 6.7?
I wasn't able to reproduce the exact same bug as the reporter, but I did see a different panic due to the above patches. Some of those fixes aren't appropriate without pulling in earlier commits, but I think it's probably best to just drop these two:
bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()") 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Greg, does that sound OK?
Sure, can you send reverts for them, or do you need me to make them?
thanks,
greg k-h
On Sat, Jan 13, 2024 at 10:35:09AM +0100, Greg KH wrote:
On Thu, Jan 11, 2024 at 12:26:39PM -0500, Jeff Layton wrote:
On Thu, 2024-01-11 at 10:30 +0100, Greg KH wrote:
On Thu, Jan 11, 2024 at 07:20:02PM +1100, email200202 wrote:
Reverting the following 3 commits fixed the problem in kernel 6.1.71:
f9a01938e07910224d4a2fd00583725d686c3f38 bb4f791cb2de1140d0fbcedfe9e791ff364021d7 03d68ffc48b94cc1e15bbf3b4f16f1e1e4fa286a
When sending us git ids, please show the full context so we have a hint as to what they are. For this, it should be:
f9a01938e079 ("NFSD: fix possible oops when nfsd/pool_stats is closed.") bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()") 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Do you also have these issues in the latest 6.6.y release? 6.7?
I wasn't able to reproduce the exact same bug as the reporter, but I did see a different panic due to the above patches. Some of those fixes aren't appropriate without pulling in earlier commits, but I think it's probably best to just drop these two:
bb4f791cb2de ("nfsd: call nfsd_last_thread() before final nfsd_put()") 03d68ffc48b9 ("nfsd: separate nfsd_last_thread() from nfsd_put()")
Greg, does that sound OK?
Sure, can you send reverts for them, or do you need me to make them?
Nevermind, I made them, thanks.
greg k-h
On 11.01.24 09:20, email200202 wrote:
#regzbot introduced: v6.1.70..v6.1.71
After kernel upgrade 6.1.70 to 6.1.71, the computer hangs during shutdown.
The problem is related to NFS service. Stopping NFS service hangs:
# /etc/init.d/nfs stop * Caching service dependencies ... [ ok ] * Stopping NFS mountd ... [ ok ] * Stopping NFS daemon ... [ ok ] [...]
#regzbot fix: b2c545c39877408a2fe2 #regzbot ignore-activity
Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat) -- Everything you wanna know about Linux kernel regression tracking: https://linux-regtracking.leemhuis.info/about/#tldr That page also explains what to do if mails like this annoy you.
linux-stable-mirror@lists.linaro.org
-
Chuck Lever III -
email200202 -
Greg KH -
Jeff Layton -
Linux regression tracking #update (Thorsten Leemhuis)