stable-rc-5.3.9-rc1: regressions detected - remove_proc_entry: removing non-empty directory 'net/dev_snmp6', leaking at least 'lo'
Linux stable rc 5.3 branch running LTP reported following test failures. While investigating these failures I have found this kernel warning from boot console. Please find detailed LTP output log in the bottom of this email.
List of regression test cases: ltp-containers-tests: * netns_breakns_ip_ipv6_ioctl * netns_breakns_ip_ipv6_netlink * netns_breakns_ns_exec_ipv6_ioctl * netns_breakns_ns_exec_ipv6_netlink * netns_comm_ip_ipv6_ioctl * netns_comm_ip_ipv6_netlink * netns_comm_ns_exec_ipv6_ioctl * netns_comm_ns_exec_ipv6_netlink
dmesg log: [ 0.000000] Linux version 5.3.9-rc1 (oe-user@oe-host) (gcc version 7.3.0 (GCC)) #1 SMP PREEMPT Mon Nov 4 12:14:24 UTC 2019 [ 0.000000] Machine model: ARM Juno development board (r2) ... [ 3.670227] ------------[ cut here ]------------ [ 3.674887] remove_proc_entry: removing non-empty directory 'net/dev_snmp6', leaking at least 'lo' [ 3.684183] WARNING: CPU: 1 PID: 1 at /usr/src/kernel/fs/proc/generic.c:684 remove_proc_entry+0x194/0x1a8 [ 3.693658] Modules linked in: [ 3.696687] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.3.9-rc1 #1 [ 3.702806] Hardware name: ARM Juno development board (r2) (DT) [ 3.708669] pstate: 40000005 (nZcv daif -PAN -UAO) [ 3.713414] pc : remove_proc_entry+0x194/0x1a8 [ 3.717814] lr : remove_proc_entry+0x194/0x1a8 [ 3.722213] sp : ffff00001003bbe0 [ 3.725494] x29: ffff00001003bbe0 x28: ffff0000119cddc0 [ 3.730757] x27: ffff000012256000 x26: ffff00001220b000 [ 3.736019] x25: ffff00001220a000 x24: ffff000012209000 [ 3.741280] x23: ffff8009754a6b00 x22: ffff800973a536db [ 3.746541] x21: ffff800973a53600 x20: ffff000011f8f000 [ 3.751803] x19: ffff8009754a6bdb x18: ffffffffffffffff [ 3.757065] x17: 0000000000000007 x16: 0000000000000000 [ 3.762326] x15: ffff000011f8f848 x14: ffff80097396f108 [ 3.767588] x13: ffff80097396f107 x12: ffff000012268b70 [ 3.772849] x11: ffff000012268000 x10: 0000000000000028 [ 3.778111] x9 : 0000000000000000 x8 : ffff000011f8f848 [ 3.783372] x7 : 00000000b2722639 x6 : ffff000011f93000 [ 3.788634] x5 : 0000000000000000 x4 : ffff800975dd8000 [ 3.793895] x3 : ffff000011f90000 x2 : 0000000000000000 [ 3.799157] x1 : 43129acc141cb700 x0 : 0000000000000000 [ 3.804419] Call trace: [ 3.806841] remove_proc_entry+0x194/0x1a8 [ 3.810900] ipv6_proc_exit_net+0x38/0x58 [ 3.814872] ops_exit_list.isra.1+0x54/0x88 [ 3.819013] unregister_pernet_operations+0xec/0x150 [ 3.823929] unregister_pernet_subsys+0x34/0x48 [ 3.828416] ipv6_misc_proc_exit+0x1c/0x28 [ 3.832473] inet6_init+0x2a4/0x33c [ 3.835929] do_one_initcall+0x94/0x458 [ 3.839727] kernel_init_freeable+0x484/0x52c [ 3.844043] kernel_init+0x18/0x110 [ 3.847498] ret_from_fork+0x10/0x1c [ 3.851037] irq event stamp: 255276 [ 3.854492] hardirqs last enabled at (255275): [<ffff00001104f828>] _raw_spin_unlock_irq+0x38/0x78 [ 3.863453] hardirqs last disabled at (255276): [<ffff0000100a5a14>] debug_exception_enter+0xac/0xe8 [ 3.872498] softirqs last enabled at (255270): [<ffff00001008210c>] __do_softirq+0x474/0x580 [ 3.880943] softirqs last disabled at (255259): [<ffff0000101018e4>] irq_exit+0x144/0x150 [ 3.889044] ---[ end trace 6cbc85548f1f4bc5 ]---
... LTP test trimmed output,
RTNETLINK answers: Operation not supported netns_breakns_ns_exec_ipv6_netlink 1 TBROK: adding address to veth0 failed tee: /proc/sys/net/ipv6/conf/veth0/accept_dad: No such file or directory tee: /proc/sys/net/ipv6/conf/veth0/accept_ra: No such file or directory tee: /proc/sys/net/ipv6/conf/veth1/accept_dad: No such file or directory tee: /proc/sys/net/ipv6/conf/veth1/accept_ra: No such file or directory No support for INET6 on this system. netns_breakns_ns_exec_ipv6_ioctl 1 TBROK: adding address to veth0 failed netns_breakns_ip_ipv6_netlink 1 TBROK: adding address to veth0 failed netns_breakns_ip_ipv6_ioctl 1 TBROK: adding address to veth0 failed netns_comm_ns_exec_ipv6_netlink 1 TBROK: adding address to veth0 failed netns_comm_ns_exec_ipv6_ioctl 1 TBROK: adding address to veth0 failed netns_comm_ip_ipv6_netlink 1 TBROK: adding address to veth0 failed netns_comm_ip_ipv6_ioctl 1 TBROK: adding address to veth0 failed
metadata: git branch: linux-5.3.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: ff21af282725ae2ebc3ac4298513816f760c929e git describe: v5.3.8-160-gff21af282725 make_kernelversion: 5.3.9-rc1 kernel-config: http://snapshots.linaro.org/openembedded/lkft/lkft/sumo/juno/lkft/linux-stab... kernel-defconfig: http://snapshots.linaro.org/openembedded/lkft/lkft/sumo/juno/lkft/linux-stab... build-url: https://ci.linaro.org/job/openembedded-lkft-linux-stable-rc-5.3/DISTRO=lkft,... build-location: http://snapshots.linaro.org/openembedded/lkft/lkft/sumo/juno/lkft/linux-stab...
We are investigating this problem.
Full test logs, https://qa-reports.linaro.org/lkft/linux-stable-rc-5.3-oe/build/v5.3.8-160-g... https://qa-reports.linaro.org/lkft/linux-stable-rc-5.3-oe/build/v5.3.8-160-g... https://qa-reports.linaro.org/lkft/linux-stable-rc-5.3-oe/build/v5.3.8-160-g... https://qa-reports.linaro.org/lkft/linux-stable-rc-5.3-oe/build/v5.3.8-160-g... https://qa-reports.linaro.org/lkft/linux-stable-rc-5.3-oe/build/v5.3.8-160-g...
-- Linaro LKFT https://lkft.linaro.org/
Hi Sasha and Greg,
On Mon, 4 Nov 2019 at 20:59, Naresh Kamboju naresh.kamboju@linaro.org wrote:
Linux stable rc 5.3 branch running LTP reported following test failures. While investigating these failures I have found this kernel warning from boot console. Please find detailed LTP output log in the bottom of this email.
List of regression test cases: ltp-containers-tests: * netns_breakns_ip_ipv6_ioctl * netns_breakns_ip_ipv6_netlink * netns_breakns_ns_exec_ipv6_ioctl * netns_breakns_ns_exec_ipv6_netlink * netns_comm_ip_ipv6_ioctl * netns_comm_ip_ipv6_netlink * netns_comm_ns_exec_ipv6_ioctl * netns_comm_ns_exec_ipv6_netlink
These reported failures got fixed on latest stable-rc 5.3.y after dropping a patch [1]. The kernel warning is also gone now.
metadata: git branch: linux-5.3.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: 75c9913bbf6e9e64cb669236571e6af45cddfd68
ref: [PATCH AUTOSEL 5.3 12/33] blackhole_netdev: fix syzkaller reported issue [1] https://lkml.org/lkml/2019/10/25/794
Best regards Naresh Kamboju
On Tue, Nov 05, 2019 at 09:44:39AM +0530, Naresh Kamboju wrote:
Hi Sasha and Greg,
On Mon, 4 Nov 2019 at 20:59, Naresh Kamboju naresh.kamboju@linaro.org wrote:
Linux stable rc 5.3 branch running LTP reported following test failures. While investigating these failures I have found this kernel warning from boot console. Please find detailed LTP output log in the bottom of this email.
List of regression test cases: ltp-containers-tests: * netns_breakns_ip_ipv6_ioctl * netns_breakns_ip_ipv6_netlink * netns_breakns_ns_exec_ipv6_ioctl * netns_breakns_ns_exec_ipv6_netlink * netns_comm_ip_ipv6_ioctl * netns_comm_ip_ipv6_netlink * netns_comm_ns_exec_ipv6_ioctl * netns_comm_ns_exec_ipv6_netlink
These reported failures got fixed on latest stable-rc 5.3.y after dropping a patch [1].
What is the subject of the patch?
The kernel warning is also gone now.
metadata: git branch: linux-5.3.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: 75c9913bbf6e9e64cb669236571e6af45cddfd68
The -rc tree is rebased all the time, can I get a "real" subject line to get a chance to figure out what you are trying to refer to here?
ref: [PATCH AUTOSEL 5.3 12/33] blackhole_netdev: fix syzkaller reported issue [1] https://lkml.org/lkml/2019/10/25/794
lore.kernel.org is much more reliable :)
thanks,
greg k-h
On Tue, 5 Nov 2019 at 13:05, Greg Kroah-Hartman gregkh@linuxfoundation.org wrote:
Linux stable rc 5.3 branch running LTP reported following test failures. While investigating these failures I have found this kernel warning from boot console. Please find detailed LTP output log in the bottom of this email.
List of regression test cases: ltp-containers-tests: * netns_breakns_ip_ipv6_ioctl
<trim>
* netns_comm_ns_exec_ipv6_netlinkThese reported failures got fixed on latest stable-rc 5.3.y after dropping a patch [1].
What is the subject of the patch?
blackhole_netdev: fix syzkaller reported issue upstream commit b0818f80c8c1bc215bba276bd61c216014fab23b
The kernel warning is also gone now.
metadata: git branch: linux-5.3.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: 75c9913bbf6e9e64cb669236571e6af45cddfd68
The -rc tree is rebased all the time, can I get a "real" subject line to get a chance to figure out what you are trying to refer to here?
Linux 5.3.9-rc1 is good candidate on branch linux-5.3.y and linux-stable-rc tree.
ref: [PATCH AUTOSEL 5.3 12/33] blackhole_netdev: fix syzkaller reported issue [1] https://lkml.org/lkml/2019/10/25/794
lore.kernel.org is much more reliable :)
Thank you.
[ Sasha Levin sashal@kernel.org wrote on Mon, 4 Nov 2019 12:13:45 -0500 ] I've dropped this patch from 5.3 too, it was reverted upstream. https://lore.kernel.org/netdev/20191104171345.GG4787@sasha-vm/
Ref:
Reverting below patch fixed this problem. --- commit b0818f80c8c1bc215bba276bd61c216014fab23b Author: Mahesh Bandewar maheshb@google.com Date: Fri Oct 11 18:14:55 2019 -0700
blackhole_netdev: fix syzkaller reported issue
While invalidating the dst, we assign backhole_netdev instead of loopback device. However, this device does not have idev pointer and hence no ip6_ptr even if IPv6 is enabled. Possibly this has triggered the syzbot reported crash.
The syzbot report does not have reproducer, however, this is the only device that doesn't have matching idev created.
Crash instruction is :
static inline bool ip6_ignore_linkdown(const struct net_device *dev) { const struct inet6_dev *idev = __in6_dev_get(dev);
return !!idev->cnf.ignore_routes_with_linkdown; <= crash }
Also ipv6 always assumes presence of idev and never checks for it being NULL (as does the above referenced code). So adding a idev for the blackhole_netdev to avoid this class of crashes in the future.
Signed-off-by: David S. Miller davem@davemloft.net
On Tue, Nov 05, 2019 at 01:28:01PM +0530, Naresh Kamboju wrote:
On Tue, 5 Nov 2019 at 13:05, Greg Kroah-Hartman gregkh@linuxfoundation.org wrote:
Linux stable rc 5.3 branch running LTP reported following test failures. While investigating these failures I have found this kernel warning from boot console. Please find detailed LTP output log in the bottom of this email.
List of regression test cases: ltp-containers-tests: * netns_breakns_ip_ipv6_ioctl
<trim> > > > * netns_comm_ns_exec_ipv6_netlink > > > > These reported failures got fixed on latest stable-rc 5.3.y after > > dropping a patch [1]. > > What is the subject of the patch?
blackhole_netdev: fix syzkaller reported issue upstream commit b0818f80c8c1bc215bba276bd61c216014fab23b
That commit is not in any stable queue or tree at the moment, are you sure this is still an issue?
The kernel warning is also gone now.
metadata: git branch: linux-5.3.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: 75c9913bbf6e9e64cb669236571e6af45cddfd68
The -rc tree is rebased all the time, can I get a "real" subject line to get a chance to figure out what you are trying to refer to here?
Linux 5.3.9-rc1 is good candidate on branch linux-5.3.y and linux-stable-rc tree.
I can not parse this, what do you mean?
thanks,
greg k-h
On Tue, 5 Nov 2019 at 13:36, Greg Kroah-Hartman gregkh@linuxfoundation.org wrote:
These reported failures got fixed on latest stable-rc 5.3.y after dropping a patch [1].
What is the subject of the patch?
blackhole_netdev: fix syzkaller reported issue upstream commit b0818f80c8c1bc215bba276bd61c216014fab23b
That commit is not in any stable queue or tree at the moment, are you sure this is still an issue?
Since it has been dropped, the issue is gone now.
The kernel warning is also gone now.
metadata: git branch: linux-5.3.y git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git commit: 75c9913bbf6e9e64cb669236571e6af45cddfd68
The -rc tree is rebased all the time, can I get a "real" subject line to get a chance to figure out what you are trying to refer to here?
Linux 5.3.9-rc1 is good candidate on branch linux-5.3.y and linux-stable-rc tree.
I can not parse this, what do you mean?
linux-stable-rc linux-5.3.y branch and make kernel version 5.3.9-rc1 is been tested and no regresion found.
- Naresh
-
Greg Kroah-Hartman -
Naresh Kamboju