Hi All,
I’m looking at the 4.14-rc5 results. I think it’s important that we establish clear green baselines so we can detect regressions.
Universally I think we want to get stuff on skip lists and then get after those skip lists working on fixes. As that happens, we basically get those fixes back ported to 4.9 and 4.4.
I haven’t captured everything since this requires manual c/n/p: (Being able to do command line queries would be really awesome)
With the triage meeting tomorrow I’d like to focus down in these areas.
x86_64
ltp-syscalls-tests - How much of this caused by NFS? Why are we still using NFS?
linkat01
open12
openat02
renameat201
renameat202
sendfile09
sendfile09_64
utime01
utime02
utime06
utimes01
ltp_containers - looks like arm64 is in better shape? Did something not get replicated on x86?
netns_breakns_ip_ipv4_ioctl
netns_breakns_ip_ipv4_netlink
netns_breakns_ip_ipv6_ioctl
netns_breakns_ip_ipv6_netlink
netns_breakns_ns_exec_ipv4_ioc
netns_breakns_ns_exec_ipv4_net
netns_breakns_ns_exec_ipv6_ioc
netns_breakns_ns_exec_ipv6_net
netns_comm_ip_ipv4_ioctl
netns_comm_ip_ipv4_netlink
netns_comm_ip_ipv6_ioctl
netns_comm_ip_ipv6_netlink
netns_comm_ns_exec_ipv4_ioctl
netns_comm_ns_exec_ipv4_netlin
netns_comm_ns_exec_ipv6_ioctl
netns_comm_ns_exec_ipv6_netlin
netns_sysfs
kselftest
ftracetest - fixed in kselftest-next
pstore_tests - extra command line options needed
run_fuse_test.sh - fixed for ARM
run.sh
run_vmtests
test_align
test_kmod.sh
test_progs
test_verifier
HiKey
quotactl01 - still missing quota
quota_remount_test01
ltp-containers is better but…. :
netns_comm_ip_ipv6_ioctl
netns_comm_ip_ipv6_netlink
netns_comm_ns_exec_ipv6_ioctl
netns_comm_ns_exec_ipv6_netlin
netns_sysfs
kselftest
breakpoint_test_arm64
ftracetest
pstore_tests
run_fuse_test.sh
run.sh
run_vmtests
seccomp_bpf
test_align
test_kmod.sh
test_maps
test_progs
test_verifier
Juno
ltp-syscall - looks like all NFS related? when can we dump NFS?
open12
openat02
quotactl01
renameat201
renameat202
sendfile09
sendfile09_64
utime01
utime02
utime06
utimensat01
utimes01
quota_remount_test01 - missing quota
ltp-containers - same problems as on Hikey
netns_comm_ip_ipv6_ioctl
netns_comm_ip_ipv6_netlink
netns_comm_ns_exec_ipv6_ioctl
netns_comm_ns_exec_ipv6_netlin
netns_sysfs
x15
perf_event_open02
quotactl01
leapsec_timer
22 failures in ltp-hugetlbfs - I thought we agreed we were going to put it on our skip list for now
ltp-containers-test
18 failures — looks to be same as intel
It’s a long list but feels like with a fix or two in the test environment we could have a number of failures switch to green.
This past RC cycle I think we exposed a weakness in what we have in LKFT where the ability to execute some key functional stacks in the system to drive the kernel would probably be useful for validation.
The networking bug involving dhclient for example.
So what if we used either Debian, Gentoo or akin that has a mechanism that has as part of it’s packaging system a test target for each package. Simplest build a package, runs ‘make test’ (or akin) for some key packages that exercises parts of the system that should help tickle the kernel in interesting ways to tease out regressions.
Thus wouldn’t work on modest boards but the socionext or Juno boards could probably work fine.
Thoughts?
Regards,
Tom
On Mon, Oct 09, 2017 at 09:44:36AM +0000, Linaro QA wrote:
> Summary
> ------------------------------------------------------------------------
>
> kernel: 4.9.54
> git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
> git branch: linux-4.9.y
> git commit: f37eb7b586f1dd24a86c50278c65322fc6787722
> git describe: v4.9.54
> Test details: https://qa-reports.linaro.org/lkft/linux-stable-rc-4.9-oe/build/v4.9.54
Hm, I don't seem to have a test result here for 4.9.55-rc1, so I'm
hijacking this thread to ask what happened to the tests there?
Turns out that 4.9.55-rc1 did not work at all for networking, yet no
tests seem to have caught it. Are we not testing something with a
network here? You said you were using NFS, how did that work?
Anything we can do to add to the tests to verify that basic dhcp works?
We should learn from our mistakes...
Heck, the simple build/boot tests that Google runs instantly found this
issue when I tried to merge 4.9.55 into the android-common trees, maybe
I should just rely on thost tests from now on?
Because of this, I had to release 4.9.56 a few hours later fixing the
issue. This doesn't make me feel like I can trust this testing effort
at all just yet, given the track record it has shown this past
week. We have had two known-bad kernel releases and none of this testing
caught it at all (both were caught by community members...) :(
Back to booting the kernels on my laptop and running 'make
allmodconfig', that's found more errors so far than anything else...
greg k-h