Looking through these kselftest failures. The large quantity were because the first run went without a skipfile. The second run still had a few failures that I was surprised by:
all architectures bpf/test_align bpf/test_verifier
x86 x86/ldt_gdt_64
The two bpf tests are passing on mainline and recently removed from our skiplist. The patches made 4.15, so I was surprised to see it failing. I believe the issue is that the test source was also updated (see i.e. 'selftests/bpf: fix test_align'), but we are running with an older version of kselftest on 4.15.
I suspect the problem is the same with x86/ldt_gdt_64; mainline runs with mainline kselftest, but 4.15 is running with 4.14 selftest. So when the test gets fixed on mainline, mainline gets fixed, but then we see the regressions again on 4.15 when it's released.
Can someone remind me again why we run mismatched kselftest? It seems to cause more confusion than it is worth.
These issues are also difficult to explain on the mailing lists, and if any enterprising engineer were to try to investigate for themselves, they would be quite surprised.
I know we did it for good reasons, I just think we need to re-evaluate periodically to be sure it still makes sense.
Dan
On Sat, Feb 03, 2018 at 03:52:10AM +0000, Linaro QA wrote:
Summary
kernel: 4.15.1-rc1 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git branch: linux-4.15.y git commit: b01b3d9519f250398695c7cc6493ba1e8fb072f4 git describe: v4.15-56-gb01b3d9519f2 Test details: https://qa-reports.linaro.org/lkft/linux-stable-rc-4.15-oe/build/v4.15-56-gb...
No regressions (compared to build )
Boards, architectures and test suites:
hi6220-hikey - arm64
- boot - pass: 38,
- kselftest - pass: 98, skip: 14, fail: 12
- libhugetlbfs - pass: 180, skip: 2,
- ltp-cap_bounds-tests - pass: 4,
- ltp-containers-tests - pass: 128,
- ltp-fcntl-locktests-tests - pass: 4,
- ltp-filecaps-tests - pass: 4,
- ltp-fs-tests - pass: 120,
- ltp-fs_bind-tests - pass: 4,
- ltp-fs_perms_simple-tests - pass: 38,
- ltp-fsx-tests - pass: 4,
- ltp-hugetlb-tests - pass: 21, skip: 1,
- ltp-io-tests - pass: 6,
- ltp-ipc-tests - pass: 18,
- ltp-math-tests - pass: 22,
- ltp-nptl-tests - pass: 4,
- ltp-pty-tests - pass: 4,
- ltp-sched-tests - pass: 20,
- ltp-securebits-tests - pass: 8,
- ltp-syscalls-tests - pass: 1968, skip: 242,
- ltp-timers-tests - pass: 24,
juno-r2 - arm64
- boot - pass: 31,
- kselftest - pass: 111, skip: 28, fail: 12
- libhugetlbfs - pass: 90, skip: 1,
- ltp-cap_bounds-tests - pass: 4,
- ltp-containers-tests - pass: 128,
- ltp-fcntl-locktests-tests - pass: 2,
- ltp-filecaps-tests - pass: 4,
- ltp-fs-tests - pass: 120,
- ltp-fs_bind-tests - pass: 2,
- ltp-fs_perms_simple-tests - pass: 38,
- ltp-fsx-tests - pass: 2,
- ltp-hugetlb-tests - pass: 44,
- ltp-io-tests - pass: 3,
- ltp-ipc-tests - pass: 18,
- ltp-math-tests - pass: 11,
- ltp-nptl-tests - pass: 4,
- ltp-pty-tests - pass: 4,
- ltp-sched-tests - pass: 10,
- ltp-securebits-tests - pass: 4,
- ltp-syscalls-tests - pass: 987, skip: 121,
- ltp-timers-tests - pass: 24,
x15 - arm
- boot - pass: 41,
- kselftest - pass: 92, skip: 32, fail: 15
- libhugetlbfs - pass: 174, skip: 2,
- ltp-cap_bounds-tests - pass: 4,
- ltp-containers-tests - pass: 124, fail: 4
- ltp-fcntl-locktests-tests - pass: 4,
- ltp-filecaps-tests - pass: 4,
- ltp-fs-tests - pass: 120,
- ltp-fs_bind-tests - pass: 4,
- ltp-fs_perms_simple-tests - pass: 38,
- ltp-fsx-tests - pass: 4,
- ltp-hugetlb-tests - pass: 40, skip: 4,
- ltp-io-tests - pass: 6,
- ltp-ipc-tests - pass: 18,
- ltp-math-tests - pass: 22,
- ltp-nptl-tests - pass: 4,
- ltp-pty-tests - pass: 8,
- ltp-sched-tests - pass: 26, skip: 2,
- ltp-securebits-tests - pass: 8,
- ltp-syscalls-tests - pass: 2076, skip: 132,
- ltp-timers-tests - pass: 24,
x86_64
- boot - pass: 40,
- kselftest - pass: 121, skip: 16, fail: 14
- libhugetlbfs - pass: 180, skip: 2,
- ltp-cap_bounds-tests - pass: 4,
- ltp-containers-tests - pass: 128,
- ltp-fcntl-locktests-tests - pass: 4,
- ltp-filecaps-tests - pass: 4,
- ltp-fs-tests - pass: 122, skip: 2,
- ltp-fs_bind-tests - pass: 4,
- ltp-fs_perms_simple-tests - pass: 38,
- ltp-fsx-tests - pass: 4,
- ltp-hugetlb-tests - pass: 44,
- ltp-io-tests - pass: 6,
- ltp-ipc-tests - pass: 18,
- ltp-math-tests - pass: 22,
- ltp-nptl-tests - pass: 4,
- ltp-pty-tests - pass: 8,
- ltp-sched-tests - pass: 18, skip: 2,
- ltp-securebits-tests - pass: 8,
- ltp-syscalls-tests - pass: 2032, skip: 232,
- ltp-timers-tests - pass: 24,
-- Linaro QA (beta) https://qa-reports.linaro.org _______________________________________________ Lkft-triage mailing list Lkft-triage@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lkft-triage