Summary ------------------------------------------------------------------------ kernel: 5.1.0 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git git branch: master git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556 git describe: next-20190510 Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
Regressions (compared to build next-20190509) ------------------------------------------------------------------------ No regressions
Fixes (compared to build next-20190509) ------------------------------------------------------------------------ No fixes
In total: ------------------------------------------------------------------------ Ran 12 total tests in the following environments and test suites. pass 11 fail 1 xfail 0 skip 0
Environments -------------- - dragonboard-410c - arm64 - hi6220-hikey - arm64 - i386 - juno-r2 - arm64 - qemu_arm - qemu_arm64 - qemu_i386 - qemu_x86_64 - x15 - arm - x86_64
Test Suites ----------- * boot-lkft-kselftests-master-519 * boot-lkft-kselftests-vsyscall-mode-native-master-519 * boot-lkft-kselftests-vsyscall-mode-none-master-519
Failures ------------------------------------------------------------------------
qemu_x86_64:
qemu_arm64:
qemu_i386:
x15:
qemu_arm:
x86:
i386:
hi6220-hikey:
juno-r2:
dragonboard-410c: * boot-lkft-kselftests-master-519/dragonboard-410c
Skips ------------------------------------------------------------------------ No skips
On Sat, May 11, 2019 at 10:14:57AM +0000, ci_notify@linaro.org wrote:
Summary
kernel: 5.1.0 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git git branch: master git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556 git describe: next-20190510 Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
Regressions (compared to build next-20190509)
No regressions Fixes (compared to build next-20190509)
No fixes
In total:
Ran 12 total tests in the following environments and test suites. pass 11 fail 1 xfail 0 skip 0
Environments
- dragonboard-410c - arm64
- hi6220-hikey - arm64
- i386
- juno-r2 - arm64
- qemu_arm
- qemu_arm64
- qemu_i386
- qemu_x86_64
- x15 - arm
- x86_64
Test Suites
- boot-lkft-kselftests-master-519
What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
$ wget https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510/testrun... $ grep '^ok ' log | wc -l 60 $ grep '^not ok' log | grep -iv '# SKIP' | wc -l 30 $ grep -i '^not ok.*# SKIP' log | wc -l 18
60 pass, 30 fail, 18 skip
On Sat, 11 May 2019 at 16:51, Kees Cook keescook@chromium.org wrote:
On Sat, May 11, 2019 at 10:14:57AM +0000, ci_notify@linaro.org wrote:
Summary
kernel: 5.1.0 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git git branch: master git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556 git describe: next-20190510 Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
Regressions (compared to build next-20190509)
No regressions
Fixes (compared to build next-20190509)
No fixes
In total:
Ran 12 total tests in the following environments and test suites. pass 11 fail 1 xfail 0 skip 0
Environments
- dragonboard-410c - arm64
- hi6220-hikey - arm64
- i386
- juno-r2 - arm64
- qemu_arm
- qemu_arm64
- qemu_i386
- qemu_x86_64
- x15 - arm
- x86_64
Test Suites
- boot-lkft-kselftests-master-519
What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
The boot is passed when the auto-login-action in suite 'lava' is passed [1].
There's is two issues: 1. We had a way that changed the run_kselftest.sh file in order to skip tests that hanged (that results in no test output from the run) various boards. Since the new change to kselftest, the patch to run_kselftest.sh got added that breaks our way to skip the tests and kernel hangs. 2. The way we parse the output doesn't match the new output format.
We will turn off the "next-* kselftest restults" until we have resolved this issue.
Cheers, Anders
$ wget https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510/testrun... $ grep '^ok ' log | wc -l 60 $ grep '^not ok' log | grep -iv '# SKIP' | wc -l 30 $ grep -i '^not ok.*# SKIP' log | wc -l 18
60 pass, 30 fail, 18 skip
[1] https://lkft.validation.linaro.org/results/testcase/141157606
On Tue, May 14, 2019 at 05:28:45PM +0200, Anders Roxell wrote:
On Sat, 11 May 2019 at 16:51, Kees Cook keescook@chromium.org wrote:
Test Suites
- boot-lkft-kselftests-master-519
What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
The boot is passed when the auto-login-action in suite 'lava' is passed [1].
There's is two issues:
- We had a way that changed the run_kselftest.sh file in order to
skip tests that hanged (that results in no test output from the run) various boards. Since the new change to kselftest, the patch to run_kselftest.sh got added that breaks our way to skip the tests and kernel hangs.
There's a fix in -next that adds an "alarm" check now so no tests should ever hang for more than 30 seconds now: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/co...
- The way we parse the output doesn't match the new output format.
We will turn off the "next-* kselftest restults" until we have resolved this issue.
I'd prefer to keep the tests running -- they're still valuable logs to have. I was just curious about the specific reporting. As long as the issue is known, that's fine. No need to disable it entirely. :)
-Kees
On Tue, 14 May 2019 at 17:46, Kees Cook keescook@chromium.org wrote:
On Tue, May 14, 2019 at 05:28:45PM +0200, Anders Roxell wrote:
On Sat, 11 May 2019 at 16:51, Kees Cook keescook@chromium.org wrote:
Test Suites
- boot-lkft-kselftests-master-519
What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
The boot is passed when the auto-login-action in suite 'lava' is passed [1].
There's is two issues:
- We had a way that changed the run_kselftest.sh file in order to
skip tests that hanged (that results in no test output from the run) various boards. Since the new change to kselftest, the patch to run_kselftest.sh got added that breaks our way to skip the tests and kernel hangs.
aha, right that's awesome. So its only scripts that we have to skip now then.
There's a fix in -next that adds an "alarm" check now so no tests should ever hang for more than 30 seconds now: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/co...
- The way we parse the output doesn't match the new output format.
We will turn off the "next-* kselftest restults" until we have resolved this issue.
I'd prefer to keep the tests running -- they're still valuable logs to have. I was just curious about the specific reporting. As long as the issue is known, that's fine. No need to disable it entirely. :)
We will still run the tests on next, the plan is just to stop sending reports until we have the parsing fixed.
Cheers, Anders
-- Kees Cook
-
Anders Roxell -
ci_notify＠linaro.org -
Kees Cook