Summary ------------------------------------------------------------------------
kernel: 4.20.0-rc5 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git git branch: master git commit: 4eaaa2b99e30305f4bc677f4abfe56c1f8b39670 git describe: v4.20-rc5-47-g4eaaa2b99e30 Test details: https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v4.20-rc5-47-g4ea...
Regressions (compared to build v4.20-rc5-44-gd08970904582) ------------------------------------------------------------------------
qemu_i386: ltp-cve-tests: * cve-2017-17053
* test src: git://github.com/linux-test-project/ltp.git
Fixes (compared to build v4.20-rc5-44-gd08970904582) ------------------------------------------------------------------------
dragonboard-410c - arm64: kselftest: * net_pmtu.sh
* test src: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
qemu_x86_64: kselftest: * bpf_test_sockmap
* test src: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ltp-fs-tests: * read_all_sys
* test src: git://github.com/linux-test-project/ltp.git
x86_64: kselftest-vsyscall-mode-none: * bpf_test_sockmap
* test src: not informed
Boards, architectures and test suites: -------------------------------------
dragonboard-410c - arm64 * boot - pass: 21, * install-android-platform-tools-r2600 - pass: 42, * kselftest - fail: 2, pass: 94, xfail: 24, skip: 13 * libhugetlbfs - pass: 89, xfail: 1, skip: 1 * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 80, * ltp-cve-tests - pass: 26, skip: 10 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 59, skip: 6 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 18, skip: 2 * ltp-io-tests - pass: 2, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 13, * ltp-securebits-tests - pass: 3, * ltp-syscalls-tests - pass: 1027, xfail: 4, skip: 134 * ltp-timers-tests - pass: 12, * spectre-meltdown-checker-test - pass: 8,
i386 * boot - pass: 22, * kselftest - fail: 6, pass: 35, xfail: 21, skip: 7 * libhugetlbfs - pass: 1, * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 80, * ltp-cve-tests - fail: 1, pass: 29, xfail: 1, skip: 5 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 59, skip: 6 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 19, skip: 1 * ltp-io-tests - pass: 1, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-open-posix-tests - pass: 1690, xfail: 4, skip: 41 * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 9, skip: 4 * ltp-securebits-tests - pass: 2, * ltp-syscalls-tests - pass: 1099, xfail: 3, skip: 63 * ltp-timers-tests - pass: 11, * spectre-meltdown-checker-test - fail: 5, pass: 3,
juno-r2 - arm64 * boot - pass: 21, * libhugetlbfs - pass: 90, skip: 1 * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 80, * ltp-cve-tests - pass: 26, skip: 10 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 59, skip: 6 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 21, * ltp-io-tests - pass: 2, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-open-posix-tests - pass: 1689, xfail: 4, skip: 42 * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 9, skip: 4 * ltp-securebits-tests - pass: 3, * ltp-timers-tests - pass: 12, * spectre-meltdown-checker-test - fail: 2, pass: 5, skip: 1
qemu_arm * boot - pass: 21, * kselftest - fail: 5, pass: 63, xfail: 30, skip: 13 * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 79, skip: 1 * ltp-cve-tests - pass: 23, skip: 13 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 60, skip: 5 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 20, skip: 1 * ltp-io-tests - pass: 2, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 7, skip: 6 * ltp-securebits-tests - pass: 3, * ltp-syscalls-tests - pass: 1059, xfail: 1, skip: 105 * ltp-timers-tests - pass: 12, * spectre-meltdown-checker-test - fail: 2, pass: 6,
qemu_arm64 * boot - pass: 21, * kselftest - fail: 2, pass: 50, xfail: 19, skip: 13 * libhugetlbfs - pass: 90, skip: 1 * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 80, * ltp-cve-tests - pass: 25, skip: 11 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 59, skip: 6 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 21, * ltp-io-tests - pass: 2, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 7, skip: 6 * ltp-securebits-tests - pass: 3, * ltp-syscalls-tests - pass: 1002, xfail: 3, skip: 160 * ltp-timers-tests - pass: 12, * spectre-meltdown-checker-test - fail: 2, pass: 5, skip: 1
qemu_i386 * boot - pass: 21, * kselftest - fail: 5, pass: 111, xfail: 32, skip: 11 * libhugetlbfs - pass: 87, skip: 1 * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 80, * ltp-cve-tests - fail: 1, pass: 29, xfail: 1, skip: 5 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 59, skip: 6 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 20, skip: 1 * ltp-io-tests - pass: 2, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 13, * ltp-securebits-tests - pass: 3, * ltp-syscalls-tests - pass: 1066, xfail: 2, skip: 97 * ltp-timers-tests - pass: 12, * spectre-meltdown-checker-test - pass: 8,
qemu_x86_64 * boot - pass: 21, * kselftest - fail: 2, pass: 99, xfail: 15, skip: 19 * libhugetlbfs - pass: 90, skip: 1 * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 80, * ltp-cve-tests - pass: 30, skip: 6 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 59, skip: 6 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 21, * ltp-io-tests - pass: 2, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 13, * ltp-securebits-tests - pass: 3, * ltp-syscalls-tests - pass: 1002, xfail: 2, skip: 161 * ltp-timers-tests - pass: 12, * spectre-meltdown-checker-test - pass: 8,
x15 - arm * boot - pass: 22, * install-android-platform-tools-r2600 - pass: 44, * kselftest - fail: 5, pass: 84, xfail: 31, skip: 12 * libhugetlbfs - pass: 87, skip: 1 * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 79, skip: 1 * ltp-cve-tests - pass: 25, skip: 11 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 60, skip: 5 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 19, skip: 2 * ltp-io-tests - pass: 2, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-open-posix-tests - pass: 1690, xfail: 4, skip: 41 * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 13, * ltp-securebits-tests - pass: 3, * ltp-syscalls-tests - pass: 1083, xfail: 1, skip: 81 * ltp-timers-tests - pass: 12, * spectre-meltdown-checker-test - fail: 2, pass: 6,
x86_64 * boot - pass: 24, * kselftest - fail: 2, pass: 115, xfail: 17, skip: 16 * kselftest-vsyscall-mode-native - fail: 4, pass: 116, xfail: 15, skip: 17 * kselftest-vsyscall-mode-none - fail: 4, pass: 115, xfail: 13, skip: 17 * libhugetlbfs - pass: 90, skip: 1 * ltp-cap_bounds-tests - pass: 1, * ltp-containers-tests - pass: 80, * ltp-cve-tests - pass: 30, skip: 6 * ltp-fcntl-locktests-tests - pass: 1, * ltp-filecaps-tests - pass: 1, * ltp-fs-tests - pass: 60, skip: 5 * ltp-fs_bind-tests - pass: 1, * ltp-fs_perms_simple-tests - pass: 18, * ltp-fsx-tests - pass: 1, * ltp-hugetlb-tests - pass: 21, * ltp-io-tests - pass: 2, * ltp-ipc-tests - pass: 8, * ltp-math-tests - pass: 10, * ltp-nptl-tests - pass: 1, * ltp-open-posix-tests - pass: 1688, xfail: 4, skip: 42 * ltp-pty-tests - pass: 4, * ltp-sched-tests - pass: 9, skip: 4 * ltp-securebits-tests - pass: 3, * ltp-syscalls-tests - pass: 1041, xfail: 3, skip: 121 * ltp-timers-tests - pass: 12, * spectre-meltdown-checker-test - fail: 3, pass: 5,
On 12/6/18 8:05 AM, Linaro QA (BETA) wrote:
Summary
kernel: 4.20.0-rc5 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git git branch: master git commit: 4eaaa2b99e30305f4bc677f4abfe56c1f8b39670 git describe: v4.20-rc5-47-g4eaaa2b99e30 Test details: https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v4.20-rc5-47-g4ea...
Regressions (compared to build v4.20-rc5-44-gd08970904582)
qemu_i386: ltp-cve-tests: * cve-2017-17053
This is failing for i386 in all tests.
i386
...
- spectre-meltdown-checker-test - fail: 5, pass: 3,
Are we testing the CVEs together with spectre-meltdown-checker-test already ?
We will have to check spectre-meltdown-checker-test results every time there is a cve-XXXX-XXXXX (related to either meltdown or spectre) test failure.. in order to know if it was a false positive.
The way I'm reading this, those:
CVE-2017-5753: OK CVE-2017-5715: OK CVE-2017-5754: VULN CVE-2018-3640: VULN CVE-2018-3639: VULN CVE-2018-3615: VULN CVE-2018-3620: VULN CVE-2018-3646: OK
are only coming from spectre-meltdown-checker, right ?
Like here:
https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v4.20-rc5-47-g4ea...
I thought we would run spectre-meltdown-checker simultaneously with ltp-cve-tests.. Are we splitting the ltp-cve-tests into ltp-cve-tests and ltp-cve-tests + spectre-meltdown-checker (this last having only spectre and meltdown cve related tests) ?
Did I miss something ?
Thanks!
On Thu, 6 Dec 2018 at 17:40, Rafael David Tinoco rafael.tinoco@linaro.org wrote:
On 12/6/18 8:05 AM, Linaro QA (BETA) wrote:
Summary
kernel: 4.20.0-rc5 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git git branch: master git commit: 4eaaa2b99e30305f4bc677f4abfe56c1f8b39670 git describe: v4.20-rc5-47-g4eaaa2b99e30 Test details: https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v4.20-rc5-47-g4ea...
Regressions (compared to build v4.20-rc5-44-gd08970904582)
qemu_i386: ltp-cve-tests: * cve-2017-17053
This is failing for i386 in all tests.
i386
...
- spectre-meltdown-checker-test - fail: 5, pass: 3,
Are we testing the CVEs together with spectre-meltdown-checker-test already ?
In sequence Boot device run LTP CVE tests collect results run Spectre meltdown checker collect results exit.
We will have to check spectre-meltdown-checker-test results every time there is a cve-XXXX-XXXXX (related to either meltdown or spectre) test failure.. in order to know if it was a false positive.
That is good plan. LTP CVE have 36 test case and SMC have 8 test case in total and there is only one single common test case CVE-2017-5754.
The way I'm reading this, those:
CVE-2017-5753: OK CVE-2017-5715: OK CVE-2017-5754: VULN CVE-2018-3640: VULN CVE-2018-3639: VULN CVE-2018-3615: VULN CVE-2018-3620: VULN CVE-2018-3646: OK
are only coming from spectre-meltdown-checker, right ?
Right !
Like here:
https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v4.20-rc5-47-g4ea...
I thought we would run spectre-meltdown-checker simultaneously with ltp-cve-tests.. Are we splitting the ltp-cve-tests into ltp-cve-tests and ltp-cve-tests + spectre-meltdown-checker (this last having only spectre and meltdown cve related tests) ?
Did I miss something ?
LTP CVE we have from past. We have just appended spectre meltdown checker test in the sequence of run like as i said the sequence above.
- Naresh
Thanks!
-- Rafael D. Tinoco Linaro - Kernel Validation
On Thu, Dec 06, 2018 at 10:10:11AM -0200, Rafael David Tinoco wrote:
On 12/6/18 8:05 AM, Linaro QA (BETA) wrote:
Summary
kernel: 4.20.0-rc5 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git git branch: master git commit: 4eaaa2b99e30305f4bc677f4abfe56c1f8b39670 git describe: v4.20-rc5-47-g4eaaa2b99e30 Test details: https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v4.20-rc5-47-g4ea...
Regressions (compared to build v4.20-rc5-44-gd08970904582)
qemu_i386: ltp-cve-tests: * cve-2017-17053
This is failing for i386 in all tests.
i386
...
- spectre-meltdown-checker-test - fail: 5, pass: 3,
Are we testing the CVEs together with spectre-meltdown-checker-test already ?
We will have to check spectre-meltdown-checker-test results every time there is a cve-XXXX-XXXXX (related to either meltdown or spectre) test failure.. in order to know if it was a false positive.
The way I'm reading this, those:
CVE-2017-5753: OK CVE-2017-5715: OK CVE-2017-5754: VULN CVE-2018-3640: VULN CVE-2018-3639: VULN CVE-2018-3615: VULN CVE-2018-3620: VULN CVE-2018-3646: OK
are only coming from spectre-meltdown-checker, right ?
Like here:
https://qa-reports.linaro.org/lkft/linux-mainline-oe/build/v4.20-rc5-47-g4ea...
I thought we would run spectre-meltdown-checker simultaneously with ltp-cve-tests.. Are we splitting the ltp-cve-tests into ltp-cve-tests and ltp-cve-tests + spectre-meltdown-checker (this last having only spectre and meltdown cve related tests) ?
Did I miss something ?
ltp-cve-tests and spectre-meltdown-checker run in the same lava job (for example https://lkft.validation.linaro.org/scheduler/job/537010 to use your example), but they report as separate test suites in qa-reports.
Does that help?
Thanks!
-- Rafael D. Tinoco Linaro - Kernel Validation
On 12/6/18 12:05 PM, Dan Rue wrote:
ltp-cve-tests and spectre-meltdown-checker run in the same lava job (for example https://lkft.validation.linaro.org/scheduler/job/537010 to use your example), but they report as separate test suites in qa-reports.
Does that help?
Yes =) That was my doubt. Thanks!
-
Dan Rue -
Linaro QA -
Naresh Kamboju -
Rafael David Tinoco