- Linux-kselftest-mirror - lists.linaro.org

[PATCH AUTOSEL 5.14 171/252] selftests/bpf: Fix flaky send_signal test

by Sasha Levin

From: Yonghong Song <yhs(a)fb.com> [ Upstream commit b16ac5bf732a5e23d164cf908ec7742d6a6120d3 ] libbpf CI has reported send_signal test is flaky although I am not able to reproduce it in my local environment. But I am able to reproduce with on-demand libbpf CI ([1]). Through code analysis, the following is possible reason. The failed subtest runs bpf program in softirq environment. Since bpf_send_signal() only sends to a fork of "test_progs" process. If the underlying current task is not "test_progs", bpf_send_signal() will not be triggered and the subtest will fail. To reduce the chances where the underlying process is not the intended one, this patch boosted scheduling priority to -20 (highest allowed by setpriority() call). And I did 10 runs with on-demand libbpf CI with this patch and I didn't observe any failures. [1] https://github.com/libbpf/libbpf/actions/workflows/ondemand.yml Signed-off-by: Yonghong Song <yhs(a)fb.com> Signed-off-by: Andrii Nakryiko <andrii(a)kernel.org> Link: https://lore.kernel.org/bpf/20210817190923.3186725-1-yhs@fb.com Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- .../selftests/bpf/prog_tests/send_signal.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/send_signal.c b/tools/testing/selftests/bpf/prog_tests/send_signal.c index 023cc532992d..839f7ddaec16 100644 --- a/tools/testing/selftests/bpf/prog_tests/send_signal.c +++ b/tools/testing/selftests/bpf/prog_tests/send_signal.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 #include <test_progs.h> +#include <sys/time.h> +#include <sys/resource.h> #include "test_send_signal_kern.skel.h" int sigusr1_received = 0; @@ -41,12 +43,23 @@ static void test_send_signal_common(struct perf_event_attr *attr, } if (pid == 0) { + int old_prio; + /* install signal handler and notify parent */ signal(SIGUSR1, sigusr1_handler); close(pipe_c2p[0]); /* close read */ close(pipe_p2c[1]); /* close write */ + /* boost with a high priority so we got a higher chance + * that if an interrupt happens, the underlying task + * is this process. + */ + errno = 0; + old_prio = getpriority(PRIO_PROCESS, 0); + ASSERT_OK(errno, "getpriority"); + ASSERT_OK(setpriority(PRIO_PROCESS, 0, -20), "setpriority"); + /* notify parent signal handler is installed */ CHECK(write(pipe_c2p[1], buf, 1) != 1, "pipe_write", "err %d\n", -errno); @@ -62,6 +75,9 @@ static void test_send_signal_common(struct perf_event_attr *attr, /* wait for parent notification and exit */ CHECK(read(pipe_p2c[0], buf, 1) != 1, "pipe_read", "err %d\n", -errno); + /* restore the old priority */ + ASSERT_OK(setpriority(PRIO_PROCESS, 0, old_prio), "setpriority"); + close(pipe_c2p[1]); close(pipe_p2c[0]); exit(0); -- 2.30.2

4 years

1
0
0 0

[PATCH AUTOSEL 5.14 170/252] selftests/bpf: Correctly display subtest skip status

by Sasha Levin

From: Yucong Sun <fallentree(a)fb.com> [ Upstream commit f667d1d66760fcb27aee6c9964eefde39a464afe ] In skip_account(), test->skip_cnt is set to 0 at the end, this makes next print statement never display SKIP status for the subtest. This patch moves the accounting logic after the print statement, fixing the issue. This patch also added SKIP status display for normal tests. Signed-off-by: Yucong Sun <fallentree(a)fb.com> Signed-off-by: Andrii Nakryiko <andrii(a)kernel.org> Link: https://lore.kernel.org/bpf/20210817044732.3263066-3-fallentree@fb.com Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/bpf/test_progs.c | 25 ++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index 6f103106a39b..bfbf2277b61a 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -148,18 +148,18 @@ void test__end_subtest() struct prog_test_def *test = env.test; int sub_error_cnt = test->error_cnt - test->old_error_cnt; - if (sub_error_cnt) - env.fail_cnt++; - else if (test->skip_cnt == 0) - env.sub_succ_cnt++; - skip_account(); - dump_test_log(test, sub_error_cnt); fprintf(env.stdout, "#%d/%d %s:%s\n", test->test_num, test->subtest_num, test->subtest_name, sub_error_cnt ? "FAIL" : (test->skip_cnt ? "SKIP" : "OK")); + if (sub_error_cnt) + env.fail_cnt++; + else if (test->skip_cnt == 0) + env.sub_succ_cnt++; + skip_account(); + free(test->subtest_name); test->subtest_name = NULL; } @@ -786,17 +786,18 @@ int main(int argc, char **argv) test__end_subtest(); test->tested = true; - if (test->error_cnt) - env.fail_cnt++; - else - env.succ_cnt++; - skip_account(); dump_test_log(test, test->error_cnt); fprintf(env.stdout, "#%d %s:%s\n", test->test_num, test->test_name, - test->error_cnt ? "FAIL" : "OK"); + test->error_cnt ? "FAIL" : (test->skip_cnt ? "SKIP" : "OK")); + + if (test->error_cnt) + env.fail_cnt++; + else + env.succ_cnt++; + skip_account(); reset_affinity(); restore_netns(); -- 2.30.2

4 years

1
0
0 0

[PATCH AUTOSEL 5.14 137/252] selftests/bpf: Fix xdp_tx.c prog section name

by Sasha Levin

From: Jussi Maki <joamaki(a)gmail.com> [ Upstream commit 95413846cca37f20000dd095cf6d91f8777129d7 ] The program type cannot be deduced from 'tx' which causes an invalid argument error when trying to load xdp_tx.o using the skeleton. Rename the section name to "xdp" so that libbpf can deduce the type. Signed-off-by: Jussi Maki <joamaki(a)gmail.com> Signed-off-by: Daniel Borkmann <daniel(a)iogearbox.net> Acked-by: Andrii Nakryiko <andrii(a)kernel.org> Link: https://lore.kernel.org/bpf/20210731055738.16820-7-joamaki@gmail.com Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/bpf/progs/xdp_tx.c | 2 +- tools/testing/selftests/bpf/test_xdp_veth.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/xdp_tx.c b/tools/testing/selftests/bpf/progs/xdp_tx.c index 94e6c2b281cb..5f725c720e00 100644 --- a/tools/testing/selftests/bpf/progs/xdp_tx.c +++ b/tools/testing/selftests/bpf/progs/xdp_tx.c @@ -3,7 +3,7 @@ #include <linux/bpf.h> #include <bpf/bpf_helpers.h> -SEC("tx") +SEC("xdp") int xdp_tx(struct xdp_md *xdp) { return XDP_TX; diff --git a/tools/testing/selftests/bpf/test_xdp_veth.sh b/tools/testing/selftests/bpf/test_xdp_veth.sh index ba8ffcdaac30..995278e684b6 100755 --- a/tools/testing/selftests/bpf/test_xdp_veth.sh +++ b/tools/testing/selftests/bpf/test_xdp_veth.sh @@ -108,7 +108,7 @@ ip link set dev veth2 xdp pinned $BPF_DIR/progs/redirect_map_1 ip link set dev veth3 xdp pinned $BPF_DIR/progs/redirect_map_2 ip -n ns1 link set dev veth11 xdp obj xdp_dummy.o sec xdp_dummy -ip -n ns2 link set dev veth22 xdp obj xdp_tx.o sec tx +ip -n ns2 link set dev veth22 xdp obj xdp_tx.o sec xdp ip -n ns3 link set dev veth33 xdp obj xdp_dummy.o sec xdp_dummy trap cleanup EXIT -- 2.30.2

4 years

1
0
0 0

[PATCH AUTOSEL 5.14 051/252] selftests: firmware: Fix ignored return val of asprintf() warn

by Sasha Levin

From: Shuah Khan <skhan(a)linuxfoundation.org> [ Upstream commit fe968ca2cac91888310b143a483123c84906e3fc ] Fix the following ingonred return val of asprintf() warn during build: cc -Wall -O2 fw_namespace.c -o ../tools/testing/selftests/firmware/fw_namespace fw_namespace.c: In function ‘main’: fw_namespace.c:132:2: warning: ignoring return value of ‘asprintf’ declared with attribute ‘warn_unused_result’ [-Wunused-result] 132 | asprintf(&fw_path, "/lib/firmware/%s", fw_name); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Link: https://lore.kernel.org/r/20210708031827.51293-1-skhan@linuxfoundation.org Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/firmware/fw_namespace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/firmware/fw_namespace.c b/tools/testing/selftests/firmware/fw_namespace.c index 0e393cb5f42d..4c6f0cd83c5b 100644 --- a/tools/testing/selftests/firmware/fw_namespace.c +++ b/tools/testing/selftests/firmware/fw_namespace.c @@ -129,7 +129,8 @@ int main(int argc, char **argv) die("mounting tmpfs to /lib/firmware failed\n"); sys_path = argv[1]; - asprintf(&fw_path, "/lib/firmware/%s", fw_name); + if (asprintf(&fw_path, "/lib/firmware/%s", fw_name) < 0) + die("error: failed to build full fw_path\n"); setup_fw(fw_path); -- 2.30.2

4 years

1
0
0 0

[REGRESSION] lkft kselftest for next-20210908

by lkft＠linaro.org

## Build * kernel: 5.14.0 * git: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next * git branch: master * git commit: 999569d59a0aa2509ae4a67ecc266c1134e37e7b * git describe: next-20210908 * test details: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210908 ## Regressions (compared to next-20210907) * x15, kselftest-rtc - rtc.rtctest Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Fixes (compared to next-20210907) * hi6220-hikey, kselftest-rtc - rtc.rtctest * qemu_arm64, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace ## Test result summary total: 6126, pass: 3608, fail: 597, skip: 1921, xfail: 0 ## Build Summary ## Test suites summary * kselftest- * kselftest-android * kselftest-arm64 * kselftest-arm64/arm64.btitest.bti_c_func * kselftest-arm64/arm64.btitest.bti_j_func * kselftest-arm64/arm64.btitest.bti_jc_func * kselftest-arm64/arm64.btitest.bti_none_func * kselftest-arm64/arm64.btitest.nohint_func * kselftest-arm64/arm64.btitest.paciasp_func * kselftest-arm64/arm64.nobtitest.bti_c_func * kselftest-arm64/arm64.nobtitest.bti_j_func * kselftest-arm64/arm64.nobtitest.bti_jc_func * kselftest-arm64/arm64.nobtitest.bti_none_func * kselftest-arm64/arm64.nobtitest.nohint_func * kselftest-arm64/arm64.nobtitest.paciasp_func * kselftest-bpf * kselftest-breakpoints * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-drivers * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kexec * kselftest-kvm * kselftest-lib * kselftest-livepatch * kselftest-lkdtm * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-tc-testing * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-vm * kselftest-x86 * kselftest-zram -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

[PATCH] kunit: fix reference count leak in kfree_at_end

by Xiyu Yang

The reference counting issue happens in the normal path of kfree_at_end(). When kunit_alloc_and_get_resource() is invoked, the function forgets to handle the returned resource object, whose refcount increased inside, causing a refcount leak. Fix this issue by calling kunit_put_resource() at the end of function. Signed-off-by: Xiyu Yang <xiyuyang19(a)fudan.edu.cn> Signed-off-by: Xin Tan <tanxin.ctf(a)gmail.com> --- lib/kunit/executor_test.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c index cdbe54b16501..3af30abad826 100644 --- a/lib/kunit/executor_test.c +++ b/lib/kunit/executor_test.c @@ -113,11 +113,13 @@ static void kfree_res_free(struct kunit_resource *res) */ static void kfree_at_end(struct kunit *test, const void *to_free) { + struct kunit_resource *res; /* kfree() handles NULL already, but avoid allocating a no-op cleanup. */ if (IS_ERR_OR_NULL(to_free)) return; - kunit_alloc_and_get_resource(test, NULL, kfree_res_free, GFP_KERNEL, + res = kunit_alloc_and_get_resource(test, NULL, kfree_res_free, GFP_KERNEL, (void *)to_free); + kunit_put_resource(res); } static struct kunit_suite *alloc_fake_suite(struct kunit *test, -- 2.7.4

4 years

2
1
0 0

[PATCH 0/3] kunit: allow running test suites/cases individually

by Daniel Latypov

Allow running each suite or each test case alone per kernel boot. The motivation for this is to debug "test hermeticity" issues. This new --run_isolated flag would be a good first step to try and narrow down root causes. Context: sometimes tests pass/fail depending on what ran before them. Memory corruption errors in particular might only cause noticeable issues later on. But you can also have the opposite, where "fixing" one test causes another to start failing. Usage: $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit --run_isolated=suite $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit --run_isolated=test $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit --run_isolated=test example The last one would provide output like ======== [PASSED] example ======== [PASSED] example_simple_test ============================================================ Testing complete. 1 tests run. 0 failed. 0 crashed. 0 skipped. Starting KUnit Kernel (2/3)... ============================================================ ======== [SKIPPED] example ======== [SKIPPED] example_skip_test # SKIP this test should be skipped ============================================================ Testing complete. 1 tests run. 0 failed. 0 crashed. 1 skipped. Starting KUnit Kernel (3/3)... ============================================================ ======== [SKIPPED] example ======== [SKIPPED] example_mark_skipped_test # SKIP this test should be skipped ============================================================ Testing complete. 1 tests run. 0 failed. 0 crashed. 1 skipped. See the last patch's description for a bit more detail. Meta: The first patch is from another series with just a reworded commit message, https://lore.kernel.org/linux-kselftest/20210805235145.2528054-2-dlatypov@g… This patch series is based on the 2 patches in https://lore.kernel.org/linux-kselftest/20210831171926.3832806-2-dlatypov@g…. (That's what adds support for us to run a single test case by itself). Daniel Latypov (3): kunit: add 'kunit.action' param to allow listing out tests kunit: tool: factor exec + parse steps into a function kunit: tool: support running each suite/test separately lib/kunit/executor.c | 38 +++++++- tools/testing/kunit/kunit.py | 127 +++++++++++++++++-------- tools/testing/kunit/kunit_tool_test.py | 40 ++++++++ 3 files changed, 160 insertions(+), 45 deletions(-) base-commit: 23fdafa5ae209688d5d5253786bab666bdb07b69 -- 2.33.0.309.g3052b89438-goog

4 years

1
3
0 0

[PATCH bpf-next v2] selftests/bpf: Fix build of task_pt_regs test for arm64

by Jean-Philippe Brucker

struct pt_regs is not exported to userspace on all archs. arm64 and s390 export "user_pt_regs" instead, which causes build failure at the moment: progs/test_task_pt_regs.c:8:16: error: variable has incomplete type 'struct pt_regs' struct pt_regs current_regs = {}; Instead of using pt_regs from ptrace.h, use the larger kernel struct from vmlinux.h directly. Since the test runner task_pt_regs.c does not have access to the kernel struct definition, copy it into a char array. Fixes: 576d47bb1a92 ("bpf: selftests: Add bpf_task_pt_regs() selftest") Suggested-by: Andrii Nakryiko <andrii.nakryiko(a)gmail.com> Signed-off-by: Jean-Philippe Brucker <jean-philippe(a)linaro.org> --- v2: Work on struct pt_regs from vmlinux.h v1: https://lore.kernel.org/bpf/20210902090925.2010528-1-jean-philippe@linaro.o… --- .../selftests/bpf/prog_tests/task_pt_regs.c | 1 - .../selftests/bpf/progs/test_task_pt_regs.c | 19 +++++++++++++------ 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/task_pt_regs.c b/tools/testing/selftests/bpf/prog_tests/task_pt_regs.c index 53f0e0fa1a53..37c20b5ffa70 100644 --- a/tools/testing/selftests/bpf/prog_tests/task_pt_regs.c +++ b/tools/testing/selftests/bpf/prog_tests/task_pt_regs.c @@ -1,7 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #define _GNU_SOURCE #include <test_progs.h> -#include <linux/ptrace.h> #include "test_task_pt_regs.skel.h" void test_task_pt_regs(void) diff --git a/tools/testing/selftests/bpf/progs/test_task_pt_regs.c b/tools/testing/selftests/bpf/progs/test_task_pt_regs.c index 6c059f1cfa1b..e6cb09259408 100644 --- a/tools/testing/selftests/bpf/progs/test_task_pt_regs.c +++ b/tools/testing/selftests/bpf/progs/test_task_pt_regs.c @@ -1,12 +1,17 @@ // SPDX-License-Identifier: GPL-2.0 -#include <linux/ptrace.h> -#include <linux/bpf.h> +#include "vmlinux.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> -struct pt_regs current_regs = {}; -struct pt_regs ctx_regs = {}; +#define PT_REGS_SIZE sizeof(struct pt_regs) + +/* + * The kernel struct pt_regs isn't exported in its entirety to userspace. + * Pass it as an array to task_pt_regs.c + */ +char current_regs[PT_REGS_SIZE] = {}; +char ctx_regs[PT_REGS_SIZE] = {}; int uprobe_res = 0; SEC("uprobe/trigger_func") @@ -17,8 +22,10 @@ int handle_uprobe(struct pt_regs *ctx) current = bpf_get_current_task_btf(); regs = (struct pt_regs *) bpf_task_pt_regs(current); - __builtin_memcpy(&current_regs, regs, sizeof(*regs)); - __builtin_memcpy(&ctx_regs, ctx, sizeof(*ctx)); + if (bpf_probe_read_kernel(current_regs, PT_REGS_SIZE, regs)) + return 0; + if (bpf_probe_read_kernel(ctx_regs, PT_REGS_SIZE, ctx)) + return 0; /* Prove that uprobe was run */ uprobe_res = 1; -- 2.33.0

4 years

3
2
0 0

[REGRESSION] lkft kselftest for next-20210907

by lkft＠linaro.org

## Build * kernel: 5.14.0 * git: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next * git branch: master * git commit: b2bb710d34d5965358777d5b3b17764fa7bc2a8a * git describe: next-20210907 * test details: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210907 ## Regressions (compared to next-20210903) * hi6220-hikey, kselftest-rtc - rtc.rtctest * qemu-arm64-gic-version2, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * qemu-arm64-gic-version3, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * qemu_arm, kselftest-rtc - rtc.rtctest * x15, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Fixes (compared to next-20210903) No fixes found. ## Test result summary total: 3480, pass: 1813, fail: 272, skip: 1395, xfail: 0 ## Build Summary ## Test suites summary * kselftest-android * kselftest-arm64 * kselftest-arm64/arm64.btitest.bti_c_func * kselftest-arm64/arm64.btitest.bti_j_func * kselftest-arm64/arm64.btitest.bti_jc_func * kselftest-arm64/arm64.btitest.bti_none_func * kselftest-arm64/arm64.btitest.nohint_func * kselftest-arm64/arm64.btitest.paciasp_func * kselftest-arm64/arm64.nobtitest.bti_c_func * kselftest-arm64/arm64.nobtitest.bti_j_func * kselftest-arm64/arm64.nobtitest.bti_jc_func * kselftest-arm64/arm64.nobtitest.bti_none_func * kselftest-arm64/arm64.nobtitest.nohint_func * kselftest-arm64/arm64.nobtitest.paciasp_func * kselftest-breakpoints * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-drivers * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kvm * kselftest-lib * kselftest-livepatch * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-vm * kselftest-x86 * kselftest-zram -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

Fixing up at least some fallout from '-Werror'

by Linus Torvalds

So I just committed these three fixes: 4b93c544e90e ("thunderbolt: test: split up test cases in tb_test_credit_alloc_all") ba7b1f861086 ("lib/test_scanf: split up number parsing test routines") 1476ff21abb4 ("iwl: fix debug printf format strings") for the fallout from -Werror that I could easily check (mainly i386 'allyesconfig' - a situation I don't normally test). The printk format string one was trivial and I hopefully didn't screw anything up, but I'd ask people to look at and verify the two other ones. I tried to be very careful, and organizing the code movement in such a way that 'git diff' shows that it's doing the same thing before and after, but hey, mistakes happen. I found those two test-based ones somewhat annoying, because they both showed how little the test infrastructure tries to follow kernel rules. I bet those warnings have been showing up for a long long time, and people went "that's not a relevant configuration" or had some other reason to ignore them. No, the test cases may not be relevant in most situations, but it's not a good thing when something that is supposed to verify kernel behavior then violates some very fundamental and core kernel rules. And maybe it was simply missed. The one thing that was clear when I did that thunderbolt thing in particular is how easy it is to create variations of those 'struct some-assertion-struct' things on stack as part of the KUNIT infrastructure. That's unfortunate. It is possible that the solution to the kernel stack usage might have been to make those structures static instead, but I didn't check whether the description structs really can be. It would be even nicer if they were 'static const'. Being fully initialized instead of generating not only code that uses up stack, but also the code to dynamically initialize them on the stack is all kinds of nasty. I took one look at the generated code, and ran away screaming. Anyway, I'm adding the Kunit maintainer and lists here too, just to see if maybe it could be possible to make those 'struct kunit_assert' things and friends be static and const, but at least for the cases that caused problems for i386, those three commits should make the build pass. The test_scanf case didn't actually use the Kunit infrastructure, the stack use explosion is because gcc doesn't seem to combine stack allocations in many situations. I know gcc *sometimes* does that stack allocation combining, but not here. I suspect it might be related to type aliasing, and only merging stack slots when they have the same types, and thus triggered by the different result buffer sizes. Maybe. Linus

4 years

4
3
0 0

[REGRESSION] lkft kselftest for next-20210906

by lkft＠linaro.org

## Build * kernel: 5.14.0 * git: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next * git branch: master * git commit: 7800ca95d0ed11b492962dc3abc2181c9d5f7f82 * git describe: next-20210906 * test details: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210906 ## Regressions (compared to next-20210903) * hi6220-hikey, kselftest-rtc - rtc.rtctest * hi6220-hikey, kselftest-timers - timers.nsleep-lat - timers.set-timer-lat * qemu-arm64-gic-version2, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * qemu-arm64-mte, kselftest-timers - timers.rtcpie * qemu_arm, kselftest-rtc - rtc.rtctest Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Fixes (compared to next-20210903) * qemu-arm64-gic-version2, kselftest-timers - timers.rtcpie * qemu_arm64, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * qemu_arm64, kselftest-timers - timers.rtcpie ## Test result summary total: 3570, pass: 1917, fail: 288, skip: 1365, xfail: 0 ## Build Summary ## Test suites summary * kselftest-android * kselftest-arm64 * kselftest-arm64/arm64.btitest.bti_c_func * kselftest-arm64/arm64.btitest.bti_j_func * kselftest-arm64/arm64.btitest.bti_jc_func * kselftest-arm64/arm64.btitest.bti_none_func * kselftest-arm64/arm64.btitest.nohint_func * kselftest-arm64/arm64.btitest.paciasp_func * kselftest-arm64/arm64.nobtitest.bti_c_func * kselftest-arm64/arm64.nobtitest.bti_j_func * kselftest-arm64/arm64.nobtitest.bti_jc_func * kselftest-arm64/arm64.nobtitest.bti_none_func * kselftest-arm64/arm64.nobtitest.nohint_func * kselftest-arm64/arm64.nobtitest.paciasp_func * kselftest-breakpoints * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-drivers * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kvm * kselftest-lib * kselftest-livepatch * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-vm * kselftest-x86 * kselftest-zram -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

[PATCH 0/6] test_hash.c: refactor into KUnit

by Isabella Basso

We refactored the lib/test_hash.c file into KUnit as part of the student group LKCAMP [1] introductory hackathon for kernel development. This test was pointed to our group by Daniel Latypov [2], so its full conversion into a pure KUnit test was our goal in this patch series, but we ran into many problems relating to it not being split as unit tests, which complicated matters a bit, as the reasoning behind the original tests is quite cryptic for those unfamiliar with hash implementations. Some interesting developments we'd like to highlight are: - In patch 1/6 we noticed that there was an unused define directive that could be removed. - In patch 5/6 we noticed how stringhash and hash tests are all under the lib/test_hash.c file, which might cause some confusion, and we also broke those kernel config entries up. Overall KUnit developments have been made in the other patches in this series: In patches 2/6 through 4/6 and 6/6 we refactored the lib/test_hash.c file so as to make it more compatible with the KUnit style, whilst preserving the original idea of the maintainer who designed it (i.e. George Spelvin), which might be undesirable for unit tests, but we assume it is enough for a first patch. This is our first patch series so we hope our contributions are interesting and also hope to get some useful criticism from the community :) [1] - https://lkcamp.dev/ [2] - https://lore.kernel.org/linux-kselftest/CAGS_qxojszgM19u=3HLwFgKX5bm5Khywvs… Isabella Basso (6): hash.h: remove unused define directive test_hash.c: move common definitions to top of file test_hash.c: split test_int_hash into arch-specific functions test_hash.c: split test_hash_init lib/Kconfig.debug: properly split hash test kernel entries test_hash.c: refactor into kunit include/linux/hash.h | 5 +- lib/Kconfig.debug | 28 ++++- lib/Makefile | 3 +- lib/test_hash.c | 249 ++++++++++++++++--------------------- tools/include/linux/hash.h | 5 +- 5 files changed, 136 insertions(+), 154 deletions(-) -- 2.33.0

4 years

6
17
0 0

proc-pid-vm.c:214:19: warning: 'str_vsyscall' defined but not used

by Naresh Kamboju

Following build warnings noticed while building stable-rc Linux 5.14.1-rc1 with gcc-11 for arm64 architecture. aarch64-linux-gnu-gcc -Wall -O2 -Wno-unused-function -D_GNU_SOURCE proc-pid-vm.c -o kselftest/proc/proc-pid-vm proc-pid-vm.c:214:19: warning: 'str_vsyscall' defined but not used [-Wunused-const-variable=] 214 | static const char str_vsyscall[] = | ^~~~~~~~~~~~ proc-pid-vm.c:212:13: warning: 'g_vsyscall' defined but not used [-Wunused-variable] 212 | static bool g_vsyscall = false; | ^~~~~~~~~~ Build config: https://builds.tuxbuild.com/1xXcUtI2INra8KaHjOXXQMOyAD0/config Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> meta data: ----------- git_describe: v5.14-rc6-389-g95dc72bb9c03, git_repo: https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable-rc, git_sha: 95dc72bb9c032093e79e628a98c927b3db73a6c3, git_short_log: 95dc72bb9c03 (\Linux 5.14.1-rc1\), kconfig: [ defconfig, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, CONFIG_ARM64_MODULE_PLTS=y ], kernel_version: 5.14.1-rc1, target_arch: arm64, targets: [ dtbs, dtbs-legacy, headers, kernel, kselftest, kselftest-merge, modules ], toolchain: gcc-11, steps to reproduce: https://builds.tuxbuild.com/1xXcUtI2INra8KaHjOXXQMOyAD0/tuxmake_reproducer.… -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

udmabuf.c:30:17: warning: implicit declaration of function 'open'

by Naresh Kamboju

Following build warnings noticed while building stable-rc Linux 5.14.1-rc1 with gcc-11 for arm64 architecture. aarch64-linux-gnu-gcc -I../../../../../usr/include/ udmabuf.c -o kselftest/drivers/dma-buf/udmabuf udmabuf.c: In function 'main': udmabuf.c:30:17: warning: implicit declaration of function 'open'; did you mean 'popen'? [-Wimplicit-function-declaration] 30 | devfd = open(/dev/udmabuf, O_RDWR); | ^~~~ | popen udmabuf.c:42:15: warning: implicit declaration of function 'fcntl' [-Wimplicit-function-declaration] 42 | ret = fcntl(memfd, F_ADD_SEALS, F_SEAL_SHRINK); | ^~~~~ make[3]: Leaving directory '/builds/linux/tools/testing/selftests/drivers/dma-buf' Build config: https://builds.tuxbuild.com/1xXcUtI2INra8KaHjOXXQMOyAD0/config Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> meta data: ----------- git_describe: v5.14-rc6-389-g95dc72bb9c03, git_repo: https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable-rc, git_sha: 95dc72bb9c032093e79e628a98c927b3db73a6c3, git_short_log: 95dc72bb9c03 (\Linux 5.14.1-rc1\), kconfig: [ defconfig, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, https://raw.githubusercontent.com/Linaro/meta-lkft/sumo/recipes-kernel/linu…, CONFIG_ARM64_MODULE_PLTS=y ], kernel_version: 5.14.1-rc1, target_arch: arm64, targets: [ dtbs, dtbs-legacy, headers, kernel, kselftest, kselftest-merge, modules ], toolchain: gcc-11, steps to reproduce: https://builds.tuxbuild.com/1xXcUtI2INra8KaHjOXXQMOyAD0/tuxmake_reproducer.… -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

[REGRESSION] lkft kselftest for next-20210903

by lkft＠linaro.org

## Build * kernel: 5.14.0 * git: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next * git branch: master * git commit: 14c6345e6e6ca0570f3897214f24748ef6009761 * git describe: next-20210903 * test details: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210903 ## Regressions (compared to next-20210902) * i386, kselftest-net - net.ip_defrag.sh * i386, kselftest-rtc - rtc.rtctest * qemu_arm64, kselftest-timers - timers.rtcpie * qemu_x86_64, kselftest-rtc - rtc.rtctest Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Fixes (compared to next-20210902) * i386, kselftest-net - net.so_txtime.sh * qemu-arm64-gic-version2, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * qemu-arm64-gic-version3, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * qemu-arm64-mte, kselftest-timers - timers.rtcpie * qemu_arm, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_migrate * qemu_arm, kselftest-rtc - rtc.rtctest * qemu_x86_64, kselftest-timers - timers.rtcpie ## Test result summary total: 5548, pass: 3276, fail: 566, skip: 1706, xfail: 0 ## Build Summary ## Test suites summary * kselftest- * kselftest-android * kselftest-arm64 * kselftest-arm64/arm64.btitest.bti_c_func * kselftest-arm64/arm64.btitest.bti_j_func * kselftest-arm64/arm64.btitest.bti_jc_func * kselftest-arm64/arm64.btitest.bti_none_func * kselftest-arm64/arm64.btitest.nohint_func * kselftest-arm64/arm64.btitest.paciasp_func * kselftest-arm64/arm64.nobtitest.bti_c_func * kselftest-arm64/arm64.nobtitest.bti_j_func * kselftest-arm64/arm64.nobtitest.bti_jc_func * kselftest-arm64/arm64.nobtitest.bti_none_func * kselftest-arm64/arm64.nobtitest.nohint_func * kselftest-arm64/arm64.nobtitest.paciasp_func * kselftest-bpf * kselftest-breakpoints * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-drivers * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kexec * kselftest-kvm * kselftest-lib * kselftest-livepatch * kselftest-lkdtm * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-tc-testing * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-vm * kselftest-x86 * kselftest-zram -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

[GIT PULL] Kselftest update for Linux 5.15-rc1

by Shuah Khan

Hi Linus, Please pull the following Kselftest update for Linux 5.15-rc1. This Kselftest update for Linux 5.15-rc1 consists of fixes to build and test failures. -- openat2 test failure for O_LARGEFILE flag on ARM64 -- x86 test build failures related to glibc 2.34 adding support for variable sized MINSIGSTKSZ and SIGSTKSZ -- removing obsolete configs in sync and cpufreq config files -- minor spelling and duplicate header include cleanups diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit 2734d6c1b1a089fb593ef6a23d4b70903526fe0c: Linux 5.14-rc2 (2021-07-18 14:13:49 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-next-5.15-rc1 for you to fetch changes up to 67d6d80d90fb27b3cc7659f464fa3b87fd67bc14: selftests/cpufreq: Rename DEBUG_PI_LIST to DEBUG_PLIST (2021-08-31 11:00:02 -0600) ---------------------------------------------------------------- linux-kselftest-next-5.15-rc1 This Kselftest update for Linux 5.15-rc1 consists of fixes to build and test failures. -- openat2 test failure for O_LARGEFILE flag on ARM64 -- x86 test build failures related to glibc 2.34 adding support for variable sized MINSIGSTKSZ and SIGSTKSZ -- removing obsolete configs in sync and cpufreq config files -- minor spelling and duplicate header include cleanups ---------------------------------------------------------------- Baolin Wang (1): selftests: openat2: Fix testing failure for O_LARGEFILE flag Changcheng Deng (1): kselftest:sched: remove duplicate include in cs_prctl_test.c Colin Ian King (1): selftests: safesetid: Fix spelling mistake "cant" -> "can't" Jun Miao (1): selftests/x86: Fix error: variably modified 'altstack_data' at file scope Li Zhijian (2): selftests/sync: Remove the deprecated config SYNC selftests/cpufreq: Rename DEBUG_PI_LIST to DEBUG_PLIST tools/testing/selftests/cpufreq/config | 2 +- tools/testing/selftests/openat2/openat2_test.c | 4 ++++ tools/testing/selftests/safesetid/safesetid-test.c | 2 +- tools/testing/selftests/sched/cs_prctl_test.c | 2 -- tools/testing/selftests/sync/config | 1 - tools/testing/selftests/x86/mov_ss_trap.c | 4 ++-- tools/testing/selftests/x86/sigreturn.c | 7 +++---- tools/testing/selftests/x86/single_step_syscall.c | 4 ++-- tools/testing/selftests/x86/syscall_arg_fault.c | 7 +++---- 9 files changed, 16 insertions(+), 17 deletions(-) ----------------------------------------------------------------

4 years

2
1
0 0

[PATCH bpf-next] selftests/bpf: Fix build of task_pt_regs tests for arm64

by Jean-Philippe Brucker

struct pt_regs is not exported to userspace on all archs. arm64 and s390 export "user_pt_regs" instead, which causes build failure at the moment: progs/test_task_pt_regs.c:8:16: error: variable has incomplete type 'struct pt_regs' struct pt_regs current_regs = {}; Use the multi-arch macros defined by tools/lib/bpf/bpf_tracing.h to copy the pt_regs into a locally-defined struct. Copying the user_pt_regs struct on arm64 wouldn't work because the struct is too large and the compiler complains about using too much stack. Fixes: 576d47bb1a92 ("bpf: selftests: Add bpf_task_pt_regs() selftest") Signed-off-by: Jean-Philippe Brucker <jean-philippe(a)linaro.org> --- .../selftests/bpf/bpf_pt_regs_helpers.h | 30 +++++++++++++++++++ .../selftests/bpf/prog_tests/task_pt_regs.c | 1 + .../selftests/bpf/progs/test_task_pt_regs.c | 10 ++++--- 3 files changed, 37 insertions(+), 4 deletions(-) create mode 100644 tools/testing/selftests/bpf/bpf_pt_regs_helpers.h diff --git a/tools/testing/selftests/bpf/bpf_pt_regs_helpers.h b/tools/testing/selftests/bpf/bpf_pt_regs_helpers.h new file mode 100644 index 000000000000..7531f4824ead --- /dev/null +++ b/tools/testing/selftests/bpf/bpf_pt_regs_helpers.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __BPF_PT_REGS_HELPERS +#define __BPF_PT_REGS_HELPERS + +#include <bpf/bpf_tracing.h> + +struct bpf_pt_regs { + unsigned long long parm[5]; + unsigned long long ret; + unsigned long long fp; + unsigned long long rc; + unsigned long long sp; + unsigned long long ip; +}; + +static inline void bpf_copy_pt_regs(struct bpf_pt_regs *dest, struct pt_regs *src) +{ + dest->parm[0] = PT_REGS_PARM1(src); + dest->parm[1] = PT_REGS_PARM2(src); + dest->parm[2] = PT_REGS_PARM3(src); + dest->parm[3] = PT_REGS_PARM4(src); + dest->parm[4] = PT_REGS_PARM5(src); + dest->ret = PT_REGS_RET(src); + dest->fp = PT_REGS_FP(src); + dest->rc = PT_REGS_RC(src); + dest->sp = PT_REGS_SP(src); + dest->ip = PT_REGS_IP(src); +} + +#endif /* __BPF_PT_REGS_HELPERS */ diff --git a/tools/testing/selftests/bpf/prog_tests/task_pt_regs.c b/tools/testing/selftests/bpf/prog_tests/task_pt_regs.c index 53f0e0fa1a53..196453b75937 100644 --- a/tools/testing/selftests/bpf/prog_tests/task_pt_regs.c +++ b/tools/testing/selftests/bpf/prog_tests/task_pt_regs.c @@ -2,6 +2,7 @@ #define _GNU_SOURCE #include <test_progs.h> #include <linux/ptrace.h> +#include "bpf_pt_regs_helpers.h" #include "test_task_pt_regs.skel.h" void test_task_pt_regs(void) diff --git a/tools/testing/selftests/bpf/progs/test_task_pt_regs.c b/tools/testing/selftests/bpf/progs/test_task_pt_regs.c index 6c059f1cfa1b..348da3509093 100644 --- a/tools/testing/selftests/bpf/progs/test_task_pt_regs.c +++ b/tools/testing/selftests/bpf/progs/test_task_pt_regs.c @@ -5,8 +5,10 @@ #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> -struct pt_regs current_regs = {}; -struct pt_regs ctx_regs = {}; +#include "bpf_pt_regs_helpers.h" + +struct bpf_pt_regs current_regs = {}; +struct bpf_pt_regs ctx_regs = {}; int uprobe_res = 0; SEC("uprobe/trigger_func") @@ -17,8 +19,8 @@ int handle_uprobe(struct pt_regs *ctx) current = bpf_get_current_task_btf(); regs = (struct pt_regs *) bpf_task_pt_regs(current); - __builtin_memcpy(&current_regs, regs, sizeof(*regs)); - __builtin_memcpy(&ctx_regs, ctx, sizeof(*ctx)); + bpf_copy_pt_regs(&current_regs, regs); + bpf_copy_pt_regs(&ctx_regs, ctx); /* Prove that uprobe was run */ uprobe_res = 1; -- 2.33.0

4 years

4
4
0 0

[RFCv3 00/15] tcp: Initial support for RFC5925 auth option

by Leonard Crestez

This is similar to TCP MD5 in functionality but it's sufficiently different that wire formats are incompatible. Compared to TCP-MD5 more algorithms are supported and multiple keys can be used on the same connection but there is still no negotiation mechanism. Expected use-case is protecting long-duration BGP/LDP connections between routers using pre-shared keys. This version is mostly functional, it incorporates ABI feedback from previous versions and adds tests to kselftests. More discussion and testing is required and obvious optimizations were skipped in favor of adding functionality. Here are several flaws: * RST and TIMEWAIT are mostly unhandled * Locking is lockdep-clean but need to be revised * Sequence Number Extension not implemented * User is responsible for ensuring keys do not overlap * Traffic key is not cached (reducing performance) Not all ABI suggestions were incorporated, they can be discussed further. However I very much want to avoid supporting algorithms beyond RFC5926. Test suite was added to tools/selftests/tcp_authopt. Tests are written in python using pytest and scapy and check the API in some detail and validate packet captures. Python code is already in linux and in kselftests but virtualenvs not very much. This test suite uses `tox` to create a private virtualenv and hide dependencies. Let me know if this is OK or how it can be improved. Limited testing support is also included in nettest and fcnal-test.sh, those tests are slow and cover much less. Changes for frr: https://github.com/FRRouting/frr/pull/9442 That PR was made early for ABI feedback, it has many issues. Changes for yabgp: https://github.com/cdleonard/yabgp/commits/tcp_authopt The patched version of yabgp can establish a BGP session protected by TCP Authentication Option with a Cisco IOS-XR router. It old now. Changes since RFCv2: * Removed local_id from ABI and match on send_id/recv_id/addr * Add all relevant out-of-tree tests to tools/testing/selftests * Return an error instead of ignoring unknown flags, hopefully this makes it easier to extend. * Check sk_family before __tcp_authopt_info_get_or_create in tcp_set_authopt_key * Use sock_owned_by_me instead of WARN_ON(!lockdep_sock_is_held(sk)) * Fix some intermediate build failures reported by kbuild robot * Improve documentation Link: https://lore.kernel.org/netdev/cover.1628544649.git.cdleonard@gmail.com/ Changes since RFC: * Split into per-topic commits for ease of review. The intermediate commits compile with a few "unused function" warnings and don't do anything useful by themselves. * Add ABI documention including kernel-doc on uapi * Fix lockdep warnings from crypto by creating pools with one shash for each cpu * Accept short options to setsockopt by padding with zeros; this approach allows increasing the size of the structs in the future. * Support for aes-128-cmac-96 * Support for binding addresses to keys in a way similar to old tcp_md5 * Add support for retrieving received keyid/rnextkeyid and controling the keyid/rnextkeyid being sent. Link: https://lore.kernel.org/netdev/01383a8751e97ef826ef2adf93bfde3a08195a43.162… Leonard Crestez (15): tcp: authopt: Initial support and key management docs: Add user documentation for tcp_authopt selftests: Initial tcp_authopt test module selftests: tcp_authopt: Initial sockopt manipulation tcp: authopt: Add crypto initialization tcp: authopt: Compute packet signatures tcp: authopt: Hook into tcp core tcp: authopt: Add snmp counters selftests: tcp_authopt: Test key address binding selftests: tcp_authopt: Capture and verify packets selftests: Initial tcp_authopt support for nettest selftests: Initial tcp_authopt support for fcnal-test selftests: Add -t tcp_authopt option for fcnal-test.sh tcp: authopt: Add key selection controls selftests: tcp_authopt: Add tests for rollover Documentation/networking/index.rst | 1 + Documentation/networking/tcp_authopt.rst | 69 + include/linux/tcp.h | 6 + include/net/tcp.h | 1 + include/net/tcp_authopt.h | 134 ++ include/uapi/linux/snmp.h | 1 + include/uapi/linux/tcp.h | 110 ++ net/ipv4/Kconfig | 14 + net/ipv4/Makefile | 1 + net/ipv4/proc.c | 1 + net/ipv4/tcp.c | 27 + net/ipv4/tcp_authopt.c | 1168 +++++++++++++++++ net/ipv4/tcp_input.c | 17 + net/ipv4/tcp_ipv4.c | 5 + net/ipv4/tcp_minisocks.c | 2 + net/ipv4/tcp_output.c | 74 +- net/ipv6/tcp_ipv6.c | 4 + tools/testing/selftests/net/fcnal-test.sh | 34 + tools/testing/selftests/net/nettest.c | 34 +- tools/testing/selftests/tcp_authopt/Makefile | 5 + .../testing/selftests/tcp_authopt/README.rst | 15 + tools/testing/selftests/tcp_authopt/config | 6 + tools/testing/selftests/tcp_authopt/run.sh | 11 + tools/testing/selftests/tcp_authopt/setup.cfg | 17 + tools/testing/selftests/tcp_authopt/setup.py | 5 + .../tcp_authopt/tcp_authopt_test/__init__.py | 0 .../tcp_authopt/tcp_authopt_test/conftest.py | 21 + .../full_tcp_sniff_session.py | 53 + .../tcp_authopt_test/linux_tcp_authopt.py | 198 +++ .../tcp_authopt_test/netns_fixture.py | 63 + .../tcp_authopt/tcp_authopt_test/server.py | 82 ++ .../tcp_authopt/tcp_authopt_test/sockaddr.py | 101 ++ .../tcp_authopt_test/tcp_authopt_alg.py | 276 ++++ .../tcp_authopt/tcp_authopt_test/test_bind.py | 143 ++ .../tcp_authopt_test/test_rollover.py | 181 +++ .../tcp_authopt_test/test_sockopt.py | 74 ++ .../tcp_authopt_test/test_vectors.py | 359 +++++ .../tcp_authopt_test/test_verify_capture.py | 123 ++ .../tcp_authopt/tcp_authopt_test/utils.py | 154 +++ .../tcp_authopt/tcp_authopt_test/validator.py | 158 +++ 40 files changed, 3746 insertions(+), 2 deletions(-) create mode 100644 Documentation/networking/tcp_authopt.rst create mode 100644 include/net/tcp_authopt.h create mode 100644 net/ipv4/tcp_authopt.c create mode 100644 tools/testing/selftests/tcp_authopt/Makefile create mode 100644 tools/testing/selftests/tcp_authopt/README.rst create mode 100644 tools/testing/selftests/tcp_authopt/config create mode 100755 tools/testing/selftests/tcp_authopt/run.sh create mode 100644 tools/testing/selftests/tcp_authopt/setup.cfg create mode 100644 tools/testing/selftests/tcp_authopt/setup.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/__init__.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/conftest.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/full_tcp_sniff_session.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/linux_tcp_authopt.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/netns_fixture.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/server.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/sockaddr.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/tcp_authopt_alg.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/test_bind.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/test_rollover.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/test_sockopt.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/test_vectors.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/test_verify_capture.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/utils.py create mode 100644 tools/testing/selftests/tcp_authopt/tcp_authopt_test/validator.py base-commit: 3a62c333497b164868fdcd241842a1dd4e331825 -- 2.25.1

4 years

7
30
0 0

[PATCH v5] lib: add basic KUnit test for lib/math

by Daniel Latypov

Add basic test coverage for files that don't require any config options: * part of math.h (what seem to be the most commonly used macros) * gcd.c * lcm.c * int_sqrt.c * reciprocal_div.c (Ignored int_pow.c since it's a simple textbook algorithm.) These tests aren't particularly interesting, but they * provide short and simple examples of parameterized tests * provide a place to add tests for any new files in this dir * are written so adding new test cases to cover edge cases should be easy * looking at code coverage, we hit all the branches in the .c files Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- Changes since v4: * add in test cases for some math.h macros (abs, round_up/round_down, div_round_down/closest) * use parameterized testing less to keep things terser Changes since v3: * fix `checkpatch.pl --strict` warnings * add test cases for gcd(0,0) and lcm(0,0) * minor: don't test both gcd(a,b) and gcd(b,a) when a == b Changes since v2: mv math_test.c => math_kunit.c Changes since v1: * Rebase and rewrite to use the new parameterized testing support. * misc: fix overflow in literal and inline int_sqrt format string. * related: commit 1f0e943df68a ("Documentation: kunit: provide guidance for testing many inputs") was merged explaining the patterns shown here. * there's an in-flight patch to update it for parameterized testing. v1: https://lore.kernel.org/lkml/20201019224556.3536790-1-dlatypov@google.com/ --- lib/math/Kconfig | 5 + lib/math/Makefile | 2 + lib/math/math_kunit.c | 264 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 271 insertions(+) create mode 100644 lib/math/math_kunit.c diff --git a/lib/math/Kconfig b/lib/math/Kconfig index f19bc9734fa7..6ba8680439c1 100644 --- a/lib/math/Kconfig +++ b/lib/math/Kconfig @@ -15,3 +15,8 @@ config PRIME_NUMBERS config RATIONAL bool + +config MATH_KUNIT_TEST + tristate "KUnit test for lib/math" if !KUNIT_ALL_TESTS + default KUNIT_ALL_TESTS + depends on KUNIT diff --git a/lib/math/Makefile b/lib/math/Makefile index be6909e943bd..30abb7a8d564 100644 --- a/lib/math/Makefile +++ b/lib/math/Makefile @@ -4,3 +4,5 @@ obj-y += div64.o gcd.o lcm.o int_pow.o int_sqrt.o reciprocal_div.o obj-$(CONFIG_CORDIC) += cordic.o obj-$(CONFIG_PRIME_NUMBERS) += prime_numbers.o obj-$(CONFIG_RATIONAL) += rational.o + +obj-$(CONFIG_MATH_KUNIT_TEST) += math_kunit.o diff --git a/lib/math/math_kunit.c b/lib/math/math_kunit.c new file mode 100644 index 000000000000..80a087a32884 --- /dev/null +++ b/lib/math/math_kunit.c @@ -0,0 +1,264 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Simple KUnit suite for math helper funcs that are always enabled. + * + * Copyright (C) 2020, Google LLC. + * Author: Daniel Latypov <dlatypov(a)google.com> + */ + +#include <kunit/test.h> +#include <linux/gcd.h> +#include <linux/kernel.h> +#include <linux/lcm.h> +#include <linux/reciprocal_div.h> + +static void abs_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, abs('\0'), '\0'); + KUNIT_EXPECT_EQ(test, abs('a'), 'a'); + KUNIT_EXPECT_EQ(test, abs(-'a'), 'a'); + + /* The expression in the macro is actually promoted to an int. */ + KUNIT_EXPECT_EQ(test, abs((short)0), 0); + KUNIT_EXPECT_EQ(test, abs((short)42), 42); + KUNIT_EXPECT_EQ(test, abs((short)-42), 42); + + KUNIT_EXPECT_EQ(test, abs(0), 0); + KUNIT_EXPECT_EQ(test, abs(42), 42); + KUNIT_EXPECT_EQ(test, abs(-42), 42); + + KUNIT_EXPECT_EQ(test, abs(0L), 0L); + KUNIT_EXPECT_EQ(test, abs(42L), 42L); + KUNIT_EXPECT_EQ(test, abs(-42L), 42L); + + KUNIT_EXPECT_EQ(test, abs(0LL), 0LL); + KUNIT_EXPECT_EQ(test, abs(42LL), 42LL); + KUNIT_EXPECT_EQ(test, abs(-42LL), 42LL); + + /* Unsigned types get casted to signed. */ + KUNIT_EXPECT_EQ(test, abs(0ULL), 0LL); + KUNIT_EXPECT_EQ(test, abs(42ULL), 42LL); +} + +static void int_sqrt_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, int_sqrt(0UL), 0UL); + KUNIT_EXPECT_EQ(test, int_sqrt(1UL), 1UL); + KUNIT_EXPECT_EQ(test, int_sqrt(4UL), 2UL); + KUNIT_EXPECT_EQ(test, int_sqrt(5UL), 2UL); + KUNIT_EXPECT_EQ(test, int_sqrt(8UL), 2UL); + KUNIT_EXPECT_EQ(test, int_sqrt(1UL << 30), 1UL << 15); +} + +static void round_up_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, round_up(0, 1), 0); + KUNIT_EXPECT_EQ(test, round_up(1, 2), 2); + KUNIT_EXPECT_EQ(test, round_up(3, 2), 4); + KUNIT_EXPECT_EQ(test, round_up((1 << 30) - 1, 2), 1 << 30); + KUNIT_EXPECT_EQ(test, round_up((1 << 30) - 1, 1 << 29), 1 << 30); +} + +static void round_down_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, round_down(0, 1), 0); + KUNIT_EXPECT_EQ(test, round_down(1, 2), 0); + KUNIT_EXPECT_EQ(test, round_down(3, 2), 2); + KUNIT_EXPECT_EQ(test, round_down((1 << 30) - 1, 2), (1 << 30) - 2); + KUNIT_EXPECT_EQ(test, round_down((1 << 30) - 1, 1 << 29), 1 << 29); +} + +static void div_round_up_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(0, 1), 0); + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(20, 10), 2); + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(21, 10), 3); + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(21, 20), 2); + KUNIT_EXPECT_EQ(test, DIV_ROUND_UP(21, 99), 1); +} + +static void div_round_closest_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, DIV_ROUND_CLOSEST(0, 1), 0); + KUNIT_EXPECT_EQ(test, DIV_ROUND_CLOSEST(20, 10), 2); + KUNIT_EXPECT_EQ(test, DIV_ROUND_CLOSEST(21, 10), 2); + KUNIT_EXPECT_EQ(test, DIV_ROUND_CLOSEST(25, 10), 3); +} + +/* Generic test case for unsigned long inputs. */ +struct test_case { + unsigned long a, b; + unsigned long result; +}; + +static struct test_case gcd_cases[] = { + { + .a = 0, .b = 0, + .result = 0, + }, + { + .a = 0, .b = 1, + .result = 1, + }, + { + .a = 2, .b = 2, + .result = 2, + }, + { + .a = 2, .b = 4, + .result = 2, + }, + { + .a = 3, .b = 5, + .result = 1, + }, + { + .a = 3 * 9, .b = 3 * 5, + .result = 3, + }, + { + .a = 3 * 5 * 7, .b = 3 * 5 * 11, + .result = 15, + }, + { + .a = 1 << 21, + .b = (1 << 21) - 1, + .result = 1, + }, +}; + +KUNIT_ARRAY_PARAM(gcd, gcd_cases, NULL); + +static void gcd_test(struct kunit *test) +{ + const char *message_fmt = "gcd(%lu, %lu)"; + const struct test_case *test_param = test->param_value; + + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + gcd(test_param->a, test_param->b), + message_fmt, test_param->a, + test_param->b); + + if (test_param->a == test_param->b) + return; + + /* gcd(a,b) == gcd(b,a) */ + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + gcd(test_param->b, test_param->a), + message_fmt, test_param->b, + test_param->a); +} + +static struct test_case lcm_cases[] = { + { + .a = 0, .b = 0, + .result = 0, + }, + { + .a = 0, .b = 1, + .result = 0, + }, + { + .a = 1, .b = 2, + .result = 2, + }, + { + .a = 2, .b = 2, + .result = 2, + }, + { + .a = 3 * 5, .b = 3 * 7, + .result = 3 * 5 * 7, + }, +}; + +KUNIT_ARRAY_PARAM(lcm, lcm_cases, NULL); + +static void lcm_test(struct kunit *test) +{ + const char *message_fmt = "lcm(%lu, %lu)"; + const struct test_case *test_param = test->param_value; + + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + lcm(test_param->a, test_param->b), + message_fmt, test_param->a, + test_param->b); + + if (test_param->a == test_param->b) + return; + + /* lcm(a,b) == lcm(b,a) */ + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + lcm(test_param->b, test_param->a), + message_fmt, test_param->b, + test_param->a); +} + +struct u32_test_case { + u32 a, b; + u32 result; +}; + +static struct u32_test_case reciprocal_div_cases[] = { + { + .a = 0, .b = 1, + .result = 0, + }, + { + .a = 42, .b = 20, + .result = 2, + }, + { + .a = 42, .b = 9999, + .result = 0, + }, + { + .a = (1 << 16), .b = (1 << 14), + .result = 1 << 2, + }, +}; + +KUNIT_ARRAY_PARAM(reciprocal_div, reciprocal_div_cases, NULL); + +static void reciprocal_div_test(struct kunit *test) +{ + const struct u32_test_case *test_param = test->param_value; + struct reciprocal_value rv = reciprocal_value(test_param->b); + + KUNIT_EXPECT_EQ_MSG(test, test_param->result, + reciprocal_divide(test_param->a, rv), + "reciprocal_divide(%u, %u)", + test_param->a, test_param->b); +} + +static void reciprocal_scale_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, reciprocal_scale(0u, 100), 0u); + KUNIT_EXPECT_EQ(test, reciprocal_scale(1u, 100), 0u); + KUNIT_EXPECT_EQ(test, reciprocal_scale(1u << 4, 1 << 28), 1u); + KUNIT_EXPECT_EQ(test, reciprocal_scale(1u << 16, 1 << 28), 1u << 12); + KUNIT_EXPECT_EQ(test, reciprocal_scale(~0u, 1 << 28), (1u << 28) - 1); +} + +static struct kunit_case math_test_cases[] = { + KUNIT_CASE(abs_test), + KUNIT_CASE(int_sqrt_test), + KUNIT_CASE(round_up_test), + KUNIT_CASE(round_down_test), + KUNIT_CASE(div_round_up_test), + KUNIT_CASE(div_round_closest_test), + KUNIT_CASE_PARAM(gcd_test, gcd_gen_params), + KUNIT_CASE_PARAM(lcm_test, lcm_gen_params), + KUNIT_CASE_PARAM(reciprocal_div_test, reciprocal_div_gen_params), + KUNIT_CASE(reciprocal_scale_test), + {} +}; + +static struct kunit_suite math_test_suite = { + .name = "lib-math", + .test_cases = math_test_cases, +}; + +kunit_test_suites(&math_test_suite); + +MODULE_LICENSE("GPL v2"); base-commit: 4fa56ad0d12e24df768c98bffe9039f915d1bc02 -- 2.31.1.295.g9ea45b61b8-goog

4 years

2
4
0 0

[PATCH] selftests/gpio: Fix gpio compiling error

by Li Zhijian

[root@iaas-rpma gpio]# make gcc gpio-mockup-cdev.c -o /home/lizhijian/linux/tools/testing/selftests/gpio/gpio-mockup-cdev gpio-mockup-cdev.c: In function ‘request_line_v2’: gpio-mockup-cdev.c:24:30: error: storage size of ‘req’ isn’t known 24 | struct gpio_v2_line_request req; | ^~~ gpio-mockup-cdev.c:32:14: error: ‘GPIO_V2_LINE_FLAG_OUTPUT’ undeclared (first use in this function); did you mean ‘GPIOLINE_FLAG_IS_OUT’? 32 | if (flags & GPIO_V2_LINE_FLAG_OUTPUT) { | ^~~~~~~~~~~~~~~~~~~~~~~~ Search headers from linux tree like others, such as sched CC: Philip Li <philip.li(a)intel.com> Reported-by: kernel test robot <lkp(a)intel.com> Signed-off-by: Li Zhijian <lizhijian(a)cn.fujitsu.com> --- tools/testing/selftests/gpio/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/gpio/Makefile b/tools/testing/selftests/gpio/Makefile index 39f2bbe8dd3d..42ea7d2aa844 100644 --- a/tools/testing/selftests/gpio/Makefile +++ b/tools/testing/selftests/gpio/Makefile @@ -3,5 +3,6 @@ TEST_PROGS := gpio-mockup.sh TEST_FILES := gpio-mockup-sysfs.sh TEST_GEN_PROGS_EXTENDED := gpio-mockup-cdev +CFLAGS += -I../../../../usr/include include ../lib.mk -- 2.31.1

4 years

3
3
0 0

[V3][PATCH] selftests/x86: Fix error: variably modified 'altstack_data' at file scope

by Jun Miao

Based on glibc 2.33 -> 2.34, there is one new feature: NEWS for version 2.34 ===================== Major new features: * Add _SC_MINSIGSTKSZ and _SC_SIGSTKSZ. When _DYNAMIC_STACK_SIZE_SOURCE or _GNU_SOURCE are defined, MINSIGSTKSZ and SIGSTKSZ are no longer constant on Linux. MINSIGSTKSZ is redefined to sysconf(_SC_MINSIGSTKSZ) and SIGSTKSZ is redefined to sysconf (_SC_SIGSTKSZ). This supports dynamic sized register sets for modern architectural features like Arm SVE. ===================== If _SC_SIGSTKSZ_SOURCE or _GNU_SOURCE are defined, MINSIGSTKSZ and SIGSTKSZ are redefined as: /* Default stack size for a signal handler: sysconf (SC_SIGSTKSZ). */ # undef SIGSTKSZ # define SIGSTKSZ sysconf (_SC_SIGSTKSZ) /* Minimum stack size for a signal handler: SIGSTKSZ. */ # undef MINSIGSTKSZ # define MINSIGSTKSZ SIGSTKSZ Compilation will fail if the source assumes constant MINSIGSTKSZ or SIGSTKSZ. Build error with the GNU C Library 2.34: DEBUG: | sigreturn.c:150:13: error: variably modified 'altstack_data' at file scope | sigreturn.c:150:13: error: variably modified 'altstack_data' at file scope DEBUG: | 150 | static char altstack_data[SIGSTKSZ]; | 150 | static char altstack_data[SIGSTKSZ]; DEBUG: | | ^~~~~~~~~~~~~ DEBUG: | single_step_syscall.c:60:22: error: variably modified 'altstack_data' at file scope DEBUG: | 60 | static unsigned char altstack_data[SIGSTKSZ]; DEBUG: | | ^~~~~~~~~~~~~ Link: https://sourceware.org/pipermail/libc-alpha/2021-January/121996.html Link: https://sourceware.org/pipermail/libc-alpha/2021-August/129718.html Suggested-by: Jianwei Hu <jianwei.hu(a)windriver.com> Signed-off-by: Jun Miao <jun.miao(a)windriver.com> --- tools/testing/selftests/x86/mov_ss_trap.c | 4 ++-- tools/testing/selftests/x86/sigreturn.c | 7 +++---- tools/testing/selftests/x86/single_step_syscall.c | 4 ++-- tools/testing/selftests/x86/syscall_arg_fault.c | 7 +++---- 4 files changed, 10 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/x86/mov_ss_trap.c b/tools/testing/selftests/x86/mov_ss_trap.c index 6da0ac3f0135..cc3de6ff9fba 100644 --- a/tools/testing/selftests/x86/mov_ss_trap.c +++ b/tools/testing/selftests/x86/mov_ss_trap.c @@ -47,7 +47,6 @@ unsigned short ss; extern unsigned char breakpoint_insn[]; sigjmp_buf jmpbuf; -static unsigned char altstack_data[SIGSTKSZ]; static void enable_watchpoint(void) { @@ -250,13 +249,14 @@ int main() if (sigsetjmp(jmpbuf, 1) == 0) { printf("[RUN]\tMOV SS; SYSENTER\n"); stack_t stack = { - .ss_sp = altstack_data, + .ss_sp = malloc(sizeof(char) * SIGSTKSZ), .ss_size = SIGSTKSZ, }; if (sigaltstack(&stack, NULL) != 0) err(1, "sigaltstack"); sethandler(SIGSEGV, handle_and_longjmp, SA_RESETHAND | SA_ONSTACK); nr = SYS_getpid; + free(stack.ss_sp); /* Clear EBP first to make sure we segfault cleanly. */ asm volatile ("xorl %%ebp, %%ebp; mov %[ss], %%ss; SYSENTER" : "+a" (nr) : [ss] "m" (ss) : "flags", "rcx" diff --git a/tools/testing/selftests/x86/sigreturn.c b/tools/testing/selftests/x86/sigreturn.c index 57c4f67f16ef..5d7961a5f7f6 100644 --- a/tools/testing/selftests/x86/sigreturn.c +++ b/tools/testing/selftests/x86/sigreturn.c @@ -138,9 +138,6 @@ static unsigned short LDT3(int idx) return (idx << 3) | 7; } -/* Our sigaltstack scratch space. */ -static char altstack_data[SIGSTKSZ]; - static void sethandler(int sig, void (*handler)(int, siginfo_t *, void *), int flags) { @@ -771,7 +768,8 @@ int main() setup_ldt(); stack_t stack = { - .ss_sp = altstack_data, + /* Our sigaltstack scratch space. */ + .ss_sp = malloc(sizeof(char) * SIGSTKSZ), .ss_size = SIGSTKSZ, }; if (sigaltstack(&stack, NULL) != 0) @@ -872,5 +870,6 @@ int main() total_nerrs += test_nonstrict_ss(); #endif + free(stack.ss_sp); return total_nerrs ? 1 : 0; } diff --git a/tools/testing/selftests/x86/single_step_syscall.c b/tools/testing/selftests/x86/single_step_syscall.c index 120ac741fe44..9a30f443e928 100644 --- a/tools/testing/selftests/x86/single_step_syscall.c +++ b/tools/testing/selftests/x86/single_step_syscall.c @@ -57,7 +57,6 @@ static void clearhandler(int sig) static volatile sig_atomic_t sig_traps, sig_eflags; sigjmp_buf jmpbuf; -static unsigned char altstack_data[SIGSTKSZ]; #ifdef __x86_64__ # define REG_IP REG_RIP @@ -210,7 +209,7 @@ int main() unsigned long nr = SYS_getpid; printf("[RUN]\tSet TF and check SYSENTER\n"); stack_t stack = { - .ss_sp = altstack_data, + .ss_sp = malloc(sizeof(char) * SIGSTKSZ), .ss_size = SIGSTKSZ, }; if (sigaltstack(&stack, NULL) != 0) @@ -219,6 +218,7 @@ int main() SA_RESETHAND | SA_ONSTACK); sethandler(SIGILL, print_and_longjmp, SA_RESETHAND); set_eflags(get_eflags() | X86_EFLAGS_TF); + free(stack.ss_sp); /* Clear EBP first to make sure we segfault cleanly. */ asm volatile ("xorl %%ebp, %%ebp; SYSENTER" : "+a" (nr) :: "flags", "rcx" #ifdef __x86_64__ diff --git a/tools/testing/selftests/x86/syscall_arg_fault.c b/tools/testing/selftests/x86/syscall_arg_fault.c index bff474b5efc6..461fa41a4d02 100644 --- a/tools/testing/selftests/x86/syscall_arg_fault.c +++ b/tools/testing/selftests/x86/syscall_arg_fault.c @@ -17,9 +17,6 @@ #include "helpers.h" -/* Our sigaltstack scratch space. */ -static unsigned char altstack_data[SIGSTKSZ]; - static void sethandler(int sig, void (*handler)(int, siginfo_t *, void *), int flags) { @@ -104,7 +101,8 @@ static void sigill(int sig, siginfo_t *info, void *ctx_void) int main() { stack_t stack = { - .ss_sp = altstack_data, + /* Our sigaltstack scratch space. */ + .ss_sp = malloc(sizeof(char) * SIGSTKSZ), .ss_size = SIGSTKSZ, }; if (sigaltstack(&stack, NULL) != 0) @@ -233,5 +231,6 @@ int main() set_eflags(get_eflags() & ~X86_EFLAGS_TF); #endif + free(stack.ss_sp); return 0; } -- 2.32.0

4 years

2
2
0 0

KernelCI Hackfest #2 - Sept 6-10 2021

by Guillaume Tucker

The first KernelCI hackfest[1] early June was successful in getting a number of kernel developers to work alongside the core KernelCI team. Test coverage was increased in particular with kselftest, LTP, KUnit and a new test suite for libcamera. We're now improving documentation and tooling to make it easier for anyone to get started. Find out more about KernelCI on https://kernelci.org. The second hackfest is scheduled for the 6th-10th September. It should be a good opportunity to start discussing and working on upstream kernel testing topics ahead of the Linux Plumbers Conference[2]. Here's the project board where anyone can already add some ideas: https://github.com/orgs/kernelci/projects/5 There is no registration system, but please reply to this email or send a message on IRC (#kernelci libera.chat) or kernelci.slack.com if you would like to take part so you'll get email updates and invitations to the meetings and open hours sessions online. You may just drop in and out at any point during the hackfest as you see fit. The hackfest features: * Daily open hours online using Big Blue Button to discuss things and get support from the KernelCI team * KernelCI team members available across most time zones to provide quick feedback * A curated list of topics and a project board to help set objectives and coordinate efforts between all contributors As always, KernelCI is at the service of the kernel community so please share any feedback you may have to help shape this upcoming hackfest in the best possible way. Thanks, Guillaume [1] https://foundation.kernelci.org/blog/2021/06/24/the-first-ever-kernelci-hac… [2] https://www.linuxplumbersconf.org/event/11/page/104-accepted-microconferenc…

4 years

2
2
0 0

Re: KernelCI Hackfest #2 - Sept 6-10 2021

by Guillaume Tucker

On 02/08/2021 10:00, Guillaume Tucker wrote: > The first KernelCI hackfest[1] early June was successful in getting > a number of kernel developers to work alongside the core KernelCI > team. Test coverage was increased in particular with kselftest, > LTP, KUnit and a new test suite for libcamera. We're now improving > documentation and tooling to make it easier for anyone to get > started. Find out more about KernelCI on https://kernelci.org. > > The second hackfest is scheduled for the 6th-10th September. It > should be a good opportunity to start discussing and working on > upstream kernel testing topics ahead of the Linux Plumbers > Conference[2]. Please find below some extra information for the KernelCI Hackfest which is taking place next week. We're expecting at least some contributors from the Civil Infrastructure Platform project, the Google Chrome OS kernel team, Collabora kernel developers and a few more from the wider Linux kernel community. If you need any direct support, please reply to this email or ask on kernelci.slack.com or IRC #kernelci (libera.chat). > Here's the project board where anyone can already add some ideas: > > https://github.com/orgs/kernelci/projects/5 In order to add an issue to the workboard, please first create one in a KernelCI GitHub repository such as kernelci-core: https://github.com/kernelci/kernelci-core/issues Each contributor to the hackfest should be added to the KernelCI "hackers" team, which has permission to edit the workboard. If you aren't part of this team yet, please ask and you'll be invited. Note: Having a GitHub account is not mandatory for taking part in the hackfest. It's mainly there to facilitate coordination, even though it is required in order to contribute to the KernelCI GitHub repositories. Contributions as part of the hackfest may also be in the kernel tree such as improvements to kselftest, KUnit or bug fixes, or other test suites such as LTP etc. > The hackfest features: > > * Daily open hours online using Big Blue Button to discuss things > and get support from the KernelCI team > > * KernelCI team members available across most time zones to provide > quick feedback > > * A curated list of topics and a project board to help set > objectives and coordinate efforts between all contributors Please see the table below with the proposed daily open hours to accommodate most time zones: Region Zone Time 1 Time 2 East Asia GMT+10 17:00-19:00 03:00-05:00 Europe GMT+2 09:00-11:00 19:00-21:00 UTC 07:00-09:00 17:00-19:00 West America GMT-7 00:00-02:00 10:00-12:00 They will be held as a Big Blue Button virtual conference with the same URL as the last hackfest. It's not being shared publicly to avoid any potential abuse, so please ask if you don't have it already. On Monday, the focus should be put on getting started and reviewing the backlog on the hackfest workboard to distribute things among people or help new contributors find topics suitable for them. Open hours are otherwise opportunities to get more direct support from the KernelCI team or discuss any topic. See you there! Best wishes, Guillaume > [1] https://foundation.kernelci.org/blog/2021/06/24/the-first-ever-kernelci-hac… > [2] https://www.linuxplumbersconf.org/event/11/page/104-accepted-microconferenc…

4 years

1
0
0 0

[REGRESSION] lkft kselftest for next-20210902

by lkft＠linaro.org

## Build * kernel: 5.14.0 * git: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next * git branch: master * git commit: 34560ced20599a01660a693f464c7c7dc83db1d9 * git describe: next-20210902 * test details: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210902 ## Regressions (compared to next-20210831) * i386, kselftest-net - net.so_txtime.sh * qemu_arm, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_migrate * qemu_arm, kselftest-rtc - rtc.rtctest * qemu_arm64, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * qemu_i386, kselftest-rtc - rtc.rtctest * qemu_x86_64, kselftest-timers - timers.rtcpie * x86, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * x86, kselftest-rtc - rtc.rtctest Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Fixes (compared to next-20210831) * dragonboard-410c, kselftest-capabilities - capabilities.test_execve * hi6220-hikey, kselftest-rtc - rtc.rtctest * i386, kselftest-rtc - rtc.rtctest * qemu_arm64, kselftest-timers - timers.rtcpie * qemu_x86_64, kselftest-rtc - rtc.rtctest * x15, kselftest-rtc - rtc.rtctest ## Test result summary total: 4712, pass: 2612, fail: 374, skip: 1726, xfail: 0 ## Build Summary ## Test suites summary * kselftest-android * kselftest-arm64 * kselftest-arm64/arm64.btitest.bti_c_func * kselftest-arm64/arm64.btitest.bti_j_func * kselftest-arm64/arm64.btitest.bti_jc_func * kselftest-arm64/arm64.btitest.bti_none_func * kselftest-arm64/arm64.btitest.nohint_func * kselftest-arm64/arm64.btitest.paciasp_func * kselftest-arm64/arm64.nobtitest.bti_c_func * kselftest-arm64/arm64.nobtitest.bti_j_func * kselftest-arm64/arm64.nobtitest.bti_jc_func * kselftest-arm64/arm64.nobtitest.bti_none_func * kselftest-arm64/arm64.nobtitest.nohint_func * kselftest-arm64/arm64.nobtitest.paciasp_func * kselftest-bpf * kselftest-breakpoints * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-drivers * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kexec * kselftest-kvm * kselftest-lib * kselftest-livepatch * kselftest-lkdtm * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-tc-testing * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-vm * kselftest-x86 * kselftest-zram -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

[GIT PULL] KUnit update for Linux 5.15-rc1

by Shuah Khan

Hi Linus, Please pull the following KUnit update for Linux 5.15-rc1. This KUnit update for Linux 5.15-rc1 adds new features and tests: tool: -- support for --kernel_args to allow setting module params -- support for --raw_output option to show just the kunit output during make tests: -- KUnit tests for checksums and timestamps -- Print test statistics on failure -- Integrates UBSAN into the KUnit testing framework. It fails KUnit tests whenever it reports undefined behavior. diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit 2734d6c1b1a089fb593ef6a23d4b70903526fe0c: Linux 5.14-rc2 (2021-07-18 14:13:49 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-kunit-5.15-rc1 for you to fetch changes up to acd8e8407b8fcc3229d6d8558cac338bea801aed: kunit: Print test statistics on failure (2021-08-13 13:38:31 -0600) ---------------------------------------------------------------- linux-kselftest-kunit-5.15-rc1 This KUnit update for Linux 5.15-rc1 adds new features and tests: tool: -- support for --kernel_args to allow setting module params -- support for --raw_output option to show just the kunit output during make tests: -- KUnit tests for checksums and timestamps -- Print test statistics on failure -- Integrates UBSAN into the KUnit testing framework. It fails KUnit tests whenever it reports undefined behavior. ---------------------------------------------------------------- Daniel Latypov (2): kunit: tool: add --kernel_args to allow setting module params kunit: tool: make --raw_output support only showing kunit output David Gow (2): fat: Add KUnit tests for checksums and timestamps kunit: Print test statistics on failure Uriel Guajardo (1): kunit: ubsan integration Documentation/dev-tools/kunit/kunit-tool.rst | 9 +- Documentation/dev-tools/kunit/running_tips.rst | 10 ++ fs/fat/.kunitconfig | 5 + fs/fat/Kconfig | 14 +- fs/fat/Makefile | 2 + fs/fat/fat_test.c | 196 +++++++++++++++++++++++++ fs/fat/misc.c | 3 + lib/kunit/test.c | 109 ++++++++++++++ lib/ubsan.c | 3 + tools/testing/kunit/kunit.py | 36 +++-- tools/testing/kunit/kunit_parser.py | 6 +- tools/testing/kunit/kunit_tool_test.py | 29 +++- 12 files changed, 398 insertions(+), 24 deletions(-) create mode 100644 fs/fat/.kunitconfig create mode 100644 fs/fat/fat_test.c ----------------------------------------------------------------

4 years

2
1
0 0

[REGRESSION] lkft kselftest for next-20210901

by lkft＠linaro.org

## Build * kernel: 5.14.0 * git: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next * git branch: master * git commit: c1b13fe76e95c0f64c9dba9876dfbdb0bd862d99 * git describe: next-20210901 * test details: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210901 ## Regressions (compared to next-20210831) * qemu_arm, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_migrate - cgroup.test_freezer.test_cgfreezer_ptrace * qemu_arm, kselftest-rtc - rtc.rtctest * qemu_arm64, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Fixes (compared to next-20210831) * dragonboard-410c, kselftest-capabilities - capabilities.test_execve * i386, kselftest-rtc - rtc.rtctest * qemu-arm64-gic-version2, kselftest-timers - timers.rtcpie * qemu-arm64-gic-version3, kselftest-timers - timers.rtcpie * x86, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * x86, kselftest-rtc - rtc.rtctest ## Test result summary total: 5353, pass: 3042, fail: 465, skip: 1846, xfail: 0 ## Build Summary ## Test suites summary * kselftest-android * kselftest-arm64 * kselftest-arm64/arm64.btitest.bti_c_func * kselftest-arm64/arm64.btitest.bti_j_func * kselftest-arm64/arm64.btitest.bti_jc_func * kselftest-arm64/arm64.btitest.bti_none_func * kselftest-arm64/arm64.btitest.nohint_func * kselftest-arm64/arm64.btitest.paciasp_func * kselftest-arm64/arm64.nobtitest.bti_c_func * kselftest-arm64/arm64.nobtitest.bti_j_func * kselftest-arm64/arm64.nobtitest.bti_jc_func * kselftest-arm64/arm64.nobtitest.bti_none_func * kselftest-arm64/arm64.nobtitest.nohint_func * kselftest-arm64/arm64.nobtitest.paciasp_func * kselftest-bpf * kselftest-breakpoints * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-drivers * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kexec * kselftest-kvm * kselftest-lib * kselftest-livepatch * kselftest-lkdtm * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-tc-testing * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-vm * kselftest-x86 * kselftest-zram -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

[PATCH v3 0/5] KVM: rseq: Fix and a test for a KVM+rseq bug

by Sean Christopherson

Patch 1 fixes a KVM+rseq bug where KVM's handling of TIF_NOTIFY_RESUME, e.g. for task migration, clears the flag without informing rseq and leads to stale data in userspace's rseq struct. Patch 2 is a cleanup to try and make future bugs less likely. It's also a baby step towards moving and renaming tracehook_notify_resume() since it has nothing to do with tracing. Patch 3 is a fix/cleanup to stop overriding x86's unistd_{32,64}.h when the include path (intentionally) omits tools' uapi headers. KVM's selftests do exactly that so that they can pick up the uapi headers from the installed kernel headers, and still use various tools/ headers that mirror kernel code, e.g. linux/types.h. This allows the new test in patch 4 to reference __NR_rseq without having to manually define it. Patch 4 is a regression test for the KVM+rseq bug. Patch 5 is a cleanup made possible by patch 3. Based on commit 835d31d319d9 ("Merge tag 'media/v5.15-1' of ..."). v3: - Collect Ack/Review. [Mathieu, Ben] - Add explicit smp_wmb() instead of relying on atomic_inc() to do a full barrier. [Mathieu] - Add lots and lots of comments in the selftest, especially around why the migration thread needs a udelay(). [Mathieu] - Delay between 1us and 10us to reduce the odds of having a hard dependency on arch/kernel behavior. [Mathieu] - Dropped an s390 change in patch 2 after a rebase to upstream master. v2: - https://lkml.kernel.org/r/20210820225002.310652-1-seanjc@google.com - Don't touch rseq_cs when handling KVM case so that rseq_syscall() will still detect a naughty userspace. [Mathieu] - Use a sequence counter + retry in the test to ensure the process isn't migrated between sched_getcpu() and reading rseq.cpu_id, i.e. to avoid a flaky test. [Mathieu] - Add Mathieu's ack for patch 2. - Add more comments in the test. v1: https://lkml.kernel.org/r/20210818001210.4073390-1-seanjc@google.com Sean Christopherson (5): KVM: rseq: Update rseq when processing NOTIFY_RESUME on xfer to KVM guest entry: rseq: Call rseq_handle_notify_resume() in tracehook_notify_resume() tools: Move x86 syscall number fallbacks to .../uapi/ KVM: selftests: Add a test for KVM_RUN+rseq to detect task migration bugs KVM: selftests: Remove __NR_userfaultfd syscall fallback arch/arm/kernel/signal.c | 1 - arch/arm64/kernel/signal.c | 1 - arch/csky/kernel/signal.c | 4 +- arch/mips/kernel/signal.c | 4 +- arch/powerpc/kernel/signal.c | 4 +- include/linux/tracehook.h | 2 + kernel/entry/common.c | 4 +- kernel/rseq.c | 14 +- .../x86/include/{ => uapi}/asm/unistd_32.h | 0 .../x86/include/{ => uapi}/asm/unistd_64.h | 3 - tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 3 + tools/testing/selftests/kvm/rseq_test.c | 236 ++++++++++++++++++ 13 files changed, 257 insertions(+), 20 deletions(-) rename tools/arch/x86/include/{ => uapi}/asm/unistd_32.h (100%) rename tools/arch/x86/include/{ => uapi}/asm/unistd_64.h (83%) create mode 100644 tools/testing/selftests/kvm/rseq_test.c -- 2.33.0.153.gba50c8fa24-goog

4 years

2
6
0 0

[PATCH] kselftests/sched: cleanup the child processes

by Li Zhijian

Previously, 'make -C sched run_tests' will block forever when it occurs something wrong where the *selftests framework* is waiting for its child processes to exit. [root@iaas-rpma sched]# ./cs_prctl_test ## Create a thread/process/process group hiearchy Not a core sched system tid=74985, / tgid=74985 / pgid=74985: ffffffffffffffff Not a core sched system tid=74986, / tgid=74986 / pgid=74985: ffffffffffffffff Not a core sched system tid=74988, / tgid=74986 / pgid=74985: ffffffffffffffff Not a core sched system tid=74989, / tgid=74986 / pgid=74985: ffffffffffffffff Not a core sched system tid=74990, / tgid=74986 / pgid=74985: ffffffffffffffff Not a core sched system tid=74987, / tgid=74987 / pgid=74985: ffffffffffffffff Not a core sched system tid=74991, / tgid=74987 / pgid=74985: ffffffffffffffff Not a core sched system tid=74992, / tgid=74987 / pgid=74985: ffffffffffffffff Not a core sched system tid=74993, / tgid=74987 / pgid=74985: ffffffffffffffff Not a core sched system (268) FAILED: get_cs_cookie(0) == 0 ## Set a cookie on entire process group -1 = prctl(62, 1, 0, 2, 0) core_sched create failed -- PGID: Invalid argument (cs_prctl_test.c:272) - [root@iaas-rpma sched]# ps PID TTY TIME CMD 4605 pts/2 00:00:00 bash 74986 pts/2 00:00:00 cs_prctl_test 74987 pts/2 00:00:00 cs_prctl_test 74999 pts/2 00:00:00 ps CC: Philip Li <philip.li(a)intel.com> Reported-by: kernel test robot <lkp(a)intel.com> Signed-off-by: Li Zhijian <lizhijian(a)cn.fujitsu.com> --- tools/testing/selftests/sched/cs_prctl_test.c | 28 ++++++++++++------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/sched/cs_prctl_test.c b/tools/testing/selftests/sched/cs_prctl_test.c index 63fe6521c56d..1829383715c6 100644 --- a/tools/testing/selftests/sched/cs_prctl_test.c +++ b/tools/testing/selftests/sched/cs_prctl_test.c @@ -64,6 +64,17 @@ enum pid_type {PIDTYPE_PID = 0, PIDTYPE_TGID, PIDTYPE_PGID}; const int THREAD_CLONE_FLAGS = CLONE_THREAD | CLONE_SIGHAND | CLONE_FS | CLONE_VM | CLONE_FILES; +struct child_args { + int num_threads; + int pfd[2]; + int cpid; + int thr_tids[MAX_THREADS]; +}; + +static struct child_args procs[MAX_PROCESSES]; +static int num_processes = 2; +static int need_cleanup = 0; + static int _prctl(int option, unsigned long arg2, unsigned long arg3, unsigned long arg4, unsigned long arg5) { @@ -80,8 +91,14 @@ static int _prctl(int option, unsigned long arg2, unsigned long arg3, unsigned l #define handle_error(msg) __handle_error(__FILE__, __LINE__, msg) static void __handle_error(char *fn, int ln, char *msg) { + int pidx; printf("(%s:%d) - ", fn, ln); perror(msg); + if (need_cleanup) { + for (pidx = 0; pidx < num_processes; ++pidx) + kill(procs[pidx].cpid, 15); + need_cleanup = 0; + } exit(EXIT_FAILURE); } @@ -108,13 +125,6 @@ static unsigned long get_cs_cookie(int pid) return cookie; } -struct child_args { - int num_threads; - int pfd[2]; - int cpid; - int thr_tids[MAX_THREADS]; -}; - static int child_func_thread(void __attribute__((unused))*arg) { while (1) @@ -214,10 +224,7 @@ void _validate(int line, int val, char *msg) int main(int argc, char *argv[]) { - struct child_args procs[MAX_PROCESSES]; - int keypress = 0; - int num_processes = 2; int num_threads = 3; int delay = 0; int res = 0; @@ -264,6 +271,7 @@ int main(int argc, char *argv[]) printf("\n## Create a thread/process/process group hiearchy\n"); create_processes(num_processes, num_threads, procs); + need_cleanup = 1; disp_processes(num_processes, procs); validate(get_cs_cookie(0) == 0); -- 2.31.1

4 years

2
1
0 0

[PATCH v3] kunit: tool: improve compatibility of kunit_parser with KTAP specification

by Rae Moar

Update to kunit_parser to improve compatibility with KTAP specification including arbitrarily nested tests. Patch accomplishes three major changes: - Use a general Test object to represent all tests rather than TestCase and TestSuite objects. This allows for easier implementation of arbitrary levels of nested tests and promotes the idea that both test suites and test cases are tests. - Print errors incrementally rather than all at once after the parsing finishes to maximize information given to the user in the case of the parser given invalid input and to increase the helpfulness of the timestamps given during printing. Note that kunit.py parse does not print incrementally yet. However, this fix brings us closer to this feature. - Increase compatibility for different formats of input. Arbitrary levels of nested tests supported. Also, test cases and test suites are now supported to be present on the same level of testing. This patch now implements the KTAP specification as described here: https://lore.kernel.org/linux-kselftest/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqa…. This patch adjusts the kunit_tool_test.py file to check for the correct outputs from the new parser and adds a new test to check the parsing for a KTAP result log with correct format for multiple nested subtests (test_is_test_passed-all_passed_nested.log). This patch also alters the kunit_json.py file to allow for arbitrarily nested tests. Signed-off-by: Rae Moar <rmoar(a)google.com> Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com> --- Change log from v2: https://lore.kernel.org/linux-kselftest/20210826195505.3066755-1-rmoar@goog… - Fixes bug of type disagreement in kunit_json.py for build_dir - Removes raw_output() - Changes docstrings in kunit_parser.py (class docstring, LineStream docstrings, add_error(), total(), get_status(), all parsing methods) - Fixes bug of not printing diagnostic log in the case of end of lines - Sets default status of all tests to TEST_CRASHED - Adds and prints empty tests with crashed status in case of missing tests - Prints 'subtest' in instance of 1 subtest instead of 'subtests' - Includes checking for 'BUG:' message in search of crash messages in log (note that parse_crash_in_log method could be removed but would require deleting tests in kunit_tool_test.py that include the crash message that is no longer used. If removed, parser would still print log in cases of test crashed or failure, which would now include missing subtests) - Fixes bug of including directives (other than SKIP) in test name when matching name in result line for subtests --- Change log from v1: https://lore.kernel.org/linux-kselftest/20210820200032.2178134-1-rmoar@goog… - Rebase onto kselftest/kunit branch - Add tests to kunit_tool_test.py to check parser is correctly stripping hyphen, producing correct json objects with nested tests, correctly passing kselftest TAP output, and correctly deals with missing test plan. - Fix bug to correctly match test name in instance of a missing test plan. - Fix bug in kunit_tool_test.py pointed out by Daniel where it was not correctly checking for a proper match to the '0 tests run!' error message. Reverts changes back to original. - A few minor changes to commit message using Daniel's comments. - Change docstrings using Daniel's comments to reduce: - Shortens some docstrings to be one-line or just description if it is self explanatory. - Remove explicit respecification of types of parameters and returns because this is already specified in the function annoations. However, some descriptions of the parameters and returns remain and some contain the type for context. Additionally, the types of public attributes of classes remain. - Remove any documentation of 'Return: None' - Remove docstrings of helper methods within other methods --- tools/testing/kunit/kunit_json.py | 56 +- tools/testing/kunit/kunit_parser.py | 1022 ++++++++++++----- tools/testing/kunit/kunit_tool_test.py | 132 ++- .../test_is_test_passed-all_passed_nested.log | 34 + .../test_is_test_passed-kselftest.log | 14 + .../test_is_test_passed-missing_plan.log | 31 + .../kunit/test_data/test_strip_hyphen.log | 16 + 7 files changed, 925 insertions(+), 380 deletions(-) create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-kselftest.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log create mode 100644 tools/testing/kunit/test_data/test_strip_hyphen.log diff --git a/tools/testing/kunit/kunit_json.py b/tools/testing/kunit/kunit_json.py index f5cca5c38cac..746bec72b9ac 100644 --- a/tools/testing/kunit/kunit_json.py +++ b/tools/testing/kunit/kunit_json.py @@ -11,47 +11,47 @@ import os import kunit_parser -from kunit_parser import TestStatus - -def get_json_result(test_result, def_config, build_dir, json_path) -> str: - sub_groups = [] - - # Each test suite is mapped to a KernelCI sub_group - for test_suite in test_result.suites: - sub_group = { - "name": test_suite.name, - "arch": "UM", - "defconfig": def_config, - "build_environment": build_dir, - "test_cases": [], - "lab_name": None, - "kernel": None, - "job": None, - "git_branch": "kselftest", - } - test_cases = [] - # TODO: Add attachments attribute in test_case with detailed - # failure message, see https://api.kernelci.org/schema-test-case.html#get - for case in test_suite.cases: - test_case = {"name": case.name, "status": "FAIL"} - if case.status == TestStatus.SUCCESS: +from kunit_parser import Test, TestResult, TestStatus +from typing import Any, Dict, Optional + +JsonObj = Dict[str, Any] + +def _get_group_json(test: Test, def_config: str, + build_dir: Optional[str]) -> JsonObj: + sub_groups = [] # List[JsonObj] + test_cases = [] # List[JsonObj] + + for subtest in test.subtests: + if len(subtest.subtests): + sub_group = _get_group_json(subtest, def_config, + build_dir) + sub_groups.append(sub_group) + else: + test_case = {"name": subtest.name, "status": "FAIL"} + if subtest.status == TestStatus.SUCCESS: test_case["status"] = "PASS" - elif case.status == TestStatus.TEST_CRASHED: + elif subtest.status == TestStatus.TEST_CRASHED: test_case["status"] = "ERROR" test_cases.append(test_case) - sub_group["test_cases"] = test_cases - sub_groups.append(sub_group) + test_group = { - "name": "KUnit Test Group", + "name": test.name, "arch": "UM", "defconfig": def_config, "build_environment": build_dir, "sub_groups": sub_groups, + "test_cases": test_cases, "lab_name": None, "kernel": None, "job": None, "git_branch": "kselftest", } + return test_group + +def get_json_result(test_result: TestResult, def_config: str, + build_dir: Optional[str], json_path: str) -> str: + test_group = _get_group_json(test_result.test, def_config, build_dir) + test_group["name"] = "KUnit Test Group" json_obj = json.dumps(test_group, indent=4) if json_path != 'stdout': with open(json_path, 'w') as result_path: diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 6310a641b151..f1b28def3e78 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -1,11 +1,15 @@ # SPDX-License-Identifier: GPL-2.0 # -# Parses test results from a kernel dmesg log. +# Parses KTAP test results from a kernel dmesg log and incrementally prints +# results with reader-friendly format. Stores and returns test results in a +# Test object. # # Copyright (C) 2019, Google LLC. # Author: Felix Guo <felixguoxiuping(a)gmail.com> # Author: Brendan Higgins <brendanhiggins(a)google.com> +# Author: Rae Moar <rmoar(a)google.com> +from __future__ import annotations import re from collections import namedtuple @@ -14,33 +18,52 @@ from enum import Enum, auto from functools import reduce from typing import Iterable, Iterator, List, Optional, Tuple -TestResult = namedtuple('TestResult', ['status','suites','log']) - -class TestSuite(object): +TestResult = namedtuple('TestResult', ['status','test','log']) + +class Test(object): + """ + A class to represent a test parsed from KTAP results. All KTAP + results within a test log are stored in a main Test object as + subtests. + + Attributes: + status : TestStatus - status of the test + name : str - name of the test + expected_count : int - expected number of subtests (0 if single + test case and None if unknown expected number of subtests) + subtests : List[Test] - list of subtests + log : List[str] - log of KTAP lines that correspond to the test + counts : TestCounts - counts of the test statuses and errors of + subtests or of the test itself if the test is a single + test case. + """ def __init__(self) -> None: - self.status = TestStatus.SUCCESS - self.name = '' - self.cases = [] # type: List[TestCase] - - def __str__(self) -> str: - return 'TestSuite(' + str(self.status) + ',' + self.name + ',' + str(self.cases) + ')' - - def __repr__(self) -> str: - return str(self) - -class TestCase(object): - def __init__(self) -> None: - self.status = TestStatus.SUCCESS + """Creates Test object with default attributes.""" + self.status = TestStatus.TEST_CRASHED self.name = '' + self.expected_count = 0 # type: Optional[int] + self.subtests = [] # type: List[Test] self.log = [] # type: List[str] + self.counts = TestCounts() def __str__(self) -> str: - return 'TestCase(' + str(self.status) + ',' + self.name + ',' + str(self.log) + ')' + """Returns string representation of a Test class object.""" + return ('Test(' + str(self.status) + ', ' + self.name + + ', ' + str(self.expected_count) + ', ' + + str(self.subtests) + ', ' + str(self.log) + ', ' + + str(self.counts) + ')') def __repr__(self) -> str: + """Returns string representation of a Test class object.""" return str(self) + def add_error(self, error_message: str) -> None: + """Records an error that occurred while parsing this test.""" + self.counts.errors += 1 + print_error('Test ' + self.name + ': ' + error_message) + class TestStatus(Enum): + """An enumeration class to represent the status of a test.""" SUCCESS = auto() FAILURE = auto() SKIPPED = auto() @@ -48,381 +71,754 @@ class TestStatus(Enum): NO_TESTS = auto() FAILURE_TO_PARSE_TESTS = auto() +class TestCounts: + """ + Tracks the counts of statuses of all test cases and any errors within + a Test. + + Attributes: + passed : int - the number of tests that have passed + failed : int - the number of tests that have failed + crashed : int - the number of tests that have crashed + skipped : int - the number of tests that have skipped + errors : int - the number of errors in the test and subtests + """ + def __init__(self): + """Creates TestCounts object with counts of all test + statuses and test errors set to 0. + """ + self.passed = 0 + self.failed = 0 + self.crashed = 0 + self.skipped = 0 + self.errors = 0 + + def __str__(self) -> str: + """Returns the string representation of a TestCounts object. + """ + return ('Passed: ' + str(self.passed) + + ', Failed: ' + str(self.failed) + + ', Crashed: ' + str(self.crashed) + + ', Skipped: ' + str(self.skipped) + + ', Errors: ' + str(self.errors)) + + def total(self) -> int: + """Returns the total number of test cases within a test + object, where a test case is a test with no subtests. + """ + return (self.passed + self.failed + self.crashed + + self.skipped) + + def add_subtest_counts(self, counts: TestCounts) -> None: + """ + Adds the counts of another TestCounts object to the current + TestCounts object. Used to add the counts of a subtest to the + parent test. + + Parameters: + counts - a different TestCounts object whose counts + will be added to the counts of the TestCounts object + """ + self.passed += counts.passed + self.failed += counts.failed + self.crashed += counts.crashed + self.skipped += counts.skipped + self.errors += counts.errors + + def get_status(self) -> TestStatus: + """Returns the aggregated status of a Test using test + counts. + """ + if self.crashed: + # If one of the subtests crash, the expected status + # of the Test is crashed. + return TestStatus.TEST_CRASHED + elif self.failed: + # Otherwise if one of the subtests fail, the + # expected status of the Test is failed. + return TestStatus.FAILURE + elif self.passed: + # Otherwise if one of the subtests pass, the + # expected status of the Test is passed. + return TestStatus.SUCCESS + else: + # Finally, if none of the subtests have failed, + # crashed, or passed, the expected status of the + # Test is skipped. + return TestStatus.SKIPPED + + def add_status(self, status: TestStatus) -> None: + """ + Increments count of inputted status. + + Parameters: + status - status to be added to the TestCounts object + """ + if status == TestStatus.SUCCESS or \ + status == TestStatus.NO_TESTS: + # if status is NO_TESTS the most appropriate + # attribute to increment is passed because + # the test did not fail, crash or get skipped. + self.passed += 1 + elif status == TestStatus.FAILURE: + self.failed += 1 + elif status == TestStatus.SKIPPED: + self.skipped += 1 + else: + self.crashed += 1 + class LineStream: - """Provides a peek()/pop() interface over an iterator of (line#, text).""" + """ + A class to represent the lines of kernel output. + Provides a peek()/pop() interface over an iterator of + (line#, text). + """ _lines: Iterator[Tuple[int, str]] _next: Tuple[int, str] _done: bool def __init__(self, lines: Iterator[Tuple[int, str]]): + """Creates a new LineStream that wraps the given iterator.""" self._lines = lines self._done = False self._next = (0, '') self._get_next() def _get_next(self) -> None: + """Advances the LineSteam to the next line or sets the _done + attribute if the LineStream has reached the end of the lines. + """ try: self._next = next(self._lines) except StopIteration: self._done = True def peek(self) -> str: + """Returns the current line, without advancing the LineStream. + """ return self._next[1] def pop(self) -> str: + """Returns the current line and advances the LineStream to + the next line. + """ n = self._next self._get_next() return n[1] def __bool__(self) -> bool: + """Returns True if stream has more lines.""" return not self._done # Only used by kunit_tool_test.py. def __iter__(self) -> Iterator[str]: + """Empties all lines stored in LineStream object into + Iterator object and returns the Iterator object. + """ while bool(self): yield self.pop() def line_number(self) -> int: + """Returns the line number of the current line.""" return self._next[0] -kunit_start_re = re.compile(r'TAP version [0-9]+$') -kunit_end_re = re.compile('(List of all partitions:|' - 'Kernel panic - not syncing: VFS:|reboot: System halted)') +# Parsing helper methods: + +KTAP_START = re.compile(r'KTAP version ([0-9]+)$') +TAP_START = re.compile(r'TAP version ([0-9]+)$') +KTAP_END = re.compile('(List of all partitions:|' + 'Kernel panic - not syncing: VFS:|reboot: System halted)') def extract_tap_lines(kernel_output: Iterable[str]) -> LineStream: - def isolate_kunit_output(kernel_output: Iterable[str]) -> Iterator[Tuple[int, str]]: + """Extracts KTAP lines from inputted kernel output in LineStream + object.""" + def isolate_ktap_output(kernel_output: Iterable[str]) \ + -> Iterator[Tuple[int, str]]: line_num = 0 started = False for line in kernel_output: line_num += 1 - line = line.rstrip() # line always has a trailing \n - if kunit_start_re.search(line): + line = line.rstrip() # remove trailing \n + if not started and KTAP_START.search(line): + # start extracting KTAP lines and set prefix + # to number of characters before version line + prefix_len = len( + line.split('KTAP version')[0]) + started = True + yield line_num, line[prefix_len:] + elif not started and TAP_START.search(line): + # start extracting KTAP lines and set prefix + # to number of characters before version line prefix_len = len(line.split('TAP version')[0]) started = True yield line_num, line[prefix_len:] - elif kunit_end_re.search(line): + elif started and KTAP_END.search(line): + # stop extracting KTAP lines break elif started: - yield line_num, line[prefix_len:] - return LineStream(lines=isolate_kunit_output(kernel_output)) - -DIVIDER = '=' * 60 - -RESET = '\033[0;0m' - -def red(text) -> str: - return '\033[1;31m' + text + RESET - -def yellow(text) -> str: - return '\033[1;33m' + text + RESET - -def green(text) -> str: - return '\033[1;32m' + text + RESET - -def print_with_timestamp(message) -> None: - print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) + # remove prefix and any indention and yield + # line with line number + line = line[prefix_len:].lstrip() + yield line_num, line + return LineStream(lines=isolate_ktap_output(kernel_output)) + +KTAP_VERSIONS = [1] +TAP_VERSIONS = [13, 14] + +def check_version(version_num: int, accepted_versions: List[int], + version_type: str, test: Test) -> None: + """ + Adds error to test object if version number is too high or too + low. + + Parameters: + version_num - The inputted version number from the parsed KTAP or TAP + header line + accepted_version - List of accepted KTAP or TAP versions + version_type - 'KTAP' or 'TAP' depending on the type of + version line. + test - Test object for current test being parsed + """ + if version_num < min(accepted_versions): + test.add_error(version_type + + ' version lower than expected!') + elif version_num > max(accepted_versions): + test.add_error( + version_type + ' version higher than expected!') + +def parse_ktap_header(lines: LineStream, test: Test) -> bool: + """ + Parses KTAP/TAP header line and checks version number. + Returns False if fails to parse KTAP/TAP header line. + + Accepted formats: + - 'KTAP version [version number]' + - 'TAP version [version number]' + + Parameters: + lines - LineStream of KTAP output to parse + test - Test object for current test being parsed + + Return: + True if successfully parsed KTAP/TAP header line + """ + ktap_match = KTAP_START.match(lines.peek()) + tap_match = TAP_START.match(lines.peek()) + if ktap_match: + version_num = int(ktap_match.group(1)) + check_version(version_num, KTAP_VERSIONS, 'KTAP', test) + elif tap_match: + version_num = int(tap_match.group(1)) + check_version(version_num, TAP_VERSIONS, 'TAP', test) + else: + return False + test.log.append(lines.pop()) + return True -def format_suite_divider(message) -> str: - return '======== ' + message + ' ========' +TEST_HEADER = re.compile(r'^# Subtest: (.*)$') -def print_suite_divider(message) -> None: - print_with_timestamp(DIVIDER) - print_with_timestamp(format_suite_divider(message)) +def parse_test_header(lines: LineStream, test: Test) -> bool: + """ + Parses test header and stores test name in test object. + Returns False if fails to parse test header line. -def print_log(log) -> None: - for m in log: - print_with_timestamp(m) + Accepted format: + - '# Subtest: [test name]' -TAP_ENTRIES = re.compile(r'^(TAP|[\s]*ok|[\s]*not ok|[\s]*[0-9]+\.\.[0-9]+|[\s]*# (Subtest:|.*: kunit test case crashed!)).*$') + Parameters: + lines - LineStream of ktap output to parse + test - Test object for current test being parsed -def consume_non_diagnostic(lines: LineStream) -> None: - while lines and not TAP_ENTRIES.match(lines.peek()): - lines.pop() - -def save_non_diagnostic(lines: LineStream, test_case: TestCase) -> None: - while lines and not TAP_ENTRIES.match(lines.peek()): - test_case.log.append(lines.peek()) - lines.pop() + Return: + True if successfully parsed test header line + """ + match = TEST_HEADER.match(lines.peek()) + if not match: + return False + test.log.append(lines.pop()) + test.name = match.group(1) + return True -OkNotOkResult = namedtuple('OkNotOkResult', ['is_ok','description', 'text']) +TEST_PLAN = re.compile(r'1\.\.([0-9]+)') -OK_NOT_OK_SKIP = re.compile(r'^[\s]*(ok|not ok) [0-9]+ - (.*) # SKIP(.*)$') +def parse_test_plan(lines: LineStream, test: Test) -> bool: + """ + Parses test plan line and stores the expected number of subtests in + test object. Reports an error if expected count is 0. + Returns False and reports missing test plan error if fails to parse + test plan. -OK_NOT_OK_SUBTEST = re.compile(r'^[\s]+(ok|not ok) [0-9]+ - (.*)$') + Accepted format: + - '1..[number of subtests]' -OK_NOT_OK_MODULE = re.compile(r'^(ok|not ok) ([0-9]+) - (.*)$') + Parameters: + lines - LineStream of ktap output to parse + test - Test object for current test being parsed -def parse_ok_not_ok_test_case(lines: LineStream, test_case: TestCase) -> bool: - save_non_diagnostic(lines, test_case) - if not lines: - test_case.status = TestStatus.TEST_CRASHED - return True - line = lines.peek() - match = OK_NOT_OK_SUBTEST.match(line) - while not match and lines: - line = lines.pop() - match = OK_NOT_OK_SUBTEST.match(line) - if match: - test_case.log.append(lines.pop()) - test_case.name = match.group(2) - skip_match = OK_NOT_OK_SKIP.match(line) - if skip_match: - test_case.status = TestStatus.SKIPPED - return True - if test_case.status == TestStatus.TEST_CRASHED: - return True - if match.group(1) == 'ok': - test_case.status = TestStatus.SUCCESS - else: - test_case.status = TestStatus.FAILURE - return True - else: + Return: + True if successfully parsed test plan line + """ + match = TEST_PLAN.match(lines.peek()) + if not match: + test.expected_count = None + test.add_error('missing plan line!') return False - -SUBTEST_DIAGNOSTIC = re.compile(r'^[\s]+# (.*)$') -DIAGNOSTIC_CRASH_MESSAGE = re.compile(r'^[\s]+# .*?: kunit test case crashed!$') - -def parse_diagnostic(lines: LineStream, test_case: TestCase) -> bool: - save_non_diagnostic(lines, test_case) - if not lines: + test.log.append(lines.pop()) + expected_count = int(match.group(1)) + test.expected_count = expected_count + if expected_count == 0: + test.status = TestStatus.NO_TESTS + test.add_error('0 tests run!') + return True + +TEST_RESULT = re.compile(r'^(ok|not ok) ([0-9]+) (- )?([^#]*)( # .*)?$') + +TEST_RESULT_SKIP = re.compile(r'^(ok|not ok) ([0-9]+) (- )?(.*) # SKIP(.*)$') + +def peek_test_name_match(lines: LineStream, test: Test) -> bool: + """ + Matches current line with the format of a test result line and checks + if the name matches the name of the current test. + Returns False if fails to match format or name. + + Accepted format: + - '[ok|not ok] [test number] [-] [test name] [optional skip + directive]' + + Parameters: + lines - LineStream of KTAP output to parse + test - Test object for current test being parsed + + Return: + True if matched a test result line and the name matching the + expected test name + """ + line = lines.peek() + match = TEST_RESULT.match(line) + if not match: return False + name = match.group(4) + return (name == test.name) + +def parse_test_result(lines: LineStream, test: Test, + expected_num: int) -> bool: + """ + Parses test result line and stores the status and name in the test + object. Reports an error if the test number does not match expected + test number. + Returns False if fails to parse test result line. + + Note that the SKIP directive is the only direction that causes a + change in status. + + Accepted format: + - '[ok|not ok] [test number] [-] [test name] [optional skip + directive]' + + Parameters: + lines - LineStream of KTAP output to parse + test - Test object for current test being parsed + expected_num - expected test number for current test + + Return: + True if successfully parsed a test result line. + """ line = lines.peek() - match = SUBTEST_DIAGNOSTIC.match(line) - if match: - test_case.log.append(lines.pop()) - crash_match = DIAGNOSTIC_CRASH_MESSAGE.match(line) - if crash_match: - test_case.status = TestStatus.TEST_CRASHED - return True - else: + match = TEST_RESULT.match(line) + skip_match = TEST_RESULT_SKIP.match(line) + + # Check if line matches test result line format + if not match: return False + test.log.append(lines.pop()) -def parse_test_case(lines: LineStream) -> Optional[TestCase]: - test_case = TestCase() - save_non_diagnostic(lines, test_case) - while parse_diagnostic(lines, test_case): - pass - if parse_ok_not_ok_test_case(lines, test_case): - return test_case + # Set name of test object + if skip_match: + test.name = skip_match.group(4) else: - return None - -SUBTEST_HEADER = re.compile(r'^[\s]+# Subtest: (.*)$') - -def parse_subtest_header(lines: LineStream) -> Optional[str]: - consume_non_diagnostic(lines) - if not lines: - return None - match = SUBTEST_HEADER.match(lines.peek()) - if match: - lines.pop() - return match.group(1) + test.name = match.group(4) + + # Check test num + num = int(match.group(2)) + if num != expected_num: + test.add_error('Expected test number ' + + str(expected_num) + ' but found ' + str(num)) + + # Set status of test object + status = match.group(1) + if skip_match: + test.status = TestStatus.SKIPPED + elif status == 'ok': + test.status = TestStatus.SUCCESS else: - return None + test.status = TestStatus.FAILURE + return True + +def parse_diagnostic(lines: LineStream) -> List[str]: + """ + Parse lines that do not match the format of a test result line or + test header line and returns them in list. + + Line formats that are not parsed: + - '# Subtest: [test name]' + - '[ok|not ok] [test number] [-] [test name] [optional skip + directive]' + + Parameters: + lines - LineStream of KTAP output to parse + + Return: + Log of diagnostic lines + """ + log = [] # type: List[str] + while lines and not TEST_RESULT.match(lines.peek()) and not \ + TEST_HEADER.match(lines.peek()): + log.append(lines.pop()) + return log + +DIAGNOSTIC_CRASH_MESSAGE = re.compile( + r'^(BUG:|# .*?: kunit test case crashed!$)') + +def parse_crash_in_log(test: Test) -> bool: + """ + Iterate through the lines of the log to parse for crash message. + If crash message found, set status to crashed and return True. + Otherwise return False. + + Parameters: + test - Test object for current test being parsed + + Return: + True if crash message found in log + """ + for line in test.log: + if DIAGNOSTIC_CRASH_MESSAGE.match(line): + test.status = TestStatus.TEST_CRASHED + return True + return False -SUBTEST_PLAN = re.compile(r'[\s]+[0-9]+\.\.([0-9]+)') -def parse_subtest_plan(lines: LineStream) -> Optional[int]: - consume_non_diagnostic(lines) - match = SUBTEST_PLAN.match(lines.peek()) - if match: - lines.pop() - return int(match.group(1)) - else: - return None - -def max_status(left: TestStatus, right: TestStatus) -> TestStatus: - if left == right: - return left - elif left == TestStatus.TEST_CRASHED or right == TestStatus.TEST_CRASHED: - return TestStatus.TEST_CRASHED - elif left == TestStatus.FAILURE or right == TestStatus.FAILURE: - return TestStatus.FAILURE - elif left == TestStatus.SKIPPED: - return right - else: - return left +# Printing helper methods: -def parse_ok_not_ok_test_suite(lines: LineStream, - test_suite: TestSuite, - expected_suite_index: int) -> bool: - consume_non_diagnostic(lines) - if not lines: - test_suite.status = TestStatus.TEST_CRASHED - return False - line = lines.peek() - match = OK_NOT_OK_MODULE.match(line) - if match: - lines.pop() - if match.group(1) == 'ok': - test_suite.status = TestStatus.SUCCESS - else: - test_suite.status = TestStatus.FAILURE - skip_match = OK_NOT_OK_SKIP.match(line) - if skip_match: - test_suite.status = TestStatus.SKIPPED - suite_index = int(match.group(2)) - if suite_index != expected_suite_index: - print_with_timestamp( - red('[ERROR] ') + 'expected_suite_index ' + - str(expected_suite_index) + ', but got ' + - str(suite_index)) - return True - else: - return False +DIVIDER = '=' * 60 -def bubble_up_errors(status_list: Iterable[TestStatus]) -> TestStatus: - return reduce(max_status, status_list, TestStatus.SKIPPED) +RESET = '\033[0;0m' -def bubble_up_test_case_errors(test_suite: TestSuite) -> TestStatus: - max_test_case_status = bubble_up_errors(x.status for x in test_suite.cases) - return max_status(max_test_case_status, test_suite.status) +def red(text: str) -> str: + """Returns inputted string with red color code.""" + return '\033[1;31m' + text + RESET -def parse_test_suite(lines: LineStream, expected_suite_index: int) -> Optional[TestSuite]: - if not lines: - return None - consume_non_diagnostic(lines) - test_suite = TestSuite() - test_suite.status = TestStatus.SUCCESS - name = parse_subtest_header(lines) - if not name: - return None - test_suite.name = name - expected_test_case_num = parse_subtest_plan(lines) - if expected_test_case_num is None: - return None - while expected_test_case_num > 0: - test_case = parse_test_case(lines) - if not test_case: - break - test_suite.cases.append(test_case) - expected_test_case_num -= 1 - if parse_ok_not_ok_test_suite(lines, test_suite, expected_suite_index): - test_suite.status = bubble_up_test_case_errors(test_suite) - return test_suite - elif not lines: - print_with_timestamp(red('[ERROR] ') + 'ran out of lines before end token') - return test_suite - else: - print(f'failed to parse end of suite "{name}", at line {lines.line_number()}: {lines.peek()}') - return None +def yellow(text: str) -> str: + """Returns inputted string with yellow color code.""" + return '\033[1;33m' + text + RESET -TAP_HEADER = re.compile(r'^TAP version 14$') +def green(text: str) -> str: + """Returns inputted string with green color code.""" + return '\033[1;32m' + text + RESET -def parse_tap_header(lines: LineStream) -> bool: - consume_non_diagnostic(lines) - if TAP_HEADER.match(lines.peek()): - lines.pop() - return True - else: - return False +ANSI_LEN = len(red('')) -TEST_PLAN = re.compile(r'[0-9]+\.\.([0-9]+)') +def print_with_timestamp(message: str) -> None: + """Prints message with timestamp at beginning.""" + print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) -def parse_test_plan(lines: LineStream) -> Optional[int]: - consume_non_diagnostic(lines) - match = TEST_PLAN.match(lines.peek()) - if match: - lines.pop() - return int(match.group(1)) - else: - return None - -def bubble_up_suite_errors(test_suites: Iterable[TestSuite]) -> TestStatus: - return bubble_up_errors(x.status for x in test_suites) - -def parse_test_result(lines: LineStream) -> TestResult: - consume_non_diagnostic(lines) - if not lines or not parse_tap_header(lines): - return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) - expected_test_suite_num = parse_test_plan(lines) - if expected_test_suite_num == 0: - return TestResult(TestStatus.NO_TESTS, [], lines) - elif expected_test_suite_num is None: - return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) - test_suites = [] - for i in range(1, expected_test_suite_num + 1): - test_suite = parse_test_suite(lines, i) - if test_suite: - test_suites.append(test_suite) +def format_test_divider(message: str, len_message: int) -> str: + """ + Returns string with message centered in fixed width divider. + + Example: + '===================== message example =====================' + + Parameters: + message - message to be centered in divider line + len_message - length of the message to be printed such that + any characters of the color codes are not counted + + Return: + String containing message centered in fixed width divider + """ + default_count = 3 # default number of dashes + len_1 = default_count + len_2 = default_count + difference = len(DIVIDER) - len_message - 2 # 2 spaces added + if difference > 0: + # calculate number of dashes for each side of the divider + len_1 = int(difference / 2) + len_2 = difference - len_1 + return ('=' * len_1) + ' ' + message + ' ' + ('=' * len_2) + +def print_test_header(test: Test) -> None: + """ + Prints test header with test name and optionally the expected number + of subtests. + + Example: + '=================== example (2 subtests) ===================' + + Parameters: + test - Test object representing current test being printed + """ + message = test.name + if test.expected_count: + if test.expected_count == 1: + message += (' (' + str(test.expected_count) + + ' subtest)') else: - print_with_timestamp( - red('[ERROR] ') + ' expected ' + - str(expected_test_suite_num) + - ' test suites, but got ' + str(i - 2)) - break - test_suite = parse_test_suite(lines, -1) - if test_suite: - print_with_timestamp(red('[ERROR] ') + - 'got unexpected test suite: ' + test_suite.name) - if test_suites: - return TestResult(bubble_up_suite_errors(test_suites), test_suites, lines) - else: - return TestResult(TestStatus.NO_TESTS, [], lines) + message += (' (' + str(test.expected_count) + + ' subtests)') + print_with_timestamp(format_test_divider(message, len(message))) -class TestCounts: - passed: int - failed: int - crashed: int - skipped: int +def print_log(log: Iterable[str]) -> None: + """ + Prints all strings in saved log for test in yellow. - def __init__(self): - self.passed = 0 - self.failed = 0 - self.crashed = 0 - self.skipped = 0 - - def total(self) -> int: - return self.passed + self.failed + self.crashed + self.skipped - -def print_and_count_results(test_result: TestResult) -> TestCounts: - counts = TestCounts() - for test_suite in test_result.suites: - if test_suite.status == TestStatus.SUCCESS: - print_suite_divider(green('[PASSED] ') + test_suite.name) - elif test_suite.status == TestStatus.SKIPPED: - print_suite_divider(yellow('[SKIPPED] ') + test_suite.name) - elif test_suite.status == TestStatus.TEST_CRASHED: - print_suite_divider(red('[CRASHED] ' + test_suite.name)) - else: - print_suite_divider(red('[FAILED] ') + test_suite.name) - for test_case in test_suite.cases: - if test_case.status == TestStatus.SUCCESS: - counts.passed += 1 - print_with_timestamp(green('[PASSED] ') + test_case.name) - elif test_case.status == TestStatus.SKIPPED: - counts.skipped += 1 - print_with_timestamp(yellow('[SKIPPED] ') + test_case.name) - elif test_case.status == TestStatus.TEST_CRASHED: - counts.crashed += 1 - print_with_timestamp(red('[CRASHED] ' + test_case.name)) - print_log(map(yellow, test_case.log)) - print_with_timestamp('') + Parameters: + log - Iterable object with all strings saved in log for test + """ + for m in log: + print_with_timestamp(yellow(m)) + +def format_test_result(test: Test) -> str: + """ + Returns string with formatted test result with colored status and test + name. + + Example: + '[PASSED] example' + + Parameters: + test - Test object representing current test being printed + + Return: + String containing formatted test result + """ + if test.status == TestStatus.SUCCESS: + return (green('[PASSED] ') + test.name) + elif test.status == TestStatus.SKIPPED: + return (yellow('[SKIPPED] ') + test.name) + elif test.status == TestStatus.TEST_CRASHED: + print_log(test.log) + return (red('[CRASHED] ') + test.name) + else: + print_log(test.log) + return (red('[FAILED] ') + test.name) + +def print_test_result(test: Test) -> None: + """ + Prints result line with status of test. + + Example: + '[PASSED] example' + + Parameters: + test - Test object representing current test being printed + """ + print_with_timestamp(format_test_result(test)) + +def print_test_footer(test: Test) -> None: + """ + Prints test footer with status of test. + + Example: + '===================== [PASSED] example =====================' + + Parameters: + test - Test object representing current test being printed + """ + message = format_test_result(test) + print_with_timestamp(format_test_divider(message, + len(message) - ANSI_LEN)) + +def print_summary_line(test: Test) -> None: + """ + Prints summary line of test object. Color of line is dependent on + status of test. Color is green if test passes, yellow if test is + skipped, and red if the test fails or crashes. Summary line contains + counts of the statuses of the tests subtests or the test itself if it + has no subtests. + + Example: + "Testing complete. Passed: 2, Failed: 0, Crashed: 0, Skipped: 0, + Errors: 0" + + test - Test object representing current test being printed + """ + if test.status == TestStatus.SUCCESS or \ + test.status == TestStatus.NO_TESTS: + color = green + elif test.status == TestStatus.SKIPPED: + color = yellow + else: + color = red + counts = test.counts + print_with_timestamp(color('Testing complete. ' + str(counts))) + +def print_error(error_message: str) -> None: + """ + Prints error message with error format. + + Example: + "[ERROR] Test example: missing test plan!" + + Parameters: + error_message - message describing error + """ + print_with_timestamp(red('[ERROR] ') + error_message) + +# Other methods: + +def bubble_up_test_results(test: Test) -> None: + """ + If the test has subtests, add the test counts of the subtests to the + test and check if any of the tests crashed and if so set the test + status to crashed. Otherwise if the test has no subtests add the + status of the test to the test counts. + + Parameters: + test - Test object for current test being parsed + """ + parse_crash_in_log(test) + subtests = test.subtests + counts = test.counts + status = test.status + for t in subtests: + counts.add_subtest_counts(t.counts) + if counts.total() == 0: + counts.add_status(status) + elif test.counts.get_status() == TestStatus.TEST_CRASHED: + test.status = TestStatus.TEST_CRASHED + +def parse_test(lines: LineStream, expected_num: int, log: List[str]) -> Test: + """ + Finds next test to parse in LineStream, creates new Test object, + parses any subtests of the test, populates Test object with all + information (status, name) about the test and the Test objects for + any subtests, and then returns the Test object. The method accepts + three formats of tests: + + Accepted test formats: + + - Main KTAP/TAP header + + Example: + + KTAP version 1 + 1..4 + [subtests] + + - Subtest header line + + Example: + + # Subtest: name + 1..3 + [subtests] + ok 1 name + + - Test result line + + Example: + + ok 1 - test + + Parameters: + lines - LineStream of KTAP output to parse + expected_num - expected test number for test to be parsed + log - list of strings containing any preceding diagnostic lines + corresponding to the current test + + Return: + Test object populated with characteristics and any subtests + """ + test = Test() + test.log.extend(log) + parent_test = False + main = parse_ktap_header(lines, test) + if main: + # If KTAP/TAP header is found, attempt to parse + # test plan + test.name = "main" + parse_test_plan(lines, test) + else: + # If KTAP/TAP header is not found, test must be subtest + # header or test result line so parse attempt to parser + # subtest header + parent_test = parse_test_header(lines, test) + if parent_test: + # If subtest header is found, attempt to parse + # test plan and print header + parse_test_plan(lines, test) + print_test_header(test) + expected_count = test.expected_count + subtests = [] + test_num = 1 + while expected_count is None or test_num <= expected_count: + # Loop to parse any subtests. + # Break after parsing expected number of tests or + # if expected number of tests is unknown break when test + # result line with matching name to subtest header is found + # or no more lines in stream. + sub_log = parse_diagnostic(lines) + sub_test = Test() + if not lines or (peek_test_name_match(lines, test) and + not main): + if expected_count and test_num <= expected_count: + # If parser reaches end of test before + # parsing expected number of subtests, print + # crashed subtest and record error + test.add_error('missing expected subtest!') + sub_test.log.extend(sub_log) + test.counts.add_status( + TestStatus.TEST_CRASHED) + print_test_result(sub_test) else: - counts.failed += 1 - print_with_timestamp(red('[FAILED] ') + test_case.name) - print_log(map(yellow, test_case.log)) - print_with_timestamp('') - return counts + test.log.extend(sub_log) + break + else: + sub_test = parse_test(lines, test_num, sub_log) + subtests.append(sub_test) + test_num += 1 + test.subtests = subtests + if not main: + # If not main test, look for test result line + test.log.extend(parse_diagnostic(lines)) + if (parent_test and peek_test_name_match(lines, test)) or \ + not parent_test: + parse_test_result(lines, test, expected_num) + else: + test.add_error('missing subtest result line!') + # Add statuses to TestCounts attribute in Test object + bubble_up_test_results(test) + if parent_test: + # If test has subtests and is not the main test object, print + # footer. + print_test_footer(test) + elif not main: + print_test_result(test) + return test def parse_run_tests(kernel_output: Iterable[str]) -> TestResult: - counts = TestCounts() + """ + Using kernel output, extract KTAP lines, parse the lines for test + results and print condensed test results and summary line . + + Parameters: + kernel_output - Iterable object contains lines of kernel output + + Return: + TestResult - Tuple containg status of main test object, main test + object with all subtests, and log of all KTAP lines. + """ + print_with_timestamp(DIVIDER) lines = extract_tap_lines(kernel_output) - test_result = parse_test_result(lines) - if test_result.status == TestStatus.NO_TESTS: - print(red('[ERROR] ') + yellow('no tests run!')) - elif test_result.status == TestStatus.FAILURE_TO_PARSE_TESTS: - print(red('[ERROR] ') + yellow('could not parse test results!')) + test = Test() + if not lines: + test.add_error('invalid KTAP input!') + test.status = TestStatus.FAILURE_TO_PARSE_TESTS else: - counts = print_and_count_results(test_result) + test = parse_test(lines, 0, []) + if test.status != TestStatus.NO_TESTS: + test.status = test.counts.get_status() print_with_timestamp(DIVIDER) - if test_result.status == TestStatus.SUCCESS: - fmt = green - elif test_result.status == TestStatus.SKIPPED: - fmt = yellow - else: - fmt =red - print_with_timestamp( - fmt('Testing complete. %d tests run. %d failed. %d crashed. %d skipped.' % - (counts.total(), counts.failed, counts.crashed, counts.skipped))) - return test_result + print_summary_line(test) + return TestResult(test.status, test, lines) diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 619c4554cbff..2a8b0b5f4269 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -106,10 +106,10 @@ class KUnitParserTest(unittest.TestCase): with open(log_path) as file: result = kunit_parser.extract_tap_lines(file.readlines()) self.assertContains('TAP version 14', result) - self.assertContains(' # Subtest: example', result) - self.assertContains(' 1..2', result) - self.assertContains(' ok 1 - example_simple_test', result) - self.assertContains(' ok 2 - example_mock_test', result) + self.assertContains('# Subtest: example', result) + self.assertContains('1..2', result) + self.assertContains('ok 1 - example_simple_test', result) + self.assertContains('ok 2 - example_mock_test', result) self.assertContains('ok 1 - example', result) def test_output_with_prefix_isolated_correctly(self): @@ -117,28 +117,28 @@ class KUnitParserTest(unittest.TestCase): with open(log_path) as file: result = kunit_parser.extract_tap_lines(file.readlines()) self.assertContains('TAP version 14', result) - self.assertContains(' # Subtest: kunit-resource-test', result) - self.assertContains(' 1..5', result) - self.assertContains(' ok 1 - kunit_resource_test_init_resources', result) - self.assertContains(' ok 2 - kunit_resource_test_alloc_resource', result) - self.assertContains(' ok 3 - kunit_resource_test_destroy_resource', result) - self.assertContains(' foo bar #', result) - self.assertContains(' ok 4 - kunit_resource_test_cleanup_resources', result) - self.assertContains(' ok 5 - kunit_resource_test_proper_free_ordering', result) + self.assertContains('# Subtest: kunit-resource-test', result) + self.assertContains('1..5', result) + self.assertContains('ok 1 - kunit_resource_test_init_resources', result) + self.assertContains('ok 2 - kunit_resource_test_alloc_resource', result) + self.assertContains('ok 3 - kunit_resource_test_destroy_resource', result) + self.assertContains('foo bar #', result) + self.assertContains('ok 4 - kunit_resource_test_cleanup_resources', result) + self.assertContains('ok 5 - kunit_resource_test_proper_free_ordering', result) self.assertContains('ok 1 - kunit-resource-test', result) - self.assertContains(' foo bar # non-kunit output', result) - self.assertContains(' # Subtest: kunit-try-catch-test', result) - self.assertContains(' 1..2', result) - self.assertContains(' ok 1 - kunit_test_try_catch_successful_try_no_catch', + self.assertContains('foo bar # non-kunit output', result) + self.assertContains('# Subtest: kunit-try-catch-test', result) + self.assertContains('1..2', result) + self.assertContains('ok 1 - kunit_test_try_catch_successful_try_no_catch', result) - self.assertContains(' ok 2 - kunit_test_try_catch_unsuccessful_try_does_catch', + self.assertContains('ok 2 - kunit_test_try_catch_unsuccessful_try_does_catch', result) self.assertContains('ok 2 - kunit-try-catch-test', result) - self.assertContains(' # Subtest: string-stream-test', result) - self.assertContains(' 1..3', result) - self.assertContains(' ok 1 - string_stream_test_empty_on_creation', result) - self.assertContains(' ok 2 - string_stream_test_not_empty_after_add', result) - self.assertContains(' ok 3 - string_stream_test_get_string', result) + self.assertContains('# Subtest: string-stream-test', result) + self.assertContains('1..3', result) + self.assertContains('ok 1 - string_stream_test_empty_on_creation', result) + self.assertContains('ok 2 - string_stream_test_not_empty_after_add', result) + self.assertContains('ok 3 - string_stream_test_get_string', result) self.assertContains('ok 3 - string-stream-test', result) def test_parse_successful_test_log(self): @@ -148,6 +148,13 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) + def test_parse_successful_nested_tests_log(self): + all_passed_log = test_data_path('test_is_test_passed-all_passed_nested.log') + with open(all_passed_log) as file: + result = kunit_parser.parse_run_tests(file.readlines()) + self.assertEqual( + kunit_parser.TestStatus.SUCCESS, + result.status) def test_parse_failed_test_log(self): failed_log = test_data_path('test_is_test_passed-failure.log') @@ -162,17 +169,31 @@ class KUnitParserTest(unittest.TestCase): with open(empty_log) as file: result = kunit_parser.parse_run_tests( kunit_parser.extract_tap_lines(file.readlines())) - self.assertEqual(0, len(result.suites)) + self.assertEqual(0, len(result.test.subtests)) self.assertEqual( kunit_parser.TestStatus.FAILURE_TO_PARSE_TESTS, result.status) + def test_missing_test_plan(self): + missing_plan_log = test_data_path('test_is_test_passed-' + 'missing_plan.log') + with open(missing_plan_log) as file: + result = kunit_parser.parse_run_tests( + kunit_parser.extract_tap_lines( + file.readlines())) + self.assertEqual(2, result.test.counts.errors) + self.assertEqual( + kunit_parser.TestStatus.SUCCESS, + result.status) + def test_no_tests(self): - empty_log = test_data_path('test_is_test_passed-no_tests_run_with_header.log') - with open(empty_log) as file: + header_log = test_data_path('test_is_test_passed-' + 'no_tests_run_with_header.log') + with open(header_log) as file: result = kunit_parser.parse_run_tests( - kunit_parser.extract_tap_lines(file.readlines())) - self.assertEqual(0, len(result.suites)) + kunit_parser.extract_tap_lines( + file.readlines())) + self.assertEqual(0, len(result.test.subtests)) self.assertEqual( kunit_parser.TestStatus.NO_TESTS, result.status) @@ -182,15 +203,17 @@ class KUnitParserTest(unittest.TestCase): print_mock = mock.patch('builtins.print').start() with open(crash_log) as file: result = kunit_parser.parse_run_tests( - kunit_parser.extract_tap_lines(file.readlines())) - print_mock.assert_any_call(StrContains('could not parse test results!')) + kunit_parser.extract_tap_lines( + file.readlines())) + print_mock.assert_any_call(StrContains('invalid KTAP input!')) print_mock.stop() file.close() def test_crashed_test(self): crashed_log = test_data_path('test_is_test_passed-crash.log') with open(crashed_log) as file: - result = kunit_parser.parse_run_tests(file.readlines()) + result = kunit_parser.parse_run_tests( + file.readlines()) self.assertEqual( kunit_parser.TestStatus.TEST_CRASHED, result.status) @@ -216,6 +239,23 @@ class KUnitParserTest(unittest.TestCase): result.status) file.close() + def test_ignores_hyphen(self): + hyphen_log = test_data_path('test_strip_hyphen.log') + file = open(hyphen_log) + result = kunit_parser.parse_run_tests(file.readlines()) + + # A skipped test does not fail the whole suite. + self.assertEqual( + kunit_parser.TestStatus.SUCCESS, + result.status) + self.assertEqual( + "sysctl_test", + result.test.subtests[0].name) + self.assertEqual( + "example", + result.test.subtests[1].name) + file.close() + def test_ignores_prefix_printk_time(self): prefix_log = test_data_path('test_config_printk_time.log') @@ -224,7 +264,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_ignores_multiple_prefixes(self): prefix_log = test_data_path('test_multiple_prefixes.log') @@ -233,7 +273,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_prefix_mixed_kernel_output(self): mixed_prefix_log = test_data_path('test_interrupted_tap_output.log') @@ -242,7 +282,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_prefix_poundsign(self): pound_log = test_data_path('test_pound_sign.log') @@ -251,7 +291,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_kernel_panic_end(self): panic_log = test_data_path('test_kernel_panic_interrupt.log') @@ -260,7 +300,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.TEST_CRASHED, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_pound_no_prefix(self): pound_log = test_data_path('test_pound_no_prefix.log') @@ -269,7 +309,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) class LinuxSourceTreeTest(unittest.TestCase): @@ -291,6 +331,14 @@ class LinuxSourceTreeTest(unittest.TestCase): pass tree = kunit_kernel.LinuxSourceTree('', kunitconfig_path=dir) + def test_kselftest_nested(self): + kselftest_log = test_data_path('test_is_test_passed-kselftest.log') + with open(kselftest_log) as file: + result = kunit_parser.parse_run_tests(file.readlines()) + self.assertEqual( + kunit_parser.TestStatus.SUCCESS, + result.status) + # TODO: add more test cases. @@ -322,6 +370,12 @@ class KUnitJsonTest(unittest.TestCase): result = self._json_for('test_is_test_passed-no_tests_run_with_header.log') self.assertEqual(0, len(result['sub_groups'])) + def test_nested_json(self): + result = self._json_for('test_is_test_passed-all_passed_nested.log') + self.assertEqual( + {'name': 'example_simple_test', 'status': 'PASS'}, + result["sub_groups"][0]["sub_groups"][0]["test_cases"][0]) + class StrContains(str): def __eq__(self, other): return self in other @@ -380,7 +434,7 @@ class KUnitMainTest(unittest.TestCase): self.assertEqual(e.exception.code, 1) self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 1) self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) - self.print_mock.assert_any_call(StrContains(' 0 tests run')) + self.print_mock.assert_any_call(StrContains('invalid KTAP input!')) def test_exec_raw_output(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) @@ -388,7 +442,7 @@ class KUnitMainTest(unittest.TestCase): self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) for call in self.print_mock.call_args_list: self.assertNotEqual(call, mock.call(StrContains('Testing complete.'))) - self.assertNotEqual(call, mock.call(StrContains(' 0 tests run'))) + self.assertNotEqual(call, mock.call(StrContains(' 0 tests run!'))) def test_run_raw_output(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) @@ -397,7 +451,7 @@ class KUnitMainTest(unittest.TestCase): self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) for call in self.print_mock.call_args_list: self.assertNotEqual(call, mock.call(StrContains('Testing complete.'))) - self.assertNotEqual(call, mock.call(StrContains(' 0 tests run'))) + self.assertNotEqual(call, mock.call(StrContains(' 0 tests run!'))) def test_run_raw_output_kunit(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) diff --git a/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log b/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log new file mode 100644 index 000000000000..9d5b04fe43a6 --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log @@ -0,0 +1,34 @@ +TAP version 14 +1..2 + # Subtest: sysctl_test + 1..4 + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed + ok 1 - sysctl_test_dointvec_null_tbl_data + # Subtest: example + 1..2 + init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 - example_simple_test + # example_mock_test: initializing + # example_mock_test: example_mock_test passed + ok 2 - example_mock_test + kunit example: all tests passed + ok 2 - example + # sysctl_test_dointvec_table_len_is_zero: sysctl_test_dointvec_table_len_is_zero passed + ok 3 - sysctl_test_dointvec_table_len_is_zero + # sysctl_test_dointvec_table_read_but_position_set: sysctl_test_dointvec_table_read_but_position_set passed + ok 4 - sysctl_test_dointvec_table_read_but_position_set +kunit sysctl_test: all tests passed +ok 1 - sysctl_test + # Subtest: example + 1..2 +init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 - example_simple_test + # example_mock_test: initializing + # example_mock_test: example_mock_test passed + ok 2 - example_mock_test +kunit example: all tests passed +ok 2 - example diff --git a/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log new file mode 100644 index 000000000000..65d3f27feaf2 --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log @@ -0,0 +1,14 @@ +TAP version 13 +1..2 +# selftests: membarrier: membarrier_test_single_thread +# TAP version 13 +# 1..2 +# ok 1 sys_membarrier available +# ok 2 sys membarrier invalid command test: command = -1, flags = 0, errno = 22. Failed as expected +ok 1 selftests: membarrier: membarrier_test_single_thread +# selftests: membarrier: membarrier_test_multi_thread +# TAP version 13 +# 1..2 +# ok 1 sys_membarrier available +# ok 2 sys membarrier invalid command test: command = -1, flags = 0, errno = 22. Failed as expected +ok 2 selftests: membarrier: membarrier_test_multi_thread diff --git a/tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log b/tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log new file mode 100644 index 000000000000..5cd17b7f818a --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log @@ -0,0 +1,31 @@ +KTAP version 1 + # Subtest: sysctl_test + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed + ok 1 - sysctl_test_dointvec_null_tbl_data + # sysctl_test_dointvec_table_maxlen_unset: sysctl_test_dointvec_table_maxlen_unset passed + ok 2 - sysctl_test_dointvec_table_maxlen_unset + # sysctl_test_dointvec_table_len_is_zero: sysctl_test_dointvec_table_len_is_zero passed + ok 3 - sysctl_test_dointvec_table_len_is_zero + # sysctl_test_dointvec_table_read_but_position_set: sysctl_test_dointvec_table_read_but_position_set passed + ok 4 - sysctl_test_dointvec_table_read_but_position_set + # sysctl_test_dointvec_happy_single_positive: sysctl_test_dointvec_happy_single_positive passed + ok 5 - sysctl_test_dointvec_happy_single_positive + # sysctl_test_dointvec_happy_single_negative: sysctl_test_dointvec_happy_single_negative passed + ok 6 - sysctl_test_dointvec_happy_single_negative + # sysctl_test_dointvec_single_less_int_min: sysctl_test_dointvec_single_less_int_min passed + ok 7 - sysctl_test_dointvec_single_less_int_min + # sysctl_test_dointvec_single_greater_int_max: sysctl_test_dointvec_single_greater_int_max passed + ok 8 - sysctl_test_dointvec_single_greater_int_max +kunit sysctl_test: all tests passed +ok 1 - sysctl_test + # Subtest: example + 1..2 +init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 - example_simple_test + # example_mock_test: initializing + # example_mock_test: example_mock_test passed + ok 2 - example_mock_test +kunit example: all tests passed +ok 2 - example diff --git a/tools/testing/kunit/test_data/test_strip_hyphen.log b/tools/testing/kunit/test_data/test_strip_hyphen.log new file mode 100644 index 000000000000..92ac7c24b374 --- /dev/null +++ b/tools/testing/kunit/test_data/test_strip_hyphen.log @@ -0,0 +1,16 @@ +KTAP version 1 +1..2 + # Subtest: sysctl_test + 1..1 + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed + ok 1 - sysctl_test_dointvec_null_tbl_data +kunit sysctl_test: all tests passed +ok 1 - sysctl_test + # Subtest: example + 1..1 +init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 example_simple_test +kunit example: all tests passed +ok 2 example -- 2.33.0.259.gc128427fd7-goog

4 years

1
0
0 0

[RFC v2 net-next 0/2] Add RTNL interface for SyncE

by Maciej Machnikowski

Synchronous Ethernet networks use a physical layer clock to syntonize the frequency across different network elements. Multiple reference clock sources can be used. Clocks recovered from PHY ports on the RX side or external sources like 1PPS GPS, etc. This patch series introduces basic interface for reading the DPLL state on a SyncE capable device. This state gives us information about the source of the syntonization signal and whether the DPLL circuit is tuned to the incoming signal. Next steps: - add interface to enable recovered clocks and get information about them v2: - removed whitespace changes - fix issues reported by test robot Maciej Machnikowski (2): rtnetlink: Add new RTM_GETSYNCESTATE message to get SyncE status ice: add support for reading SyncE DPLL state drivers/net/ethernet/intel/ice/ice.h | 5 ++ .../net/ethernet/intel/ice/ice_adminq_cmd.h | 34 ++++++++ drivers/net/ethernet/intel/ice/ice_common.c | 62 +++++++++++++++ drivers/net/ethernet/intel/ice/ice_common.h | 4 + drivers/net/ethernet/intel/ice/ice_devids.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 55 +++++++++++++ drivers/net/ethernet/intel/ice/ice_ptp.c | 35 +++++++++ drivers/net/ethernet/intel/ice/ice_ptp_hw.c | 44 +++++++++++ drivers/net/ethernet/intel/ice/ice_ptp_hw.h | 22 ++++++ include/linux/netdevice.h | 6 ++ include/uapi/linux/if_link.h | 43 +++++++++++ include/uapi/linux/rtnetlink.h | 11 ++- net/core/rtnetlink.c | 77 +++++++++++++++++++ security/selinux/nlmsgtab.c | 3 +- 14 files changed, 399 insertions(+), 5 deletions(-) -- 2.26.3

4 years

5
20
0 0

[PATCH v3 1/2] kunit: drop assumption in kunit-log-test about current suite

by Daniel Latypov

This test assumes that the declared kunit_suite object is the exact one which is being executed, which KUnit will not guarantee [1]. Specifically, `suite->log` is not initialized until a suite object is executed. So if KUnit makes a copy of the suite and runs that instead, this test dereferences an invalid pointer and (hopefully) segfaults. N.B. since we no longer assume this, we can no longer verify that `suite->log` is *not* allocated during normal execution. An alternative to this patch that would allow us to test that would require exposing an API for the current test to get its current suite. Exposing that for one internal kunit test seems like overkill, and grants users more footguns (e.g. reusing a test case in multiple suites and changing behavior based on the suite name, dynamically modifying the setup/cleanup funcs, storing/reading stuff out of the suite->log, etc.). [1] In a subsequent patch, KUnit will allow running subsets of test cases within a suite by making a copy of the suite w/ the filtered test list. But there are other reasons KUnit might execute a copy, e.g. if it ever wants to support parallel execution of different suites, recovering from errors and restarting suites Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- lib/kunit/kunit-test.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/lib/kunit/kunit-test.c b/lib/kunit/kunit-test.c index d69efcbed624..555601d17f79 100644 --- a/lib/kunit/kunit-test.c +++ b/lib/kunit/kunit-test.c @@ -415,12 +415,15 @@ static struct kunit_suite kunit_log_test_suite = { static void kunit_log_test(struct kunit *test) { - struct kunit_suite *suite = &kunit_log_test_suite; + struct kunit_suite suite; + + suite.log = kunit_kzalloc(test, KUNIT_LOG_SIZE, GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, suite.log); kunit_log(KERN_INFO, test, "put this in log."); kunit_log(KERN_INFO, test, "this too."); - kunit_log(KERN_INFO, suite, "add to suite log."); - kunit_log(KERN_INFO, suite, "along with this."); + kunit_log(KERN_INFO, &suite, "add to suite log."); + kunit_log(KERN_INFO, &suite, "along with this."); #ifdef CONFIG_KUNIT_DEBUGFS KUNIT_EXPECT_NOT_ERR_OR_NULL(test, @@ -428,12 +431,11 @@ static void kunit_log_test(struct kunit *test) KUNIT_EXPECT_NOT_ERR_OR_NULL(test, strstr(test->log, "this too.")); KUNIT_EXPECT_NOT_ERR_OR_NULL(test, - strstr(suite->log, "add to suite log.")); + strstr(suite.log, "add to suite log.")); KUNIT_EXPECT_NOT_ERR_OR_NULL(test, - strstr(suite->log, "along with this.")); + strstr(suite.log, "along with this.")); #else KUNIT_EXPECT_PTR_EQ(test, test->log, (char *)NULL); - KUNIT_EXPECT_PTR_EQ(test, suite->log, (char *)NULL); #endif } base-commit: 9c849ce86e0fa93a218614eac562ace44053d7ce -- 2.33.0.259.gc128427fd7-goog

4 years

2
3
0 0

[PATCH v3 0/2] kselftests: clean configs

by Li Zhijian

0Day will check if all configs listing under selftests are able to be enabled properly. For the missing configs, it will report something like: LKP WARN miss config CONFIG_SYNC= of sync/config CC: "Rafael J. Wysocki" <rjw(a)rjwysocki.net> CC: Viresh Kumar <viresh.kumar(a)linaro.org> CC: linux-pm(a)vger.kernel.org Reported-by: kernel test robot <lkp(a)intel.com> Li Zhijian (2): selftests/sync: Remove the deprecated config SYNC selftests/cpufreq: Rename DEBUG_PI_LIST to DEBUG_PLIST tools/testing/selftests/cpufreq/config | 2 +- tools/testing/selftests/sync/config | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) -- 2.31.1

4 years

3
4
0 0

[REGRESSION] lkft kselftest for next-20210831

by lkft＠linaro.org

## Build * kernel: 5.14.0 * git: https://gitlab.com/Linaro/lkft/mirrors/next/linux-next * git branch: master * git commit: 52c7b727581fe725f8b8a283af21fe0651c73c48 * git describe: next-20210831 * test details: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210831 ## Regressions (compared to next-20210830) * dragonboard-410c, kselftest-capabilities - capabilities.test_execve * i386, kselftest-rtc - rtc.rtctest * qemu-arm64-gic-version2, kselftest-timers - timers.rtcpie * qemu-arm64-mte, kselftest-timers - timers.rtcpie * qemu_x86_64, kselftest-rtc - rtc.rtctest * x86, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * x86, kselftest-rtc - rtc.rtctest Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Fixes (compared to next-20210830) * qemu_arm, kselftest-rtc - rtc.rtctest - rtc.rtctest.rtc.alarm_alm_set_minute * qemu_arm64, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace ## Test result summary total: 6489, pass: 3757, fail: 565, skip: 2167, xfail: 0 ## Build Summary ## Test suites summary * kselftest-android * kselftest-arm64 * kselftest-arm64/arm64.btitest.bti_c_func * kselftest-arm64/arm64.btitest.bti_j_func * kselftest-arm64/arm64.btitest.bti_jc_func * kselftest-arm64/arm64.btitest.bti_none_func * kselftest-arm64/arm64.btitest.nohint_func * kselftest-arm64/arm64.btitest.paciasp_func * kselftest-arm64/arm64.nobtitest.bti_c_func * kselftest-arm64/arm64.nobtitest.bti_j_func * kselftest-arm64/arm64.nobtitest.bti_jc_func * kselftest-arm64/arm64.nobtitest.bti_none_func * kselftest-arm64/arm64.nobtitest.nohint_func * kselftest-arm64/arm64.nobtitest.paciasp_func * kselftest-bpf * kselftest-breakpoints * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-drivers * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kexec * kselftest-kvm * kselftest-lib * kselftest-livepatch * kselftest-lkdtm * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-tc-testing * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-vm * kselftest-x86 * kselftest-zram -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

[PATCH net-next 0/2] Add RTNL interface for SyncE EEC state

by Maciej Machnikowski

Synchronous Ethernet networks use a physical layer clock to syntonize the frequency across different network elements. Basic SyncE node defined in the ITU-T G.8264 consist of an Ethernet Equipment Clock (EEC) and have the ability to recover synchronization from the synchronization inputs - either traffic interfaces or external frequency sources. The EEC can synchronize its frequency (syntonize) to any of those sources. It is also able to select synchronization source through priority tables and synchronization status messaging. It also provides neccessary filtering and holdover capabilities This patch series introduces basic interface for reading the Ethernet Equipment Clock (EEC) state on a SyncE capable device. This state gives information about the source of the syntonization signal and the state of EEC. This interface is required to implement Synchronization Status Messaging on upper layers. Maciej Machnikowski (2): rtnetlink: Add new RTM_GETEECSTATE message to get SyncE status ice: add support for reading SyncE DPLL state drivers/net/ethernet/intel/ice/ice.h | 5 ++ .../net/ethernet/intel/ice/ice_adminq_cmd.h | 34 ++++++++++ drivers/net/ethernet/intel/ice/ice_common.c | 62 ++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_common.h | 4 ++ drivers/net/ethernet/intel/ice/ice_devids.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 57 +++++++++++++++++ drivers/net/ethernet/intel/ice/ice_ptp.c | 35 ++++++++++ drivers/net/ethernet/intel/ice/ice_ptp_hw.c | 44 +++++++++++++ drivers/net/ethernet/intel/ice/ice_ptp_hw.h | 22 +++++++ include/linux/netdevice.h | 5 ++ include/uapi/linux/if_link.h | 46 +++++++++++++ include/uapi/linux/rtnetlink.h | 3 + net/core/rtnetlink.c | 64 +++++++++++++++++++ security/selinux/nlmsgtab.c | 3 +- 14 files changed, 386 insertions(+), 1 deletion(-) -- 2.26.3

4 years

2
3
0 0

[RFC v3 net-next 0/2] Add RTNL interface for SyncE

by Maciej Machnikowski

Synchronous Ethernet networks use a physical layer clock to syntonize the frequency across different network elements. Basic SyncE node defined in the ITU-T G.8264 consist of an Ethernet Equipment Clock (EEC) and have the ability to recover synchronization from the synchronization inputs - either traffic interfaces or external frequency sources. The EEC can synchronize its frequency (syntonize) to any of those sources. It is also able to select synchronization source through priority tables and synchronization status messaging. It also provides neccessary filtering and holdover capabilities This patch series introduces basic interface for reading the Ethernet Equipment Clock (EEC) state on a SyncE capable device. This state gives information about the source of the syntonization signal and the state of EEC. This interface is required to implement Synchronization Status Messaging on upper layers. Next steps: - add interface to enable source clocks and get information about them v2: - removed whitespace changes - fix issues reported by test robot v3: - Changed naming from SyncE to EEC - Clarify cover letter and commit message for patch 1 Maciej Machnikowski (2): rtnetlink: Add new RTM_GETEECSTATE message to get SyncE status ice: add support for reading SyncE DPLL state drivers/net/ethernet/intel/ice/ice.h | 5 ++ .../net/ethernet/intel/ice/ice_adminq_cmd.h | 34 +++++++++ drivers/net/ethernet/intel/ice/ice_common.c | 62 ++++++++++++++++ drivers/net/ethernet/intel/ice/ice_common.h | 4 + drivers/net/ethernet/intel/ice/ice_devids.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 55 ++++++++++++++ drivers/net/ethernet/intel/ice/ice_ptp.c | 35 +++++++++ drivers/net/ethernet/intel/ice/ice_ptp_hw.c | 44 +++++++++++ drivers/net/ethernet/intel/ice/ice_ptp_hw.h | 22 ++++++ include/linux/netdevice.h | 6 ++ include/uapi/linux/if_link.h | 43 +++++++++++ include/uapi/linux/rtnetlink.h | 3 + net/core/rtnetlink.c | 74 +++++++++++++++++++ security/selinux/nlmsgtab.c | 3 +- 14 files changed, 392 insertions(+), 1 deletion(-) -- 2.26.3

4 years

4
5
0 0

[PATCH] selftest: gpio: undeclared variable

by cgel.zte＠gmail.com

From: Chi Minghao <chi.minghao(a)zte.com.cn> Add undeclared variable Fixes: shortid ("undeclared variable") Reported-by: Zeal Robot <zealci(a)zte.com.cn> Signed-off-by: Chi Minghao <chi.minghao(a)zte.com.cn> --- .../testing/selftests/gpio/gpio-mockup-cdev.c | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/tools/testing/selftests/gpio/gpio-mockup-cdev.c b/tools/testing/selftests/gpio/gpio-mockup-cdev.c index e83eac71621a..83a32b8835d3 100644 --- a/tools/testing/selftests/gpio/gpio-mockup-cdev.c +++ b/tools/testing/selftests/gpio/gpio-mockup-cdev.c @@ -15,9 +15,66 @@ #include <unistd.h> #include <sys/ioctl.h> #include <linux/gpio.h> +#include <linux/const.h> +#include <linux/ioctl.h> +#include <linux/types.h> #define CONSUMER "gpio-mockup-cdev" +#define GPIO_V2_LINE_NUM_ATTRS_MAX 10 +#define GPIO_V2_LINES_MAX 64 +#define GPIO_MAX_NAME_SIZE 32 + +#define GPIOHANDLE_REQUEST_BIAS_PULL_UP (1UL << 5) +#define GPIOHANDLE_REQUEST_BIAS_DISABLE (1UL << 7) +#define GPIOHANDLE_REQUEST_BIAS_PULL_DOWN (1UL << 6) +#define GPIO_V2_LINE_GET_VALUES_IOCTL _IOWR(0xB4, 0x0E, struct gpio_v2_line_values) +#define GPIO_V2_GET_LINE_IOCTL _IOWR(0xB4, 0x07, struct gpio_v2_line_request) + +enum gpio_v2_line_flag { + GPIO_V2_LINE_FLAG_ACTIVE_LOW = _BITULL(1), + GPIO_V2_LINE_FLAG_INPUT = _BITULL(2), + GPIO_V2_LINE_FLAG_OUTPUT = _BITULL(3), + GPIO_V2_LINE_FLAG_BIAS_PULL_UP = _BITULL(8), + GPIO_V2_LINE_FLAG_BIAS_PULL_DOWN = _BITULL(9), + GPIO_V2_LINE_FLAG_BIAS_DISABLED = _BITULL(10), +}; +enum gpio_v2_line_attr_id { + GPIO_V2_LINE_ATTR_ID_OUTPUT_VALUES = 2, +}; +struct gpio_v2_line_values { + __aligned_u64 bits; + __aligned_u64 mask; +}; +struct gpio_v2_line_attribute { + __u32 id; + __u32 padding; + union { + __aligned_u64 flags; + __aligned_u64 values; + __u32 debounce_period_us; + }; +}; +struct gpio_v2_line_config_attribute { + struct gpio_v2_line_attribute attr; + __aligned_u64 mask; +}; +struct gpio_v2_line_config { + __aligned_u64 flags; + __u32 num_attrs; + __u32 padding[5]; + struct gpio_v2_line_config_attribute attrs[GPIO_V2_LINE_NUM_ATTRS_MAX]; +}; +struct gpio_v2_line_request { + __u32 offsets[GPIO_V2_LINES_MAX]; + char consumer[GPIO_MAX_NAME_SIZE]; + struct gpio_v2_line_config config; + __u32 num_lines; + __u32 event_buffer_size; + __u32 padding[5]; + __s32 fd; +}; + static int request_line_v2(int cfd, unsigned int offset, uint64_t flags, unsigned int val) { -- 2.25.1

4 years

1
0
0 0

[RFC net-next 0/7] Add basic SyncE interfaces

by Arkadiusz Kubalewski

SyncE - Synchronous Ethernet is defined in ITU-T Rec. G.8264 (https://www.itu.int/rec/T-REC-G.8264) SyncE allows synchronizing the frequency of ethernet PHY clock signal (the frequency used to send the data onto wire), to some reference clock signal. Multiple reference clock sources can be available. PHY ports recover the frequency at which the transmitter sent the data on the RX side. Alternatively, we can use external sources like 1PPS GPS, etc. This patch series introduces basic interfaces for communication with a SyncE capable device. The first part of the interface allows acquiring the synchronization state of DPLL (Digital Phase Locked Loop). DPLL LOCKED state means that the frequency generated by it is locked to the input frequency. As a result, PHYs connected to it are synchronized to the chosen input frequency signal. The second part can be used to select the port from which the clock gets recovered. Each PHY chip can have multiple pins on which the recovered clock can be propagated. For example, a SyncE-capable PHY can recover the carrier frequency of the first port, divide it internally, and output it as a reference clock on PIN 0. When such a signal is enabled, the DPLL can LOCK to the frequency recovered on PIN 0. Next steps: - Add CONFIG_SYNCE definition into Kconfig - Add more configuration interfaces. Aiming at devlink, since this would be device-wide configuration Arkadiusz Kubalewski (7): ptp: Add interface for acquiring DPLL state selftests/ptp: Add usage of PTP_DPLL_GETSTATE ioctl in testptp ice: add get_dpll_state ptp interface usage net: add ioctl interface for recover reference clock on netdev selftests/net: Add test app for SIOC{S|G}SYNCE ice: add SIOC{S|G}SYNCE interface usage to recover reference signal ice: add sysfs interface to configure PHY recovered reference signal .../net/ethernet/intel/ice/ice_adminq_cmd.h | 62 +++++ drivers/net/ethernet/intel/ice/ice_common.c | 101 ++++++++ drivers/net/ethernet/intel/ice/ice_common.h | 9 + drivers/net/ethernet/intel/ice/ice_main.c | 4 + drivers/net/ethernet/intel/ice/ice_ptp.c | 234 +++++++++++++++++- drivers/net/ethernet/intel/ice/ice_ptp.h | 9 + drivers/net/ethernet/intel/ice/ice_ptp_hw.h | 6 + drivers/ptp/ptp_chardev.c | 15 ++ drivers/ptp/ptp_clockmatrix.h | 12 - drivers/ptp/ptp_private.h | 2 + drivers/ptp/ptp_sysfs.c | 48 ++++ include/linux/ptp_clock_kernel.h | 9 + include/uapi/linux/net_synce.h | 21 ++ include/uapi/linux/ptp_clock.h | 27 ++ include/uapi/linux/sockios.h | 4 + net/core/dev_ioctl.c | 6 +- tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/phy_ref_clk.c | 138 +++++++++++ tools/testing/selftests/ptp/testptp.c | 27 +- 19 files changed, 720 insertions(+), 15 deletions(-) create mode 100644 include/uapi/linux/net_synce.h create mode 100644 tools/testing/selftests/net/phy_ref_clk.c base-commit: aba1e4adb54e020d3ca85a4df3ef0f8febe87548 -- 2.24.0

4 years

6
27
0 0

[REGRESSION] lkft kselftest for next-20210830

by lkft＠linaro.org

## Build * kernel: 5.14.0 * git: ['https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git', 'https://gitlab.com/Linaro/lkft/mirrors/next/linux-next'] * git branch: master * git commit: 93717cde744f9d26aa1b4561f7d9ba2a230459eb * git describe: next-20210830 * test details: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210830 ## Regressions (compared to next-20210827) * dragonboard-410c, kselftest-net - net.udpgso_bench.sh * qemu-arm64-gic-version3, kselftest-cgroup - cgroup.test_freezer - cgroup.test_freezer.test_cgfreezer_ptrace * qemu-arm64-gic-version3, kselftest-timers - timers.rtcpie * qemu_arm, kselftest-rtc - rtc.rtctest.rtc.alarm_alm_set_minute Reported-by: Linux Kernel Functional Testing <lkft(a)linaro.org> ## Fixes (compared to next-20210827) * qemu_arm, kselftest-timers - timers.rtcpie * qemu_i386, kselftest-rtc - rtc.rtctest * qemu_i386, kselftest-seccomp - seccomp.seccomp_benchmark ## Test result summary total: 5374, pass: 3047, fail: 465, skip: 1862, xfail: 0 ## Build Summary ## Test suites summary * kselftest-android * kselftest-arm64 * kselftest-arm64/arm64.btitest.bti_c_func * kselftest-arm64/arm64.btitest.bti_j_func * kselftest-arm64/arm64.btitest.bti_jc_func * kselftest-arm64/arm64.btitest.bti_none_func * kselftest-arm64/arm64.btitest.nohint_func * kselftest-arm64/arm64.btitest.paciasp_func * kselftest-arm64/arm64.nobtitest.bti_c_func * kselftest-arm64/arm64.nobtitest.bti_j_func * kselftest-arm64/arm64.nobtitest.bti_jc_func * kselftest-arm64/arm64.nobtitest.bti_none_func * kselftest-arm64/arm64.nobtitest.nohint_func * kselftest-arm64/arm64.nobtitest.paciasp_func * kselftest-bpf * kselftest-breakpoints * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-drivers * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kexec * kselftest-kvm * kselftest-lib * kselftest-livepatch * kselftest-lkdtm * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-tc-testing * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-vm * kselftest-x86 * kselftest-zram -- Linaro LKFT https://lkft.linaro.org

4 years

1
0
0 0

[PATCH 0/2] selftests: tool: update format of kselftest TAP results to improve compatibility with KTAP

by Rae Moar

This series of patches updates the format of kselftest TAP results to improve compatibility with the proposed KTAP specification (https://lore.kernel.org/linux-kselftest/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqa…). Three changes: - Change from "# " to " " for indentation of nested tests - Add subtest header line at start of tests with subtests. Line format is "# Subtest: [name of test]". - Remove TAP header in nested tests Standardizing TAP results would not only allow for clearer documentation and ease of reading but by standardizing the format across different testing frameworks, we could also share the use of tools. As an example: This is a truncated version of TAP results from the kselftest ptrace with the new format changes: TAP version 13 1..1 # selftests: ptrace: get_syscall_info # Subtest: selftests: ptrace: get_syscall_info 1..1 # Starting 1 tests from 1 test cases. # RUN global.get_syscall_info ... # OK global.get_syscall_info ok 1 global.get_syscall_info # PASSED: 1 / 1 tests passed. # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0 ok 1 selftests: ptrace: get_syscall_info With the new patch to update the KUnit parser to improve compatibility with the proposed KTAP specification, (https://lore.kernel.org/linux-kselftest/20210826195505.3066755-1-rmoar@goog…) the above TAP results would be parsed as the following: [20:46:09] ============================================================ [20:46:09] ===== selftests: ptrace: get_syscall_info (1 subtest) ====== [20:46:09] [PASSED] global.get_syscall_info [20:46:09] ======= [PASSED] selftests: ptrace: get_syscall_info ======= [20:46:09] ============================================================ [20:46:09] Testing complete. Passed: 1, Failed: 0, Crashed: 0, Skipped: 0, Errors: 0 Thus, the kunit parser could become a useful tool for kselftest users. Rae Moar (2): selftests: tool: Add subtest header line and change indentation format in TAP results Revert "selftests: Remove KSFT_TAP_LEVEL" tools/testing/selftests/Makefile | 6 ++++++ tools/testing/selftests/kselftest/prefix.pl | 2 +- tools/testing/selftests/kselftest/runner.sh | 7 ++++--- 3 files changed, 11 insertions(+), 4 deletions(-) -- 2.33.0.259.gc128427fd7-goog

4 years

2
6
0 0

Re: [PATCH v2] kunit: tool: improve compatibility of kunit_parser with KTAP specification

by Rae Moar

On Mon, Aug 30, 2021 at 4:08 PM <Tim.Bird(a)sony.com> wrote: > > > > > -----Original Message----- > > From: Rae Moar <rmoar(a)google.com> > > > > On Thu, Aug 26, 2021 at 7:49 PM Daniel Latypov <dlatypov(a)google.com> wrote: > > > > > > On Thu, Aug 26, 2021 at 12:55 PM 'Rae Moar' via KUnit Development > > > <kunit-dev(a)googlegroups.com> wrote: > > > > > > > > Update to kunit_parser to improve compatibility with KTAP > > > > specification including arbitrarily nested tests. Patch accomplishes > > > > three major changes: > > > > > > > > - Use a general Test object to represent all tests rather than TestCase > > > > and TestSuite objects. This allows for easier implementation of arbitrary > > > > levels of nested tests and promotes the idea that both test suites and test > > > > cases are tests. > > > > > > > > - Print errors incrementally rather than all at once after the > > > > parsing finishes to maximize information given to the user in the > > > > case of the parser given invalid input and to increase the helpfulness > > > > of the timestamps given during printing. Note that kunit.py parse does > > > > not print incrementally yet. However, this fix brings us closer to > > > > this feature. > > > > > > > > - Increase compatibility for different formats of input. Arbitrary levels > > > > of nested tests supported. Also, test cases and test suites are now > > > > supported to be present on the same level of testing. > > > > > > > > This patch now implements the KTAP specification as described here: > > > > https://lore.kernel.org/linux-kselftest/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqa…. > > > > > > > > This patch adjusts the kunit_tool_test.py file to check for > > > > the correct outputs from the new parser and adds a new test to check > > > > the parsing for a KTAP result log with correct format for multiple nested > > > > subtests (test_is_test_passed-all_passed_nested.log). > > > > > > > > This patch also alters the kunit_json.py file to allow for arbitrarily > > > > nested tests. > > > > > > > > Signed-off-by: Rae Moar <rmoar(a)google.com> > > > > Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com> > > > > > > Acked-by: Daniel Latypov <dlatypov(a)google.com> > > > > > > Looks good to me, some more bikeshedding about comments below. > > > pytype does find a minor error (which was actually my fault since I'd > > > suggested you annotate `build_dir: str` off-list). > > > Also, this patch accidentally adds kunit_parser.raw_output() back in. > > > > > > > Thanks for pointing out the pytype error! I will change the annotation in > > v3. I will also remove kunit_parser.raw_output() in v3. > > > > > > > > > > --- > > > > > > > > Change log from v1: > > > > https://lore.kernel.org/linux-kselftest/20210820200032.2178134-1-rmoar@goog… > > > > - Rebase onto kselftest/kunit branch > > > > - Add tests to kunit_tool_test.py to check parser is correctly stripping > > > > hyphen, producing correct json objects with nested tests, correctly > > > > passing kselftest TAP output, and correctly deals with missing test plan. > > > > - Fix bug to correctly match test name in instance of a missing test plan. > > > > - Fix bug in kunit_tool_test.py pointed out by Daniel where it was not > > > > correctly checking for a proper match to the '0 tests run!' error > > > > message. Reverts changes back to original. > > > > - A few minor changes to commit message using Daniel's comments. > > > > - Change docstrings using Daniel's comments to reduce: > > > > - Shortens some docstrings to be one-line or just description if it is > > > > self explanatory. > > > > - Remove explicit respecification of types of parameters and returns > > > > this is already specified in the function annoations. However, > > > > some descriptions of the parameters and returns remain and some contain > > > > the type for context. Additionally, the types of public attributes of > > > > classes remain. > > > > - Remove any documentation of 'Return: None' > > > > - Remove docstrings of helper methods within other methods. > > > > --- > > > > tools/testing/kunit/kunit_json.py | 55 +- > > > > tools/testing/kunit/kunit_parser.py | 1056 ++++++++++++----- > > > > tools/testing/kunit/kunit_tool_test.py | 134 ++- > > > > .../test_is_test_passed-all_passed_nested.log | 34 + > > > > .../test_is_test_passed-kselftest.log | 14 + > > > > .../test_is_test_passed-missing_plan.log | 31 + > > > > .../kunit/test_data/test_strip_hyphen.log | 16 + > > > > 7 files changed, 951 insertions(+), 389 deletions(-) > > > > create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log > > > > create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-kselftest.log > > > > create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log > > > > create mode 100644 tools/testing/kunit/test_data/test_strip_hyphen.log > > > > > > > > diff --git a/tools/testing/kunit/kunit_json.py b/tools/testing/kunit/kunit_json.py > > > > index f5cca5c38cac..e7317b4fad9d 100644 > > > > --- a/tools/testing/kunit/kunit_json.py > > > > +++ b/tools/testing/kunit/kunit_json.py > > > > @@ -11,47 +11,46 @@ import os > > > > > > > > import kunit_parser > > > > > > > > -from kunit_parser import TestStatus > > > > - > > > > -def get_json_result(test_result, def_config, build_dir, json_path) -> str: > > > > - sub_groups = [] > > > > - > > > > - # Each test suite is mapped to a KernelCI sub_group > > > > - for test_suite in test_result.suites: > > > > - sub_group = { > > > > - "name": test_suite.name, > > > > - "arch": "UM", > > > > - "defconfig": def_config, > > > > - "build_environment": build_dir, > > > > - "test_cases": [], > > > > - "lab_name": None, > > > > - "kernel": None, > > > > - "job": None, > > > > - "git_branch": "kselftest", > > > > - } > > > > - test_cases = [] > > > > - # TODO: Add attachments attribute in test_case with detailed > > > > - # failure message, see https://api.kernelci.org/schema-test-case.html#get > > > > - for case in test_suite.cases: > > > > - test_case = {"name": case.name, "status": "FAIL"} > > > > - if case.status == TestStatus.SUCCESS: > > > > +from kunit_parser import Test, TestResult, TestStatus > > > > +from typing import Any, Dict > > > > + > > > > +JsonObj = Dict[str, Any] > > > > + > > > > +def _get_group_json(test: Test, def_config: str, build_dir: str) -> JsonObj: > > > > > > > > > Ah, pytype complains about this. > > > Looks like the unit test passes None here. > > > > > > We can either make this annotation say `build_dir: Optional[str]` or > > > change the unit test to pass '' instead of None. > > > The former might be preferable to keep this change more focused. > > > > > > > Thanks for finding this! I also like 'build_dir: Optional[str]' to keep the > > changes limited to the files already being updated. > > > > > > > > > + sub_groups = [] # List[JsonObj] > > > > + test_cases = [] # List[JsonObj] > > > > + > > > > + for subtest in test.subtests: > > > > + if len(subtest.subtests): > > > > + sub_group = _get_group_json(subtest, def_config, > > > > + build_dir) > > > > + sub_groups.append(sub_group) > > > > + else: > > > > + test_case = {"name": subtest.name, "status": "FAIL"} > > > > + if subtest.status == TestStatus.SUCCESS: > > > > test_case["status"] = "PASS" > > > > - elif case.status == TestStatus.TEST_CRASHED: > > > > + elif subtest.status == TestStatus.TEST_CRASHED: > > > > test_case["status"] = "ERROR" > > > > test_cases.append(test_case) > > > > - sub_group["test_cases"] = test_cases > > > > - sub_groups.append(sub_group) > > > > + > > > > test_group = { > > > > - "name": "KUnit Test Group", > > > > + "name": test.name, > > > > "arch": "UM", > > > > "defconfig": def_config, > > > > "build_environment": build_dir, > > > > "sub_groups": sub_groups, > > > > + "test_cases": test_cases, > > > > "lab_name": None, > > > > "kernel": None, > > > > "job": None, > > > > "git_branch": "kselftest", > > > > } > > > > + return test_group > > > > + > > > > +def get_json_result(test_result: TestResult, def_config: str, build_dir: str, > > > > + json_path: str) -> str: > > > > + test_group = _get_group_json(test_result.test, def_config, build_dir) > > > > + test_group["name"] = "KUnit Test Group" > > > > json_obj = json.dumps(test_group, indent=4) > > > > if json_path != 'stdout': > > > > with open(json_path, 'w') as result_path: > > > > diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py > > > > index 6310a641b151..4b6086159c7f 100644 > > > > --- a/tools/testing/kunit/kunit_parser.py > > > > +++ b/tools/testing/kunit/kunit_parser.py > > > > @@ -1,11 +1,15 @@ > > > > # SPDX-License-Identifier: GPL-2.0 > > > > # > > > > -# Parses test results from a kernel dmesg log. > > > > +# Parses KTAP test results from a kernel dmesg log and incrementally prints > > > > +# results with reader-friendly format. Stores and returns test results in a > > > > +# Test object. > > > > # > > > > # Copyright (C) 2019, Google LLC. > > > > # Author: Felix Guo <felixguoxiuping(a)gmail.com> > > > > # Author: Brendan Higgins <brendanhiggins(a)google.com> > > > > +# Author: Rae Moar <rmoar(a)google.com> > > > > > > > > +from __future__ import annotations > > > > import re > > > > > > > > from collections import namedtuple > > > > @@ -14,33 +18,55 @@ from enum import Enum, auto > > > > from functools import reduce > > > > from typing import Iterable, Iterator, List, Optional, Tuple > > > > > > > > -TestResult = namedtuple('TestResult', ['status','suites','log']) > > > > - > > > > -class TestSuite(object): > > > > - def __init__(self) -> None: > > > > - self.status = TestStatus.SUCCESS > > > > - self.name = '' > > > > - self.cases = [] # type: List[TestCase] > > > > - > > > > - def __str__(self) -> str: > > > > - return 'TestSuite(' + str(self.status) + ',' + self.name + ',' + str(self.cases) + ')' > > > > - > > > > - def __repr__(self) -> str: > > > > - return str(self) > > > > - > > > > -class TestCase(object): > > > > +TestResult = namedtuple('TestResult', ['status','test','log']) > > > > + > > > > +class Test(object): > > > > + """ > > > > + A class to represent a test parsed from KTAP results. All KTAP > > > > + results within a test log are stored in a main Test object as > > > > + subtests. > > > > + > > > > + Attributes: > > > > + status : TestStatus - status of the test > > > > + name : str - name of the test > > > > + expected_count : int - expected number of subtests (0 if single > > > > + test case and None if unknown expected number of subtests) > > > > + subtests : List[Test] - list of subtests > > > > + log : List[str] - log of KTAP lines that correspond to the test > > > > + counts : TestCounts - counts of the test statuses and errors of > > > > + subtests or of the test itself if the test is a single > > > > + test case. > > > > + """ > > > > def __init__(self) -> None: > > > > + """Constructs the default attributes of a Test class object. > > > > + """ > > > > > > Optional: I think we can still drop documents like this on __init__() > > > and __str__() since these are well-known functions. > > > __init__() could have a comment if it did anything non-obvious, but > > > that's not the case here. > > > > > > > I would prefer to keep docstrings for all public methods in classes, > > including __init__() and __str__(). This best follows the docstring > > conventions (https://www.python.org/dev/peps/pep-0257/#specification). > > I have attempted to make the docstrings for simple methods quite simplistic. > > I may even change this docstring to be: 'Creates Test object with default > > attributes' in order to not refer to the low-level details of constructing the > > attributes. > > > > > > self.status = TestStatus.SUCCESS > > > > self.name = '' > > > > + self.expected_count = 0 # type: Optional[int] > > > > + self.subtests = [] # type: List[Test] > > > > self.log = [] # type: List[str] > > > > + self.counts = TestCounts() > > > > > > > > def __str__(self) -> str: > > > > - return 'TestCase(' + str(self.status) + ',' + self.name + ',' + str(self.log) + ')' > > > > + """Returns string representation of a Test class object.""" > > > > + return ('Test(' + str(self.status) + ', ' + self.name + > > > > + ', ' + str(self.expected_count) + ', ' + > > > > + str(self.subtests) + ', ' + str(self.log) + ', ' + > > > > + str(self.counts) + ')') > > > > > > > > def __repr__(self) -> str: > > > > + """Returns string representation of a Test class object.""" > > > > return str(self) > > > > > > > > + def add_error(self, error_message: str) -> None: > > > > + """Adds error to test object by incrementing the error count > > > > + and printing the error message. > > > > + """ > > > > > > nit: this comment explains the low-level details of what the function > > > does, which isn't as necessary. > > > > > > We'd want to document more what the intent of the function is, e.g. perhaps: > > > Records an error that occured while parsing this test. > > > > > > In the future, we might want to change how errors are printed, hence > > > why I don't want to include that in a comment, per se. > > > > > > E.g. I've seen a case where a user was getting truncated TAP output, > > > but it was hard to tell. > > > The error message was logged at the top but got lost after the parser > > > went on to print out all the test results. > > > > > > The parser will behave the same way after this patch. > > > So we might eventually want to have a follow-up that includes a > > > summary of all the encountered errors at the end, for example. > > > > I like this change. I will update the docstring to be: > > 'Records an error that occurred while parsing this test' in v3. > > > > > > > > > + self.counts.errors += 1 > > > > + print_error('Test ' + self.name + ': ' + error_message) > > > > + > > > > class TestStatus(Enum): > > > > + """An enumeration class to represent the status of a test.""" > > > > SUCCESS = auto() > > > > FAILURE = auto() > > > > SKIPPED = auto() > > > > @@ -48,381 +74,769 @@ class TestStatus(Enum): > > > > NO_TESTS = auto() > > > > FAILURE_TO_PARSE_TESTS = auto() > > > > > > > > +class TestCounts: > > > > + """ > > > > + A class to represent the counts of statuses and test errors of > > > > + subtests or of the test itself if the test is a single test case with > > > > + no subtests. Note that the sum of the counts of passed, failed, > > > > + crashed, and skipped should sum to the total number of subtests for > > > > + the test. > > > > > > optional: can tersen this a bit. e.g. I don't think we need to > > > reiterate that this is a class, for example :) > > > I'd personally write it a bit more like > > > > > > Tracks the number of tests that had each status. > > > This includes the test itself and all its subtests. > > > > > > > I agree this can be shortened. I will definitely be removing the > > second sentence. > > That can be described in the total() method. I will likely change the > > docstring to be: > > 'Tracks the counts of statuses of all test cases and any errors within a Test.' > > > > This correctly describes that the TestCounts object only tracks the > > statuses of test > > cases with no subtests. So if the test itself had subtests, it's > > status would not be > > counted. However, this does use the phrase 'test case', which we were attempting > > to move away from. What do you think of this change? > > > > > > + > > > > + Attributes: > > > > + passed : int - the number of tests that have passed > > > > + failed : int - the number of tests that have failed > > > > + crashed : int - the number of tests that have crashed > > > > + skipped : int - the number of tests that have skipped > > > > + errors : int - the number of errors in the test and subtests > > > > + """ > > > > + def __init__(self): > > > > + """Contructs the default attributes of a TestCounts class > > > > + object. Sets the counts of all test statuses and test > > > > + errors to be 0. > > > > + """ > > > > + self.passed = 0 > > > > + self.failed = 0 > > > > + self.crashed = 0 > > > > + self.skipped = 0 > > > > + self.errors = 0 > > > > + > > > > + def __str__(self) -> str: > > > > + """Returns the string representation of a TestCounts object. > > > > + """ > > > > > > Echoing the comment above, I think comments can be dropped for > > > functions like __str__. > > > > See comment above on docstrings. > > > > > > > > > + return ('Passed: ' + str(self.passed) + > > > > + ', Failed: ' + str(self.failed) + > > > > + ', Crashed: ' + str(self.crashed) + > > > > + ', Skipped: ' + str(self.skipped) + > > > > + ', Errors: ' + str(self.errors)) > > > > + > > > > + def total(self) -> int: > > > > + """Returns total number of subtests or 1 if the test object > > > > + has no subtests to represent the test itself. This number is > > > > + calculated by the sum of the passed, failed, crashed, and > > > > + skipped subtests. > > > > > > I think the first sentence would be sufficient here, e.g. > > > Returns the number of subtests, or 1 if the test had none. > > > > I will either just use the first sentence. Or I will change this to be: > > 'Returns the total number of test cases within a test object.' > > I think this more clearly depicts what this method returns. > > I may add: 'where a test case is a test with no subtests.' See > > above comment on class docstring. Again > > is this an issue to be using the term test case? > > > > > > > > > + """ > > > > + return (self.passed + self.failed + self.crashed + > > > > + self.skipped) > > > > + > > > > + def add_subtest_counts(self, counts: TestCounts) -> None: > > > > + """ > > > > + Adds the counts of another TestCounts object to the current > > > > + TestCounts object. Used to add the counts of a subtest to the > > > > + parent test. > > > > + > > > > + Parameters: > > > > + counts - a different TestCounts object whose counts > > > > + will be added to the counts of the TestCounts object > > > > + """ > > > > + self.passed += counts.passed > > > > + self.failed += counts.failed > > > > + self.crashed += counts.crashed > > > > + self.skipped += counts.skipped > > > > + self.errors += counts.errors > > > > + > > > > + def get_status(self) -> TestStatus: > > > > + """Returns the expected status of a Test using test counts.""" > > > > > > nit: would "aggregated status" be a bit more accurate than "expected status" ? > > > > > > > I like the change to "aggregated status". I will change this is v3 :) > > > > > > + if self.crashed: > > > > + # If one of the subtests crash, the expected status > > > > + # of the Test is crashed. > > > > + return TestStatus.TEST_CRASHED > > > > + elif self.failed: > > > > + # Otherwise if one of the subtests fail, the > > > > + # expected status of the Test is failed. > > > > + return TestStatus.FAILURE > > > > + elif self.passed: > > > > + # Otherwise if one of the subtests pass, the > > > > + # expected status of the Test is passed. > > > > + return TestStatus.SUCCESS > > > > + else: > > > > + # Finally, if none of the subtests have failed, > > > > + # crashed, or passed, the expected status of the > > > > + # Test is skipped. > > > > + return TestStatus.SKIPPED > > > > + > > > > + def add_status(self, status: TestStatus) -> None: > > > > + """ > > > > + Given inputted status, increments corresponding attribute of > > > > + TestCounts object. > > > > + > > > > + Parameters: > > > > + status - status to be added to the TestCounts object > > > > + """ > > > > + if status == TestStatus.SUCCESS or \ > > > > + status == TestStatus.NO_TESTS: > > > > + # if status is NO_TESTS the most appropriate > > > > + # attribute to increment is passed because > > > > + # the test did not fail, crash or get skipped. > > > > + self.passed += 1 > > > > + elif status == TestStatus.FAILURE: > > > > + self.failed += 1 > > > > + elif status == TestStatus.SKIPPED: > > > > + self.skipped += 1 > > > > + else: > > > > + self.crashed += 1 > > > > + > > > > class LineStream: > > > > - """Provides a peek()/pop() interface over an iterator of (line#, text).""" > > > > + """ > > > > + A class to represent the lines of kernel output. > > > > + Provides a peek()/pop() interface over an iterator of > > > > + (line#, text). > > > > + """ > > > > _lines: Iterator[Tuple[int, str]] > > > > _next: Tuple[int, str] > > > > _done: bool > > > > > > > > def __init__(self, lines: Iterator[Tuple[int, str]]): > > > > + """Set defaults for LineStream object and sets _lines > > > > + attribute to lines parameter. > > > > > > nit: this "initializes" more than "set[s] defaults". > > > Perhaps: > > > Creates a new LineStream that wraps the given iterator. > > > > > > > Thanks for this suggestion! I will definitely change this in v3. > > I prefer this description much more. > > > > > > > > > + """ > > > > self._lines = lines > > > > self._done = False > > > > self._next = (0, '') > > > > self._get_next() > > > > > > > > def _get_next(self) -> None: > > > > + """Advances the LineSteam to the next line or sets the _done > > > > + attribute if the LineStream has reached the end of the lines. > > > > + """ > > > > try: > > > > self._next = next(self._lines) > > > > except StopIteration: > > > > self._done = True > > > > > > > > def peek(self) -> str: > > > > + """Returns the next line in the LineStream without advancing > > > > + the LineStream. > > > > > > Looks good to me, thanks! > > > > > > But perhaps saying "the current line" would be more clear? > > > > > > I.e. > > > peek() > > > Returns the current line, without advancing the LineStream. > > > > > > pop() > > > Returns the current line and advances the LineStream to the next line. > > > > > > line_number() > > > Returns the line number of the current line. > > > > > > > I like all of these changes. Will change in v3. > > > > > > + """ > > > > return self._next[1] > > > > > > > > def pop(self) -> str: > > > > + """Returns the next line in the LineStream and advances the > > > > + LineStream to the next line. > > > > + """ > > > > n = self._next > > > > self._get_next() > > > > return n[1] > > > > > > > > def __bool__(self) -> bool: > > > > + """Returns whether the LineStream has reached the end of the > > > > + lines. > > > > + """ > > > > > > Perhaps this (a bit shorter): > > > Checks whether there's any more lines in the stream. > > > > I do think this could be shorter. However, I think 'checks' is a bit inexact. > > I prefer 'Returns if stream has any more lines' or > > 'Returns if stream is at end of lines.' > > These have opposite meanings. Does True mean there are moremy m > lines, or there are no more lines? You should pick the wording > that makes sense according to the value. I would recommend > to include the word "True" in the description. As in: > "Returns True if stream has more lines". > My mistake. Thanks for catching this! I like 'Returns True if stream has more lines.' > > > > > > > > > return not self._done > > > > > > > > # Only used by kunit_tool_test.py. > > > > def __iter__(self) -> Iterator[str]: > > > > + """Empties all lines stored in LineStream object into > > > > + Iterator object and returns the Iterator object. > > > > + """ > > > > > > Nit: it doesn't really return an iterator object. > > > Given this is another __ function, I think we can leave off the comment. > > > > > > The behavior is about what one would expect for __iter__(). > > > > > > > I think it is best to keep the simplistic docstrings for all public methods > > (see comment above). However, I thought this function did return an Iterator. > > I may be mistaken but since this method uses the yield keyword, this > > function would return a generator, which is a type of iterator. Since I > > would like to keep the docstring, could you explain further? > > > > > > while bool(self): > > > > yield self.pop() > > > > > > > > def line_number(self) -> int: > > > > + """Returns the line number of the next line in the > > > > + LineStream. > > > > + """ > > > > return self._next[0] > > > > > > > > -kunit_start_re = re.compile(r'TAP version [0-9]+$') > > > > -kunit_end_re = re.compile('(List of all partitions:|' > > > > - 'Kernel panic - not syncing: VFS:|reboot: System halted)') > > > > +# Parsing helper methods: > > > > + > > > > +KTAP_START = re.compile(r'KTAP version ([0-9]+)$') > > > > +TAP_START = re.compile(r'TAP version ([0-9]+)$') > > > > +KTAP_END = re.compile('(List of all partitions:|' > > > > + 'Kernel panic - not syncing: VFS:|reboot: System halted)') > > > > > > > > def extract_tap_lines(kernel_output: Iterable[str]) -> LineStream: > > > > - def isolate_kunit_output(kernel_output: Iterable[str]) -> Iterator[Tuple[int, str]]: > > > > + """Extracts KTAP lines from inputted kernel output in LineStream > > > > + object.""" > > > > + def isolate_ktap_output(kernel_output: Iterable[str]) \ > > > > + -> Iterator[Tuple[int, str]]: > > > > line_num = 0 > > > > started = False > > > > for line in kernel_output: > > > > line_num += 1 > > > > - line = line.rstrip() # line always has a trailing \n > > > > - if kunit_start_re.search(line): > > > > + line = line.rstrip() # remove trailing \n > > > > + if not started and KTAP_START.search(line): > > > > + # start extracting KTAP lines and set prefix > > > > + # to number of characters before version line > > > > + prefix_len = len( > > > > + line.split('KTAP version')[0]) > > > > + started = True > > > > + yield line_num, line[prefix_len:] > > > > + elif not started and TAP_START.search(line): > > > > + # start extracting KTAP lines and set prefix > > > > + # to number of characters before version line > > > > prefix_len = len(line.split('TAP version')[0]) > > > > started = True > > > > yield line_num, line[prefix_len:] > > > > - elif kunit_end_re.search(line): > > > > + elif started and KTAP_END.search(line): > > > > + # stop extracting KTAP lines > > > > break > > > > elif started: > > > > - yield line_num, line[prefix_len:] > > > > - return LineStream(lines=isolate_kunit_output(kernel_output)) > > > > - > > > > -DIVIDER = '=' * 60 > > > > - > > > > -RESET = '\033[0;0m' > > > > - > > > > -def red(text) -> str: > > > > - return '\033[1;31m' + text + RESET> - > > > > -def yellow(text) -> str: > > > > - return '\033[1;33m' + text + RESET > > > > - > > > > -def green(text) -> str: > > > > - return '\033[1;32m' + text + RESET > > > > - > > > > -def print_with_timestamp(message) -> None: > > > > - print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) > > > > - > > > > -def format_suite_divider(message) -> str: > > > > - return '======== ' + message + ' ========' > > > > - > > > > -def print_suite_divider(message) -> None: > > > > - print_with_timestamp(DIVIDER) > > > > - print_with_timestamp(format_suite_divider(message)) > > > > - > > > > -def print_log(log) -> None: > > > > - for m in log: > > > > - print_with_timestamp(m) > > > > - > > > > -TAP_ENTRIES = re.compile(r'^(TAP|[\s]*ok|[\s]*not ok|[\s]*[0-9]+\.\.[0-9]+|[\s]*# (Subtest:|.*: kunit test case crashed!)).*$') > > > > - > > > > -def consume_non_diagnostic(lines: LineStream) -> None: > > > > - while lines and not TAP_ENTRIES.match(lines.peek()): > > > > - lines.pop() > > > > - > > > > -def save_non_diagnostic(lines: LineStream, test_case: TestCase) -> None: > > > > - while lines and not TAP_ENTRIES.match(lines.peek()): > > > > - test_case.log.append(lines.peek()) > > > > - lines.pop() > > > > - > > > > -OkNotOkResult = namedtuple('OkNotOkResult', ['is_ok','description', 'text']) > > > > - > > > > -OK_NOT_OK_SKIP = re.compile(r'^[\s]*(ok|not ok) [0-9]+ - (.*) # SKIP(.*)$') > > > > - > > > > -OK_NOT_OK_SUBTEST = re.compile(r'^[\s]+(ok|not ok) [0-9]+ - (.*)$') > > > > - > > > > -OK_NOT_OK_MODULE = re.compile(r'^(ok|not ok) ([0-9]+) - (.*)$') > > > > - > > > > -def parse_ok_not_ok_test_case(lines: LineStream, test_case: TestCase) -> bool: > > > > - save_non_diagnostic(lines, test_case) > > > > - if not lines: > > > > - test_case.status = TestStatus.TEST_CRASHED > > > > - return True > > > > - line = lines.peek() > > > > - match = OK_NOT_OK_SUBTEST.match(line) > > > > - while not match and lines: > > > > - line = lines.pop() > > > > - match = OK_NOT_OK_SUBTEST.match(line) > > > > - if match: > > > > - test_case.log.append(lines.pop()) > > > > - test_case.name = match.group(2) > > > > - skip_match = OK_NOT_OK_SKIP.match(line) > > > > - if skip_match: > > > > - test_case.status = TestStatus.SKIPPED > > > > - return True > > > > - if test_case.status == TestStatus.TEST_CRASHED: > > > > - return True > > > > - if match.group(1) == 'ok': > > > > - test_case.status = TestStatus.SUCCESS > > > > - else: > > > > - test_case.status = TestStatus.FAILURE > > > > - return True > > > > + # remove prefix and any indention and yield > > > > + # line with line number > > > > + line = line[prefix_len:].lstrip() > > > > + yield line_num, line > > > > + return LineStream(lines=isolate_ktap_output(kernel_output)) > > > > + > > > > +def raw_output(kernel_output: Iterable[str]) -> None: > > > > + """Prints all lines of kernel output.""" > > > > + for line in kernel_output: > > > > + print(line.rstrip()) > > > > > > This accidentally adds back in the raw_output() function. > > > We'll want to drop it, since it's no longer used by kunit.py. > > > > > > > Great catch! Will remove this for v3. Didn't realize this was > > supposed to be removed. > > > > > > + > > > > +KTAP_VERSIONS = [1] > > > > +TAP_VERSIONS = [13, 14] > > > > + > > > > +def check_version(version_num: int, accepted_versions: List[int], > > > > + version_type: str, test: Test) -> None: > > > > > > Optional: we can make it clearer that this is an internal function by > > > prefixing with a _ > > > E.g. def _check_version > > > > > > Same can go for all the other helpers here. > > > pytype should be able to catch any missed names, so it should be low risk. > > > > > > But it's also fairly low reward, since we don't really have any other > > > users, so the need to clamp down on what's "visible" to other packages > > > is small. > > > > > > > I might leave off the _ prefix because it would require almost all of the > > methods to be prefixed. However, this is reasonable so if others think > > this is necessary, I am happy to change all of the methods. > > > > > > + """ > > > > + Adds error to test object if version number is too high or too > > > > + low. > > > > + > > > > + Parameters: > > > > + version_num - The inputted version number from the parsed KTAP or TAP > > > > + header line > > > > + accepted_version - List of accepted KTAP or TAP versions > > > > + version_type - 'KTAP' or 'TAP' depending on the type of > > > > + version line. > > > > + test - Test object for current test being parsed > > > > + """ > > > > + if version_num < min(accepted_versions): > > > > + test.add_error(version_type + > > > > + ' version lower than expected!') > > > > + elif version_num > max(accepted_versions): > > > > + test.add_error( > > > > + version_type + ' version higher than expected!') > > > > + > > > > +def parse_ktap_header(lines: LineStream, test: Test) -> bool: > > > > + """ > > > > + If the next line in LineStream matches the format of KTAP or TAP > > > > + header line, the version number is checked, the line is popped, > > > > + and returns True. Otherwise the method returns False. > > > > + > > > > + Accepted formats: > > > > + - 'KTAP version [version number]' > > > > + - 'TAP version [version number]' > > > > + > > > > + Parameters: > > > > + lines - LineStream of KTAP output to parse > > > > + test - Test object for current test being parsed > > > > + > > > > + Return: > > > > + Boolean that represents if the next line in the LineStream was parsed > > > > + as the KTAP or TAP header line > > > > > > I think we can simplify these "Boolean that represents" with > > > > > > Return: > > > True if we parsed a KTAP or header line > > > > > > > I like this change a lot. I was struggling to get the right wording > > with the methods > > that returned booleans. I will change this for v3. > > > > > > + """ > > > > + ktap_match = KTAP_START.match(lines.peek()) > > > > + tap_match = TAP_START.match(lines.peek()) > > > > + if ktap_match: > > > > + version_num = int(ktap_match.group(1)) > > > > + check_version(version_num, KTAP_VERSIONS, 'KTAP', test) > > > > + elif tap_match: > > > > + version_num = int(tap_match.group(1)) > > > > + check_version(version_num, TAP_VERSIONS, 'TAP', test) > > > > else: > > > > return False > > > > - > > > > -SUBTEST_DIAGNOSTIC = re.compile(r'^[\s]+# (.*)$') > > > > -DIAGNOSTIC_CRASH_MESSAGE = re.compile(r'^[\s]+# .*?: kunit test case crashed!$') > > > > - > > > > -def parse_diagnostic(lines: LineStream, test_case: TestCase) -> bool: > > > > - save_non_diagnostic(lines, test_case) > > > > - if not lines: > > > > + test.log.append(lines.pop()) > > > > + return True > > > > + > > > > +TEST_HEADER = re.compile(r'^# Subtest: (.*)$') > > > > + > > > > +def parse_test_header(lines: LineStream, test: Test) -> bool: > > > > + """ > > > > + If the next line in LineStream matches the format of a test > > > > + header line, the name of test is set, the line is popped, > > > > + and returns True. Otherwise the method returns False. > > > > + > > > > + Accepted format: > > > > + - '# Subtest: [test name]' > > > > + > > > > + Parameters: > > > > + lines - LineStream of ktap output to parse > > > > + test - Test object for current test being parsed > > > > + > > > > + Return: > > > > + Boolean that represents if the next line in the LineStream was parsed > > > > + as a test header > > > > > > ditto above, this could be > > > True if we parsed a test header line. > > > > See above. Will change this for v3. > > > > > > > > > + """ > > > > + match = TEST_HEADER.match(lines.peek()) > > > > + if not match: > > > > return False > > > > + test.log.append(lines.pop()) > > > > + test.name = match.group(1) > > > > + return True > > > > + > > > > +TEST_PLAN = re.compile(r'1\.\.([0-9]+)') > > > > + > > > > +def parse_test_plan(lines: LineStream, test: Test) -> bool: > > > > + """ > > > > + If the next line in LineStream matches the format of a test > > > > + plan line, the expected number of subtests is set in test object, an > > > > + error is thrown if there are 0 tests, the line is popped, > > > > + and returns True. Otherwise the method adds an error that the test > > > > + plan is missing to the test object and returns False. > > > > > > nit: "missing to the test object" => missing for? > > > > > > Perhaps we can rewrite this as something more like: > > > > > > Parses a test plan and stores the expected number of subtests in `test`. > > > Returns False if the current line doesn't look like a test plan. > > > > > > It reports an error if the expected count is 0. > > > > > > > I definitely agree that 'missing for' is better than 'missing to'. > > However, overall > > this comment does seem confusing and does not match some of the updated > > docstrings. This confusing format is shared by all of the parsing methods. > > Thus, I would like to update the parsing method docstrings in v3. For example, > > this docstring would be changed to: > > > > Parses test plan line and stores the expected number of subtests in `test`. > > Reports an error if expected count is 0. > > Returns False and reports missing test plan error if fails to parse test plan. > > > > > > + > > > > + Accepted format: > > > > + - '1..[number of subtests]' > > > > + > > > > + Parameters: > > > > + lines - LineStream of ktap output to parse > > > > + test - Test object for current test being parsed > > > > + > > > > + Return: > > > > + Boolean that represents if the next line in the LineStream was parsed > > > > + as a test plan > > > > + """ > > > > + match = TEST_PLAN.match(lines.peek()) > > > > + if not match: > > > > + test.expected_count = None > > > > + test.add_error('missing plan line!') > > > > + return False > > > > + test.log.append(lines.pop()) > > > > + expected_count = int(match.group(1)) > > > > + test.expected_count = expected_count > > > > + if expected_count == 0: > > > > + test.status = TestStatus.NO_TESTS > > > > + test.add_error('0 tests run!') > > > > + return True > > > > + > > > > +TEST_RESULT = re.compile(r'^(ok|not ok) ([0-9]+) (- )?(.*)$') > > > > + > > > > +TEST_RESULT_SKIP = re.compile(r'^(ok|not ok) ([0-9]+) (- )?(.*) # SKIP(.*)$') > > > > + > > > > +def peek_test_name_match(lines: LineStream, test: Test) -> bool: > > > > + """ > > > > + If the next line in LineStream matches the format of a test > > > > + result line and the name of the result line matches the name of the > > > > + current test, the method returns True. Otherwise it returns False. > > > > + > > > > + Accepted format: > > > > + - '[ok|not ok] [test number] [-] [test name] [optional skip > > > > + directive]' > > > > + > > > > + Parameters: > > > > + lines - LineStream of KTAP output to parse > > > > + test - Test object for current test being parsed > > > > + > > > > + Return: > > > > + Boolean that represents if the next line in the LineStream matched a > > > > + test result line and the name matched the expected test name > > > > + """ > > > > line = lines.peek() > > > > - match = SUBTEST_DIAGNOSTIC.match(line) > > > > - if match: > > > > - test_case.log.append(lines.pop()) > > > > - crash_match = DIAGNOSTIC_CRASH_MESSAGE.match(line) > > > > - if crash_match: > > > > - test_case.status = TestStatus.TEST_CRASHED > > > > - return True > > > > - else: > > > > + match = TEST_RESULT.match(line) > > > > + if not match: > > > > return False > > > > + name = match.group(4) > > > > + return (name == test.name) > > > > + > > > > +def parse_test_result(lines: LineStream, test: Test, > > > > + expected_num: int) -> bool: > > > > + """ > > > > + If the next line in LineStream matches the format of a test > > > > + result line, the status in the result line is added to the test > > > > + object, the test number is checked to match the expected test number > > > > + and if not an error is added to the test object, and returns True. > > > > + Otherwise it returns False. > > > > + > > > > + Note that the skip diirective is the only > > > > > > nit: diir => dir > > > > > > > Oops. Thanks for catching that. Will change in v3. > > > > > > > > > + directive that causes a change in status and otherwise the directive > > > > + is included in the name of the test. > > > > + > > > > + Accepted format: > > > > + - '[ok|not ok] [test number] [-] [test name] [optional skip > > > > + directive]' > > > > + > > > > + Parameters: > > > > + lines - LineStream of KTAP output to parse > > > > + test - Test object for current test being parsed > > > > + expected_num - expected test number for current test > > > > + > > > > + Return: > > > > + Boolean that represents if the next line in the LineStream was parsed > > > > + as a test result line. > > > > + """ > > > > + line = lines.peek() > > > > + match = TEST_RESULT.match(line) > > > > + skip_match = TEST_RESULT_SKIP.match(line) > > > > > > > > -def parse_test_case(lines: LineStream) -> Optional[TestCase]: > > > > - test_case = TestCase() > > > > - save_non_diagnostic(lines, test_case) > > > > - while parse_diagnostic(lines, test_case): > > > > - pass > > > > - if parse_ok_not_ok_test_case(lines, test_case): > > > > - return test_case > > > > - else: > > > > - return None > > > > - > > > > -SUBTEST_HEADER = re.compile(r'^[\s]+# Subtest: (.*)$') > > > > + # Check if line matches test result line format > > > > + if not match: > > > > + return False > > > > + test.log.append(lines.pop()) > > > > > > > > -def parse_subtest_header(lines: LineStream) -> Optional[str]: > > > > - consume_non_diagnostic(lines) > > > > - if not lines: > > > > - return None > > > > - match = SUBTEST_HEADER.match(lines.peek()) > > > > - if match: > > > > - lines.pop() > > > > - return match.group(1) > > > > + # Set name of test object > > > > + if skip_match: > > > > + test.name = skip_match.group(4) > > > > else: > > > > - return None > > > > + test.name = match.group(4) > > > > > > > > -SUBTEST_PLAN = re.compile(r'[\s]+[0-9]+\.\.([0-9]+)') > > > > + # Check test num > > > > + num = int(match.group(2)) > > > > + if num != expected_num: > > > > + test.add_error('Expected test number ' + > > > > + str(expected_num) + ' but found ' + str(num)) > > > > > > > > -def parse_subtest_plan(lines: LineStream) -> Optional[int]: > > > > - consume_non_diagnostic(lines) > > > > - match = SUBTEST_PLAN.match(lines.peek()) > > > > - if match: > > > > - lines.pop() > > > > - return int(match.group(1)) > > > > - else: > > > > - return None > > > > - > > > > -def max_status(left: TestStatus, right: TestStatus) -> TestStatus: > > > > - if left == right: > > > > - return left > > > > - elif left == TestStatus.TEST_CRASHED or right == TestStatus.TEST_CRASHED: > > > > - return TestStatus.TEST_CRASHED > > > > - elif left == TestStatus.FAILURE or right == TestStatus.FAILURE: > > > > - return TestStatus.FAILURE > > > > - elif left == TestStatus.SKIPPED: > > > > - return right > > > > - else: > > > > - return left > > > > - > > > > -def parse_ok_not_ok_test_suite(lines: LineStream, > > > > - test_suite: TestSuite, > > > > - expected_suite_index: int) -> bool: > > > > - consume_non_diagnostic(lines) > > > > - if not lines: > > > > - test_suite.status = TestStatus.TEST_CRASHED > > > > - return False > > > > - line = lines.peek() > > > > - match = OK_NOT_OK_MODULE.match(line) > > > > - if match: > > > > - lines.pop() > > > > - if match.group(1) == 'ok': > > > > - test_suite.status = TestStatus.SUCCESS > > > > - else: > > > > - test_suite.status = TestStatus.FAILURE > > > > - skip_match = OK_NOT_OK_SKIP.match(line) > > > > - if skip_match: > > > > - test_suite.status = TestStatus.SKIPPED > > > > - suite_index = int(match.group(2)) > > > > - if suite_index != expected_suite_index: > > > > - print_with_timestamp( > > > > - red('[ERROR] ') + 'expected_suite_index ' + > > > > - str(expected_suite_index) + ', but got ' + > > > > - str(suite_index)) > > > > + # Set status of test object > > > > + status = match.group(1) > > > > + if test.status == TestStatus.TEST_CRASHED: > > > > return True > > > > + elif skip_match: > > > > + test.status = TestStatus.SKIPPED > > > > + elif status == 'ok': > > > > + test.status = TestStatus.SUCCESS > > > > else: > > > > - return False > > > > - > > > > -def bubble_up_errors(status_list: Iterable[TestStatus]) -> TestStatus: > > > > - return reduce(max_status, status_list, TestStatus.SKIPPED) > > > > + test.status = TestStatus.FAILURE > > > > + return True > > > > + > > > > +def parse_diagnostic(lines: LineStream) -> List[str]: > > > > + """ > > > > + If the next line in LineStream does not match the format of a test > > > > + case line or test header line, the line is checked if the test has > > > > + crashed and if so adds an error message, pops the line and adds it to > > > > + the log. > > > > + > > > > + Line formats that are not parsed: > > > > + - '# Subtest: [test name]' > > > > + - '[ok|not ok] [test number] [-] [test name] [optional skip > > > > + directive]' > > > > + > > > > + Parameters: > > > > + lines - LineStream of KTAP output to parse > > > > + > > > > + Return: > > > > + Log of diagnostic lines > > > > + """ > > > > + log = [] # type: List[str] > > > > + while lines and not TEST_RESULT.match(lines.peek()) and not \ > > > > + TEST_HEADER.match(lines.peek()): > > > > + log.append(lines.pop()) > > > > + return log > > > > + > > > > +DIAGNOSTIC_CRASH_MESSAGE = re.compile(r'^# .*?: kunit test case crashed!$') > > > > + > > > > +def parse_crash_in_log(test: Test) -> bool: > > > > + """ > > > > + Iterate through the lines of the log to parse for crash message. > > > > + If crash message found, set status to crashed and return True. > > > > + Otherwise return False. > > > > + > > > > + Parameters: > > > > + test - Test object for current test being parsed > > > > + > > > > + Return: > > > > + Boolean that represents if crash message found in log > > > > + """ > > > > + for line in test.log: > > > > + if DIAGNOSTIC_CRASH_MESSAGE.match(line): > > > > + test.status = TestStatus.TEST_CRASHED > > > > + return True > > > > + return False > > > > > > > > -def bubble_up_test_case_errors(test_suite: TestSuite) -> TestStatus: > > > > - max_test_case_status = bubble_up_errors(x.status for x in test_suite.cases) > > > > - return max_status(max_test_case_status, test_suite.status) > > > > +# Printing helper methods: > > > > > > > > -def parse_test_suite(lines: LineStream, expected_suite_index: int) -> Optional[TestSuite]: > > > > - if not lines: > > > > - return None > > > > - consume_non_diagnostic(lines) > > > > - test_suite = TestSuite() > > > > - test_suite.status = TestStatus.SUCCESS > > > > - name = parse_subtest_header(lines) > > > > - if not name: > > > > - return None > > > > - test_suite.name = name > > > > - expected_test_case_num = parse_subtest_plan(lines) > > > > - if expected_test_case_num is None: > > > > - return None > > > > - while expected_test_case_num > 0: > > > > - test_case = parse_test_case(lines) > > > > - if not test_case: > > > > - break > > > > - test_suite.cases.append(test_case) > > > > - expected_test_case_num -= 1 > > > > - if parse_ok_not_ok_test_suite(lines, test_suite, expected_suite_index): > > > > - test_suite.status = bubble_up_test_case_errors(test_suite) > > > > - return test_suite > > > > - elif not lines: > > > > - print_with_timestamp(red('[ERROR] ') + 'ran out of lines before end token') > > > > - return test_suite > > > > - else: > > > > - print(f'failed to parse end of suite "{name}", at line {lines.line_number()}: {lines.peek()}') > > > > - return None > > > > +DIVIDER = '=' * 60 > > > > > > > > -TAP_HEADER = re.compile(r'^TAP version 14$') > > > > +RESET = '\033[0;0m' > > > > > > > > -def parse_tap_header(lines: LineStream) -> bool: > > > > - consume_non_diagnostic(lines) > > > > - if TAP_HEADER.match(lines.peek()): > > > > - lines.pop() > > > > - return True > > > > - else: > > > > - return False > > > > +def red(text: str) -> str: > > > > + """Returns inputted string with red color code.""" > > > > + return '\033[1;31m' + text + RESET > > > > > > > > -TEST_PLAN = re.compile(r'[0-9]+\.\.([0-9]+)') > > > > +def yellow(text: str) -> str: > > > > + """Returns inputted string with yellow color code.""" > > > > + return '\033[1;33m' + text + RESET > > > > > > > > -def parse_test_plan(lines: LineStream) -> Optional[int]: > > > > - consume_non_diagnostic(lines) > > > > - match = TEST_PLAN.match(lines.peek()) > > > > - if match: > > > > - lines.pop() > > > > - return int(match.group(1)) > > > > - else: > > > > - return None > > > > - > > > > -def bubble_up_suite_errors(test_suites: Iterable[TestSuite]) -> TestStatus: > > > > - return bubble_up_errors(x.status for x in test_suites) > > > > - > > > > -def parse_test_result(lines: LineStream) -> TestResult: > > > > - consume_non_diagnostic(lines) > > > > - if not lines or not parse_tap_header(lines): > > > > - return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) > > > > - expected_test_suite_num = parse_test_plan(lines) > > > > - if expected_test_suite_num == 0: > > > > - return TestResult(TestStatus.NO_TESTS, [], lines) > > > > - elif expected_test_suite_num is None: > > > > - return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) > > > > - test_suites = [] > > > > - for i in range(1, expected_test_suite_num + 1): > > > > - test_suite = parse_test_suite(lines, i) > > > > - if test_suite: > > > > - test_suites.append(test_suite) > > > > - else: > > > > - print_with_timestamp( > > > > - red('[ERROR] ') + ' expected ' + > > > > - str(expected_test_suite_num) + > > > > - ' test suites, but got ' + str(i - 2)) > > > > - break > > > > - test_suite = parse_test_suite(lines, -1) > > > > - if test_suite: > > > > - print_with_timestamp(red('[ERROR] ') + > > > > - 'got unexpected test suite: ' + test_suite.name) > > > > - if test_suites: > > > > - return TestResult(bubble_up_suite_errors(test_suites), test_suites, lines) > > > > - else: > > > > - return TestResult(TestStatus.NO_TESTS, [], lines) > > > > +def green(text: str) -> str: > > > > + """Returns inputted string with green color code.""" > > > > + return '\033[1;32m' + text + RESET > > > > > > > > -class TestCounts: > > > > - passed: int > > > > - failed: int > > > > - crashed: int > > > > - skipped: int > > > > +ANSI_LEN = len(red('')) > > > > > > > > - def __init__(self): > > > > - self.passed = 0 > > > > - self.failed = 0 > > > > - self.crashed = 0 > > > > - self.skipped = 0 > > > > +def print_with_timestamp(message: str) -> None: > > > > + """Prints message with timestamp at beginning.""" > > > > + print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) > > > > > > > > - def total(self) -> int: > > > > - return self.passed + self.failed + self.crashed + self.skipped > > > > - > > > > -def print_and_count_results(test_result: TestResult) -> TestCounts: > > > > - counts = TestCounts() > > > > - for test_suite in test_result.suites: > > > > - if test_suite.status == TestStatus.SUCCESS: > > > > - print_suite_divider(green('[PASSED] ') + test_suite.name) > > > > - elif test_suite.status == TestStatus.SKIPPED: > > > > - print_suite_divider(yellow('[SKIPPED] ') + test_suite.name) > > > > - elif test_suite.status == TestStatus.TEST_CRASHED: > > > > - print_suite_divider(red('[CRASHED] ' + test_suite.name)) > > > > +def format_test_divider(message: str, len_message: int) -> str: > > > > + """ > > > > + Returns string with message centered in fixed width divider. > > > > + > > > > + Example: > > > > + '===================== message example =====================' > > > > + > > > > + Parameters: > > > > + message - message to be centered in divider line > > > > + len_message - length of the message to be printed such that > > > > + any characters of the color codes are not counted > > > > + > > > > + Return: > > > > + String containing message centered in fixed width divider > > > > + """ > > > > + default_count = 3 # default number of dashes > > > > + len_1 = default_count > > > > + len_2 = default_count > > > > + difference = len(DIVIDER) - len_message - 2 # 2 spaces added > > > > + if difference > 0: > > > > + # calculate number of dashes for each side of the divider > > > > + len_1 = int(difference / 2) > > > > + len_2 = difference - len_1 > > > > + return ('=' * len_1) + ' ' + message + ' ' + ('=' * len_2) > > > > + > > > > +def print_test_header(test: Test) -> None: > > > > + """ > > > > + Prints test header with test name and optionally the expected number > > > > + of subtests. > > > > + > > > > + Example: > > > > + '=================== example (2 subtests) ===================' > > > > + > > > > + Parameters: > > > > + test - Test object representing current test being printed > > > > + """ > > > > + message = test.name > > > > + if test.expected_count: > > > > + message += ' (' + str(test.expected_count) + ' subtests)' > > > > + print_with_timestamp(format_test_divider(message, len(message))) > > > > + > > > > +def print_log(log: Iterable[str]) -> None: > > > > + """ > > > > + Prints all strings in saved log for test in yellow. > > > > + > > > > + Parameters: > > > > + log - Iterable object with all strings saved in log for test > > > > + """ > > > > + for m in log: > > > > + print_with_timestamp(yellow(m)) > > > > + print_with_timestamp('') > > > > + > > > > +def format_test_result(test: Test) -> str: > > > > + """ > > > > + Returns string with formatted test result with colored status and test > > > > + name. > > > > + > > > > + Example: > > > > + '[PASSED] example' > > > > + > > > > + Parameters: > > > > + test - Test object representing current test being printed > > > > + > > > > + Return: > > > > + String containing formatted test result > > > > + """ > > > > + if test.status == TestStatus.SUCCESS: > > > > + return (green('[PASSED] ') + test.name) > > > > + elif test.status == TestStatus.SKIPPED: > > > > + return (yellow('[SKIPPED] ') + test.name) > > > > + elif test.status == TestStatus.TEST_CRASHED: > > > > + print_log(test.log) > > > > + return (red('[CRASHED] ') + test.name) > > > > + else: > > > > + print_log(test.log) > > > > + return (red('[FAILED] ') + test.name) > > > > + > > > > +def print_test_result(test: Test) -> None: > > > > + """ > > > > + Prints result line with status of test. > > > > + > > > > + Example: > > > > + '[PASSED] example' > > > > + > > > > + Parameters: > > > > + test - Test object representing current test being printed > > > > + """ > > > > + print_with_timestamp(format_test_result(test)) > > > > + > > > > +def print_test_footer(test: Test) -> None: > > > > + """ > > > > + Prints test footer with status of test. > > > > + > > > > + Example: > > > > + '===================== [PASSED] example =====================' > > > > + > > > > + Parameters: > > > > + test - Test object representing current test being printed > > > > + """ > > > > + message = format_test_result(test) > > > > + print_with_timestamp(format_test_divider(message, > > > > + len(message) - ANSI_LEN)) > > > > + > > > > +def print_summary_line(test: Test) -> None: > > > > + """ > > > > + Prints summary line of test object. Color of line is dependent on > > > > + status of test. Color is green if test passes, yellow if test is > > > > + skipped, and red if the test fails or crashes. Summary line contains > > > > + counts of the statuses of the tests subtests or the test itself if it > > > > + has no subtests. > > > > + > > > > + Example: > > > > + "Testing complete. Passed: 2, Failed: 0, Crashed: 0, Skipped: 0, > > > > + Errors: 0" > > > > + > > > > + test - Test object representing current test being printed > > > > + """ > > > > + if test.status == TestStatus.SUCCESS or \ > > > > + test.status == TestStatus.NO_TESTS: > > > > + color = green > > > > + elif test.status == TestStatus.SKIPPED: > > > > + color = yellow > > > > + else: > > > > + color = red > > > > + counts = test.counts > > > > + print_with_timestamp(color('Testing complete. ' + str(counts))) > > > > + > > > > +def print_error(error_message: str) -> None: > > > > + """ > > > > + Prints error message with error format. > > > > + > > > > + Example: > > > > + "[ERROR] Test example: missing test plan!" > > > > + > > > > + Parameters: > > > > + error_message - message describing error > > > > + """ > > > > + print_with_timestamp(red('[ERROR] ') + error_message) > > > > + > > > > +# Other methods: > > > > + > > > > +def bubble_up_test_results(test: Test) -> None: > > > > + """ > > > > + If the test has subtests, add the test counts of the subtests to the > > > > + test and check if any of the tests crashed and if so set the test > > > > + status to crashed. Otherwise if the test has no subtests add the > > > > + status of the test to the test counts. > > > > + > > > > + Parameters: > > > > + test - Test object for current test being parsed > > > > + """ > > > > + parse_crash_in_log(test) > > > > + subtests = test.subtests > > > > + counts = test.counts > > > > + status = test.status > > > > + for t in subtests: > > > > + counts.add_subtest_counts(t.counts) > > > > + if counts.total() == 0: > > > > + counts.add_status(status) > > > > + elif test.counts.get_status() == TestStatus.TEST_CRASHED: > > > > + test.status = TestStatus.TEST_CRASHED > > > > + > > > > +def parse_test(lines: LineStream, expected_num: int, log: List[str]) -> Test: > > > > + """ > > > > + Finds next test to parse in LineStream, creates new Test object, > > > > + parses any subtests of the test, populates Test object with all > > > > + information (status, name) about the test and the Test objects for > > > > + any subtests, and then returns the Test object. The method accepts > > > > + three formats of tests: > > > > + > > > > + Accepted test formats: > > > > + > > > > + - Main KTAP/TAP header > > > > + > > > > + Example: > > > > + > > > > + KTAP version 1 > > > > + 1..4 > > > > + [subtests] > > > > + > > > > + - Subtest header line > > > > + > > > > + Example: > > > > + > > > > + # Subtest: name > > > > + 1..3 > > > > + [subtests] > > > > + ok 1 name > > > > + > > > > + - Test result line > > > > + > > > > + Example: > > > > + > > > > + ok 1 - test > > > > + > > > > + Parameters: > > > > + lines - LineStream of KTAP output to parse > > > > + expected_num - expected test number for test to be parsed > > > > + log - list of strings containing any preceding diagnostic lines > > > > + corresponding to the current test > > > > + > > > > + Return: > > > > + Test object populated with characteristics and any subtests > > > > + """ > > > > + test = Test() > > > > + test.log.extend(log) > > > > + parent_test = False > > > > + main = parse_ktap_header(lines, test) > > > > + if main: > > > > + # If KTAP/TAP header is found, attempt to parse > > > > + # test plan > > > > + test.name = "main" > > > > + parse_test_plan(lines, test) > > > > + else: > > > > + # If KTAP/TAP header is not found, test must be subtest > > > > + # header or test result line so parse attempt to parser > > > > + # subtest header > > > > + parent_test = parse_test_header(lines, test) > > > > + if parent_test: > > > > + # If subtest header is found, attempt to parse > > > > + # test plan and print header > > > > + parse_test_plan(lines, test) > > > > + print_test_header(test) > > > > + expected_count = test.expected_count > > > > + subtests = [] > > > > + test_num = 1 > > > > + while main or expected_count is None or test_num <= expected_count: > > > > + # Loop to parse any subtests. > > > > + # If test is main test, do not break until no lines left. > > > > + # Otherwise, break after parsing expected number of tests or > > > > + # if expected number of tests is unknown break when found > > > > + # test result line with matching name to subtest header. > > > > + if not lines: > > > > + if expected_count and test_num <= expected_count: > > > > + test.add_error('missing expected subtests!') > > > > + break > > > > + sub_log = parse_diagnostic(lines) > > > > + if not expected_count and not main and \ > > > > + peek_test_name_match(lines, test): > > > > + test.log.extend(sub_log) > > > > + break > > > > + subtests.append(parse_test(lines, test_num, sub_log)) > > > > + test_num += 1 > > > > + test.subtests = subtests > > > > + if not main: > > > > + # If not main test, look for test result line > > > > + test.log.extend(parse_diagnostic(lines)) > > > > + if (parent_test and peek_test_name_match(lines, test)) or \ > > > > + not parent_test: > > > > + parse_test_result(lines, test, expected_num) > > > > else: > > > > - print_suite_divider(red('[FAILED] ') + test_suite.name) > > > > - for test_case in test_suite.cases: > > > > - if test_case.status == TestStatus.SUCCESS: > > > > - counts.passed += 1 > > > > - print_with_timestamp(green('[PASSED] ') + test_case.name) > > > > - elif test_case.status == TestStatus.SKIPPED: > > > > - counts.skipped += 1 > > > > - print_with_timestamp(yellow('[SKIPPED] ') + test_case.name) > > > > - elif test_case.status == TestStatus.TEST_CRASHED: > > > > - counts.crashed += 1 > > > > - print_with_timestamp(red('[CRASHED] ' + test_case.name)) > > > > - print_log(map(yellow, test_case.log)) > > > > - print_with_timestamp('') > > > > - else: > > > > - counts.failed += 1 > > > > - print_with_timestamp(red('[FAILED] ') + test_case.name) > > > > - print_log(map(yellow, test_case.log)) > > > > - print_with_timestamp('') > > > > - return counts > > > > + test.add_error('missing subtest result line!') > > > > + # Add statuses to TestCounts attribute in Test object > > > > + bubble_up_test_results(test) > > > > + if parent_test: > > > > + # If test has subtests and is not the main test object, print > > > > + # footer. > > > > + print_test_footer(test) > > > > + elif not main: > > > > + print_test_result(test) > > > > + return test > > > > > > > > def parse_run_tests(kernel_output: Iterable[str]) -> TestResult: > > > > - counts = TestCounts() > > > > + """ > > > > + Using kernel output, extract KTAP lines, parse the lines for test > > > > + results and print condensed test results and summary line . > > > > + > > > > + Parameters: > > > > + kernel_output - Iterable object contains lines of kernel output > > > > + > > > > + Return: > > > > + TestResult - Tuple containg status of main test object, main test > > > > + object with all subtests, and log of all KTAP lines. > > > > + """ > > > > + print_with_timestamp(DIVIDER) > > > > lines = extract_tap_lines(kernel_output) > > > > - test_result = parse_test_result(lines) > > > > - if test_result.status == TestStatus.NO_TESTS: > > > > - print(red('[ERROR] ') + yellow('no tests run!')) > > > > - elif test_result.status == TestStatus.FAILURE_TO_PARSE_TESTS: > > > > - print(red('[ERROR] ') + yellow('could not parse test results!')) > > > > + test = Test() > > > > + if not lines: > > > > + test.add_error('invalid KTAP input!') > > > > + test.status = TestStatus.FAILURE_TO_PARSE_TESTS > > > > else: > > > > - counts = print_and_count_results(test_result) > > > > + test = parse_test(lines, 0, []) > > > > + if test.status != TestStatus.NO_TESTS: > > > > + test.status = test.counts.get_status() > > > > print_with_timestamp(DIVIDER) > > > > - if test_result.status == TestStatus.SUCCESS: > > > > - fmt = green > > > > - elif test_result.status == TestStatus.SKIPPED: > > > > - fmt = yellow > > > > - else: > > > > - fmt =red > > > > - print_with_timestamp( > > > > - fmt('Testing complete. %d tests run. %d failed. %d crashed. %d skipped.' % > > > > - (counts.total(), counts.failed, counts.crashed, counts.skipped))) > > > > - return test_result > > > > + print_summary_line(test) > > > > + return TestResult(test.status, test, lines) > > > > diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py > > > > index 619c4554cbff..e527b90de8ea 100755 > > > > --- a/tools/testing/kunit/kunit_tool_test.py > > > > +++ b/tools/testing/kunit/kunit_tool_test.py > > > > @@ -106,10 +106,10 @@ class KUnitParserTest(unittest.TestCase): > > > > with open(log_path) as file: > > > > result = kunit_parser.extract_tap_lines(file.readlines()) > > > > self.assertContains('TAP version 14', result) > > > > - self.assertContains(' # Subtest: example', result) > > > > - self.assertContains(' 1..2', result) > > > > - self.assertContains(' ok 1 - example_simple_test', result) > > > > - self.assertContains(' ok 2 - example_mock_test', result) > > > > + self.assertContains('# Subtest: example', result) > > > > + self.assertContains('1..2', result) > > > > + self.assertContains('ok 1 - example_simple_test', result) > > > > + self.assertContains('ok 2 - example_mock_test', result) > > > > self.assertContains('ok 1 - example', result) > > > > > > > > def test_output_with_prefix_isolated_correctly(self): > > > > @@ -117,28 +117,28 @@ class KUnitParserTest(unittest.TestCase): > > > > with open(log_path) as file: > > > > result = kunit_parser.extract_tap_lines(file.readlines()) > > > > self.assertContains('TAP version 14', result) > > > > - self.assertContains(' # Subtest: kunit-resource-test', result) > > > > - self.assertContains(' 1..5', result) > > > > - self.assertContains(' ok 1 - kunit_resource_test_init_resources', result) > > > > - self.assertContains(' ok 2 - kunit_resource_test_alloc_resource', result) > > > > - self.assertContains(' ok 3 - kunit_resource_test_destroy_resource', result) > > > > - self.assertContains(' foo bar #', result) > > > > - self.assertContains(' ok 4 - kunit_resource_test_cleanup_resources', result) > > > > - self.assertContains(' ok 5 - kunit_resource_test_proper_free_ordering', result) > > > > + self.assertContains('# Subtest: kunit-resource-test', result) > > > > + self.assertContains('1..5', result) > > > > + self.assertContains('ok 1 - kunit_resource_test_init_resources', result) > > > > + self.assertContains('ok 2 - kunit_resource_test_alloc_resource', result) > > > > + self.assertContains('ok 3 - kunit_resource_test_destroy_resource', result) > > > > + self.assertContains('foo bar #', result) > > > > + self.assertContains('ok 4 - kunit_resource_test_cleanup_resources', result) > > > > + self.assertContains('ok 5 - kunit_resource_test_proper_free_ordering', result) > > > > self.assertContains('ok 1 - kunit-resource-test', result) > > > > - self.assertContains(' foo bar # non-kunit output', result) > > > > - self.assertContains(' # Subtest: kunit-try-catch-test', result) > > > > - self.assertContains(' 1..2', result) > > > > - self.assertContains(' ok 1 - kunit_test_try_catch_successful_try_no_catch', > > > > + self.assertContains('foo bar # non-kunit output', result) > > > > + self.assertContains('# Subtest: kunit-try-catch-test', result) > > > > + self.assertContains('1..2', result) > > > > + self.assertContains('ok 1 - kunit_test_try_catch_successful_try_no_catch', > > > > result) > > > > - self.assertContains(' ok 2 - kunit_test_try_catch_unsuccessful_try_does_catch', > > > > + self.assertContains('ok 2 - kunit_test_try_catch_unsuccessful_try_does_catch', > > > > result) > > > > self.assertContains('ok 2 - kunit-try-catch-test', result) > > > > - self.assertContains(' # Subtest: string-stream-test', result) > > > > - self.assertContains(' 1..3', result) > > > > - self.assertContains(' ok 1 - string_stream_test_empty_on_creation', result) > > > > - self.assertContains(' ok 2 - string_stream_test_not_empty_after_add', result) > > > > - self.assertContains(' ok 3 - string_stream_test_get_string', result) > > > > + self.assertContains('# Subtest: string-stream-test', result) > > > > + self.assertContains('1..3', result) > > > > + self.assertContains('ok 1 - string_stream_test_empty_on_creation', result) > > > > + self.assertContains('ok 2 - string_stream_test_not_empty_after_add', result) > > > > + self.assertContains('ok 3 - string_stream_test_get_string', result) > > > > self.assertContains('ok 3 - string-stream-test', result) > > > > > > > > def test_parse_successful_test_log(self): > > > > @@ -148,6 +148,13 @@ class KUnitParserTest(unittest.TestCase): > > > > self.assertEqual( > > > > kunit_parser.TestStatus.SUCCESS, > > > > result.status) > > > > + def test_parse_successful_nested_tests_log(self): > > > > + all_passed_log = test_data_path('test_is_test_passed-all_passed_nested.log') > > > > + with open(all_passed_log) as file: > > > > + result = kunit_parser.parse_run_tests(file.readlines()) > > > > + self.assertEqual( > > > > + kunit_parser.TestStatus.SUCCESS, > > > > + result.status) > > > > > > > > def test_parse_failed_test_log(self): > > > > failed_log = test_data_path('test_is_test_passed-failure.log') > > > > @@ -162,17 +169,31 @@ class KUnitParserTest(unittest.TestCase): > > > > with open(empty_log) as file: > > > > result = kunit_parser.parse_run_tests( > > > > kunit_parser.extract_tap_lines(file.readlines())) > > > > - self.assertEqual(0, len(result.suites)) > > > > + self.assertEqual(0, len(result.test.subtests)) > > > > self.assertEqual( > > > > kunit_parser.TestStatus.FAILURE_TO_PARSE_TESTS, > > > > result.status) > > > > > > > > + def test_missing_test_plan(self): > > > > + missing_plan_log = test_data_path('test_is_test_passed-' > > > > + 'missing_plan.log') > > > > + with open(missing_plan_log) as file: > > > > + result = kunit_parser.parse_run_tests( > > > > + kunit_parser.extract_tap_lines( > > > > + file.readlines())) > > > > + self.assertEqual(2, result.test.counts.errors) > > > > + self.assertEqual( > > > > + kunit_parser.TestStatus.SUCCESS, > > > > + result.status) > > > > + > > > > def test_no_tests(self): > > > > - empty_log = test_data_path('test_is_test_passed-no_tests_run_with_header.log') > > > > - with open(empty_log) as file: > > > > + header_log = test_data_path('test_is_test_passed-' > > > > + 'no_tests_run_with_header.log') > > > > + with open(header_log) as file: > > > > result = kunit_parser.parse_run_tests( > > > > - kunit_parser.extract_tap_lines(file.readlines())) > > > > - self.assertEqual(0, len(result.suites)) > > > > + kunit_parser.extract_tap_lines( > > > > + file.readlines())) > > > > + self.assertEqual(0, len(result.test.subtests)) > > > > self.assertEqual( > > > > kunit_parser.TestStatus.NO_TESTS, > > > > result.status) > > > > @@ -182,15 +203,17 @@ class KUnitParserTest(unittest.TestCase): > > > > print_mock = mock.patch('builtins.print').start() > > > > with open(crash_log) as file: > > > > result = kunit_parser.parse_run_tests( > > > > - kunit_parser.extract_tap_lines(file.readlines())) > > > > - print_mock.assert_any_call(StrContains('could not parse test results!')) > > > > + kunit_parser.extract_tap_lines( > > > > + file.readlines())) > > > > + print_mock.assert_any_call(StrContains('invalid KTAP input!')) > > > > print_mock.stop() > > > > file.close() > > > > > > > > def test_crashed_test(self): > > > > crashed_log = test_data_path('test_is_test_passed-crash.log') > > > > with open(crashed_log) as file: > > > > - result = kunit_parser.parse_run_tests(file.readlines()) > > > > + result = kunit_parser.parse_run_tests( > > > > + file.readlines()) > > > > self.assertEqual( > > > > kunit_parser.TestStatus.TEST_CRASHED, > > > > result.status) > > > > @@ -216,6 +239,23 @@ class KUnitParserTest(unittest.TestCase): > > > > result.status) > > > > file.close() > > > > > > > > + def test_ignores_hyphen(self): > > > > + hyphen_log = test_data_path('test_strip_hyphen.log') > > > > + file = open(hyphen_log) > > > > + result = kunit_parser.parse_run_tests(file.readlines()) > > > > + > > > > + # A skipped test does not fail the whole suite. > > > > + self.assertEqual( > > > > + kunit_parser.TestStatus.SUCCESS, > > > > + result.status) > > > > + self.assertEqual( > > > > + "sysctl_test", > > > > + result.test.subtests[0].name) > > > > + self.assertEqual( > > > > + "example", > > > > + result.test.subtests[1].name) > > > > + file.close() > > > > + > > > > > > > > def test_ignores_prefix_printk_time(self): > > > > prefix_log = test_data_path('test_config_printk_time.log') > > > > @@ -224,7 +264,7 @@ class KUnitParserTest(unittest.TestCase): > > > > self.assertEqual( > > > > kunit_parser.TestStatus.SUCCESS, > > > > result.status) > > > > - self.assertEqual('kunit-resource-test', result.suites[0].name) > > > > + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) > > > > > > > > def test_ignores_multiple_prefixes(self): > > > > prefix_log = test_data_path('test_multiple_prefixes.log') > > > > @@ -233,7 +273,7 @@ class KUnitParserTest(unittest.TestCase): > > > > self.assertEqual( > > > > kunit_parser.TestStatus.SUCCESS, > > > > result.status) > > > > - self.assertEqual('kunit-resource-test', result.suites[0].name) > > > > + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) > > > > > > > > def test_prefix_mixed_kernel_output(self): > > > > mixed_prefix_log = test_data_path('test_interrupted_tap_output.log') > > > > @@ -242,7 +282,7 @@ class KUnitParserTest(unittest.TestCase): > > > > self.assertEqual( > > > > kunit_parser.TestStatus.SUCCESS, > > > > result.status) > > > > - self.assertEqual('kunit-resource-test', result.suites[0].name) > > > > + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) > > > > > > > > def test_prefix_poundsign(self): > > > > pound_log = test_data_path('test_pound_sign.log') > > > > @@ -251,16 +291,16 @@ class KUnitParserTest(unittest.TestCase): > > > > self.assertEqual( > > > > kunit_parser.TestStatus.SUCCESS, > > > > result.status) > > > > - self.assertEqual('kunit-resource-test', result.suites[0].name) > > > > + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) > > > > > > > > def test_kernel_panic_end(self): > > > > panic_log = test_data_path('test_kernel_panic_interrupt.log') > > > > with open(panic_log) as file: > > > > result = kunit_parser.parse_run_tests(file.readlines()) > > > > self.assertEqual( > > > > - kunit_parser.TestStatus.TEST_CRASHED, > > > > + kunit_parser.TestStatus.SUCCESS, > > > > result.status) > > > > - self.assertEqual('kunit-resource-test', result.suites[0].name) > > > > + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) > > > > > > > > def test_pound_no_prefix(self): > > > > pound_log = test_data_path('test_pound_no_prefix.log') > > > > @@ -269,7 +309,7 @@ class KUnitParserTest(unittest.TestCase): > > > > self.assertEqual( > > > > kunit_parser.TestStatus.SUCCESS, > > > > result.status) > > > > - self.assertEqual('kunit-resource-test', result.suites[0].name) > > > > + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) > > > > > > > > class LinuxSourceTreeTest(unittest.TestCase): > > > > > > > > @@ -291,6 +331,14 @@ class LinuxSourceTreeTest(unittest.TestCase): > > > > pass > > > > tree = kunit_kernel.LinuxSourceTree('', kunitconfig_path=dir) > > > > > > > > + def test_kselftest_nested(self): > > > > + kselftest_log = test_data_path('test_is_test_passed-kselftest.log') > > > > + with open(kselftest_log) as file: > > > > + result = kunit_parser.parse_run_tests(file.readlines()) > > > > + self.assertEqual( > > > > + kunit_parser.TestStatus.SUCCESS, > > > > + result.status) > > > > + > > > > # TODO: add more test cases. > > > > > > > > > > > > @@ -322,6 +370,12 @@ class KUnitJsonTest(unittest.TestCase): > > > > result = self._json_for('test_is_test_passed-no_tests_run_with_header.log') > > > > self.assertEqual(0, len(result['sub_groups'])) > > > > > > > > + def test_nested_json(self): > > > > + result = self._json_for('test_is_test_passed-all_passed_nested.log') > > > > + self.assertEqual( > > > > + {'name': 'example_simple_test', 'status': 'PASS'}, > > > > + result["sub_groups"][0]["sub_groups"][0]["test_cases"][0]) > > > > + > > > > class StrContains(str): > > > > def __eq__(self, other): > > > > return self in other > > > > @@ -380,7 +434,7 @@ class KUnitMainTest(unittest.TestCase): > > > > self.assertEqual(e.exception.code, 1) > > > > self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 1) > > > > self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) > > > > - self.print_mock.assert_any_call(StrContains(' 0 tests run')) > > > > + self.print_mock.assert_any_call(StrContains('invalid KTAP input!')) > > > > > > > > def test_exec_raw_output(self): > > > > self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) > > > > @@ -388,7 +442,7 @@ class KUnitMainTest(unittest.TestCase): > > > > self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) > > > > for call in self.print_mock.call_args_list: > > > > self.assertNotEqual(call, mock.call(StrContains('Testing complete.'))) > > > > - self.assertNotEqual(call, mock.call(StrContains(' 0 tests run'))) > > > > + self.assertNotEqual(call, mock.call(StrContains(' 0 tests run!'))) > > > > > > > > def test_run_raw_output(self): > > > > self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) > > > > @@ -397,7 +451,7 @@ class KUnitMainTest(unittest.TestCase): > > > > self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) > > > > for call in self.print_mock.call_args_list: > > > > self.assertNotEqual(call, mock.call(StrContains('Testing complete.'))) > > > > - self.assertNotEqual(call, mock.call(StrContains(' 0 tests run'))) > > > > + self.assertNotEqual(call, mock.call(StrContains(' 0 tests run!'))) > > > > > > > > def test_run_raw_output_kunit(self): > > > > self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) > > > > diff --git a/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log > > b/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log > > > > new file mode 100644 > > > > index 000000000000..9d5b04fe43a6 > > > > --- /dev/null > > > > +++ b/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log > > > > @@ -0,0 +1,34 @@ > > > > +TAP version 14 > > > > +1..2 > > > > + # Subtest: sysctl_test > > > > + 1..4 > > > > + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed > > > > + ok 1 - sysctl_test_dointvec_null_tbl_data > > > > + # Subtest: example > > > > + 1..2 > > > > + init_suite > > > > + # example_simple_test: initializing > > > > + # example_simple_test: example_simple_test passed > > > > + ok 1 - example_simple_test > > > > + # example_mock_test: initializing > > > > + # example_mock_test: example_mock_test passed > > > > + ok 2 - example_mock_test > > > > + kunit example: all tests passed > > > > + ok 2 - example > > > > + # sysctl_test_dointvec_table_len_is_zero: sysctl_test_dointvec_table_len_is_zero passed > > > > + ok 3 - sysctl_test_dointvec_table_len_is_zero > > > > + # sysctl_test_dointvec_table_read_but_position_set: sysctl_test_dointvec_table_read_but_position_set passed > > > > + ok 4 - sysctl_test_dointvec_table_read_but_position_set > > > > +kunit sysctl_test: all tests passed > > > > +ok 1 - sysctl_test > > > > + # Subtest: example > > > > + 1..2 > > > > +init_suite > > > > + # example_simple_test: initializing > > > > + # example_simple_test: example_simple_test passed > > > > + ok 1 - example_simple_test > > > > + # example_mock_test: initializing > > > > + # example_mock_test: example_mock_test passed > > > > + ok 2 - example_mock_test > > > > +kunit example: all tests passed > > > > +ok 2 - example > > > > diff --git a/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log b/tools/testing/kunit/test_data/test_is_test_passed- > > kselftest.log > > > > new file mode 100644 > > > > index 000000000000..65d3f27feaf2 > > > > --- /dev/null > > > > +++ b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log > > > > @@ -0,0 +1,14 @@ > > > > +TAP version 13 > > > > +1..2 > > > > +# selftests: membarrier: membarrier_test_single_thread > > > > +# TAP version 13 > > > > +# 1..2 > > > > +# ok 1 sys_membarrier available > > > > +# ok 2 sys membarrier invalid command test: command = -1, flags = 0, errno = 22. Failed as expected > > > > +ok 1 selftests: membarrier: membarrier_test_single_thread > > > > +# selftests: membarrier: membarrier_test_multi_thread > > > > +# TAP version 13 > > > > +# 1..2 > > > > +# ok 1 sys_membarrier available > > > > +# ok 2 sys membarrier invalid command test: command = -1, flags = 0, errno = 22. Failed as expected > > > > +ok 2 selftests: membarrier: membarrier_test_multi_thread > > > > diff --git a/tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log b/tools/testing/kunit/test_data/test_is_test_passed- > > missing_plan.log > > > > new file mode 100644 > > > > index 000000000000..5cd17b7f818a > > > > --- /dev/null > > > > +++ b/tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log > > > > @@ -0,0 +1,31 @@ > > > > +KTAP version 1 > > > > + # Subtest: sysctl_test > > > > + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed > > > > + ok 1 - sysctl_test_dointvec_null_tbl_data > > > > + # sysctl_test_dointvec_table_maxlen_unset: sysctl_test_dointvec_table_maxlen_unset passed > > > > + ok 2 - sysctl_test_dointvec_table_maxlen_unset > > > > + # sysctl_test_dointvec_table_len_is_zero: sysctl_test_dointvec_table_len_is_zero passed > > > > + ok 3 - sysctl_test_dointvec_table_len_is_zero > > > > + # sysctl_test_dointvec_table_read_but_position_set: sysctl_test_dointvec_table_read_but_position_set passed > > > > + ok 4 - sysctl_test_dointvec_table_read_but_position_set > > > > + # sysctl_test_dointvec_happy_single_positive: sysctl_test_dointvec_happy_single_positive passed > > > > + ok 5 - sysctl_test_dointvec_happy_single_positive > > > > + # sysctl_test_dointvec_happy_single_negative: sysctl_test_dointvec_happy_single_negative passed > > > > + ok 6 - sysctl_test_dointvec_happy_single_negative > > > > + # sysctl_test_dointvec_single_less_int_min: sysctl_test_dointvec_single_less_int_min passed > > > > + ok 7 - sysctl_test_dointvec_single_less_int_min > > > > + # sysctl_test_dointvec_single_greater_int_max: sysctl_test_dointvec_single_greater_int_max passed > > > > + ok 8 - sysctl_test_dointvec_single_greater_int_max > > > > +kunit sysctl_test: all tests passed > > > > +ok 1 - sysctl_test > > > > + # Subtest: example > > > > + 1..2 > > > > +init_suite > > > > + # example_simple_test: initializing > > > > + # example_simple_test: example_simple_test passed > > > > + ok 1 - example_simple_test > > > > + # example_mock_test: initializing > > > > + # example_mock_test: example_mock_test passed > > > > + ok 2 - example_mock_test > > > > +kunit example: all tests passed > > > > +ok 2 - example > > > > diff --git a/tools/testing/kunit/test_data/test_strip_hyphen.log b/tools/testing/kunit/test_data/test_strip_hyphen.log > > > > new file mode 100644 > > > > index 000000000000..92ac7c24b374 > > > > --- /dev/null > > > > +++ b/tools/testing/kunit/test_data/test_strip_hyphen.log > > > > @@ -0,0 +1,16 @@ > > > > +KTAP version 1 > > > > +1..2 > > > > + # Subtest: sysctl_test > > > > + 1..1 > > > > + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed > > > > + ok 1 - sysctl_test_dointvec_null_tbl_data > > > > +kunit sysctl_test: all tests passed > > > > +ok 1 - sysctl_test > > > > + # Subtest: example > > > > + 1..1 > > > > +init_suite > > > > + # example_simple_test: initializing > > > > + # example_simple_test: example_simple_test passed > > > > + ok 1 example_simple_test > > > > +kunit example: all tests passed > > > > +ok 2 example > > > > -- > > > > 2.33.0.259.gc128427fd7-goog > > > > > > > > -- > > > > You received this message because you are subscribed to the Google Groups "KUnit Development" group. > > > > To unsubscribe from this group and stop receiving emails from it, send an email to kunit-dev+unsubscribe(a)googlegroups.com. > > > > To view this discussion on the web visit https://groups.google.com/d/msgid/kunit-dev/20210826195505.3066755-1- > > rmoar%40google.com.

4 years

1
0
0 0

[PATCH v2] kunit: tool: improve compatibility of kunit_parser with KTAP specification

by Rae Moar

Update to kunit_parser to improve compatibility with KTAP specification including arbitrarily nested tests. Patch accomplishes three major changes: - Use a general Test object to represent all tests rather than TestCase and TestSuite objects. This allows for easier implementation of arbitrary levels of nested tests and promotes the idea that both test suites and test cases are tests. - Print errors incrementally rather than all at once after the parsing finishes to maximize information given to the user in the case of the parser given invalid input and to increase the helpfulness of the timestamps given during printing. Note that kunit.py parse does not print incrementally yet. However, this fix brings us closer to this feature. - Increase compatibility for different formats of input. Arbitrary levels of nested tests supported. Also, test cases and test suites are now supported to be present on the same level of testing. This patch now implements the KTAP specification as described here: https://lore.kernel.org/linux-kselftest/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqa…. This patch adjusts the kunit_tool_test.py file to check for the correct outputs from the new parser and adds a new test to check the parsing for a KTAP result log with correct format for multiple nested subtests (test_is_test_passed-all_passed_nested.log). This patch also alters the kunit_json.py file to allow for arbitrarily nested tests. Signed-off-by: Rae Moar <rmoar(a)google.com> Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com> --- Change log from v1: https://lore.kernel.org/linux-kselftest/20210820200032.2178134-1-rmoar@goog… - Rebase onto kselftest/kunit branch - Add tests to kunit_tool_test.py to check parser is correctly stripping hyphen, producing correct json objects with nested tests, correctly passing kselftest TAP output, and correctly deals with missing test plan. - Fix bug to correctly match test name in instance of a missing test plan. - Fix bug in kunit_tool_test.py pointed out by Daniel where it was not correctly checking for a proper match to the '0 tests run!' error message. Reverts changes back to original. - A few minor changes to commit message using Daniel's comments. - Change docstrings using Daniel's comments to reduce: - Shortens some docstrings to be one-line or just description if it is self explanatory. - Remove explicit respecification of types of parameters and returns because this is already specified in the function annoations. However, some descriptions of the parameters and returns remain and some contain the type for context. Additionally, the types of public attributes of classes remain. - Remove any documentation of 'Return: None' - Remove docstrings of helper methods within other methods. --- tools/testing/kunit/kunit_json.py | 55 +- tools/testing/kunit/kunit_parser.py | 1056 ++++++++++++----- tools/testing/kunit/kunit_tool_test.py | 134 ++- .../test_is_test_passed-all_passed_nested.log | 34 + .../test_is_test_passed-kselftest.log | 14 + .../test_is_test_passed-missing_plan.log | 31 + .../kunit/test_data/test_strip_hyphen.log | 16 + 7 files changed, 951 insertions(+), 389 deletions(-) create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-kselftest.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log create mode 100644 tools/testing/kunit/test_data/test_strip_hyphen.log diff --git a/tools/testing/kunit/kunit_json.py b/tools/testing/kunit/kunit_json.py index f5cca5c38cac..e7317b4fad9d 100644 --- a/tools/testing/kunit/kunit_json.py +++ b/tools/testing/kunit/kunit_json.py @@ -11,47 +11,46 @@ import os import kunit_parser -from kunit_parser import TestStatus - -def get_json_result(test_result, def_config, build_dir, json_path) -> str: - sub_groups = [] - - # Each test suite is mapped to a KernelCI sub_group - for test_suite in test_result.suites: - sub_group = { - "name": test_suite.name, - "arch": "UM", - "defconfig": def_config, - "build_environment": build_dir, - "test_cases": [], - "lab_name": None, - "kernel": None, - "job": None, - "git_branch": "kselftest", - } - test_cases = [] - # TODO: Add attachments attribute in test_case with detailed - # failure message, see https://api.kernelci.org/schema-test-case.html#get - for case in test_suite.cases: - test_case = {"name": case.name, "status": "FAIL"} - if case.status == TestStatus.SUCCESS: +from kunit_parser import Test, TestResult, TestStatus +from typing import Any, Dict + +JsonObj = Dict[str, Any] + +def _get_group_json(test: Test, def_config: str, build_dir: str) -> JsonObj: + sub_groups = [] # List[JsonObj] + test_cases = [] # List[JsonObj] + + for subtest in test.subtests: + if len(subtest.subtests): + sub_group = _get_group_json(subtest, def_config, + build_dir) + sub_groups.append(sub_group) + else: + test_case = {"name": subtest.name, "status": "FAIL"} + if subtest.status == TestStatus.SUCCESS: test_case["status"] = "PASS" - elif case.status == TestStatus.TEST_CRASHED: + elif subtest.status == TestStatus.TEST_CRASHED: test_case["status"] = "ERROR" test_cases.append(test_case) - sub_group["test_cases"] = test_cases - sub_groups.append(sub_group) + test_group = { - "name": "KUnit Test Group", + "name": test.name, "arch": "UM", "defconfig": def_config, "build_environment": build_dir, "sub_groups": sub_groups, + "test_cases": test_cases, "lab_name": None, "kernel": None, "job": None, "git_branch": "kselftest", } + return test_group + +def get_json_result(test_result: TestResult, def_config: str, build_dir: str, + json_path: str) -> str: + test_group = _get_group_json(test_result.test, def_config, build_dir) + test_group["name"] = "KUnit Test Group" json_obj = json.dumps(test_group, indent=4) if json_path != 'stdout': with open(json_path, 'w') as result_path: diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 6310a641b151..4b6086159c7f 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -1,11 +1,15 @@ # SPDX-License-Identifier: GPL-2.0 # -# Parses test results from a kernel dmesg log. +# Parses KTAP test results from a kernel dmesg log and incrementally prints +# results with reader-friendly format. Stores and returns test results in a +# Test object. # # Copyright (C) 2019, Google LLC. # Author: Felix Guo <felixguoxiuping(a)gmail.com> # Author: Brendan Higgins <brendanhiggins(a)google.com> +# Author: Rae Moar <rmoar(a)google.com> +from __future__ import annotations import re from collections import namedtuple @@ -14,33 +18,55 @@ from enum import Enum, auto from functools import reduce from typing import Iterable, Iterator, List, Optional, Tuple -TestResult = namedtuple('TestResult', ['status','suites','log']) - -class TestSuite(object): - def __init__(self) -> None: - self.status = TestStatus.SUCCESS - self.name = '' - self.cases = [] # type: List[TestCase] - - def __str__(self) -> str: - return 'TestSuite(' + str(self.status) + ',' + self.name + ',' + str(self.cases) + ')' - - def __repr__(self) -> str: - return str(self) - -class TestCase(object): +TestResult = namedtuple('TestResult', ['status','test','log']) + +class Test(object): + """ + A class to represent a test parsed from KTAP results. All KTAP + results within a test log are stored in a main Test object as + subtests. + + Attributes: + status : TestStatus - status of the test + name : str - name of the test + expected_count : int - expected number of subtests (0 if single + test case and None if unknown expected number of subtests) + subtests : List[Test] - list of subtests + log : List[str] - log of KTAP lines that correspond to the test + counts : TestCounts - counts of the test statuses and errors of + subtests or of the test itself if the test is a single + test case. + """ def __init__(self) -> None: + """Constructs the default attributes of a Test class object. + """ self.status = TestStatus.SUCCESS self.name = '' + self.expected_count = 0 # type: Optional[int] + self.subtests = [] # type: List[Test] self.log = [] # type: List[str] + self.counts = TestCounts() def __str__(self) -> str: - return 'TestCase(' + str(self.status) + ',' + self.name + ',' + str(self.log) + ')' + """Returns string representation of a Test class object.""" + return ('Test(' + str(self.status) + ', ' + self.name + + ', ' + str(self.expected_count) + ', ' + + str(self.subtests) + ', ' + str(self.log) + ', ' + + str(self.counts) + ')') def __repr__(self) -> str: + """Returns string representation of a Test class object.""" return str(self) + def add_error(self, error_message: str) -> None: + """Adds error to test object by incrementing the error count + and printing the error message. + """ + self.counts.errors += 1 + print_error('Test ' + self.name + ': ' + error_message) + class TestStatus(Enum): + """An enumeration class to represent the status of a test.""" SUCCESS = auto() FAILURE = auto() SKIPPED = auto() @@ -48,381 +74,769 @@ class TestStatus(Enum): NO_TESTS = auto() FAILURE_TO_PARSE_TESTS = auto() +class TestCounts: + """ + A class to represent the counts of statuses and test errors of + subtests or of the test itself if the test is a single test case with + no subtests. Note that the sum of the counts of passed, failed, + crashed, and skipped should sum to the total number of subtests for + the test. + + Attributes: + passed : int - the number of tests that have passed + failed : int - the number of tests that have failed + crashed : int - the number of tests that have crashed + skipped : int - the number of tests that have skipped + errors : int - the number of errors in the test and subtests + """ + def __init__(self): + """Contructs the default attributes of a TestCounts class + object. Sets the counts of all test statuses and test + errors to be 0. + """ + self.passed = 0 + self.failed = 0 + self.crashed = 0 + self.skipped = 0 + self.errors = 0 + + def __str__(self) -> str: + """Returns the string representation of a TestCounts object. + """ + return ('Passed: ' + str(self.passed) + + ', Failed: ' + str(self.failed) + + ', Crashed: ' + str(self.crashed) + + ', Skipped: ' + str(self.skipped) + + ', Errors: ' + str(self.errors)) + + def total(self) -> int: + """Returns total number of subtests or 1 if the test object + has no subtests to represent the test itself. This number is + calculated by the sum of the passed, failed, crashed, and + skipped subtests. + """ + return (self.passed + self.failed + self.crashed + + self.skipped) + + def add_subtest_counts(self, counts: TestCounts) -> None: + """ + Adds the counts of another TestCounts object to the current + TestCounts object. Used to add the counts of a subtest to the + parent test. + + Parameters: + counts - a different TestCounts object whose counts + will be added to the counts of the TestCounts object + """ + self.passed += counts.passed + self.failed += counts.failed + self.crashed += counts.crashed + self.skipped += counts.skipped + self.errors += counts.errors + + def get_status(self) -> TestStatus: + """Returns the expected status of a Test using test counts.""" + if self.crashed: + # If one of the subtests crash, the expected status + # of the Test is crashed. + return TestStatus.TEST_CRASHED + elif self.failed: + # Otherwise if one of the subtests fail, the + # expected status of the Test is failed. + return TestStatus.FAILURE + elif self.passed: + # Otherwise if one of the subtests pass, the + # expected status of the Test is passed. + return TestStatus.SUCCESS + else: + # Finally, if none of the subtests have failed, + # crashed, or passed, the expected status of the + # Test is skipped. + return TestStatus.SKIPPED + + def add_status(self, status: TestStatus) -> None: + """ + Given inputted status, increments corresponding attribute of + TestCounts object. + + Parameters: + status - status to be added to the TestCounts object + """ + if status == TestStatus.SUCCESS or \ + status == TestStatus.NO_TESTS: + # if status is NO_TESTS the most appropriate + # attribute to increment is passed because + # the test did not fail, crash or get skipped. + self.passed += 1 + elif status == TestStatus.FAILURE: + self.failed += 1 + elif status == TestStatus.SKIPPED: + self.skipped += 1 + else: + self.crashed += 1 + class LineStream: - """Provides a peek()/pop() interface over an iterator of (line#, text).""" + """ + A class to represent the lines of kernel output. + Provides a peek()/pop() interface over an iterator of + (line#, text). + """ _lines: Iterator[Tuple[int, str]] _next: Tuple[int, str] _done: bool def __init__(self, lines: Iterator[Tuple[int, str]]): + """Set defaults for LineStream object and sets _lines + attribute to lines parameter. + """ self._lines = lines self._done = False self._next = (0, '') self._get_next() def _get_next(self) -> None: + """Advances the LineSteam to the next line or sets the _done + attribute if the LineStream has reached the end of the lines. + """ try: self._next = next(self._lines) except StopIteration: self._done = True def peek(self) -> str: + """Returns the next line in the LineStream without advancing + the LineStream. + """ return self._next[1] def pop(self) -> str: + """Returns the next line in the LineStream and advances the + LineStream to the next line. + """ n = self._next self._get_next() return n[1] def __bool__(self) -> bool: + """Returns whether the LineStream has reached the end of the + lines. + """ return not self._done # Only used by kunit_tool_test.py. def __iter__(self) -> Iterator[str]: + """Empties all lines stored in LineStream object into + Iterator object and returns the Iterator object. + """ while bool(self): yield self.pop() def line_number(self) -> int: + """Returns the line number of the next line in the + LineStream. + """ return self._next[0] -kunit_start_re = re.compile(r'TAP version [0-9]+$') -kunit_end_re = re.compile('(List of all partitions:|' - 'Kernel panic - not syncing: VFS:|reboot: System halted)') +# Parsing helper methods: + +KTAP_START = re.compile(r'KTAP version ([0-9]+)$') +TAP_START = re.compile(r'TAP version ([0-9]+)$') +KTAP_END = re.compile('(List of all partitions:|' + 'Kernel panic - not syncing: VFS:|reboot: System halted)') def extract_tap_lines(kernel_output: Iterable[str]) -> LineStream: - def isolate_kunit_output(kernel_output: Iterable[str]) -> Iterator[Tuple[int, str]]: + """Extracts KTAP lines from inputted kernel output in LineStream + object.""" + def isolate_ktap_output(kernel_output: Iterable[str]) \ + -> Iterator[Tuple[int, str]]: line_num = 0 started = False for line in kernel_output: line_num += 1 - line = line.rstrip() # line always has a trailing \n - if kunit_start_re.search(line): + line = line.rstrip() # remove trailing \n + if not started and KTAP_START.search(line): + # start extracting KTAP lines and set prefix + # to number of characters before version line + prefix_len = len( + line.split('KTAP version')[0]) + started = True + yield line_num, line[prefix_len:] + elif not started and TAP_START.search(line): + # start extracting KTAP lines and set prefix + # to number of characters before version line prefix_len = len(line.split('TAP version')[0]) started = True yield line_num, line[prefix_len:] - elif kunit_end_re.search(line): + elif started and KTAP_END.search(line): + # stop extracting KTAP lines break elif started: - yield line_num, line[prefix_len:] - return LineStream(lines=isolate_kunit_output(kernel_output)) - -DIVIDER = '=' * 60 - -RESET = '\033[0;0m' - -def red(text) -> str: - return '\033[1;31m' + text + RESET - -def yellow(text) -> str: - return '\033[1;33m' + text + RESET - -def green(text) -> str: - return '\033[1;32m' + text + RESET - -def print_with_timestamp(message) -> None: - print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) - -def format_suite_divider(message) -> str: - return '======== ' + message + ' ========' - -def print_suite_divider(message) -> None: - print_with_timestamp(DIVIDER) - print_with_timestamp(format_suite_divider(message)) - -def print_log(log) -> None: - for m in log: - print_with_timestamp(m) - -TAP_ENTRIES = re.compile(r'^(TAP|[\s]*ok|[\s]*not ok|[\s]*[0-9]+\.\.[0-9]+|[\s]*# (Subtest:|.*: kunit test case crashed!)).*$') - -def consume_non_diagnostic(lines: LineStream) -> None: - while lines and not TAP_ENTRIES.match(lines.peek()): - lines.pop() - -def save_non_diagnostic(lines: LineStream, test_case: TestCase) -> None: - while lines and not TAP_ENTRIES.match(lines.peek()): - test_case.log.append(lines.peek()) - lines.pop() - -OkNotOkResult = namedtuple('OkNotOkResult', ['is_ok','description', 'text']) - -OK_NOT_OK_SKIP = re.compile(r'^[\s]*(ok|not ok) [0-9]+ - (.*) # SKIP(.*)$') - -OK_NOT_OK_SUBTEST = re.compile(r'^[\s]+(ok|not ok) [0-9]+ - (.*)$') - -OK_NOT_OK_MODULE = re.compile(r'^(ok|not ok) ([0-9]+) - (.*)$') - -def parse_ok_not_ok_test_case(lines: LineStream, test_case: TestCase) -> bool: - save_non_diagnostic(lines, test_case) - if not lines: - test_case.status = TestStatus.TEST_CRASHED - return True - line = lines.peek() - match = OK_NOT_OK_SUBTEST.match(line) - while not match and lines: - line = lines.pop() - match = OK_NOT_OK_SUBTEST.match(line) - if match: - test_case.log.append(lines.pop()) - test_case.name = match.group(2) - skip_match = OK_NOT_OK_SKIP.match(line) - if skip_match: - test_case.status = TestStatus.SKIPPED - return True - if test_case.status == TestStatus.TEST_CRASHED: - return True - if match.group(1) == 'ok': - test_case.status = TestStatus.SUCCESS - else: - test_case.status = TestStatus.FAILURE - return True + # remove prefix and any indention and yield + # line with line number + line = line[prefix_len:].lstrip() + yield line_num, line + return LineStream(lines=isolate_ktap_output(kernel_output)) + +def raw_output(kernel_output: Iterable[str]) -> None: + """Prints all lines of kernel output.""" + for line in kernel_output: + print(line.rstrip()) + +KTAP_VERSIONS = [1] +TAP_VERSIONS = [13, 14] + +def check_version(version_num: int, accepted_versions: List[int], + version_type: str, test: Test) -> None: + """ + Adds error to test object if version number is too high or too + low. + + Parameters: + version_num - The inputted version number from the parsed KTAP or TAP + header line + accepted_version - List of accepted KTAP or TAP versions + version_type - 'KTAP' or 'TAP' depending on the type of + version line. + test - Test object for current test being parsed + """ + if version_num < min(accepted_versions): + test.add_error(version_type + + ' version lower than expected!') + elif version_num > max(accepted_versions): + test.add_error( + version_type + ' version higher than expected!') + +def parse_ktap_header(lines: LineStream, test: Test) -> bool: + """ + If the next line in LineStream matches the format of KTAP or TAP + header line, the version number is checked, the line is popped, + and returns True. Otherwise the method returns False. + + Accepted formats: + - 'KTAP version [version number]' + - 'TAP version [version number]' + + Parameters: + lines - LineStream of KTAP output to parse + test - Test object for current test being parsed + + Return: + Boolean that represents if the next line in the LineStream was parsed + as the KTAP or TAP header line + """ + ktap_match = KTAP_START.match(lines.peek()) + tap_match = TAP_START.match(lines.peek()) + if ktap_match: + version_num = int(ktap_match.group(1)) + check_version(version_num, KTAP_VERSIONS, 'KTAP', test) + elif tap_match: + version_num = int(tap_match.group(1)) + check_version(version_num, TAP_VERSIONS, 'TAP', test) else: return False - -SUBTEST_DIAGNOSTIC = re.compile(r'^[\s]+# (.*)$') -DIAGNOSTIC_CRASH_MESSAGE = re.compile(r'^[\s]+# .*?: kunit test case crashed!$') - -def parse_diagnostic(lines: LineStream, test_case: TestCase) -> bool: - save_non_diagnostic(lines, test_case) - if not lines: + test.log.append(lines.pop()) + return True + +TEST_HEADER = re.compile(r'^# Subtest: (.*)$') + +def parse_test_header(lines: LineStream, test: Test) -> bool: + """ + If the next line in LineStream matches the format of a test + header line, the name of test is set, the line is popped, + and returns True. Otherwise the method returns False. + + Accepted format: + - '# Subtest: [test name]' + + Parameters: + lines - LineStream of ktap output to parse + test - Test object for current test being parsed + + Return: + Boolean that represents if the next line in the LineStream was parsed + as a test header + """ + match = TEST_HEADER.match(lines.peek()) + if not match: return False + test.log.append(lines.pop()) + test.name = match.group(1) + return True + +TEST_PLAN = re.compile(r'1\.\.([0-9]+)') + +def parse_test_plan(lines: LineStream, test: Test) -> bool: + """ + If the next line in LineStream matches the format of a test + plan line, the expected number of subtests is set in test object, an + error is thrown if there are 0 tests, the line is popped, + and returns True. Otherwise the method adds an error that the test + plan is missing to the test object and returns False. + + Accepted format: + - '1..[number of subtests]' + + Parameters: + lines - LineStream of ktap output to parse + test - Test object for current test being parsed + + Return: + Boolean that represents if the next line in the LineStream was parsed + as a test plan + """ + match = TEST_PLAN.match(lines.peek()) + if not match: + test.expected_count = None + test.add_error('missing plan line!') + return False + test.log.append(lines.pop()) + expected_count = int(match.group(1)) + test.expected_count = expected_count + if expected_count == 0: + test.status = TestStatus.NO_TESTS + test.add_error('0 tests run!') + return True + +TEST_RESULT = re.compile(r'^(ok|not ok) ([0-9]+) (- )?(.*)$') + +TEST_RESULT_SKIP = re.compile(r'^(ok|not ok) ([0-9]+) (- )?(.*) # SKIP(.*)$') + +def peek_test_name_match(lines: LineStream, test: Test) -> bool: + """ + If the next line in LineStream matches the format of a test + result line and the name of the result line matches the name of the + current test, the method returns True. Otherwise it returns False. + + Accepted format: + - '[ok|not ok] [test number] [-] [test name] [optional skip + directive]' + + Parameters: + lines - LineStream of KTAP output to parse + test - Test object for current test being parsed + + Return: + Boolean that represents if the next line in the LineStream matched a + test result line and the name matched the expected test name + """ line = lines.peek() - match = SUBTEST_DIAGNOSTIC.match(line) - if match: - test_case.log.append(lines.pop()) - crash_match = DIAGNOSTIC_CRASH_MESSAGE.match(line) - if crash_match: - test_case.status = TestStatus.TEST_CRASHED - return True - else: + match = TEST_RESULT.match(line) + if not match: return False + name = match.group(4) + return (name == test.name) + +def parse_test_result(lines: LineStream, test: Test, + expected_num: int) -> bool: + """ + If the next line in LineStream matches the format of a test + result line, the status in the result line is added to the test + object, the test number is checked to match the expected test number + and if not an error is added to the test object, and returns True. + Otherwise it returns False. + + Note that the skip diirective is the only + directive that causes a change in status and otherwise the directive + is included in the name of the test. + + Accepted format: + - '[ok|not ok] [test number] [-] [test name] [optional skip + directive]' + + Parameters: + lines - LineStream of KTAP output to parse + test - Test object for current test being parsed + expected_num - expected test number for current test + + Return: + Boolean that represents if the next line in the LineStream was parsed + as a test result line. + """ + line = lines.peek() + match = TEST_RESULT.match(line) + skip_match = TEST_RESULT_SKIP.match(line) -def parse_test_case(lines: LineStream) -> Optional[TestCase]: - test_case = TestCase() - save_non_diagnostic(lines, test_case) - while parse_diagnostic(lines, test_case): - pass - if parse_ok_not_ok_test_case(lines, test_case): - return test_case - else: - return None - -SUBTEST_HEADER = re.compile(r'^[\s]+# Subtest: (.*)$') + # Check if line matches test result line format + if not match: + return False + test.log.append(lines.pop()) -def parse_subtest_header(lines: LineStream) -> Optional[str]: - consume_non_diagnostic(lines) - if not lines: - return None - match = SUBTEST_HEADER.match(lines.peek()) - if match: - lines.pop() - return match.group(1) + # Set name of test object + if skip_match: + test.name = skip_match.group(4) else: - return None + test.name = match.group(4) -SUBTEST_PLAN = re.compile(r'[\s]+[0-9]+\.\.([0-9]+)') + # Check test num + num = int(match.group(2)) + if num != expected_num: + test.add_error('Expected test number ' + + str(expected_num) + ' but found ' + str(num)) -def parse_subtest_plan(lines: LineStream) -> Optional[int]: - consume_non_diagnostic(lines) - match = SUBTEST_PLAN.match(lines.peek()) - if match: - lines.pop() - return int(match.group(1)) - else: - return None - -def max_status(left: TestStatus, right: TestStatus) -> TestStatus: - if left == right: - return left - elif left == TestStatus.TEST_CRASHED or right == TestStatus.TEST_CRASHED: - return TestStatus.TEST_CRASHED - elif left == TestStatus.FAILURE or right == TestStatus.FAILURE: - return TestStatus.FAILURE - elif left == TestStatus.SKIPPED: - return right - else: - return left - -def parse_ok_not_ok_test_suite(lines: LineStream, - test_suite: TestSuite, - expected_suite_index: int) -> bool: - consume_non_diagnostic(lines) - if not lines: - test_suite.status = TestStatus.TEST_CRASHED - return False - line = lines.peek() - match = OK_NOT_OK_MODULE.match(line) - if match: - lines.pop() - if match.group(1) == 'ok': - test_suite.status = TestStatus.SUCCESS - else: - test_suite.status = TestStatus.FAILURE - skip_match = OK_NOT_OK_SKIP.match(line) - if skip_match: - test_suite.status = TestStatus.SKIPPED - suite_index = int(match.group(2)) - if suite_index != expected_suite_index: - print_with_timestamp( - red('[ERROR] ') + 'expected_suite_index ' + - str(expected_suite_index) + ', but got ' + - str(suite_index)) + # Set status of test object + status = match.group(1) + if test.status == TestStatus.TEST_CRASHED: return True + elif skip_match: + test.status = TestStatus.SKIPPED + elif status == 'ok': + test.status = TestStatus.SUCCESS else: - return False - -def bubble_up_errors(status_list: Iterable[TestStatus]) -> TestStatus: - return reduce(max_status, status_list, TestStatus.SKIPPED) + test.status = TestStatus.FAILURE + return True + +def parse_diagnostic(lines: LineStream) -> List[str]: + """ + If the next line in LineStream does not match the format of a test + case line or test header line, the line is checked if the test has + crashed and if so adds an error message, pops the line and adds it to + the log. + + Line formats that are not parsed: + - '# Subtest: [test name]' + - '[ok|not ok] [test number] [-] [test name] [optional skip + directive]' + + Parameters: + lines - LineStream of KTAP output to parse + + Return: + Log of diagnostic lines + """ + log = [] # type: List[str] + while lines and not TEST_RESULT.match(lines.peek()) and not \ + TEST_HEADER.match(lines.peek()): + log.append(lines.pop()) + return log + +DIAGNOSTIC_CRASH_MESSAGE = re.compile(r'^# .*?: kunit test case crashed!$') + +def parse_crash_in_log(test: Test) -> bool: + """ + Iterate through the lines of the log to parse for crash message. + If crash message found, set status to crashed and return True. + Otherwise return False. + + Parameters: + test - Test object for current test being parsed + + Return: + Boolean that represents if crash message found in log + """ + for line in test.log: + if DIAGNOSTIC_CRASH_MESSAGE.match(line): + test.status = TestStatus.TEST_CRASHED + return True + return False -def bubble_up_test_case_errors(test_suite: TestSuite) -> TestStatus: - max_test_case_status = bubble_up_errors(x.status for x in test_suite.cases) - return max_status(max_test_case_status, test_suite.status) +# Printing helper methods: -def parse_test_suite(lines: LineStream, expected_suite_index: int) -> Optional[TestSuite]: - if not lines: - return None - consume_non_diagnostic(lines) - test_suite = TestSuite() - test_suite.status = TestStatus.SUCCESS - name = parse_subtest_header(lines) - if not name: - return None - test_suite.name = name - expected_test_case_num = parse_subtest_plan(lines) - if expected_test_case_num is None: - return None - while expected_test_case_num > 0: - test_case = parse_test_case(lines) - if not test_case: - break - test_suite.cases.append(test_case) - expected_test_case_num -= 1 - if parse_ok_not_ok_test_suite(lines, test_suite, expected_suite_index): - test_suite.status = bubble_up_test_case_errors(test_suite) - return test_suite - elif not lines: - print_with_timestamp(red('[ERROR] ') + 'ran out of lines before end token') - return test_suite - else: - print(f'failed to parse end of suite "{name}", at line {lines.line_number()}: {lines.peek()}') - return None +DIVIDER = '=' * 60 -TAP_HEADER = re.compile(r'^TAP version 14$') +RESET = '\033[0;0m' -def parse_tap_header(lines: LineStream) -> bool: - consume_non_diagnostic(lines) - if TAP_HEADER.match(lines.peek()): - lines.pop() - return True - else: - return False +def red(text: str) -> str: + """Returns inputted string with red color code.""" + return '\033[1;31m' + text + RESET -TEST_PLAN = re.compile(r'[0-9]+\.\.([0-9]+)') +def yellow(text: str) -> str: + """Returns inputted string with yellow color code.""" + return '\033[1;33m' + text + RESET -def parse_test_plan(lines: LineStream) -> Optional[int]: - consume_non_diagnostic(lines) - match = TEST_PLAN.match(lines.peek()) - if match: - lines.pop() - return int(match.group(1)) - else: - return None - -def bubble_up_suite_errors(test_suites: Iterable[TestSuite]) -> TestStatus: - return bubble_up_errors(x.status for x in test_suites) - -def parse_test_result(lines: LineStream) -> TestResult: - consume_non_diagnostic(lines) - if not lines or not parse_tap_header(lines): - return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) - expected_test_suite_num = parse_test_plan(lines) - if expected_test_suite_num == 0: - return TestResult(TestStatus.NO_TESTS, [], lines) - elif expected_test_suite_num is None: - return TestResult(TestStatus.FAILURE_TO_PARSE_TESTS, [], lines) - test_suites = [] - for i in range(1, expected_test_suite_num + 1): - test_suite = parse_test_suite(lines, i) - if test_suite: - test_suites.append(test_suite) - else: - print_with_timestamp( - red('[ERROR] ') + ' expected ' + - str(expected_test_suite_num) + - ' test suites, but got ' + str(i - 2)) - break - test_suite = parse_test_suite(lines, -1) - if test_suite: - print_with_timestamp(red('[ERROR] ') + - 'got unexpected test suite: ' + test_suite.name) - if test_suites: - return TestResult(bubble_up_suite_errors(test_suites), test_suites, lines) - else: - return TestResult(TestStatus.NO_TESTS, [], lines) +def green(text: str) -> str: + """Returns inputted string with green color code.""" + return '\033[1;32m' + text + RESET -class TestCounts: - passed: int - failed: int - crashed: int - skipped: int +ANSI_LEN = len(red('')) - def __init__(self): - self.passed = 0 - self.failed = 0 - self.crashed = 0 - self.skipped = 0 +def print_with_timestamp(message: str) -> None: + """Prints message with timestamp at beginning.""" + print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) - def total(self) -> int: - return self.passed + self.failed + self.crashed + self.skipped - -def print_and_count_results(test_result: TestResult) -> TestCounts: - counts = TestCounts() - for test_suite in test_result.suites: - if test_suite.status == TestStatus.SUCCESS: - print_suite_divider(green('[PASSED] ') + test_suite.name) - elif test_suite.status == TestStatus.SKIPPED: - print_suite_divider(yellow('[SKIPPED] ') + test_suite.name) - elif test_suite.status == TestStatus.TEST_CRASHED: - print_suite_divider(red('[CRASHED] ' + test_suite.name)) +def format_test_divider(message: str, len_message: int) -> str: + """ + Returns string with message centered in fixed width divider. + + Example: + '===================== message example =====================' + + Parameters: + message - message to be centered in divider line + len_message - length of the message to be printed such that + any characters of the color codes are not counted + + Return: + String containing message centered in fixed width divider + """ + default_count = 3 # default number of dashes + len_1 = default_count + len_2 = default_count + difference = len(DIVIDER) - len_message - 2 # 2 spaces added + if difference > 0: + # calculate number of dashes for each side of the divider + len_1 = int(difference / 2) + len_2 = difference - len_1 + return ('=' * len_1) + ' ' + message + ' ' + ('=' * len_2) + +def print_test_header(test: Test) -> None: + """ + Prints test header with test name and optionally the expected number + of subtests. + + Example: + '=================== example (2 subtests) ===================' + + Parameters: + test - Test object representing current test being printed + """ + message = test.name + if test.expected_count: + message += ' (' + str(test.expected_count) + ' subtests)' + print_with_timestamp(format_test_divider(message, len(message))) + +def print_log(log: Iterable[str]) -> None: + """ + Prints all strings in saved log for test in yellow. + + Parameters: + log - Iterable object with all strings saved in log for test + """ + for m in log: + print_with_timestamp(yellow(m)) + print_with_timestamp('') + +def format_test_result(test: Test) -> str: + """ + Returns string with formatted test result with colored status and test + name. + + Example: + '[PASSED] example' + + Parameters: + test - Test object representing current test being printed + + Return: + String containing formatted test result + """ + if test.status == TestStatus.SUCCESS: + return (green('[PASSED] ') + test.name) + elif test.status == TestStatus.SKIPPED: + return (yellow('[SKIPPED] ') + test.name) + elif test.status == TestStatus.TEST_CRASHED: + print_log(test.log) + return (red('[CRASHED] ') + test.name) + else: + print_log(test.log) + return (red('[FAILED] ') + test.name) + +def print_test_result(test: Test) -> None: + """ + Prints result line with status of test. + + Example: + '[PASSED] example' + + Parameters: + test - Test object representing current test being printed + """ + print_with_timestamp(format_test_result(test)) + +def print_test_footer(test: Test) -> None: + """ + Prints test footer with status of test. + + Example: + '===================== [PASSED] example =====================' + + Parameters: + test - Test object representing current test being printed + """ + message = format_test_result(test) + print_with_timestamp(format_test_divider(message, + len(message) - ANSI_LEN)) + +def print_summary_line(test: Test) -> None: + """ + Prints summary line of test object. Color of line is dependent on + status of test. Color is green if test passes, yellow if test is + skipped, and red if the test fails or crashes. Summary line contains + counts of the statuses of the tests subtests or the test itself if it + has no subtests. + + Example: + "Testing complete. Passed: 2, Failed: 0, Crashed: 0, Skipped: 0, + Errors: 0" + + test - Test object representing current test being printed + """ + if test.status == TestStatus.SUCCESS or \ + test.status == TestStatus.NO_TESTS: + color = green + elif test.status == TestStatus.SKIPPED: + color = yellow + else: + color = red + counts = test.counts + print_with_timestamp(color('Testing complete. ' + str(counts))) + +def print_error(error_message: str) -> None: + """ + Prints error message with error format. + + Example: + "[ERROR] Test example: missing test plan!" + + Parameters: + error_message - message describing error + """ + print_with_timestamp(red('[ERROR] ') + error_message) + +# Other methods: + +def bubble_up_test_results(test: Test) -> None: + """ + If the test has subtests, add the test counts of the subtests to the + test and check if any of the tests crashed and if so set the test + status to crashed. Otherwise if the test has no subtests add the + status of the test to the test counts. + + Parameters: + test - Test object for current test being parsed + """ + parse_crash_in_log(test) + subtests = test.subtests + counts = test.counts + status = test.status + for t in subtests: + counts.add_subtest_counts(t.counts) + if counts.total() == 0: + counts.add_status(status) + elif test.counts.get_status() == TestStatus.TEST_CRASHED: + test.status = TestStatus.TEST_CRASHED + +def parse_test(lines: LineStream, expected_num: int, log: List[str]) -> Test: + """ + Finds next test to parse in LineStream, creates new Test object, + parses any subtests of the test, populates Test object with all + information (status, name) about the test and the Test objects for + any subtests, and then returns the Test object. The method accepts + three formats of tests: + + Accepted test formats: + + - Main KTAP/TAP header + + Example: + + KTAP version 1 + 1..4 + [subtests] + + - Subtest header line + + Example: + + # Subtest: name + 1..3 + [subtests] + ok 1 name + + - Test result line + + Example: + + ok 1 - test + + Parameters: + lines - LineStream of KTAP output to parse + expected_num - expected test number for test to be parsed + log - list of strings containing any preceding diagnostic lines + corresponding to the current test + + Return: + Test object populated with characteristics and any subtests + """ + test = Test() + test.log.extend(log) + parent_test = False + main = parse_ktap_header(lines, test) + if main: + # If KTAP/TAP header is found, attempt to parse + # test plan + test.name = "main" + parse_test_plan(lines, test) + else: + # If KTAP/TAP header is not found, test must be subtest + # header or test result line so parse attempt to parser + # subtest header + parent_test = parse_test_header(lines, test) + if parent_test: + # If subtest header is found, attempt to parse + # test plan and print header + parse_test_plan(lines, test) + print_test_header(test) + expected_count = test.expected_count + subtests = [] + test_num = 1 + while main or expected_count is None or test_num <= expected_count: + # Loop to parse any subtests. + # If test is main test, do not break until no lines left. + # Otherwise, break after parsing expected number of tests or + # if expected number of tests is unknown break when found + # test result line with matching name to subtest header. + if not lines: + if expected_count and test_num <= expected_count: + test.add_error('missing expected subtests!') + break + sub_log = parse_diagnostic(lines) + if not expected_count and not main and \ + peek_test_name_match(lines, test): + test.log.extend(sub_log) + break + subtests.append(parse_test(lines, test_num, sub_log)) + test_num += 1 + test.subtests = subtests + if not main: + # If not main test, look for test result line + test.log.extend(parse_diagnostic(lines)) + if (parent_test and peek_test_name_match(lines, test)) or \ + not parent_test: + parse_test_result(lines, test, expected_num) else: - print_suite_divider(red('[FAILED] ') + test_suite.name) - for test_case in test_suite.cases: - if test_case.status == TestStatus.SUCCESS: - counts.passed += 1 - print_with_timestamp(green('[PASSED] ') + test_case.name) - elif test_case.status == TestStatus.SKIPPED: - counts.skipped += 1 - print_with_timestamp(yellow('[SKIPPED] ') + test_case.name) - elif test_case.status == TestStatus.TEST_CRASHED: - counts.crashed += 1 - print_with_timestamp(red('[CRASHED] ' + test_case.name)) - print_log(map(yellow, test_case.log)) - print_with_timestamp('') - else: - counts.failed += 1 - print_with_timestamp(red('[FAILED] ') + test_case.name) - print_log(map(yellow, test_case.log)) - print_with_timestamp('') - return counts + test.add_error('missing subtest result line!') + # Add statuses to TestCounts attribute in Test object + bubble_up_test_results(test) + if parent_test: + # If test has subtests and is not the main test object, print + # footer. + print_test_footer(test) + elif not main: + print_test_result(test) + return test def parse_run_tests(kernel_output: Iterable[str]) -> TestResult: - counts = TestCounts() + """ + Using kernel output, extract KTAP lines, parse the lines for test + results and print condensed test results and summary line . + + Parameters: + kernel_output - Iterable object contains lines of kernel output + + Return: + TestResult - Tuple containg status of main test object, main test + object with all subtests, and log of all KTAP lines. + """ + print_with_timestamp(DIVIDER) lines = extract_tap_lines(kernel_output) - test_result = parse_test_result(lines) - if test_result.status == TestStatus.NO_TESTS: - print(red('[ERROR] ') + yellow('no tests run!')) - elif test_result.status == TestStatus.FAILURE_TO_PARSE_TESTS: - print(red('[ERROR] ') + yellow('could not parse test results!')) + test = Test() + if not lines: + test.add_error('invalid KTAP input!') + test.status = TestStatus.FAILURE_TO_PARSE_TESTS else: - counts = print_and_count_results(test_result) + test = parse_test(lines, 0, []) + if test.status != TestStatus.NO_TESTS: + test.status = test.counts.get_status() print_with_timestamp(DIVIDER) - if test_result.status == TestStatus.SUCCESS: - fmt = green - elif test_result.status == TestStatus.SKIPPED: - fmt = yellow - else: - fmt =red - print_with_timestamp( - fmt('Testing complete. %d tests run. %d failed. %d crashed. %d skipped.' % - (counts.total(), counts.failed, counts.crashed, counts.skipped))) - return test_result + print_summary_line(test) + return TestResult(test.status, test, lines) diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 619c4554cbff..e527b90de8ea 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -106,10 +106,10 @@ class KUnitParserTest(unittest.TestCase): with open(log_path) as file: result = kunit_parser.extract_tap_lines(file.readlines()) self.assertContains('TAP version 14', result) - self.assertContains(' # Subtest: example', result) - self.assertContains(' 1..2', result) - self.assertContains(' ok 1 - example_simple_test', result) - self.assertContains(' ok 2 - example_mock_test', result) + self.assertContains('# Subtest: example', result) + self.assertContains('1..2', result) + self.assertContains('ok 1 - example_simple_test', result) + self.assertContains('ok 2 - example_mock_test', result) self.assertContains('ok 1 - example', result) def test_output_with_prefix_isolated_correctly(self): @@ -117,28 +117,28 @@ class KUnitParserTest(unittest.TestCase): with open(log_path) as file: result = kunit_parser.extract_tap_lines(file.readlines()) self.assertContains('TAP version 14', result) - self.assertContains(' # Subtest: kunit-resource-test', result) - self.assertContains(' 1..5', result) - self.assertContains(' ok 1 - kunit_resource_test_init_resources', result) - self.assertContains(' ok 2 - kunit_resource_test_alloc_resource', result) - self.assertContains(' ok 3 - kunit_resource_test_destroy_resource', result) - self.assertContains(' foo bar #', result) - self.assertContains(' ok 4 - kunit_resource_test_cleanup_resources', result) - self.assertContains(' ok 5 - kunit_resource_test_proper_free_ordering', result) + self.assertContains('# Subtest: kunit-resource-test', result) + self.assertContains('1..5', result) + self.assertContains('ok 1 - kunit_resource_test_init_resources', result) + self.assertContains('ok 2 - kunit_resource_test_alloc_resource', result) + self.assertContains('ok 3 - kunit_resource_test_destroy_resource', result) + self.assertContains('foo bar #', result) + self.assertContains('ok 4 - kunit_resource_test_cleanup_resources', result) + self.assertContains('ok 5 - kunit_resource_test_proper_free_ordering', result) self.assertContains('ok 1 - kunit-resource-test', result) - self.assertContains(' foo bar # non-kunit output', result) - self.assertContains(' # Subtest: kunit-try-catch-test', result) - self.assertContains(' 1..2', result) - self.assertContains(' ok 1 - kunit_test_try_catch_successful_try_no_catch', + self.assertContains('foo bar # non-kunit output', result) + self.assertContains('# Subtest: kunit-try-catch-test', result) + self.assertContains('1..2', result) + self.assertContains('ok 1 - kunit_test_try_catch_successful_try_no_catch', result) - self.assertContains(' ok 2 - kunit_test_try_catch_unsuccessful_try_does_catch', + self.assertContains('ok 2 - kunit_test_try_catch_unsuccessful_try_does_catch', result) self.assertContains('ok 2 - kunit-try-catch-test', result) - self.assertContains(' # Subtest: string-stream-test', result) - self.assertContains(' 1..3', result) - self.assertContains(' ok 1 - string_stream_test_empty_on_creation', result) - self.assertContains(' ok 2 - string_stream_test_not_empty_after_add', result) - self.assertContains(' ok 3 - string_stream_test_get_string', result) + self.assertContains('# Subtest: string-stream-test', result) + self.assertContains('1..3', result) + self.assertContains('ok 1 - string_stream_test_empty_on_creation', result) + self.assertContains('ok 2 - string_stream_test_not_empty_after_add', result) + self.assertContains('ok 3 - string_stream_test_get_string', result) self.assertContains('ok 3 - string-stream-test', result) def test_parse_successful_test_log(self): @@ -148,6 +148,13 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) + def test_parse_successful_nested_tests_log(self): + all_passed_log = test_data_path('test_is_test_passed-all_passed_nested.log') + with open(all_passed_log) as file: + result = kunit_parser.parse_run_tests(file.readlines()) + self.assertEqual( + kunit_parser.TestStatus.SUCCESS, + result.status) def test_parse_failed_test_log(self): failed_log = test_data_path('test_is_test_passed-failure.log') @@ -162,17 +169,31 @@ class KUnitParserTest(unittest.TestCase): with open(empty_log) as file: result = kunit_parser.parse_run_tests( kunit_parser.extract_tap_lines(file.readlines())) - self.assertEqual(0, len(result.suites)) + self.assertEqual(0, len(result.test.subtests)) self.assertEqual( kunit_parser.TestStatus.FAILURE_TO_PARSE_TESTS, result.status) + def test_missing_test_plan(self): + missing_plan_log = test_data_path('test_is_test_passed-' + 'missing_plan.log') + with open(missing_plan_log) as file: + result = kunit_parser.parse_run_tests( + kunit_parser.extract_tap_lines( + file.readlines())) + self.assertEqual(2, result.test.counts.errors) + self.assertEqual( + kunit_parser.TestStatus.SUCCESS, + result.status) + def test_no_tests(self): - empty_log = test_data_path('test_is_test_passed-no_tests_run_with_header.log') - with open(empty_log) as file: + header_log = test_data_path('test_is_test_passed-' + 'no_tests_run_with_header.log') + with open(header_log) as file: result = kunit_parser.parse_run_tests( - kunit_parser.extract_tap_lines(file.readlines())) - self.assertEqual(0, len(result.suites)) + kunit_parser.extract_tap_lines( + file.readlines())) + self.assertEqual(0, len(result.test.subtests)) self.assertEqual( kunit_parser.TestStatus.NO_TESTS, result.status) @@ -182,15 +203,17 @@ class KUnitParserTest(unittest.TestCase): print_mock = mock.patch('builtins.print').start() with open(crash_log) as file: result = kunit_parser.parse_run_tests( - kunit_parser.extract_tap_lines(file.readlines())) - print_mock.assert_any_call(StrContains('could not parse test results!')) + kunit_parser.extract_tap_lines( + file.readlines())) + print_mock.assert_any_call(StrContains('invalid KTAP input!')) print_mock.stop() file.close() def test_crashed_test(self): crashed_log = test_data_path('test_is_test_passed-crash.log') with open(crashed_log) as file: - result = kunit_parser.parse_run_tests(file.readlines()) + result = kunit_parser.parse_run_tests( + file.readlines()) self.assertEqual( kunit_parser.TestStatus.TEST_CRASHED, result.status) @@ -216,6 +239,23 @@ class KUnitParserTest(unittest.TestCase): result.status) file.close() + def test_ignores_hyphen(self): + hyphen_log = test_data_path('test_strip_hyphen.log') + file = open(hyphen_log) + result = kunit_parser.parse_run_tests(file.readlines()) + + # A skipped test does not fail the whole suite. + self.assertEqual( + kunit_parser.TestStatus.SUCCESS, + result.status) + self.assertEqual( + "sysctl_test", + result.test.subtests[0].name) + self.assertEqual( + "example", + result.test.subtests[1].name) + file.close() + def test_ignores_prefix_printk_time(self): prefix_log = test_data_path('test_config_printk_time.log') @@ -224,7 +264,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_ignores_multiple_prefixes(self): prefix_log = test_data_path('test_multiple_prefixes.log') @@ -233,7 +273,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_prefix_mixed_kernel_output(self): mixed_prefix_log = test_data_path('test_interrupted_tap_output.log') @@ -242,7 +282,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_prefix_poundsign(self): pound_log = test_data_path('test_pound_sign.log') @@ -251,16 +291,16 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_kernel_panic_end(self): panic_log = test_data_path('test_kernel_panic_interrupt.log') with open(panic_log) as file: result = kunit_parser.parse_run_tests(file.readlines()) self.assertEqual( - kunit_parser.TestStatus.TEST_CRASHED, + kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) def test_pound_no_prefix(self): pound_log = test_data_path('test_pound_no_prefix.log') @@ -269,7 +309,7 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual( kunit_parser.TestStatus.SUCCESS, result.status) - self.assertEqual('kunit-resource-test', result.suites[0].name) + self.assertEqual('kunit-resource-test', result.test.subtests[0].name) class LinuxSourceTreeTest(unittest.TestCase): @@ -291,6 +331,14 @@ class LinuxSourceTreeTest(unittest.TestCase): pass tree = kunit_kernel.LinuxSourceTree('', kunitconfig_path=dir) + def test_kselftest_nested(self): + kselftest_log = test_data_path('test_is_test_passed-kselftest.log') + with open(kselftest_log) as file: + result = kunit_parser.parse_run_tests(file.readlines()) + self.assertEqual( + kunit_parser.TestStatus.SUCCESS, + result.status) + # TODO: add more test cases. @@ -322,6 +370,12 @@ class KUnitJsonTest(unittest.TestCase): result = self._json_for('test_is_test_passed-no_tests_run_with_header.log') self.assertEqual(0, len(result['sub_groups'])) + def test_nested_json(self): + result = self._json_for('test_is_test_passed-all_passed_nested.log') + self.assertEqual( + {'name': 'example_simple_test', 'status': 'PASS'}, + result["sub_groups"][0]["sub_groups"][0]["test_cases"][0]) + class StrContains(str): def __eq__(self, other): return self in other @@ -380,7 +434,7 @@ class KUnitMainTest(unittest.TestCase): self.assertEqual(e.exception.code, 1) self.assertEqual(self.linux_source_mock.build_reconfig.call_count, 1) self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) - self.print_mock.assert_any_call(StrContains(' 0 tests run')) + self.print_mock.assert_any_call(StrContains('invalid KTAP input!')) def test_exec_raw_output(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) @@ -388,7 +442,7 @@ class KUnitMainTest(unittest.TestCase): self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) for call in self.print_mock.call_args_list: self.assertNotEqual(call, mock.call(StrContains('Testing complete.'))) - self.assertNotEqual(call, mock.call(StrContains(' 0 tests run'))) + self.assertNotEqual(call, mock.call(StrContains(' 0 tests run!'))) def test_run_raw_output(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) @@ -397,7 +451,7 @@ class KUnitMainTest(unittest.TestCase): self.assertEqual(self.linux_source_mock.run_kernel.call_count, 1) for call in self.print_mock.call_args_list: self.assertNotEqual(call, mock.call(StrContains('Testing complete.'))) - self.assertNotEqual(call, mock.call(StrContains(' 0 tests run'))) + self.assertNotEqual(call, mock.call(StrContains(' 0 tests run!'))) def test_run_raw_output_kunit(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) diff --git a/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log b/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log new file mode 100644 index 000000000000..9d5b04fe43a6 --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-all_passed_nested.log @@ -0,0 +1,34 @@ +TAP version 14 +1..2 + # Subtest: sysctl_test + 1..4 + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed + ok 1 - sysctl_test_dointvec_null_tbl_data + # Subtest: example + 1..2 + init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 - example_simple_test + # example_mock_test: initializing + # example_mock_test: example_mock_test passed + ok 2 - example_mock_test + kunit example: all tests passed + ok 2 - example + # sysctl_test_dointvec_table_len_is_zero: sysctl_test_dointvec_table_len_is_zero passed + ok 3 - sysctl_test_dointvec_table_len_is_zero + # sysctl_test_dointvec_table_read_but_position_set: sysctl_test_dointvec_table_read_but_position_set passed + ok 4 - sysctl_test_dointvec_table_read_but_position_set +kunit sysctl_test: all tests passed +ok 1 - sysctl_test + # Subtest: example + 1..2 +init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 - example_simple_test + # example_mock_test: initializing + # example_mock_test: example_mock_test passed + ok 2 - example_mock_test +kunit example: all tests passed +ok 2 - example diff --git a/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log new file mode 100644 index 000000000000..65d3f27feaf2 --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log @@ -0,0 +1,14 @@ +TAP version 13 +1..2 +# selftests: membarrier: membarrier_test_single_thread +# TAP version 13 +# 1..2 +# ok 1 sys_membarrier available +# ok 2 sys membarrier invalid command test: command = -1, flags = 0, errno = 22. Failed as expected +ok 1 selftests: membarrier: membarrier_test_single_thread +# selftests: membarrier: membarrier_test_multi_thread +# TAP version 13 +# 1..2 +# ok 1 sys_membarrier available +# ok 2 sys membarrier invalid command test: command = -1, flags = 0, errno = 22. Failed as expected +ok 2 selftests: membarrier: membarrier_test_multi_thread diff --git a/tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log b/tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log new file mode 100644 index 000000000000..5cd17b7f818a --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-missing_plan.log @@ -0,0 +1,31 @@ +KTAP version 1 + # Subtest: sysctl_test + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed + ok 1 - sysctl_test_dointvec_null_tbl_data + # sysctl_test_dointvec_table_maxlen_unset: sysctl_test_dointvec_table_maxlen_unset passed + ok 2 - sysctl_test_dointvec_table_maxlen_unset + # sysctl_test_dointvec_table_len_is_zero: sysctl_test_dointvec_table_len_is_zero passed + ok 3 - sysctl_test_dointvec_table_len_is_zero + # sysctl_test_dointvec_table_read_but_position_set: sysctl_test_dointvec_table_read_but_position_set passed + ok 4 - sysctl_test_dointvec_table_read_but_position_set + # sysctl_test_dointvec_happy_single_positive: sysctl_test_dointvec_happy_single_positive passed + ok 5 - sysctl_test_dointvec_happy_single_positive + # sysctl_test_dointvec_happy_single_negative: sysctl_test_dointvec_happy_single_negative passed + ok 6 - sysctl_test_dointvec_happy_single_negative + # sysctl_test_dointvec_single_less_int_min: sysctl_test_dointvec_single_less_int_min passed + ok 7 - sysctl_test_dointvec_single_less_int_min + # sysctl_test_dointvec_single_greater_int_max: sysctl_test_dointvec_single_greater_int_max passed + ok 8 - sysctl_test_dointvec_single_greater_int_max +kunit sysctl_test: all tests passed +ok 1 - sysctl_test + # Subtest: example + 1..2 +init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 - example_simple_test + # example_mock_test: initializing + # example_mock_test: example_mock_test passed + ok 2 - example_mock_test +kunit example: all tests passed +ok 2 - example diff --git a/tools/testing/kunit/test_data/test_strip_hyphen.log b/tools/testing/kunit/test_data/test_strip_hyphen.log new file mode 100644 index 000000000000..92ac7c24b374 --- /dev/null +++ b/tools/testing/kunit/test_data/test_strip_hyphen.log @@ -0,0 +1,16 @@ +KTAP version 1 +1..2 + # Subtest: sysctl_test + 1..1 + # sysctl_test_dointvec_null_tbl_data: sysctl_test_dointvec_null_tbl_data passed + ok 1 - sysctl_test_dointvec_null_tbl_data +kunit sysctl_test: all tests passed +ok 1 - sysctl_test + # Subtest: example + 1..1 +init_suite + # example_simple_test: initializing + # example_simple_test: example_simple_test passed + ok 1 example_simple_test +kunit example: all tests passed +ok 2 example -- 2.33.0.259.gc128427fd7-goog

4 years

2
2
0 0

[PATCH v7 0/6] cgroup/cpuset: Add new cpuset partition type & empty effecitve cpus

by Waiman Long

v7: - Simplify the documentation patch (patch 5) as suggested by Tejun. - Fix a typo in patch 2 and improper commit log in patch 3. v6: - Remove duplicated tmpmask from update_prstate() which should fix the frame size too large problem reported by kernel test robot. v5: - Rebased to the latest for-5.15 branch of cgroup git tree and drop the 1st v4 patch as it has been merged. - Update patch 1 to always allow changing partition root back to member even if it invalidates child partitions undeneath it. - Adjust the empty effective cpu partition patch to not allow 0 effective cpu for terminal partition which will make it invalid). - Add a new patch to enable reading of cpuset.cpus.partition to display the reason that causes invalid partition. - Adjust the documentation and testing patch accordingly. This patchset makes four enhancements to the cpuset v2 code. Patch 1: Properly handle partition root tree and make partition invalid in case changes to cpuset.cpus violate any of the partition root constraints. Patch 2: Enable the "cpuset.cpus.partition" file to show the reason that causes invalid partition like "root invalid (No cpu available due to hotplug)". Patch 3: Add a new partition state "isolated" to create a partition root without load balancing. This is for handling intermitten workloads that have a strict low latency requirement. Patch 4: Allow partition roots that are not the top cpuset to distribute all its cpus to child partitions as long as there is no task associated with that partition root. This allows more flexibility for middleware to manage multiple partitions. Patch 5 updates the cgroup-v2.rst file accordingly. Patch 6 adds a new cpuset test to test the new cpuset partition code. Waiman Long (6): cgroup/cpuset: Properly transition to invalid partition cgroup/cpuset: Show invalid partition reason string cgroup/cpuset: Add a new isolated cpus.partition type cgroup/cpuset: Allow non-top parent partition to distribute out all CPUs cgroup/cpuset: Update description of cpuset.cpus.partition in cgroup-v2.rst kselftest/cgroup: Add cpuset v2 partition root state test Documentation/admin-guide/cgroup-v2.rst | 112 +-- kernel/cgroup/cpuset.c | 337 ++++++--- tools/testing/selftests/cgroup/Makefile | 5 +- .../selftests/cgroup/test_cpuset_prs.sh | 663 ++++++++++++++++++ tools/testing/selftests/cgroup/wait_inotify.c | 86 +++ 5 files changed, 1050 insertions(+), 153 deletions(-) create mode 100755 tools/testing/selftests/cgroup/test_cpuset_prs.sh create mode 100644 tools/testing/selftests/cgroup/wait_inotify.c -- 2.18.1

4 years

4
15
0 0

[PATCH RESEND 00/17] Resend bitmap patches

by Yury Norov

This is a resend of previously submitted series: https://lore.kernel.org/patchwork/cover/1462071/ https://lore.kernel.org/patchwork/patch/1458703/ https://lore.kernel.org/lkml/YPG8SdsbQ+sxjk0w@yury-ThinkPad/T/ https://lore.kernel.org/lkml/YMVSHCY9yEocmfVD@yury-ThinkPad/T/ Most of the patches received testing and review. If I missed to add someone's review tag putting all together - my kind apologise. Please resend it here. I believe I addessed all comments except Joe's one. In comment to patch 3, Joe Perches suggested to rename include/linux/find.h, but didn't give a new name, so I leave it as is. Since this header is not for direct inclusion, I'm OK with any reasonable name, and we can change it later. Andrew, can you please take this series in linux-next? Andy Shevchenko (1): tools: Rename bitmap_alloc() to bitmap_zalloc() Yury Norov (16): bitops: protect find_first_{,zero}_bit properly bitops: move find_bit_*_le functions from le.h to find.h include: move find.h from asm_generic to linux arch: remove GENERIC_FIND_FIRST_BIT entirely lib: add find_first_and_bit() cpumask: use find_first_and_bit() all: replace find_next{,_zero}_bit with find_first{,_zero}_bit where appropriate tools: sync tools/bitmap with mother linux cpumask: replace cpumask_next_* with cpumask_first_* where appropriate include/linux: move for_each_bit() macros from bitops.h to find.h find: micro-optimize for_each_{set,clear}_bit() Replace for_each_*_bit_from() with for_each_*_bit() where appropriate mm/percpu: micro-optimize pcpu_is_populated() bitmap: unify find_bit operations lib: bitmap: add performance test for bitmap_print_to_pagebuf vsprintf: rework bitmap_list_string MAINTAINERS | 4 +- arch/alpha/include/asm/bitops.h | 2 - arch/arc/Kconfig | 1 - arch/arc/include/asm/bitops.h | 1 - arch/arm/include/asm/bitops.h | 1 - arch/arm64/Kconfig | 1 - arch/arm64/include/asm/bitops.h | 1 - arch/csky/include/asm/bitops.h | 1 - arch/h8300/include/asm/bitops.h | 1 - arch/hexagon/include/asm/bitops.h | 1 - arch/ia64/include/asm/bitops.h | 2 - arch/m68k/include/asm/bitops.h | 2 - arch/mips/Kconfig | 1 - arch/mips/include/asm/bitops.h | 1 - arch/openrisc/include/asm/bitops.h | 1 - arch/parisc/include/asm/bitops.h | 2 - arch/powerpc/include/asm/bitops.h | 2 - arch/powerpc/include/asm/cputhreads.h | 2 +- arch/powerpc/platforms/pasemi/dma_lib.c | 4 +- arch/riscv/include/asm/bitops.h | 1 - arch/s390/Kconfig | 1 - arch/s390/include/asm/bitops.h | 1 - arch/s390/kvm/kvm-s390.c | 2 +- arch/sh/include/asm/bitops.h | 1 - arch/sparc/include/asm/bitops_32.h | 1 - arch/sparc/include/asm/bitops_64.h | 2 - arch/x86/Kconfig | 1 - arch/x86/include/asm/bitops.h | 2 - arch/x86/kernel/apic/vector.c | 4 +- arch/x86/um/Kconfig | 1 - arch/xtensa/include/asm/bitops.h | 1 - block/blk-mq.c | 2 +- drivers/block/rnbd/rnbd-clt.c | 2 +- drivers/dma/ti/edma.c | 2 +- drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 4 +- drivers/hwmon/ltc2992.c | 3 +- drivers/iio/adc/ad7124.c | 2 +- drivers/infiniband/hw/irdma/hw.c | 16 +- drivers/media/cec/core/cec-core.c | 2 +- drivers/media/mc/mc-devnode.c | 2 +- drivers/mmc/host/renesas_sdhi_core.c | 2 +- drivers/net/virtio_net.c | 2 +- drivers/pci/controller/dwc/pci-dra7xx.c | 2 +- drivers/scsi/lpfc/lpfc_sli.c | 10 +- drivers/soc/fsl/qbman/bman_portal.c | 2 +- drivers/soc/fsl/qbman/qman_portal.c | 2 +- drivers/soc/ti/k3-ringacc.c | 4 +- drivers/tty/n_tty.c | 2 +- drivers/virt/acrn/ioreq.c | 3 +- fs/f2fs/segment.c | 8 +- fs/ocfs2/cluster/heartbeat.c | 2 +- fs/ocfs2/dlm/dlmdomain.c | 4 +- fs/ocfs2/dlm/dlmmaster.c | 18 +- fs/ocfs2/dlm/dlmrecovery.c | 2 +- fs/ocfs2/dlm/dlmthread.c | 2 +- include/asm-generic/bitops.h | 1 - include/asm-generic/bitops/le.h | 64 --- include/linux/bitmap.h | 34 +- include/linux/bitops.h | 34 -- include/linux/cpumask.h | 46 ++- include/linux/find.h | 372 ++++++++++++++++++ kernel/time/clocksource.c | 4 +- lib/Kconfig | 3 - lib/find_bit.c | 21 + lib/find_bit_benchmark.c | 21 + lib/genalloc.c | 2 +- lib/test_bitmap.c | 37 ++ lib/vsprintf.c | 24 +- mm/percpu.c | 35 +- net/ncsi/ncsi-manage.c | 4 +- tools/include/asm-generic/bitops.h | 1 - tools/include/asm-generic/bitops/find.h | 145 ------- tools/include/linux/bitmap.h | 11 +- .../bitops => tools/include/linux}/find.h | 54 ++- tools/lib/find_bit.c | 20 + tools/perf/bench/find-bit-bench.c | 2 +- tools/perf/builtin-c2c.c | 6 +- tools/perf/builtin-record.c | 2 +- tools/perf/tests/bitmap.c | 2 +- tools/perf/tests/mem2node.c | 2 +- tools/perf/util/affinity.c | 4 +- tools/perf/util/header.c | 4 +- tools/perf/util/metricgroup.c | 2 +- tools/perf/util/mmap.c | 4 +- .../selftests/kvm/dirty_log_perf_test.c | 2 +- tools/testing/selftests/kvm/dirty_log_test.c | 4 +- .../selftests/kvm/x86_64/vmx_dirty_log_test.c | 2 +- 87 files changed, 657 insertions(+), 461 deletions(-) create mode 100644 include/linux/find.h delete mode 100644 tools/include/asm-generic/bitops/find.h rename {include/asm-generic/bitops => tools/include/linux}/find.h (83%) -- 2.30.2

4 years

3
25
0 0

[PATCH v2] lib: use of kunit in test_parman.c

by José Aquiles Guedes de Rezende

Convert the parman test module to use the KUnit test framework. This makes the test clearer by leveraging KUnit's assertion macros and test case definitions,as well as helps standardize on a testing framework. Co-developed-by: Matheus Henrique de Souza Silva <matheushenriquedesouzasilva(a)protonmail.com> Signed-off-by: Matheus Henrique de Souza Silva <matheushenriquedesouzasilva(a)protonmail.com> Signed-off-by: José Aquiles Guedes de Rezende <jjoseaquiless(a)gmail.com> --- Changes in v2: - Rename TEST_PARMAN config item to PARMAN_KUNIT_TEST and make it work with the kunit framework. - Change KUNIT_ASSERT_EQ to KUNIT_ASSERT_EQ_MSG. - Call test_parman_resize(test_parman, 0) when parman_create fail - Remove kunit_kfree. - Remove "\n" in error messages - Remove casts to unsigned long lib/Kconfig.debug | 13 +++-- lib/Makefile | 2 +- lib/test_parman.c | 145 +++++++++++++++++++--------------------------- 3 files changed, 70 insertions(+), 90 deletions(-) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 831212722924..e68a27e5e5b0 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2231,12 +2231,15 @@ config TEST_HASH config TEST_IDA tristate "Perform selftest on IDA functions" -config TEST_PARMAN - tristate "Perform selftest on priority array manager" - depends on PARMAN +config PARMAN_KUNIT_TEST + tristate "Kunit test for priority array manager" if !KUNIT_ALL_TESTS + select PARMAN + depends on KUNIT + default KUNIT_ALL_TESTS help - Enable this option to test priority array manager on boot - (or module load). + Enable this option to test priority array manager on boot. + For more information on KUnit and unit tests in general please refer + to the KUnit documentation in Documentation/dev-tools/kunit/. If unsure, say N. diff --git a/lib/Makefile b/lib/Makefile index 5efd1b435a37..deb8946735e8 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -88,7 +88,7 @@ obj-$(CONFIG_TEST_BITMAP) += test_bitmap.o obj-$(CONFIG_TEST_STRSCPY) += test_strscpy.o obj-$(CONFIG_TEST_UUID) += test_uuid.o obj-$(CONFIG_TEST_XARRAY) += test_xarray.o -obj-$(CONFIG_TEST_PARMAN) += test_parman.o +obj-$(CONFIG_PARMAN_KUNIT_TEST) += test_parman.o obj-$(CONFIG_TEST_KMOD) += test_kmod.o obj-$(CONFIG_TEST_DEBUG_VIRTUAL) += test_debug_virtual.o obj-$(CONFIG_TEST_MEMCAT_P) += test_memcat_p.o diff --git a/lib/test_parman.c b/lib/test_parman.c index 35e32243693c..512f874bc71c 100644 --- a/lib/test_parman.c +++ b/lib/test_parman.c @@ -41,6 +41,8 @@ #include <linux/err.h> #include <linux/random.h> #include <linux/parman.h> +#include <linux/sched.h> +#include <kunit/test.h> #define TEST_PARMAN_PRIO_SHIFT 7 /* defines number of prios for testing */ #define TEST_PARMAN_PRIO_COUNT BIT(TEST_PARMAN_PRIO_SHIFT) @@ -91,12 +93,14 @@ struct test_parman { static int test_parman_resize(void *priv, unsigned long new_count) { + struct kunit *test = current->kunit_test; struct test_parman *test_parman = priv; struct test_parman_item **prio_array; unsigned long old_count; prio_array = krealloc(test_parman->prio_array, ITEM_PTRS_SIZE(new_count), GFP_KERNEL); + KUNIT_EXPECT_NOT_ERR_OR_NULL(test, prio_array); if (new_count == 0) return 0; if (!prio_array) @@ -214,42 +218,41 @@ static void test_parman_items_fini(struct test_parman *test_parman) } } -static struct test_parman *test_parman_create(const struct parman_ops *ops) +static int test_parman_create(struct kunit *test) { struct test_parman *test_parman; int err; - test_parman = kzalloc(sizeof(*test_parman), GFP_KERNEL); - if (!test_parman) - return ERR_PTR(-ENOMEM); + test_parman = kunit_kzalloc(test, sizeof(*test_parman), GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, test_parman); + err = test_parman_resize(test_parman, TEST_PARMAN_BASE_COUNT); - if (err) - goto err_resize; - test_parman->parman = parman_create(ops, test_parman); - if (!test_parman->parman) { - err = -ENOMEM; - goto err_parman_create; + KUNIT_ASSERT_EQ_MSG(test, err, 0, "test_parman_resize failed"); + + test_parman->parman = parman_create(&test_parman_lsort_ops, test_parman); + if (IS_ERR_OR_NULL(test_parman->parman)) { + test_parman_resize(test_parman, 0); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, test_parman->parman); } + test_parman_rnd_init(test_parman); test_parman_prios_init(test_parman); test_parman_items_init(test_parman); test_parman->run_budget = TEST_PARMAN_RUN_BUDGET; - return test_parman; - -err_parman_create: - test_parman_resize(test_parman, 0); -err_resize: - kfree(test_parman); - return ERR_PTR(err); + test->priv = test_parman; + return 0; } -static void test_parman_destroy(struct test_parman *test_parman) +static void test_parman_destroy(struct kunit *test) { + struct test_parman *test_parman = test->priv; + + if (!test_parman) + return; test_parman_items_fini(test_parman); test_parman_prios_fini(test_parman); parman_destroy(test_parman->parman); test_parman_resize(test_parman, 0); - kfree(test_parman); } static bool test_parman_run_check_budgets(struct test_parman *test_parman) @@ -265,8 +268,9 @@ static bool test_parman_run_check_budgets(struct test_parman *test_parman) return true; } -static int test_parman_run(struct test_parman *test_parman) +static void test_parman_run(struct kunit *test) { + struct test_parman *test_parman = test->priv; unsigned int i = test_parman_rnd_get(test_parman); int err; @@ -281,8 +285,8 @@ static int test_parman_run(struct test_parman *test_parman) err = parman_item_add(test_parman->parman, &item->prio->parman_prio, &item->parman_item); - if (err) - return err; + KUNIT_ASSERT_EQ_MSG(test, err, 0, "parman_item_add failed"); + test_parman->prio_array[item->parman_item.index] = item; test_parman->used_items++; } else { @@ -294,22 +298,19 @@ static int test_parman_run(struct test_parman *test_parman) } item->used = !item->used; } - return 0; } -static int test_parman_check_array(struct test_parman *test_parman, - bool gaps_allowed) +static void test_parman_check_array(struct kunit *test, bool gaps_allowed) { unsigned int last_unused_items = 0; unsigned long last_priority = 0; unsigned int used_items = 0; int i; + struct test_parman *test_parman = test->priv; - if (test_parman->prio_array_limit < TEST_PARMAN_BASE_COUNT) { - pr_err("Array limit is lower than the base count (%lu < %lu)\n", - test_parman->prio_array_limit, TEST_PARMAN_BASE_COUNT); - return -EINVAL; - } + KUNIT_ASSERT_GE_MSG(test, test_parman->prio_array_limit, TEST_PARMAN_BASE_COUNT, + "Array limit is lower than the base count (%lu < %lu)", + test_parman->prio_array_limit, TEST_PARMAN_BASE_COUNT); for (i = 0; i < test_parman->prio_array_limit; i++) { struct test_parman_item *item = test_parman->prio_array[i]; @@ -318,77 +319,53 @@ static int test_parman_check_array(struct test_parman *test_parman, last_unused_items++; continue; } - if (last_unused_items && !gaps_allowed) { - pr_err("Gap found in array even though they are forbidden\n"); - return -EINVAL; - } + + KUNIT_ASSERT_FALSE_MSG(test, last_unused_items && !gaps_allowed, + "Gap found in array even though they are forbidden"); last_unused_items = 0; used_items++; - if (item->prio->priority < last_priority) { - pr_err("Item belongs under higher priority then the last one (current: %lu, previous: %lu)\n", - item->prio->priority, last_priority); - return -EINVAL; - } - last_priority = item->prio->priority; + KUNIT_ASSERT_GE_MSG(test, item->prio->priority, last_priority, + "Item belongs under higher priority then the last one (current: %lu, previous: %lu)", + item->prio->priority, last_priority); - if (item->parman_item.index != i) { - pr_err("Item has different index in compare to where it actually is (%lu != %d)\n", - item->parman_item.index, i); - return -EINVAL; - } - } + last_priority = item->prio->priority; - if (used_items != test_parman->used_items) { - pr_err("Number of used items in array does not match (%u != %u)\n", - used_items, test_parman->used_items); - return -EINVAL; - } + KUNIT_ASSERT_EQ_MSG(test, item->parman_item.index, i, + "Item has different index in compare to where it actually is (%lu != %d)", + item->parman_item.index, i); - if (last_unused_items >= TEST_PARMAN_RESIZE_STEP_COUNT) { - pr_err("Number of unused item at the end of array is bigger than resize step (%u >= %lu)\n", - last_unused_items, TEST_PARMAN_RESIZE_STEP_COUNT); - return -EINVAL; } - pr_info("Priority array check successful\n"); + KUNIT_ASSERT_EQ_MSG(test, used_items, test_parman->used_items, + "Number of used items in array does not match (%u != %u)", + used_items, test_parman->used_items); - return 0; + KUNIT_ASSERT_LT_MSG(test, last_unused_items, TEST_PARMAN_RESIZE_STEP_COUNT, + "Number of unused item at the end of array is bigger than resize step (%u >= %lu)", + last_unused_items, TEST_PARMAN_RESIZE_STEP_COUNT); } -static int test_parman_lsort(void) +static void test_parman_lsort(struct kunit *test) { - struct test_parman *test_parman; - int err; - - test_parman = test_parman_create(&test_parman_lsort_ops); - if (IS_ERR(test_parman)) - return PTR_ERR(test_parman); - - err = test_parman_run(test_parman); - if (err) - goto out; - - err = test_parman_check_array(test_parman, false); - if (err) - goto out; -out: - test_parman_destroy(test_parman); - return err; + test_parman_run(test); + test_parman_check_array(test, false); } -static int __init test_parman_init(void) -{ - return test_parman_lsort(); -} +static struct kunit_case parman_test_case[] = { + KUNIT_CASE(test_parman_lsort), + {} +}; -static void __exit test_parman_exit(void) -{ -} +static struct kunit_suite parman_test_suite = { + .name = "parman", + .init = test_parman_create, + .exit = test_parman_destroy, + .test_cases = parman_test_case, +}; -module_init(test_parman_init); -module_exit(test_parman_exit); +kunit_test_suite(parman_test_suite); MODULE_LICENSE("Dual BSD/GPL"); MODULE_AUTHOR("Jiri Pirko <jiri(a)mellanox.com>"); -- 2.32.0

4 years

3
2
0 0

[PATCH 0/3] Add RFC-7597 Section 5.1 PSID support

by Cole Dishington

Thanks for your time reviewing! Changes in v4: - Handle special case of no offset bits (a=0 / A=2^16). Cole Dishington (3): net: netfilter: Add RFC-7597 Section 5.1 PSID support xtables API net: netfilter: Add RFC-7597 Section 5.1 PSID support selftests: netfilter: Add RFC-7597 Section 5.1 PSID selftests include/uapi/linux/netfilter/nf_nat.h | 3 +- net/netfilter/nf_nat_core.c | 39 +++- net/netfilter/nf_nat_masquerade.c | 20 +- net/netfilter/xt_MASQUERADE.c | 44 ++++- .../netfilter/nat_masquerade_psid.sh | 182 ++++++++++++++++++ 5 files changed, 276 insertions(+), 12 deletions(-) create mode 100644 tools/testing/selftests/netfilter/nat_masquerade_psid.sh -- 2.32.0

4 years

3
14
0 0

[net-next, v4, 00/11] ptp: support virtual clocks and timestamping

by Yangbo Lu

Current PTP driver exposes one PTP device to user which binds network interface/interfaces to provide timestamping. Actually we have a way utilizing timecounter/cyclecounter to virtualize any number of PTP clocks based on a same free running physical clock for using. The purpose of having multiple PTP virtual clocks is for user space to directly/easily use them for multiple domains synchronization. user space: ^ ^ | SO_TIMESTAMPING new flag: | Packets with | SOF_TIMESTAMPING_BIND_PHC | TX/RX HW timestamps v v +--------------------------------------------+ sock: | sock (new member sk_bind_phc) | +--------------------------------------------+ ^ ^ | ethtool_get_phc_vclocks | Convert HW timestamps | | to sk_bind_phc v v +--------------+--------------+--------------+ vclock: | ptp1 | ptp2 | ptpN | +--------------+--------------+--------------+ pclock: | ptp0 free running | +--------------------------------------------+ The block diagram may explain how it works. Besides the PTP virtual clocks, the packet HW timestamp converting to the bound PHC is also done in sock driver. For user space, PTP virtual clocks can be created via sysfs, and extended SO_TIMESTAMPING API (new flag SOF_TIMESTAMPING_BIND_PHC) can be used to bind one PTP virtual clock for timestamping. The test tool timestamping.c (together with linuxptp phc_ctl tool) can be used to verify: # echo 4 > /sys/class/ptp/ptp0/n_vclocks [ 129.399472] ptp ptp0: new virtual clock ptp2 [ 129.404234] ptp ptp0: new virtual clock ptp3 [ 129.409532] ptp ptp0: new virtual clock ptp4 [ 129.413942] ptp ptp0: new virtual clock ptp5 [ 129.418257] ptp ptp0: guarantee physical clock free running # # phc_ctl /dev/ptp2 set 10000 # phc_ctl /dev/ptp3 set 20000 # # timestamping eno0 2 SOF_TIMESTAMPING_TX_HARDWARE SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_BIND_PHC # timestamping eno0 2 SOF_TIMESTAMPING_RX_HARDWARE SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_BIND_PHC # timestamping eno0 3 SOF_TIMESTAMPING_TX_HARDWARE SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_BIND_PHC # timestamping eno0 3 SOF_TIMESTAMPING_RX_HARDWARE SOF_TIMESTAMPING_RAW_HARDWARE SOF_TIMESTAMPING_BIND_PHC Changes for v2: - Converted to num_vclocks for creating virtual clocks. - Guranteed physical clock free running when using virtual clocks. - Fixed build warning. - Updated copyright. Changes for v3: - Supported PTP virtual clock in default in PTP driver. - Protected concurrency of ptp->num_vclocks accessing. - Supported PHC vclocks query via ethtool. - Extended SO_TIMESTAMPING API for PHC binding. - Converted HW timestamps to PHC bound, instead of previous binding domain value to PHC idea. - Other minor fixes. Changes for v4: - Used do_aux_work callback for vclock refreshing instead. - Used unsigned int for vclocks number, and max_vclocks for limitiation. - Fixed mutex locking. - Dynamically allocated memory for vclock index storage. - Removed ethtool ioctl command for vclocks getting. - Updated doc for ethtool phc vclocks get. - Converted to mptcp_setsockopt_sol_socket_timestamping(). - Passed so_timestamping for sock_set_timestamping. - Fixed checkpatch/build. - Other minor fixed. Yangbo Lu (11): ptp: add ptp virtual clock driver framework ptp: support ptp physical/virtual clocks conversion ptp: track available ptp vclocks information ptp: add kernel API ptp_get_vclocks_index() ethtool: add a new command for getting PHC virtual clocks ptp: add kernel API ptp_convert_timestamp() mptcp: setsockopt: convert to mptcp_setsockopt_sol_socket_timestamping() net: sock: extend SO_TIMESTAMPING for PHC binding net: socket: support hardware timestamp conversion to PHC bound selftests/net: timestamping: support binding PHC MAINTAINERS: add entry for PTP virtual clock driver Documentation/ABI/testing/sysfs-ptp | 20 ++ Documentation/networking/ethtool-netlink.rst | 22 ++ MAINTAINERS | 7 + drivers/ptp/Makefile | 2 +- drivers/ptp/ptp_clock.c | 41 +++- drivers/ptp/ptp_private.h | 39 ++++ drivers/ptp/ptp_sysfs.c | 160 ++++++++++++++ drivers/ptp/ptp_vclock.c | 219 +++++++++++++++++++ include/linux/ethtool.h | 10 + include/linux/ptp_clock_kernel.h | 31 ++- include/net/sock.h | 8 +- include/uapi/linux/ethtool_netlink.h | 15 ++ include/uapi/linux/net_tstamp.h | 17 +- net/core/sock.c | 65 +++++- net/ethtool/Makefile | 2 +- net/ethtool/common.c | 14 ++ net/ethtool/netlink.c | 10 + net/ethtool/netlink.h | 2 + net/ethtool/phc_vclocks.c | 94 ++++++++ net/mptcp/sockopt.c | 69 ++++-- net/socket.c | 19 +- tools/testing/selftests/net/timestamping.c | 62 ++++-- 22 files changed, 875 insertions(+), 53 deletions(-) create mode 100644 drivers/ptp/ptp_vclock.c create mode 100644 net/ethtool/phc_vclocks.c base-commit: 19938bafa7ae8fc0a4a2c1c1430abb1a04668da1 -- 2.25.1

4 years

5
16
0 0

[RFC net-next 0/2] Add RTNL interface for SyncE

by Maciej Machnikowski

Synchronous Ethernet networks use a physical layer clock to syntonize the frequency across different network elements. Multiple reference clock sources can be used. Clocks recovered from PHY ports on the RX side or external sources like 1PPS GPS, etc. This patch series introduces basic interface for reading the DPLL state on a SyncE capable device. This state gives us information about the source of the syntonization signal and whether the DPLL circuit is tuned to the incoming signal. Next steps: - add interface to enable recovered clocks and get information about them Maciej Machnikowski (2): rtnetlink: Add new RTM_GETSYNCESTATE message to get SyncE status ice: add support for reading SyncE DPLL state drivers/net/ethernet/intel/ice/ice.h | 5 ++ .../net/ethernet/intel/ice/ice_adminq_cmd.h | 34 ++++++++ drivers/net/ethernet/intel/ice/ice_common.c | 62 +++++++++++++++ drivers/net/ethernet/intel/ice/ice_common.h | 4 + drivers/net/ethernet/intel/ice/ice_devids.h | 3 + drivers/net/ethernet/intel/ice/ice_main.c | 55 +++++++++++++ drivers/net/ethernet/intel/ice/ice_ptp.c | 35 +++++++++ drivers/net/ethernet/intel/ice/ice_ptp_hw.c | 44 +++++++++++ drivers/net/ethernet/intel/ice/ice_ptp_hw.h | 22 ++++++ include/linux/netdevice.h | 6 ++ include/uapi/linux/if_link.h | 43 +++++++++++ include/uapi/linux/rtnetlink.h | 11 ++- net/core/rtnetlink.c | 77 +++++++++++++++++++ security/selinux/nlmsgtab.c | 3 +- 14 files changed, 399 insertions(+), 5 deletions(-) -- 2.26.3

4 years

3
4
0 0

[PATCH linux-next] tests: remove unneeded conversions to bool

by CGEL

From: Changcheng Deng <deng.changcheng(a)zte.com.cn> tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c: 225:18-23:WARNING: conversion to bool not needed here Because the definition of function is "static int move_mount_set_group_supported(void)", the return type should be int. Reported-by: Zeal Robot <zealci(a)zte.com.cn> Signed-off-by: Changcheng Deng <deng.changcheng(a)zte.com.cn> --- .../testing/selftests/move_mount_set_group/move_mount_set_group_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c b/tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c index 860198f..beade21 100644 --- a/tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c +++ b/tools/testing/selftests/move_mount_set_group/move_mount_set_group_test.c @@ -222,7 +222,7 @@ static int move_mount_set_group_supported(void) AT_FDCWD, SET_GROUP_TO, MOVE_MOUNT_SET_GROUP); umount2("/tmp", MNT_DETACH); - return ret < 0 ? false : true; + return ret < 0 ? 0 : 1; } FIXTURE(move_mount_set_group) { -- 1.8.3.1

4 years

1
0
0 0

[PATCH v2 0/5] KVM: rseq: Fix and a test for a KVM+rseq bug

by Sean Christopherson

Patch 1 fixes a KVM+rseq bug where KVM's handling of TIF_NOTIFY_RESUME, e.g. for task migration, clears the flag without informing rseq and leads to stale data in userspace's rseq struct. Patch 2 is a cleanup to try and make future bugs less likely. It's also a baby step towards moving and renaming tracehook_notify_resume() since it has nothing to do with tracing. Patch 3 is a fix/cleanup to stop overriding x86's unistd_{32,64}.h when the include path (intentionally) omits tools' uapi headers. KVM's selftests do exactly that so that they can pick up the uapi headers from the installed kernel headers, and still use various tools/ headers that mirror kernel code, e.g. linux/types.h. This allows the new test in patch 4 to reference __NR_rseq without having to manually define it. Patch 4 is a regression test for the KVM+rseq bug. Patch 5 is a cleanup made possible by patch 3. v2: - Don't touch rseq_cs when handling KVM case so that rseq_syscall() will still detect a naughty userspace. [Mathieu] - Use a sequence counter + retry in the test to ensure the process isn't migrated between sched_getcpu() and reading rseq.cpu_id, i.e. to avoid a flaky test. [Mathieu] - Add Mathieu's ack for patch 2. - Add more comments in the test. v1: https://lkml.kernel.org/r/20210818001210.4073390-1-seanjc@google.com Sean Christopherson (5): KVM: rseq: Update rseq when processing NOTIFY_RESUME on xfer to KVM guest entry: rseq: Call rseq_handle_notify_resume() in tracehook_notify_resume() tools: Move x86 syscall number fallbacks to .../uapi/ KVM: selftests: Add a test for KVM_RUN+rseq to detect task migration bugs KVM: selftests: Remove __NR_userfaultfd syscall fallback arch/arm/kernel/signal.c | 1 - arch/arm64/kernel/signal.c | 1 - arch/csky/kernel/signal.c | 4 +- arch/mips/kernel/signal.c | 4 +- arch/powerpc/kernel/signal.c | 4 +- arch/s390/kernel/signal.c | 1 - include/linux/tracehook.h | 2 + kernel/entry/common.c | 4 +- kernel/rseq.c | 14 +- .../x86/include/{ => uapi}/asm/unistd_32.h | 0 .../x86/include/{ => uapi}/asm/unistd_64.h | 3 - tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 3 + tools/testing/selftests/kvm/rseq_test.c | 154 ++++++++++++++++++ 14 files changed, 175 insertions(+), 21 deletions(-) rename tools/arch/x86/include/{ => uapi}/asm/unistd_32.h (100%) rename tools/arch/x86/include/{ => uapi}/asm/unistd_64.h (83%) create mode 100644 tools/testing/selftests/kvm/rseq_test.c -- 2.33.0.rc2.250.ged5fa647cd-goog

4 years

3
15
0 0

[PATCH v2 0/3] kselftests: clean configs

by Li Zhijian

0Day will check if all configs listing under selftests are able to be enabled properly. For the missing configs, it will report something like: LKP WARN miss config CONFIG_SYNC= of sync/config CC: kernel test robot <lkp(a)intel.com> CC: "Jason A. Donenfeld" <Jason(a)zx2c4.com> CC: Nick Desaulniers <ndesaulniers(a)google.com> CC: Masahiro Yamada <masahiroy(a)kernel.org> CC: wireguard(a)lists.zx2c4.com CC: netdev(a)vger.kernel.org CC: "Rafael J. Wysocki" <rjw(a)rjwysocki.net> CC: Viresh Kumar <viresh.kumar(a)linaro.org> CC: linux-pm(a)vger.kernel.org Reported-by: kernel test robot <lkp(a)intel.com> Li Zhijian (3): selftests/sync: Remove the deprecated config SYNC selftests/cpufreq: Rename DEBUG_PI_LIST to DEBUG_PLIST selftests/wireguard: Rename DEBUG_PI_LIST to DEBUG_PLIST tools/testing/selftests/cpufreq/config | 2 +- tools/testing/selftests/sync/config | 1 - tools/testing/selftests/wireguard/qemu/debug.config | 2 +- 3 files changed, 2 insertions(+), 3 deletions(-) -- 2.31.1

4 years

4
9
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror