- Linux-kselftest-mirror - lists.linaro.org

[PATCH] Documentation: kunit: Fix verification command

by SeongJae Park

kunit wrapper script ('kunit.py') receives a sub-command (only 'run' for now) as its argument. If no sub-command is given, it prints help message and just quit. However, an example command in the kunit documentation for a verification of kunit is missing the sub-command. This commit fixes the example. Signed-off-by: SeongJae Park <sj38.park(a)gmail.com> --- Documentation/dev-tools/kunit/start.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 6dc229e..aeeddfa 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -43,7 +43,7 @@ wrapper from your kernel repo: .. code-block:: bash - ./tools/testing/kunit/kunit.py + ./tools/testing/kunit/kunit.py run .. note:: You may want to run ``make mrproper`` first. -- 2.7.4

5 years, 9 months

3
3
0 0

[PATCH] kunit: Fix '--build_dir' option

by SeongJae Park

kunit fails to run with '--build_dir' option because the option is not properly sent to kernel running procedure. This commit fixes the problem. Signed-off-by: SeongJae Park <sj38.park(a)gmail.com> --- tools/testing/kunit/kunit.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 8d33980..e016430 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -62,9 +62,11 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, 'Tests not Parsed.') if request.raw_output: kunit_parser.raw_output( - linux.run_kernel(timeout=request.timeout)) + linux.run_kernel(timeout=request.timeout, + build_dir=request.build_dir)) else: - kunit_output = linux.run_kernel(timeout=request.timeout) + kunit_output = linux.run_kernel(timeout=request.timeout, + build_dir=request.build_dir) test_result = kunit_parser.parse_run_tests(kunit_output) test_end = time.time() -- 2.7.4

5 years, 9 months

3
8
0 0

[PATCH v18 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework

by Brendan Higgins

## TL;DR This revision addresses comments from Linus[1] and Randy[2], by moving top level `kunit/` directory to `lib/kunit/` and likewise moves top level Kconfig entry under lib/Kconfig.debug, so the KUnit submenu now shows up under the "Kernel Hacking" menu. As a consequence of this, I rewrote patch 06/18 (kbuild: enable building KUnit) - now 06/19 (lib: enable building KUnit in lib/), and now needs to be re-acked/reviewed. ## Background This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel. Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM (however, KUnit still allows you to run tests on test machines or in VMs if you want[3]) and does not require tests to be written in userspace running on a host kernel. Additionally, KUnit is fast: From invocation to completion KUnit can run several dozen tests in about a second. Currently, the entire KUnit test suite for KUnit runs in under a second from the initial invocation (build time excluded). KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, mocking, spying, and much more. ### What's so special about unit testing? A unit test is supposed to test a single unit of code in isolation, hence the name. There should be no dependencies outside the control of the test; this means no external dependencies, which makes tests orders of magnitudes faster. Likewise, since there are no external dependencies, there are no hoops to jump through to run the tests. Additionally, this makes unit tests deterministic: a failing unit test always indicates a problem. Finally, because unit tests necessarily have finer granularity, they are able to test all code paths easily solving the classic problem of difficulty in exercising error handling code. ### Is KUnit trying to replace other testing frameworks for the kernel? No. Most existing tests for the Linux kernel are end-to-end tests, which have their place. A well tested system has lots of unit tests, a reasonable number of integration tests, and some end-to-end tests. KUnit is just trying to address the unit test space which is currently not being addressed. ### More information on KUnit There is a bunch of documentation near the end of this patch set that describes how to use KUnit and best practices for writing unit tests. For convenience I am hosting the compiled docs here[4]. Additionally for convenience, I have applied these patches to a branch[5]. The repo may be cloned with: git clone https://kunit.googlesource.com/linux This patchset is on the kunit/initial/v5.3/v18 branch. ## History since v15 ### v18 - Addrssed comments on 07/19 (kunit: test: add initial tests) from Randy Dunlap by removing redundant dependencies from Kconfig entries. ### v17 - Addressed comments on 06/19 (lib: enable building KUnit in lib/) from Stephen Boyd by moving KUnit submenu ahead of Runtime Testing submenu. ### v16 - Addressed comments from Linus Torvalds by moving all kunit/ paths to lib/kunit/. - Addressed comments by Randy Dunlap by moving KUnit Kconfig under lib/Kconfig.debug so the KUnit submenu shows up under the "Kernel Hacking" menu. [1] https://www.lkml.org/lkml/2019/9/20/696 [2] https://www.lkml.org/lkml/2019/9/20/738 [3] https://google.github.io/kunit-docs/third_party/kernel/docs/usage.html#kuni… [4] https://google.github.io/kunit-docs/third_party/kernel/docs/ [5] https://kunit.googlesource.com/linux/+/kunit/initial/v5.3/v18 --- Avinash Kondareddy (1): kunit: test: add tests for KUnit managed resources Brendan Higgins (16): kunit: test: add KUnit test runner core kunit: test: add test resource management API kunit: test: add string_stream a std::stream like string builder kunit: test: add assertion printing library kunit: test: add the concept of expectations lib: enable building KUnit in lib/ kunit: test: add initial tests objtool: add kunit_try_catch_throw to the noreturn list kunit: test: add support for test abort kunit: test: add tests for kunit test abort kunit: test: add the concept of assertions kunit: defconfig: add defconfigs for building KUnit tests Documentation: kunit: add documentation for KUnit MAINTAINERS: add entry for KUnit the unit testing framework MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section kunit: fix failure to build without printk Felix Guo (1): kunit: tool: add Python wrappers for running KUnit tests Iurii Zaikin (1): kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec() Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/kunit/api/index.rst | 16 + Documentation/dev-tools/kunit/api/test.rst | 11 + Documentation/dev-tools/kunit/faq.rst | 62 + Documentation/dev-tools/kunit/index.rst | 79 + Documentation/dev-tools/kunit/start.rst | 180 ++ Documentation/dev-tools/kunit/usage.rst | 576 +++++++ MAINTAINERS | 13 + arch/um/configs/kunit_defconfig | 3 + include/kunit/assert.h | 356 ++++ include/kunit/string-stream.h | 51 + include/kunit/test.h | 1490 +++++++++++++++++ include/kunit/try-catch.h | 75 + kernel/Makefile | 2 + kernel/sysctl-test.c | 392 +++++ lib/Kconfig.debug | 13 + lib/Makefile | 2 + lib/kunit/Kconfig | 36 + lib/kunit/Makefile | 9 + lib/kunit/assert.c | 141 ++ lib/kunit/example-test.c | 88 + lib/kunit/string-stream-test.c | 52 + lib/kunit/string-stream.c | 217 +++ lib/kunit/test-test.c | 331 ++++ lib/kunit/test.c | 478 ++++++ lib/kunit/try-catch.c | 118 ++ tools/objtool/check.c | 1 + tools/testing/kunit/.gitignore | 3 + tools/testing/kunit/configs/all_tests.config | 3 + tools/testing/kunit/kunit.py | 136 ++ tools/testing/kunit/kunit_config.py | 66 + tools/testing/kunit/kunit_kernel.py | 149 ++ tools/testing/kunit/kunit_parser.py | 310 ++++ tools/testing/kunit/kunit_tool_test.py | 206 +++ .../test_is_test_passed-all_passed.log | 32 + .../test_data/test_is_test_passed-crash.log | 69 + .../test_data/test_is_test_passed-failure.log | 36 + .../test_is_test_passed-no_tests_run.log | 75 + .../test_output_isolated_correctly.log | 106 ++ .../test_data/test_read_from_file.kconfig | 17 + 40 files changed, 6001 insertions(+) create mode 100644 Documentation/dev-tools/kunit/api/index.rst create mode 100644 Documentation/dev-tools/kunit/api/test.rst create mode 100644 Documentation/dev-tools/kunit/faq.rst create mode 100644 Documentation/dev-tools/kunit/index.rst create mode 100644 Documentation/dev-tools/kunit/start.rst create mode 100644 Documentation/dev-tools/kunit/usage.rst create mode 100644 arch/um/configs/kunit_defconfig create mode 100644 include/kunit/assert.h create mode 100644 include/kunit/string-stream.h create mode 100644 include/kunit/test.h create mode 100644 include/kunit/try-catch.h create mode 100644 kernel/sysctl-test.c create mode 100644 lib/kunit/Kconfig create mode 100644 lib/kunit/Makefile create mode 100644 lib/kunit/assert.c create mode 100644 lib/kunit/example-test.c create mode 100644 lib/kunit/string-stream-test.c create mode 100644 lib/kunit/string-stream.c create mode 100644 lib/kunit/test-test.c create mode 100644 lib/kunit/test.c create mode 100644 lib/kunit/try-catch.c create mode 100644 tools/testing/kunit/.gitignore create mode 100644 tools/testing/kunit/configs/all_tests.config create mode 100755 tools/testing/kunit/kunit.py create mode 100644 tools/testing/kunit/kunit_config.py create mode 100644 tools/testing/kunit/kunit_kernel.py create mode 100644 tools/testing/kunit/kunit_parser.py create mode 100755 tools/testing/kunit/kunit_tool_test.py create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-crash.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-failure.log create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log create mode 100644 tools/testing/kunit/test_data/test_output_isolated_correctly.log create mode 100644 tools/testing/kunit/test_data/test_read_from_file.kconfig -- 2.23.0.351.gc4317032e6-goog

5 years, 9 months

7
46
0 0

[RFC PATCH] selftests: fix build targets behaviour on failures

by Cristian Marussi

Currently, when some of the KSFT subsystems specified in TARGETS fails to build, the toplevel Makefile just keeps carrying on with the build process. This behaviour is expected and desirable especially in the context of a CI system running KSelfTest where is not always easy to guarantee that the most recent and esoteric toolchain's dependencies are respected across all KSFT TARGETS in a timely manner. Unfortunately, as of now, this holds true only if the very last of the subsystems listed in TARGETS happened to be one that could be successfully built: if the last listed TARGETS fails, it is taken as the whole outcome of the Makefile target and the complete build/install halts even though many other preceding subsytems were in fact already successfully built. Fix the KSFT Makefile behaviour related to all/install targets in order to fail as a whole only when the all/install targets have failed for all of the requested TARGETS, while succeeding when at least one of TARGETS has been successfully built. Signed-off-by: Cristian Marussi <cristian.marussi(a)arm.com> --- This patch is based on ksft/fixes branch from: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git on top of commit: ce3a67780212 selftests: watchdog: Add command line option to show watchdog_info --- tools/testing/selftests/Makefile | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 4cdbae6f4e61..d907b050e929 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -140,11 +140,13 @@ else endif all: khdr - @for TARGET in $(TARGETS); do \ - BUILD_TARGET=$$BUILD/$$TARGET; \ - mkdir $$BUILD_TARGET -p; \ - $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET;\ - done; + @ret=1; \ + for TARGET in $(TARGETS); do \ + BUILD_TARGET=$$BUILD/$$TARGET; \ + mkdir $$BUILD_TARGET -p; \ + $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET; \ + ret=$$((ret * $$?)); \ + done; exit $$ret; run_tests: all @for TARGET in $(TARGETS); do \ @@ -192,10 +194,12 @@ ifdef INSTALL_PATH mkdir -p $(INSTALL_PATH)/kselftest install -m 744 kselftest/runner.sh $(INSTALL_PATH)/kselftest/ install -m 744 kselftest/prefix.pl $(INSTALL_PATH)/kselftest/ - @for TARGET in $(TARGETS); do \ + @ret=1; \ + for TARGET in $(TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET INSTALL_PATH=$(INSTALL_PATH)/$$TARGET install; \ - done; + ret=$$((ret * $$?)); \ + done; exit $$ret @# Ask all targets to emit their test scripts echo "#!/bin/sh" > $(ALL_SCRIPT) -- 2.17.1

5 years, 9 months

1
0
0 0

Re: [PATCH v3 0/3] selftests: netfilter: introduce test cases for ipvs

by Duncan Roe

On Thu, Oct 03, 2019 at 10:41:06PM +0800, Haishuang Yan wrote: > > > > On 2019??10??2??, at ????9:27, Duncan Roe <duncan_roe(a)optusnet.com.au> wrote: > > > > On Tue, Oct 01, 2019 at 09:34:13PM +0300, Julian Anastasov wrote: > >> > >> Hello, > >> > >> On Tue, 1 Oct 2019, Haishuang Yan wrote: > >> > >>> This series patch include test cases for ipvs. > >>> > >>> The test topology is who as below: > >>> +--------------------------------------------------------------+ > >>> | | | > >>> | ns0 | ns1 | > >>> | ----------- | ----------- ----------- | > >>> | | veth01 | --------- | veth10 | | veth12 | | > >>> | ----------- peer ----------- ----------- | > >>> | | | | | > >>> | ----------- | | | > >>> | | br0 | |----------------- peer |--------------| > >>> | ----------- | | | > >>> | | | | | > >>> | ---------- peer ---------- ----------- | > >>> | | veth02 | --------- | veth20 | | veth12 | | > >>> | ---------- | ---------- ----------- | > >>> | | ns2 | > >>> | | | > >>> +--------------------------------------------------------------+ > >>> > >>> Test results: > >>> # selftests: netfilter: ipvs.sh > >>> # Testing DR mode... > >>> # Testing NAT mode... > >>> # Testing Tunnel mode... > >>> # ipvs.sh: PASS > >>> ok 6 selftests: netfilter: ipvs.sh > >>> > >>> Haishuang Yan (3): > >>> selftests: netfilter: add ipvs test script > >>> selftests: netfilter: add ipvs nat test case > >>> selftests: netfilter: add ipvs tunnel test case > >> > >> Acked-by: Julian Anastasov <ja(a)ssi.bg> > >> > >>> tools/testing/selftests/netfilter/Makefile | 2 +- > >>> tools/testing/selftests/netfilter/ipvs.sh | 234 +++++++++++++++++++++++++++++ > >>> 2 files changed, 235 insertions(+), 1 deletion(-) > >>> create mode 100755 tools/testing/selftests/netfilter/ipvs.sh > >> > >> Regards > >> > >> -- > >> Julian Anastasov <ja(a)ssi.bg> > > > > I still prefer #!/bin/sh in 1/3. You never know what's in someone's environment > > > > Cheers ... Duncan. > > > > It??s also my preference too. "_" > > I have tested both #!/bin/bash and #!/bin/sh script, they all works properly. Enter these 2 lines: > ip(){ return 0; } > export -f ip Now try the #!/bin/bash script. If that now fails, try again with #!/bin/bash changed to #!/bin/bash -p Any better now? Cheers ... Duncan.

5 years, 9 months

2
1
0 0

Linux 5.4 kselftest known issues

by Shuah Khan

Here are the know kselftest issues on Linux 5.4 with top commit commit 619e17cf75dd58905aa67ccd494a6ba5f19d6cc6 on x86_64: The goal is to get these addressed before 5.4 comes out. 3 build failures and status: pidfd - undefined reference to `pthread_create' collect2: error: ld returned 1 exit status Fixed: https://patchwork.kernel.org/patch/11159517/ bfp (two issues) 1. "make TARGETS=bpf kselftest" build fails Makefile:127: tools/build/Makefile.include: No such file or directory This is due to recent kbuild changes and I have a patch ready to send. 2. Related to llvm latest version dependency. This is a hard dependency. Unless users upgrade to latest llvvm, bpf test won't run. The new llvm might not be supported on all distros yet, in which case bpf will not get tested in some rings and on some architectures. gpio "make TARGETS=gpio kselftest" build fails Makefile:23: tools/build/Makefile.include: No such file or directory This is due to recent kbuild changes and I have a patch ready to send. kvm "make TARGETS=kvm kselftest" build fails due --no-pie flags. I am working on a fix for this. no-pie-option defines aren't working correctly and I suspect try-run miht not be defined in this kselftest build case. thanks, -- Shuah

5 years, 9 months

3
2
0 0

[PATCH] kvm: selftests: fix spelling mistake: missmatch -> mismatch

by Colin King

From: Colin Ian King <colin.king(a)canonical.com> There is a spelling mistake in in an error message, fix it. Signed-off-by: Colin Ian King <colin.king(a)canonical.com> --- tools/testing/selftests/kvm/lib/sparsebit.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/lib/sparsebit.c b/tools/testing/selftests/kvm/lib/sparsebit.c index 031ba3c932ed..59ffba902e61 100644 --- a/tools/testing/selftests/kvm/lib/sparsebit.c +++ b/tools/testing/selftests/kvm/lib/sparsebit.c @@ -1866,7 +1866,7 @@ void sparsebit_validate_internal(struct sparsebit *s) * of total bits set. */ if (s->num_set != total_bits_set) { - fprintf(stderr, "Number of bits set missmatch,\n" + fprintf(stderr, "Number of bits set mismatch,\n" " s->num_set: 0x%lx total_bits_set: 0x%lx", s->num_set, total_bits_set); -- 2.20.1

5 years, 9 months

1
0
0 0

next-20191004 kselftest results

by ci_notify＠linaro.org

Summary ------------------------------------------------------------------------ kernel: 5.4.0-rc1 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git git branch: master git commit: 311ef88adfa3b69c40234bf3000d1269e718919a git describe: next-20191004 Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20191004 Regressions (compared to build next-20191003) ------------------------------------------------------------------------ No regressions Fixes (compared to build next-20191003) ------------------------------------------------------------------------ No fixes In total: ------------------------------------------------------------------------ Ran 0 total tests in the following environments and test suites. pass 0 fail 0 xfail 0 skip 0 Environments -------------- - dragonboard-410c - arm64 - hi6220-hikey - arm64 - i386 - juno-r2 - arm64 - x15 - arm - x86_64 Test Suites ----------- Failures ------------------------------------------------------------------------ hi6220-hikey: juno-r2: i386: x86: x15: dragonboard-410c: Skips ------------------------------------------------------------------------ No skips -- Linaro LKFT https://lkft.linaro.org

5 years, 9 months

1
0
0 0

[PATCH bpf-next 0/2] selftest/bpf: remove warns for enable_all_controllers

by Ivan Khoronzhuk

This micro series fixes annoying warn described in patches while samples/bpf build. Second patch fixes new warn that comes after fixing warn of first patch, that was masked. Ivan Khoronzhuk (2): selftests/bpf: add static to enable_all_controllers() selftests/bpf: correct path to include msg + path tools/testing/selftests/bpf/cgroup_helpers.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.17.1

5 years, 9 months

3
5
0 0

[PATCH v3] selftests: kvm: Fix libkvm build error

by Shuah Khan

Fix the following build error from "make TARGETS=kvm kselftest": libkvm.a(assert.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIC This error is seen when build is done from the main Makefile using kselftest target. In this case KBUILD_CPPFLAGS and CC_OPTION_CFLAGS are defined. When build is invoked using: "make -C tools/testing/selftests/kvm" KBUILD_CPPFLAGS and CC_OPTION_CFLAGS aren't defined. There is no need to pass in KBUILD_CPPFLAGS and CC_OPTION_CFLAGS for the check to determine if --no-pie is necessary, which is the case when these two aren't defined when "make -C tools/testing/selftests/kvm" runs. Fix it by simplifying the no-pie-option logic. With this change, both build variations work. "make TARGETS=kvm kselftest" "make -C tools/testing/selftests/kvm" Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> --- Changes since v2: -- Removed extra blank line added by accident. -- Fixed commit log. tools/testing/selftests/kvm/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 62c591f87dab..7ee097658ef0 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -48,7 +48,7 @@ CFLAGS += -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 \ -I$(LINUX_HDR_PATH) -Iinclude -I$(<D) -Iinclude/$(UNAME_M) -I.. no-pie-option := $(call try-run, echo 'int main() { return 0; }' | \ - $(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -no-pie -x c - -o "$$TMP", -no-pie) + $(CC) -Werror -no-pie -x c - -o "$$TMP", -no-pie) # On s390, build the testcases KVM-enabled pgste-option = $(call try-run, echo 'int main() { return 0; }' | \ -- 2.20.1

5 years, 9 months

2
1
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror