kunit wrapper script ('kunit.py') receives a sub-command (only 'run' for
now) as its argument. If no sub-command is given, it prints help
message and just quit. However, an example command in the kunit
documentation for a verification of kunit is missing the sub-command.
This commit fixes the example.
Signed-off-by: SeongJae Park <sj38.park(a)gmail.com>
---
Documentation/dev-tools/kunit/start.rst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index 6dc229e..aeeddfa 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -43,7 +43,7 @@ wrapper from your kernel repo:
.. code-block:: bash
- ./tools/testing/kunit/kunit.py
+ ./tools/testing/kunit/kunit.py run
.. note::
You may want to run ``make mrproper`` first.
--
2.7.4
## TL;DR
This revision addresses comments from Linus[1] and Randy[2], by moving
top level `kunit/` directory to `lib/kunit/` and likewise moves top
level Kconfig entry under lib/Kconfig.debug, so the KUnit submenu now
shows up under the "Kernel Hacking" menu.
As a consequence of this, I rewrote patch 06/18 (kbuild: enable building
KUnit) - now 06/19 (lib: enable building KUnit in lib/), and now needs
to be re-acked/reviewed.
## Background
This patch set proposes KUnit, a lightweight unit testing and mocking
framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework;
it does not require installing the kernel on a test machine or in a VM
(however, KUnit still allows you to run tests on test machines or in VMs
if you want[3]) and does not require tests to be written in userspace
running on a host kernel. Additionally, KUnit is fast: From invocation
to completion KUnit can run several dozen tests in about a second.
Currently, the entire KUnit test suite for KUnit runs in under a second
from the initial invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and
Googletest/Googlemock for C++. KUnit provides facilities for defining
unit test cases, grouping related test cases into test suites, providing
common infrastructure for running tests, mocking, spying, and much more.
### What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation,
hence the name. There should be no dependencies outside the control of
the test; this means no external dependencies, which makes tests orders
of magnitudes faster. Likewise, since there are no external dependencies,
there are no hoops to jump through to run the tests. Additionally, this
makes unit tests deterministic: a failing unit test always indicates a
problem. Finally, because unit tests necessarily have finer granularity,
they are able to test all code paths easily solving the classic problem
of difficulty in exercising error handling code.
### Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which
have their place. A well tested system has lots of unit tests, a
reasonable number of integration tests, and some end-to-end tests. KUnit
is just trying to address the unit test space which is currently not
being addressed.
### More information on KUnit
There is a bunch of documentation near the end of this patch set that
describes how to use KUnit and best practices for writing unit tests.
For convenience I am hosting the compiled docs here[4].
Additionally for convenience, I have applied these patches to a
branch[5]. The repo may be cloned with:
git clone https://kunit.googlesource.com/linux
This patchset is on the kunit/initial/v5.3/v18 branch.
## History since v15
### v18
- Addrssed comments on 07/19 (kunit: test: add initial tests) from
Randy Dunlap by removing redundant dependencies from Kconfig entries.
### v17
- Addressed comments on 06/19 (lib: enable building KUnit in lib/) from
Stephen Boyd by moving KUnit submenu ahead of Runtime Testing
submenu.
### v16
- Addressed comments from Linus Torvalds by moving all kunit/ paths to
lib/kunit/.
- Addressed comments by Randy Dunlap by moving KUnit Kconfig under
lib/Kconfig.debug so the KUnit submenu shows up under the "Kernel
Hacking" menu.
[1] https://www.lkml.org/lkml/2019/9/20/696
[2] https://www.lkml.org/lkml/2019/9/20/738
[3] https://google.github.io/kunit-docs/third_party/kernel/docs/usage.html#kuni…
[4] https://google.github.io/kunit-docs/third_party/kernel/docs/
[5] https://kunit.googlesource.com/linux/+/kunit/initial/v5.3/v18
---
Avinash Kondareddy (1):
kunit: test: add tests for KUnit managed resources
Brendan Higgins (16):
kunit: test: add KUnit test runner core
kunit: test: add test resource management API
kunit: test: add string_stream a std::stream like string builder
kunit: test: add assertion printing library
kunit: test: add the concept of expectations
lib: enable building KUnit in lib/
kunit: test: add initial tests
objtool: add kunit_try_catch_throw to the noreturn list
kunit: test: add support for test abort
kunit: test: add tests for kunit test abort
kunit: test: add the concept of assertions
kunit: defconfig: add defconfigs for building KUnit tests
Documentation: kunit: add documentation for KUnit
MAINTAINERS: add entry for KUnit the unit testing framework
MAINTAINERS: add proc sysctl KUnit test to PROC SYSCTL section
kunit: fix failure to build without printk
Felix Guo (1):
kunit: tool: add Python wrappers for running KUnit tests
Iurii Zaikin (1):
kernel/sysctl-test: Add null pointer test for sysctl.c:proc_dointvec()
Documentation/dev-tools/index.rst | 1 +
Documentation/dev-tools/kunit/api/index.rst | 16 +
Documentation/dev-tools/kunit/api/test.rst | 11 +
Documentation/dev-tools/kunit/faq.rst | 62 +
Documentation/dev-tools/kunit/index.rst | 79 +
Documentation/dev-tools/kunit/start.rst | 180 ++
Documentation/dev-tools/kunit/usage.rst | 576 +++++++
MAINTAINERS | 13 +
arch/um/configs/kunit_defconfig | 3 +
include/kunit/assert.h | 356 ++++
include/kunit/string-stream.h | 51 +
include/kunit/test.h | 1490 +++++++++++++++++
include/kunit/try-catch.h | 75 +
kernel/Makefile | 2 +
kernel/sysctl-test.c | 392 +++++
lib/Kconfig.debug | 13 +
lib/Makefile | 2 +
lib/kunit/Kconfig | 36 +
lib/kunit/Makefile | 9 +
lib/kunit/assert.c | 141 ++
lib/kunit/example-test.c | 88 +
lib/kunit/string-stream-test.c | 52 +
lib/kunit/string-stream.c | 217 +++
lib/kunit/test-test.c | 331 ++++
lib/kunit/test.c | 478 ++++++
lib/kunit/try-catch.c | 118 ++
tools/objtool/check.c | 1 +
tools/testing/kunit/.gitignore | 3 +
tools/testing/kunit/configs/all_tests.config | 3 +
tools/testing/kunit/kunit.py | 136 ++
tools/testing/kunit/kunit_config.py | 66 +
tools/testing/kunit/kunit_kernel.py | 149 ++
tools/testing/kunit/kunit_parser.py | 310 ++++
tools/testing/kunit/kunit_tool_test.py | 206 +++
.../test_is_test_passed-all_passed.log | 32 +
.../test_data/test_is_test_passed-crash.log | 69 +
.../test_data/test_is_test_passed-failure.log | 36 +
.../test_is_test_passed-no_tests_run.log | 75 +
.../test_output_isolated_correctly.log | 106 ++
.../test_data/test_read_from_file.kconfig | 17 +
40 files changed, 6001 insertions(+)
create mode 100644 Documentation/dev-tools/kunit/api/index.rst
create mode 100644 Documentation/dev-tools/kunit/api/test.rst
create mode 100644 Documentation/dev-tools/kunit/faq.rst
create mode 100644 Documentation/dev-tools/kunit/index.rst
create mode 100644 Documentation/dev-tools/kunit/start.rst
create mode 100644 Documentation/dev-tools/kunit/usage.rst
create mode 100644 arch/um/configs/kunit_defconfig
create mode 100644 include/kunit/assert.h
create mode 100644 include/kunit/string-stream.h
create mode 100644 include/kunit/test.h
create mode 100644 include/kunit/try-catch.h
create mode 100644 kernel/sysctl-test.c
create mode 100644 lib/kunit/Kconfig
create mode 100644 lib/kunit/Makefile
create mode 100644 lib/kunit/assert.c
create mode 100644 lib/kunit/example-test.c
create mode 100644 lib/kunit/string-stream-test.c
create mode 100644 lib/kunit/string-stream.c
create mode 100644 lib/kunit/test-test.c
create mode 100644 lib/kunit/test.c
create mode 100644 lib/kunit/try-catch.c
create mode 100644 tools/testing/kunit/.gitignore
create mode 100644 tools/testing/kunit/configs/all_tests.config
create mode 100755 tools/testing/kunit/kunit.py
create mode 100644 tools/testing/kunit/kunit_config.py
create mode 100644 tools/testing/kunit/kunit_kernel.py
create mode 100644 tools/testing/kunit/kunit_parser.py
create mode 100755 tools/testing/kunit/kunit_tool_test.py
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-all_passed.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-crash.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-failure.log
create mode 100644 tools/testing/kunit/test_data/test_is_test_passed-no_tests_run.log
create mode 100644 tools/testing/kunit/test_data/test_output_isolated_correctly.log
create mode 100644 tools/testing/kunit/test_data/test_read_from_file.kconfig
--
2.23.0.351.gc4317032e6-goog
Currently, when some of the KSFT subsystems specified in TARGETS fails to
build, the toplevel Makefile just keeps carrying on with the build process.
This behaviour is expected and desirable especially in the context of a CI
system running KSelfTest where is not always easy to guarantee that the
most recent and esoteric toolchain's dependencies are respected across all
KSFT TARGETS in a timely manner.
Unfortunately, as of now, this holds true only if the very last of the
subsystems listed in TARGETS happened to be one that could be successfully
built: if the last listed TARGETS fails, it is taken as the whole outcome
of the Makefile target and the complete build/install halts even though
many other preceding subsytems were in fact already successfully built.
Fix the KSFT Makefile behaviour related to all/install targets in order
to fail as a whole only when the all/install targets have failed for all
of the requested TARGETS, while succeeding when at least one of TARGETS
has been successfully built.
Signed-off-by: Cristian Marussi <cristian.marussi(a)arm.com>
---
This patch is based on ksft/fixes branch from:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git
on top of commit:
ce3a67780212 selftests: watchdog: Add command line option to show watchdog_info
---
tools/testing/selftests/Makefile | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 4cdbae6f4e61..d907b050e929 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -140,11 +140,13 @@ else
endif
all: khdr
- @for TARGET in $(TARGETS); do \
- BUILD_TARGET=$$BUILD/$$TARGET; \
- mkdir $$BUILD_TARGET -p; \
- $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET;\
- done;
+ @ret=1; \
+ for TARGET in $(TARGETS); do \
+ BUILD_TARGET=$$BUILD/$$TARGET; \
+ mkdir $$BUILD_TARGET -p; \
+ $(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET; \
+ ret=$$((ret * $$?)); \
+ done; exit $$ret;
run_tests: all
@for TARGET in $(TARGETS); do \
@@ -192,10 +194,12 @@ ifdef INSTALL_PATH
mkdir -p $(INSTALL_PATH)/kselftest
install -m 744 kselftest/runner.sh $(INSTALL_PATH)/kselftest/
install -m 744 kselftest/prefix.pl $(INSTALL_PATH)/kselftest/
- @for TARGET in $(TARGETS); do \
+ @ret=1; \
+ for TARGET in $(TARGETS); do \
BUILD_TARGET=$$BUILD/$$TARGET; \
$(MAKE) OUTPUT=$$BUILD_TARGET -C $$TARGET INSTALL_PATH=$(INSTALL_PATH)/$$TARGET install; \
- done;
+ ret=$$((ret * $$?)); \
+ done; exit $$ret
@# Ask all targets to emit their test scripts
echo "#!/bin/sh" > $(ALL_SCRIPT)
--
2.17.1
Here are the know kselftest issues on Linux 5.4 with
top commit commit 619e17cf75dd58905aa67ccd494a6ba5f19d6cc6
on x86_64:
The goal is to get these addressed before 5.4 comes out.
3 build failures and status:
pidfd - undefined reference to `pthread_create' collect2: error: ld
returned 1 exit status
Fixed: https://patchwork.kernel.org/patch/11159517/
bfp (two issues)
1. "make TARGETS=bpf kselftest" build fails
Makefile:127: tools/build/Makefile.include: No such file or directory
This is due to recent kbuild changes and I have a patch ready to send.
2. Related to llvm latest version dependency. This is a hard dependency.
Unless users upgrade to latest llvvm, bpf test won't run. The new llvm
might not be supported on all distros yet, in which case bpf will not
get tested in some rings and on some architectures.
gpio
"make TARGETS=gpio kselftest" build fails
Makefile:23: tools/build/Makefile.include: No such file or directory
This is due to recent kbuild changes and I have a patch ready to send.
kvm
"make TARGETS=kvm kselftest" build fails due --no-pie flags.
I am working on a fix for this. no-pie-option defines aren't working
correctly and I suspect try-run miht not be defined in this kselftest
build case.
thanks,
-- Shuah
From: Colin Ian King <colin.king(a)canonical.com>
There is a spelling mistake in in an error message, fix it.
Signed-off-by: Colin Ian King <colin.king(a)canonical.com>
---
tools/testing/selftests/kvm/lib/sparsebit.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kvm/lib/sparsebit.c b/tools/testing/selftests/kvm/lib/sparsebit.c
index 031ba3c932ed..59ffba902e61 100644
--- a/tools/testing/selftests/kvm/lib/sparsebit.c
+++ b/tools/testing/selftests/kvm/lib/sparsebit.c
@@ -1866,7 +1866,7 @@ void sparsebit_validate_internal(struct sparsebit *s)
* of total bits set.
*/
if (s->num_set != total_bits_set) {
- fprintf(stderr, "Number of bits set missmatch,\n"
+ fprintf(stderr, "Number of bits set mismatch,\n"
" s->num_set: 0x%lx total_bits_set: 0x%lx",
s->num_set, total_bits_set);
--
2.20.1
This micro series fixes annoying warn described in patches
while samples/bpf build. Second patch fixes new warn that
comes after fixing warn of first patch, that was masked.
Ivan Khoronzhuk (2):
selftests/bpf: add static to enable_all_controllers()
selftests/bpf: correct path to include msg + path
tools/testing/selftests/bpf/cgroup_helpers.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
2.17.1
Fix the following build error from "make TARGETS=kvm kselftest":
libkvm.a(assert.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIC
This error is seen when build is done from the main Makefile using
kselftest target. In this case KBUILD_CPPFLAGS and CC_OPTION_CFLAGS
are defined.
When build is invoked using:
"make -C tools/testing/selftests/kvm" KBUILD_CPPFLAGS and CC_OPTION_CFLAGS
aren't defined.
There is no need to pass in KBUILD_CPPFLAGS and CC_OPTION_CFLAGS for the
check to determine if --no-pie is necessary, which is the case when these
two aren't defined when "make -C tools/testing/selftests/kvm" runs.
Fix it by simplifying the no-pie-option logic. With this change, both
build variations work.
"make TARGETS=kvm kselftest"
"make -C tools/testing/selftests/kvm"
Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org>
---
Changes since v2:
-- Removed extra blank line added by accident.
-- Fixed commit log.
tools/testing/selftests/kvm/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 62c591f87dab..7ee097658ef0 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -48,7 +48,7 @@ CFLAGS += -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 \
-I$(LINUX_HDR_PATH) -Iinclude -I$(<D) -Iinclude/$(UNAME_M) -I..
no-pie-option := $(call try-run, echo 'int main() { return 0; }' | \
- $(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -no-pie -x c - -o "$$TMP", -no-pie)
+ $(CC) -Werror -no-pie -x c - -o "$$TMP", -no-pie)
# On s390, build the testcases KVM-enabled
pgste-option = $(call try-run, echo 'int main() { return 0; }' | \
--
2.20.1