Currently testing of userspace and in-kernel API use two different frameworks. kselftests for the userspace ones and Kunit for the in-kernel ones. Besides their different scopes, both have different strengths and limitations:
Kunit: * Tests are normal kernel code. * They use the regular kernel toolchain. * They can be packaged and distributed as modules conveniently.
Kselftests: * Tests are normal userspace code * They need a userspace toolchain. A kernel cross toolchain is likely not enough. * A fair amout of userland is required to run the tests, which means a full distro or handcrafted rootfs. * There is no way to conveniently package and run kselftests with a given kernel image. * The kselftests makefiles are not as powerful as regular kbuild. For example they are missing proper header dependency tracking or more complex compiler option modifications.
Therefore kunit is much easier to run against different kernel configurations and architectures. This series aims to combine kselftests and kunit, avoiding both their limitations. It works by compiling the userspace kselftests as part of the regular kernel build, embedding them into the kunit kernel or module and executing them from there. If the kernel toolchain is not fit to produce userspace because of a missing libc, the kernel's own nolibc can be used instead. The structured TAP output from the kselftest is integrated into the kunit KTAP output transparently, the kunit parser can parse the combined logs together.
Further room for improvements: * Call each test in its completely dedicated namespace * Handle additional test files besides the test executable through archives. CPIO, cramfs, etc. * Compatibility with kselftest_harness.h (in progress) * Expose the blobs in debugfs * Provide some convience wrappers around compat userprogs * Figure out a migration path/coexistence solution for kunit UAPI and tools/testing/selftests/
Output from the kunit example testcase, note the output of "example_uapi_tests".
$ ./tools/testing/kunit/kunit.py run --kunitconfig lib/kunit example ... Running tests with: $ .kunit/linux kunit.filter_glob=example kunit.enable=1 mem=1G console=tty kunit_shutdown=halt [11:53:53] ================== example (10 subtests) =================== [11:53:53] [PASSED] example_simple_test [11:53:53] [SKIPPED] example_skip_test [11:53:53] [SKIPPED] example_mark_skipped_test [11:53:53] [PASSED] example_all_expect_macros_test [11:53:53] [PASSED] example_static_stub_test [11:53:53] [PASSED] example_static_stub_using_fn_ptr_test [11:53:53] [PASSED] example_priv_test [11:53:53] =================== example_params_test =================== [11:53:53] [SKIPPED] example value 3 [11:53:53] [PASSED] example value 2 [11:53:53] [PASSED] example value 1 [11:53:53] [SKIPPED] example value 0 [11:53:53] =============== [PASSED] example_params_test =============== [11:53:53] [PASSED] example_slow_test [11:53:53] ======================= (4 subtests) ======================= [11:53:53] [PASSED] procfs [11:53:53] [PASSED] userspace test 2 [11:53:53] [SKIPPED] userspace test 3: some reason [11:53:53] [PASSED] userspace test 4 [11:53:53] ================ [PASSED] example_uapi_test ================ [11:53:53] ===================== [PASSED] example ===================== [11:53:53] ============================================================ [11:53:53] Testing complete. Ran 16 tests: passed: 11, skipped: 5 [11:53:53] Elapsed time: 67.543s total, 1.823s configuring, 65.655s building, 0.058s running
Based on v6.16-rc1.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de --- Changes in v4: - Move Kconfig.nolibc from tools/ to init/ - Drop generic userprogs nolibc integration - Drop generic blob framework - Pick up review tags from David - Extend new kunit TAP parser tests - Add MAINTAINERS entry - Allow CONFIG_KUNIT_UAPI=m - Split /proc validation into dedicated UAPI test - Trim recipient list a bit - Use KUNIT_FAIL_AND_ABORT() over KUNIT_FAIL() - Link to v3: https://lore.kernel.org/r/20250611-kunit-kselftests-v3-0-55e3d148cbc6@linutr...
Changes in v3: - Reintroduce CONFIG_CC_CAN_LINK_STATIC - Enable CONFIG_ARCH_HAS_NOLIBC for m68k and SPARC - Properly handle 'clean' target for userprogs - Use ramfs over tmpfs to reduce dependencies - Inherit userprogs byte order and ABI from kernel - Drop now unnecessary "#ifndef NOLIBC" - Pick up review tags - Drop usage of __private in blob.h, sparse complains and it is not really necessary - Fix execution on loongarch when using clang - Drop userprogs libgcc handling, it was ugly and is not yet necessary - Link to v2: https://lore.kernel.org/r/20250407-kunit-kselftests-v2-0-454114e287fd@linutr...
Changes in v2: - Rebase onto v6.15-rc1 - Add documentation and kernel docs - Resolve invalid kconfig breakages - Drop already applied patch "kbuild: implement CONFIG_HEADERS_INSTALL for Usermode Linux" - Drop userprogs CONFIG_WERROR integration, it doesn't need to be part of this series - Replace patch prefix "kconfig" with "kbuild" - Rename kunit_uapi_run_executable() to kunit_uapi_run_kselftest() - Generate private, conflict-free symbols in the blob framework - Handle kselftest exit codes - Handle SIGABRT - Forward output also to kunit debugfs log - Install a fd=0 stdin filedescriptor - Link to v1: https://lore.kernel.org/r/20250217-kunit-kselftests-v1-0-42b4524c3b0a@linutr...
--- Thomas Weißschuh (15): kbuild: userprogs: avoid duplication of flags inherited from kernel kbuild: userprogs: also inherit byte order and ABI from kernel kbuild: doc: add label for userprogs section init: re-add CONFIG_CC_CAN_LINK_STATIC init: add nolibc build support fs,fork,exit: export symbols necessary for KUnit UAPI support kunit: tool: Add test for nested test result reporting kunit: tool: Don't overwrite test status based on subtest counts kunit: tool: Parse skipped tests from kselftest.h kunit: Always descend into kunit directory during build kunit: qemu_configs: loongarch: Enable LSX/LSAX kunit: Introduce UAPI testing framework kunit: uapi: Add example for UAPI tests kunit: uapi: Introduce preinit executable kunit: uapi: Validate usability of /proc
Documentation/dev-tools/kunit/api/index.rst | 5 + Documentation/dev-tools/kunit/api/uapi.rst | 14 + Documentation/kbuild/makefiles.rst | 2 + MAINTAINERS | 11 + Makefile | 7 +- fs/exec.c | 2 + fs/file.c | 1 + fs/filesystems.c | 2 + fs/fs_struct.c | 1 + fs/pipe.c | 2 + include/kunit/uapi.h | 77 ++++++ init/Kconfig | 7 + init/Kconfig.nolibc | 15 + init/Makefile.nolibc | 13 + kernel/exit.c | 3 + kernel/fork.c | 2 + lib/Makefile | 4 - lib/kunit/Kconfig | 14 + lib/kunit/Makefile | 27 +- lib/kunit/kunit-example-test.c | 15 + lib/kunit/kunit-example-uapi.c | 22 ++ lib/kunit/kunit-test-uapi.c | 51 ++++ lib/kunit/kunit-test.c | 23 +- lib/kunit/kunit-uapi.c | 305 +++++++++++++++++++++ lib/kunit/uapi-preinit.c | 63 +++++ tools/testing/kunit/kunit_parser.py | 13 +- tools/testing/kunit/kunit_tool_test.py | 11 + tools/testing/kunit/qemu_configs/loongarch.py | 2 + .../test_is_test_passed-failure-nested.log | 10 + .../test_data/test_is_test_passed-kselftest.log | 3 +- 30 files changed, 714 insertions(+), 13 deletions(-) --- base-commit: 9d5898b413d17510b2a41664a42390a2c79f8bf4 change-id: 20241015-kunit-kselftests-56273bc40442
Best regards,
The duplication makes maintenance harder. Changes need to be done in two places and the lines will grow overly long.
Use an intermediary variable instead.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de --- Makefile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile index 35e6e5240c61a8a329011929fcd0352b881dccdc..c4293cf91e968ca8ee64452841fb266e24df63f6 100644 --- a/Makefile +++ b/Makefile @@ -1130,8 +1130,9 @@ LDFLAGS_vmlinux += --emit-relocs --discard-none endif
# Align the bit size of userspace programs with the kernel -KBUILD_USERCFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) -KBUILD_USERLDFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) +USERFLAGS_FROM_KERNEL := -m32 -m64 --target=% +KBUILD_USERCFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) +KBUILD_USERLDFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
# userspace programs are linked via the compiler, use the correct linker ifeq ($(CONFIG_CC_IS_CLANG)$(CONFIG_LD_IS_LLD),yy)
On Thu, Jun 26, 2025 at 08:10:09AM +0200, Thomas Weißschuh wrote:
The duplication makes maintenance harder. Changes need to be done in two places and the lines will grow overly long.
Use an intermediary variable instead.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
Makefile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Reviewed-by: Nicolas Schier n.schier@avm.de
On Thu, Jun 26, 2025 at 3:10 PM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
The duplication makes maintenance harder. Changes need to be done in two places and the lines will grow overly long.
Use an intermediary variable instead.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
Makefile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile index 35e6e5240c61a8a329011929fcd0352b881dccdc..c4293cf91e968ca8ee64452841fb266e24df63f6 100644 --- a/Makefile +++ b/Makefile @@ -1130,8 +1130,9 @@ LDFLAGS_vmlinux += --emit-relocs --discard-none endif
# Align the bit size of userspace programs with the kernel -KBUILD_USERCFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) -KBUILD_USERLDFLAGS += $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) +USERFLAGS_FROM_KERNEL := -m32 -m64 --target=% +KBUILD_USERCFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) +KBUILD_USERLDFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
Acked-by: Masahiro Yamada masahiroy@kernel.org
However, as I mentioned before, the following code is shorter.
USERFLAGS_FROM_KERNEL := $(filter -m32 -m64 --target=%, $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) KBUILD_USERCFLAGS += $(USERFLAGS_FROM_KERNEL) KBUILD_USERLDFLAGS += $(USERFLAGS_FROM_KERNEL)
Make sure the byte order and ABI of the userprogs matches the one of the kernel, similar to how the bit size is handled. Otherwise the userprogs may not be executable. This happens for example on powerpc little endian, or riscv32.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de --- Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile index c4293cf91e968ca8ee64452841fb266e24df63f6..b9aa1058321dabd3b3dd5610e45a2807dfa257f4 100644 --- a/Makefile +++ b/Makefile @@ -1129,8 +1129,8 @@ ifneq ($(CONFIG_ARCH_VMLINUX_NEEDS_RELOCS),) LDFLAGS_vmlinux += --emit-relocs --discard-none endif
-# Align the bit size of userspace programs with the kernel -USERFLAGS_FROM_KERNEL := -m32 -m64 --target=% +# Align the bit size, byte order and architecture of userspace programs with the kernel +USERFLAGS_FROM_KERNEL := -m32 -m64 -mlittle-endian -mbig-endian --target=% -march=% -mabi=% KBUILD_USERCFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) KBUILD_USERLDFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
On Thu, Jun 26, 2025 at 08:10:10AM +0200, Thomas Weißschuh wrote:
Make sure the byte order and ABI of the userprogs matches the one of the kernel, similar to how the bit size is handled. Otherwise the userprogs may not be executable. This happens for example on powerpc little endian, or riscv32.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Reviewed-by: Nicolas Schier n.schier@avm.de
On Thu, Jun 26, 2025 at 3:10 PM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
Make sure the byte order and ABI of the userprogs matches the one of the kernel, similar to how the bit size is handled. Otherwise the userprogs may not be executable. This happens for example on powerpc little endian, or riscv32.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile index c4293cf91e968ca8ee64452841fb266e24df63f6..b9aa1058321dabd3b3dd5610e45a2807dfa257f4 100644 --- a/Makefile +++ b/Makefile @@ -1129,8 +1129,8 @@ ifneq ($(CONFIG_ARCH_VMLINUX_NEEDS_RELOCS),) LDFLAGS_vmlinux += --emit-relocs --discard-none endif
-# Align the bit size of userspace programs with the kernel -USERFLAGS_FROM_KERNEL := -m32 -m64 --target=% +# Align the bit size, byte order and architecture of userspace programs with the kernel +USERFLAGS_FROM_KERNEL := -m32 -m64 -mlittle-endian -mbig-endian --target=% -march=% -mabi=% KBUILD_USERCFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS)) KBUILD_USERLDFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
Acked-by: Masahiro Yamada masahiroy@kernel.org
Some upcoming new documentation should link directly to the userprogs section.
Add a label to the section so it can be referenced.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de --- Documentation/kbuild/makefiles.rst | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst index 8aef3650c1f32b6b197e0dc777e26775d371a081..c14c1f632f6069c8751c8388a35bef539e19f9e8 100644 --- a/Documentation/kbuild/makefiles.rst +++ b/Documentation/kbuild/makefiles.rst @@ -891,6 +891,8 @@ This is possible in two ways: This will tell kbuild to build lxdialog even if not referenced in any rule.
+.. _kbuild_userprogs: + Userspace Program support =========================
On Thu, Jun 26, 2025 at 08:10:11AM +0200, Thomas Weißschuh wrote:
Some upcoming new documentation should link directly to the userprogs section.
Add a label to the section so it can be referenced.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
Documentation/kbuild/makefiles.rst | 2 ++ 1 file changed, 2 insertions(+)
Reviewed-by: Nicolas Schier n.schier@avm.de
On Thu, Jun 26, 2025 at 3:10 PM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
Some upcoming new documentation should link directly to the userprogs section.
Add a label to the section so it can be referenced.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
Documentation/kbuild/makefiles.rst | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst index 8aef3650c1f32b6b197e0dc777e26775d371a081..c14c1f632f6069c8751c8388a35bef539e19f9e8 100644 --- a/Documentation/kbuild/makefiles.rst +++ b/Documentation/kbuild/makefiles.rst @@ -891,6 +891,8 @@ This is possible in two ways: This will tell kbuild to build lxdialog even if not referenced in any rule.
+.. _kbuild_userprogs:
Userspace Program support
Acked-by: Masahiro Yamada masahiroy@kernel.org
For the kunit UAPI functionality this feature is needed.
This reverts commit d1b99cdf22e0 ("init: remove unused CONFIG_CC_CAN_LINK_STATIC")
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de --- init/Kconfig | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/init/Kconfig b/init/Kconfig index af4c2f0854554bbcdf193852cf5c1d2c2accc64f..26cafbad4f1560fb56b4bef31ae29baf54175661 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -87,6 +87,11 @@ config CC_CAN_LINK default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag)) if 64BIT default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m32-flag))
+config CC_CAN_LINK_STATIC + bool + default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag) -static) if 64BIT + default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m32-flag) -static) + # Fixed in GCC 14, 13.3, 12.4 and 11.5 # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113921 config GCC_ASM_GOTO_OUTPUT_BROKEN
On Thu, Jun 26, 2025 at 08:10:12AM +0200, Thomas Weißschuh wrote:
For the kunit UAPI functionality this feature is needed.
This reverts commit d1b99cdf22e0 ("init: remove unused CONFIG_CC_CAN_LINK_STATIC")
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
init/Kconfig | 5 +++++ 1 file changed, 5 insertions(+)
Reviewed-by: Nicolas Schier n.schier@avm.de
On Thu, Jun 26, 2025 at 3:10 PM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
For the kunit UAPI functionality this feature is needed.
This reverts commit d1b99cdf22e0 ("init: remove unused CONFIG_CC_CAN_LINK_STATIC")
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
init/Kconfig | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/init/Kconfig b/init/Kconfig index af4c2f0854554bbcdf193852cf5c1d2c2accc64f..26cafbad4f1560fb56b4bef31ae29baf54175661 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -87,6 +87,11 @@ config CC_CAN_LINK default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag)) if 64BIT default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m32-flag))
+config CC_CAN_LINK_STATIC
booldefault $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag) -static) if 64BITdefault $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m32-flag) -static)# Fixed in GCC 14, 13.3, 12.4 and 11.5 # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113921 config GCC_ASM_GOTO_OUTPUT_BROKEN
-- 2.50.0
Reviewed-by: Masahiro Yamada masahiroy@kernel.org
Building userspace applications through the kbuild "userprogs" framework requires a libc. Kernel toolchains often do not contain a libc. In this case it is useful to use the nolibc library from the kernel tree. Nolibc does not support all architectures and requires compiler flags.
Add a kconfig option, so users can know where it is available and provide a variable for common options.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de --- MAINTAINERS | 2 ++ init/Kconfig | 2 ++ init/Kconfig.nolibc | 15 +++++++++++++++ init/Makefile.nolibc | 13 +++++++++++++ 4 files changed, 32 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS index a92290fffa163f9fe8fe3f04bf66426f9a894409..e806158cc6798cf97a4aab58c038fb5351d469aa 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -17702,6 +17702,8 @@ M: Willy Tarreau w@1wt.eu M: Thomas Weißschuh linux@weissschuh.net S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/nolibc/linux-nolibc.git +F: init/Kconfig.nolibc +F: init/Makefile.nolibc F: tools/include/nolibc/ F: tools/testing/selftests/nolibc/
diff --git a/init/Kconfig b/init/Kconfig index 26cafbad4f1560fb56b4bef31ae29baf54175661..0af62f135192e0470e16eb6bb2fbb45ac38f4b81 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -92,6 +92,8 @@ config CC_CAN_LINK_STATIC default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag) -static) if 64BIT default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m32-flag) -static)
+source "init/Kconfig.nolibc" + # Fixed in GCC 14, 13.3, 12.4 and 11.5 # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113921 config GCC_ASM_GOTO_OUTPUT_BROKEN diff --git a/init/Kconfig.nolibc b/init/Kconfig.nolibc new file mode 100644 index 0000000000000000000000000000000000000000..29cbc5437e70cbc5e256f00b74d0ab4801b40de7 --- /dev/null +++ b/init/Kconfig.nolibc @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0 + +config ARCH_HAS_NOLIBC + bool + default y if ARM + default y if ARM64 + default y if LOONGARCH + default y if M68K + default y if MIPS + default y if PPC + default y if RISCV + default y if S390 + default y if SPARC + default y if UML_X86 + default y if X86 diff --git a/init/Makefile.nolibc b/init/Makefile.nolibc new file mode 100644 index 0000000000000000000000000000000000000000..a8a193f78e34e0c4a6004e871bc206e06925f88a --- /dev/null +++ b/init/Makefile.nolibc @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0 +# Compiler flags, which are necessary to build userspace applications with the +# in-kernel libc "nolibc". + +ifeq ($(and $(CONFIG_ARCH_HAS_NOLIBC),$(CONFIG_HEADERS_INSTALL)),y) + +NOLIBC_USERCFLAGS := -nostdlib -nostdinc -static -ffreestanding \ + -fno-asynchronous-unwind-tables -fno-stack-protector \ + -isystem $(objtree)/usr/include -isystem $(srctree)/tools/include/nolibc/ + +NOLIBC_USERLDFLAGS := -nostdlib -nostdinc -static + +endif # CONFIG_ARCH_HAS_NOLIBC && CONFIG_HEADERS_INSTALL
On Thu, Jun 26, 2025 at 08:10:13AM +0200, Thomas Weißschuh wrote:
Building userspace applications through the kbuild "userprogs" framework requires a libc. Kernel toolchains often do not contain a libc. In this case it is useful to use the nolibc library from the kernel tree. Nolibc does not support all architectures and requires compiler flags.
Add a kconfig option, so users can know where it is available and provide a variable for common options.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
MAINTAINERS | 2 ++ init/Kconfig | 2 ++ init/Kconfig.nolibc | 15 +++++++++++++++ init/Makefile.nolibc | 13 +++++++++++++ 4 files changed, 32 insertions(+)
Reviewed-by: Nicolas Schier n.schier@avm.de
The KUnit UAPI infrastructure starts userspace processes. As it should be able to be built as a module, export the necessary symbols.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
--- To: Kees Cook kees@kernel.org To: Alexander Viro viro@zeniv.linux.org.uk To: Christian Brauner brauner@kernel.org To: Jan Kara jack@suse.cz Cc: linux-mm@kvack.org Cc: linux-fsdevel@vger.kernel.org
./get_maintainer.pl would have also Cc-ed all of the memory management and scheduler maintainers. I trimmed the list to only BINFMT/EXEC and VFS. --- fs/exec.c | 2 ++ fs/file.c | 1 + fs/filesystems.c | 2 ++ fs/fs_struct.c | 1 + fs/pipe.c | 2 ++ kernel/exit.c | 3 +++ kernel/fork.c | 2 ++ 7 files changed, 13 insertions(+)
diff --git a/fs/exec.c b/fs/exec.c index 1f5fdd2e096e392b342f122d35aba4cf035441c7..13f7f27641942eddcb179bdd93d99b799d155813 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -68,6 +68,7 @@ #include <linux/user_events.h> #include <linux/rseq.h> #include <linux/ksm.h> +#include <linux/export.h>
#include <linux/uaccess.h> #include <asm/mmu_context.h> @@ -1919,6 +1920,7 @@ int kernel_execve(const char *kernel_filename, putname(filename); return retval; } +EXPORT_SYMBOL_GPL_FOR_MODULES(kernel_execve, "kunit-uapi");
static int do_execve(struct filename *filename, const char __user *const __user *__argv, diff --git a/fs/file.c b/fs/file.c index 3a3146664cf37115624e12f7f06826d48827e9d7..89d07feb9c328337451ce40cb0f368b6cb986c2c 100644 --- a/fs/file.c +++ b/fs/file.c @@ -1317,6 +1317,7 @@ int replace_fd(unsigned fd, struct file *file, unsigned flags) spin_unlock(&files->file_lock); return err; } +EXPORT_SYMBOL_GPL_FOR_MODULES(replace_fd, "kunit-uapi");
/** * receive_fd() - Install received file into file descriptor table diff --git a/fs/filesystems.c b/fs/filesystems.c index 95e5256821a53494d88f496193305a2e50e04444..a3a588f387bbd8268246d1026389deaadf265d0b 100644 --- a/fs/filesystems.c +++ b/fs/filesystems.c @@ -17,6 +17,7 @@ #include <linux/slab.h> #include <linux/uaccess.h> #include <linux/fs_parser.h> +#include <linux/export.h>
/* * Handling of filesystem drivers list. @@ -45,6 +46,7 @@ void put_filesystem(struct file_system_type *fs) { module_put(fs->owner); } +EXPORT_SYMBOL_GPL_FOR_MODULES(put_filesystem, "kunit-uapi");
static struct file_system_type **find_filesystem(const char *name, unsigned len) { diff --git a/fs/fs_struct.c b/fs/fs_struct.c index 64c2d0814ed6889cc12603410e6e9dc44089586f..26340d225deba3f2ec30252293fdf417235a6a4a 100644 --- a/fs/fs_struct.c +++ b/fs/fs_struct.c @@ -46,6 +46,7 @@ void set_fs_pwd(struct fs_struct *fs, const struct path *path) if (old_pwd.dentry) path_put(&old_pwd); } +EXPORT_SYMBOL_GPL_FOR_MODULES(set_fs_pwd, "kunit-uapi");
static inline int replace_path(struct path *p, const struct path *old, const struct path *new) { diff --git a/fs/pipe.c b/fs/pipe.c index 45077c37bad154ef146b047834d35d489fcc4d8d..d6cb743d2cfc041f08b498a5a764e9a96dc34069 100644 --- a/fs/pipe.c +++ b/fs/pipe.c @@ -27,6 +27,7 @@ #include <linux/watch_queue.h> #include <linux/sysctl.h> #include <linux/sort.h> +#include <linux/export.h>
#include <linux/uaccess.h> #include <asm/ioctls.h> @@ -971,6 +972,7 @@ int create_pipe_files(struct file **res, int flags) file_set_fsnotify_mode(res[1], FMODE_NONOTIFY_PERM); return 0; } +EXPORT_SYMBOL_GPL_FOR_MODULES(create_pipe_files, "kunit-uapi");
static int __do_pipe_flags(int *fd, struct file **files, int flags) { diff --git a/kernel/exit.c b/kernel/exit.c index bd743900354ca5fc6c550f80e30393a632eb9a4e..610dffb1276ac60b475708587ca053f315fea9c3 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -70,6 +70,7 @@ #include <linux/user_events.h> #include <linux/uaccess.h> #include <linux/pidfs.h> +#include <linux/export.h>
#include <uapi/linux/wait.h>
@@ -1005,6 +1006,7 @@ void __noreturn do_exit(long code) lockdep_free_task(tsk); do_task_dead(); } +EXPORT_SYMBOL_GPL_FOR_MODULES(do_exit, "kunit-uapi");
void __noreturn make_task_dead(int signr) { @@ -1887,6 +1889,7 @@ int kernel_wait(pid_t pid, int *stat) put_pid(wo.wo_pid); return ret; } +EXPORT_SYMBOL_GPL_FOR_MODULES(kernel_wait, "kunit-uapi");
SYSCALL_DEFINE4(wait4, pid_t, upid, int __user *, stat_addr, int, options, struct rusage __user *, ru) diff --git a/kernel/fork.c b/kernel/fork.c index 1ee8eb11f38bae1d2eb6de9494aea94b7a19e6c3..5de7a9bc005ade6dcfbdfe1a63cadbef8782658c 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -105,6 +105,7 @@ #include <uapi/linux/pidfd.h> #include <linux/pidfs.h> #include <linux/tick.h> +#include <linux/export.h>
#include <asm/pgalloc.h> #include <linux/uaccess.h> @@ -2676,6 +2677,7 @@ pid_t user_mode_thread(int (*fn)(void *), void *arg, unsigned long flags)
return kernel_clone(&args); } +EXPORT_SYMBOL_GPL_FOR_MODULES(user_mode_thread, "kunit-uapi");
#ifdef __ARCH_WANT_SYS_FORK SYSCALL_DEFINE0(fork)
Currently there is no test validating the result reporting from nested tests. Add one, it will also be used to validate upcoming changes to the nested test parsing.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com --- tools/testing/kunit/kunit_tool_test.py | 10 ++++++++++ .../kunit/test_data/test_is_test_passed-failure-nested.log | 7 +++++++ 2 files changed, 17 insertions(+)
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index bbba921e0eacb18663abfcabb2bccf330d8666f5..b74dc05fc2fe5b3ff629172fc7aafeb5c3d29fb3 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -165,6 +165,16 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status) self.assertEqual(result.counts.errors, 0)
+ def test_parse_failed_nested_tests_log(self): + nested_log = test_data_path('test_is_test_passed-failure-nested.log') + with open(nested_log) as file: + result = kunit_parser.parse_run_tests(file.readlines(), stdout) + self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status) + self.assertEqual(result.counts.failed, 2) + self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[0].status) + self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].status) + self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].subtests[0].status) + def test_no_header(self): empty_log = test_data_path('test_is_test_passed-no_tests_run_no_header.log') with open(empty_log) as file: diff --git a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log new file mode 100644 index 0000000000000000000000000000000000000000..2e528da39ab5b2be0fca6cf9160c10929fba3c9e --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log @@ -0,0 +1,7 @@ +KTAP version 1 +1..2 +not ok 1 subtest 1 + KTAP version 1 + 1..1 + not ok 1 subsubtest 1 +not ok 2 subtest 2
On Thu, Jun 26, 2025 at 2:10 AM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
Currently there is no test validating the result reporting from nested tests. Add one, it will also be used to validate upcoming changes to the nested test parsing.
Hello!
This patch looks good to me! However, most of the tests in kunit-tool-test do check nested test output but we do lack checks for failing tests. Could we change this commit description to be something like: "Currently there is a lack of tests validating failed results reporting from nested tests."?
Reviewed-by: Rae Moar rmoar@google.com
Thanks! -Rae
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com
tools/testing/kunit/kunit_tool_test.py | 10 ++++++++++ .../kunit/test_data/test_is_test_passed-failure-nested.log | 7 +++++++ 2 files changed, 17 insertions(+)
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index bbba921e0eacb18663abfcabb2bccf330d8666f5..b74dc05fc2fe5b3ff629172fc7aafeb5c3d29fb3 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -165,6 +165,16 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status) self.assertEqual(result.counts.errors, 0)
def test_parse_failed_nested_tests_log(self):nested_log = test_data_path('test_is_test_passed-failure-nested.log')with open(nested_log) as file:result = kunit_parser.parse_run_tests(file.readlines(), stdout)self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status)self.assertEqual(result.counts.failed, 2)self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[0].status)self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].status)self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].subtests[0].status)def test_no_header(self): empty_log = test_data_path('test_is_test_passed-no_tests_run_no_header.log') with open(empty_log) as file:diff --git a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log new file mode 100644 index 0000000000000000000000000000000000000000..2e528da39ab5b2be0fca6cf9160c10929fba3c9e --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log @@ -0,0 +1,7 @@ +KTAP version 1 +1..2 +not ok 1 subtest 1
- KTAP version 1
- 1..1
not ok 1 subsubtest 1+not ok 2 subtest 2
-- 2.50.0
If a subtest itself reports success, but the outer testcase fails, the whole testcase should be reported as a failure. However the status is recalculated based on the test counts, overwriting the outer test result. Synthesize a failed test in this case to make sure the failure is not swallowed.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com --- tools/testing/kunit/kunit_parser.py | 5 +++++ tools/testing/kunit/kunit_tool_test.py | 3 ++- tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log | 3 +++ 3 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index c176487356e6c94882046b19ea696d750905b8d5..2478beb28fc3db825855ad46200340e884da7df1 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -686,6 +686,11 @@ def bubble_up_test_results(test: Test) -> None: counts.add_status(status) elif test.counts.get_status() == TestStatus.TEST_CRASHED: test.status = TestStatus.TEST_CRASHED + if not test.ok_status(): + for t in subtests: + if not t.ok_status(): + counts.add_status(t.status) + break
def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest: bool, printer: Printer) -> Test: """ diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index b74dc05fc2fe5b3ff629172fc7aafeb5c3d29fb3..48a0dd0f9c87caf9f018aade161db90a613fc407 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -170,8 +170,9 @@ class KUnitParserTest(unittest.TestCase): with open(nested_log) as file: result = kunit_parser.parse_run_tests(file.readlines(), stdout) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status) - self.assertEqual(result.counts.failed, 2) + self.assertEqual(result.counts.failed, 3) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[0].status) + self.assertEqual(kunit_parser.TestStatus.SUCCESS, result.subtests[0].subtests[0].status) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].status) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].subtests[0].status)
diff --git a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log index 2e528da39ab5b2be0fca6cf9160c10929fba3c9e..5498dfd0b0db24663e1a1e9bf78c587de6746522 100644 --- a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log +++ b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log @@ -1,5 +1,8 @@ KTAP version 1 1..2 + KTAP version 1 + 1..1 + ok 1 test 1 not ok 1 subtest 1 KTAP version 1 1..1
On Thu, Jun 26, 2025 at 2:10 AM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
If a subtest itself reports success, but the outer testcase fails, the whole testcase should be reported as a failure. However the status is recalculated based on the test counts, overwriting the outer test result. Synthesize a failed test in this case to make sure the failure is not swallowed.
Hello!
This is a very exciting patch series! However, I have a few concerns with this patch.
When I parse the following KTAP with this change:
KTAP version 1 1..2 KTAP version 1 1..2 ok 1 test 1 not ok 2 test 2 not ok 1 subtest 1 KTAP version 1 1..1 not ok 1 subsubtest 1 not ok 2 subtest 2
The output is:
[20:54:12] ============================================================ [20:54:12] ======================= (2 subtests) ======================= [20:54:12] [PASSED] test 1 [20:54:12] [FAILED] test 2 [20:54:12] ==================== [FAILED] subtest 1 ==================== [20:54:12] ======================= (1 subtest) ======================== [20:54:12] [FAILED] subsubtest 1 [20:54:12] ==================== [FAILED] subtest 2 ==================== [20:54:12] ============================================================ [20:54:12] Testing complete. Ran 6 tests: passed: 1, failed: 5
This reports a total of 6 tests, which is not equivalent to the three subtests plus the two suites. I believe this is because the change to bubble_up_test_results below double counts the failed test case.
Historically, the KUnit parser only counts the results of test cases, not the suites. I would like to stay as close to this as possible so as to not inflate existing testing numbers. However, I believe the main concern here is the case where if there is a suite reporting failure but all subtests pass, it will not appear in the summary line. For example,
KTAP version 1 1..1 KTAP version 1 1..1 ok 1 test 1 not ok 1 subtest 1
Reporting: All passing: Tests run: 1, passed: 1
This is absolutely an important edge case to cover. Therefore, we should add 1 failure count to the suite count if the bubbled up results indicate it should instead pass.
Thanks! -Rae
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com
tools/testing/kunit/kunit_parser.py | 5 +++++ tools/testing/kunit/kunit_tool_test.py | 3 ++- tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log | 3 +++ 3 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index c176487356e6c94882046b19ea696d750905b8d5..2478beb28fc3db825855ad46200340e884da7df1 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -686,6 +686,11 @@ def bubble_up_test_results(test: Test) -> None: counts.add_status(status) elif test.counts.get_status() == TestStatus.TEST_CRASHED: test.status = TestStatus.TEST_CRASHED
if not test.ok_status():for t in subtests:if not t.ok_status():counts.add_status(t.status)break
Here instead I recommend checking if not test.ok_status() and test.counts.get_status() == TestStatus.SUCCESS and if so counts.add_status(status)
def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest: bool, printer: Printer) -> Test: """ diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index b74dc05fc2fe5b3ff629172fc7aafeb5c3d29fb3..48a0dd0f9c87caf9f018aade161db90a613fc407 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -170,8 +170,9 @@ class KUnitParserTest(unittest.TestCase): with open(nested_log) as file: result = kunit_parser.parse_run_tests(file.readlines(), stdout) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status)
self.assertEqual(result.counts.failed, 2)
self.assertEqual(result.counts.failed, 3) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[0].status)self.assertEqual(kunit_parser.TestStatus.SUCCESS, result.subtests[0].subtests[0].status) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].status) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].subtests[0].status)diff --git a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log index 2e528da39ab5b2be0fca6cf9160c10929fba3c9e..5498dfd0b0db24663e1a1e9bf78c587de6746522 100644 --- a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log +++ b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log @@ -1,5 +1,8 @@ KTAP version 1 1..2
- KTAP version 1
- 1..1
ok 1 test 1not ok 1 subtest 1 KTAP version 1 1..1
-- 2.50.0
-- You received this message because you are subscribed to the Google Groups "KUnit Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to kunit-dev+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/kunit-dev/20250626-kunit-kselftests-v4-8-4....
On Tue, Jul 01, 2025 at 05:11:59PM -0400, Rae Moar wrote:
On Thu, Jun 26, 2025 at 2:10 AM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
If a subtest itself reports success, but the outer testcase fails, the whole testcase should be reported as a failure. However the status is recalculated based on the test counts, overwriting the outer test result. Synthesize a failed test in this case to make sure the failure is not swallowed.
This is a very exciting patch series! However, I have a few concerns with this patch.
Thanks for the review!
When I parse the following KTAP with this change:
KTAP version 1 1..2 KTAP version 1 1..2 ok 1 test 1 not ok 2 test 2 not ok 1 subtest 1 KTAP version 1 1..1 not ok 1 subsubtest 1 not ok 2 subtest 2
The output is:
[20:54:12] ============================================================ [20:54:12] ======================= (2 subtests) ======================= [20:54:12] [PASSED] test 1 [20:54:12] [FAILED] test 2 [20:54:12] ==================== [FAILED] subtest 1 ==================== [20:54:12] ======================= (1 subtest) ======================== [20:54:12] [FAILED] subsubtest 1 [20:54:12] ==================== [FAILED] subtest 2 ==================== [20:54:12] ============================================================ [20:54:12] Testing complete. Ran 6 tests: passed: 1, failed: 5
This reports a total of 6 tests, which is not equivalent to the three subtests plus the two suites. I believe this is because the change to bubble_up_test_results below double counts the failed test case.
Historically, the KUnit parser only counts the results of test cases, not the suites. I would like to stay as close to this as possible so as to not inflate existing testing numbers. However, I believe the main concern here is the case where if there is a suite reporting failure but all subtests pass, it will not appear in the summary line. For example,
KTAP version 1 1..1 KTAP version 1 1..1 ok 1 test 1 not ok 1 subtest 1
Reporting: All passing: Tests run: 1, passed: 1
This is absolutely an important edge case to cover. Therefore, we should add 1 failure count to the suite count if the bubbled up results indicate it should instead pass.
Makes sense.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com
tools/testing/kunit/kunit_parser.py | 5 +++++ tools/testing/kunit/kunit_tool_test.py | 3 ++- tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log | 3 +++ 3 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index c176487356e6c94882046b19ea696d750905b8d5..2478beb28fc3db825855ad46200340e884da7df1 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -686,6 +686,11 @@ def bubble_up_test_results(test: Test) -> None: counts.add_status(status) elif test.counts.get_status() == TestStatus.TEST_CRASHED: test.status = TestStatus.TEST_CRASHED
if not test.ok_status():for t in subtests:if not t.ok_status():counts.add_status(t.status)breakHere instead I recommend checking if not test.ok_status() and test.counts.get_status() == TestStatus.SUCCESS and if so counts.add_status(status)
Thanks for the recommendation. I tried this and it works well for this specific testcase, but unfortunately all kinds of othes tests are now broken. I'll look into it some more, but any hints are highly appreciated. It has been a while since I looked at the code.
def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest: bool, printer: Printer) -> Test: """ diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index b74dc05fc2fe5b3ff629172fc7aafeb5c3d29fb3..48a0dd0f9c87caf9f018aade161db90a613fc407 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -170,8 +170,9 @@ class KUnitParserTest(unittest.TestCase): with open(nested_log) as file: result = kunit_parser.parse_run_tests(file.readlines(), stdout) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status)
self.assertEqual(result.counts.failed, 2)
self.assertEqual(result.counts.failed, 3) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[0].status)self.assertEqual(kunit_parser.TestStatus.SUCCESS, result.subtests[0].subtests[0].status) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].status) self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].subtests[0].status)diff --git a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log index 2e528da39ab5b2be0fca6cf9160c10929fba3c9e..5498dfd0b0db24663e1a1e9bf78c587de6746522 100644 --- a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log +++ b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log @@ -1,5 +1,8 @@ KTAP version 1 1..2
- KTAP version 1
- 1..1
ok 1 test 1not ok 1 subtest 1 KTAP version 1 1..1
-- 2.50.0
-- You received this message because you are subscribed to the Google Groups "KUnit Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to kunit-dev+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/kunit-dev/20250626-kunit-kselftests-v4-8-4....
Hi Rae,
On Thu, Jul 03, 2025 at 05:30:02PM +0200, Thomas Weißschuh wrote:
On Tue, Jul 01, 2025 at 05:11:59PM -0400, Rae Moar wrote:
<snip>
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com
tools/testing/kunit/kunit_parser.py | 5 +++++ tools/testing/kunit/kunit_tool_test.py | 3 ++- tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log | 3 +++ 3 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index c176487356e6c94882046b19ea696d750905b8d5..2478beb28fc3db825855ad46200340e884da7df1 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -686,6 +686,11 @@ def bubble_up_test_results(test: Test) -> None: counts.add_status(status) elif test.counts.get_status() == TestStatus.TEST_CRASHED: test.status = TestStatus.TEST_CRASHED
if not test.ok_status():for t in subtests:if not t.ok_status():counts.add_status(t.status)breakHere instead I recommend checking if not test.ok_status() and test.counts.get_status() == TestStatus.SUCCESS and if so counts.add_status(status)
Thanks for the recommendation. I tried this and it works well for this specific testcase, but unfortunately all kinds of othes tests are now broken. I'll look into it some more, but any hints are highly appreciated. It has been a while since I looked at the code.
The following variant passes all tests. What do you think?
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 333cd3a4a56b..5338489dcbe4 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -689,6 +689,9 @@ def bubble_up_test_results(test: Test) -> None: elif test.counts.get_status() == TestStatus.TEST_CRASHED: test.status = TestStatus.TEST_CRASHED
+ if status == TestStatus.FAILURE and test.counts.get_status() == TestStatus.SUCCESS: + counts.add_status(status) +
Skipped tests reported by kselftest.h use a different format than KTAP, there is no explicit test name. Normally the test name is part of the free-form string after the SKIP keyword:
ok 3 # SKIP test: some reason
Extend the parser to handle those correctly. Use the free-form string as test name instead.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com --- tools/testing/kunit/kunit_parser.py | 8 +++++--- tools/testing/kunit/test_data/test_is_test_passed-kselftest.log | 3 ++- 2 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 2478beb28fc3db825855ad46200340e884da7df1..4599d23c79b79f0e219d655c7053c8c3b34f8152 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -352,9 +352,9 @@ def parse_test_plan(lines: LineStream, test: Test) -> bool: lines.pop() return True
-TEST_RESULT = re.compile(r'^\s*(ok|not ok) ([0-9]+) (- )?([^#]*)( # .*)?$') +TEST_RESULT = re.compile(r'^\s*(ok|not ok) ([0-9]+)? ?(- )?([^#]*)( # .*)?$')
-TEST_RESULT_SKIP = re.compile(r'^\s*(ok|not ok) ([0-9]+) (- )?(.*) # SKIP(.*)$') +TEST_RESULT_SKIP = re.compile(r'^\s*(ok|not ok) ([0-9]+)? ?(- )?(.*) # SKIP ?(.*)$')
def peek_test_name_match(lines: LineStream, test: Test) -> bool: """ @@ -379,6 +379,8 @@ def peek_test_name_match(lines: LineStream, test: Test) -> bool: if not match: return False name = match.group(4) + if not name: + return False return name == test.name
def parse_test_result(lines: LineStream, test: Test, @@ -416,7 +418,7 @@ def parse_test_result(lines: LineStream, test: Test,
# Set name of test object if skip_match: - test.name = skip_match.group(4) + test.name = skip_match.group(4) or skip_match.group(5) else: test.name = match.group(4)
diff --git a/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log index 65d3f27feaf22a3f47ed831c4c24f6f11c625a92..30d9ef18bcec177067288d5242771236f29b7d56 100644 --- a/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log +++ b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log @@ -1,5 +1,5 @@ TAP version 13 -1..2 +1..3 # selftests: membarrier: membarrier_test_single_thread # TAP version 13 # 1..2 @@ -12,3 +12,4 @@ ok 1 selftests: membarrier: membarrier_test_single_thread # ok 1 sys_membarrier available # ok 2 sys membarrier invalid command test: command = -1, flags = 0, errno = 22. Failed as expected ok 2 selftests: membarrier: membarrier_test_multi_thread +ok 3 # SKIP selftests: membarrier: membarrier_test_multi_thread
On Thu, Jun 26, 2025 at 2:10 AM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
Skipped tests reported by kselftest.h use a different format than KTAP, there is no explicit test name. Normally the test name is part of the free-form string after the SKIP keyword:
ok 3 # SKIP test: some reasonExtend the parser to handle those correctly. Use the free-form string as test name instead.
Hello!
I am happy to see that the kunit_parser will be used to parse kselftest.h patches!
This patch looks pretty good to me! However, I do have one concern where it does cause the parser to break.
Thanks! -Rae
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com
tools/testing/kunit/kunit_parser.py | 8 +++++--- tools/testing/kunit/test_data/test_is_test_passed-kselftest.log | 3 ++- 2 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 2478beb28fc3db825855ad46200340e884da7df1..4599d23c79b79f0e219d655c7053c8c3b34f8152 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -352,9 +352,9 @@ def parse_test_plan(lines: LineStream, test: Test) -> bool: lines.pop() return True
-TEST_RESULT = re.compile(r'^\s*(ok|not ok) ([0-9]+) (- )?([^#]*)( # .*)?$') +TEST_RESULT = re.compile(r'^\s*(ok|not ok) ([0-9]+)? ?(- )?([^#]*)( # .*)?$')
-TEST_RESULT_SKIP = re.compile(r'^\s*(ok|not ok) ([0-9]+) (- )?(.*) # SKIP(.*)$') +TEST_RESULT_SKIP = re.compile(r'^\s*(ok|not ok) ([0-9]+)? ?(- )?(.*) # SKIP ?(.*)$')
My concern with this patch is that this makes the test number optional ([0-9]+) -> ([0-9]+)? for both the test result regex matches. But if I do pass in KTAP containing a line where the test number is missing, it causes the parser to crash because other lines of the code rely on the match with the test number.
I would prefer if we keep the test number as required. This is integral to the KTAP specification and seems to be largely shared by different frameworks test output.
def peek_test_name_match(lines: LineStream, test: Test) -> bool: """ @@ -379,6 +379,8 @@ def peek_test_name_match(lines: LineStream, test: Test) -> bool: if not match: return False name = match.group(4)
if not name:return False return name == test.namedef parse_test_result(lines: LineStream, test: Test, @@ -416,7 +418,7 @@ def parse_test_result(lines: LineStream, test: Test,
# Set name of test object if skip_match:
test.name = skip_match.group(4)
test.name = skip_match.group(4) or skip_match.group(5) else: test.name = match.group(4)diff --git a/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log index 65d3f27feaf22a3f47ed831c4c24f6f11c625a92..30d9ef18bcec177067288d5242771236f29b7d56 100644 --- a/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log +++ b/tools/testing/kunit/test_data/test_is_test_passed-kselftest.log @@ -1,5 +1,5 @@ TAP version 13 -1..2 +1..3 # selftests: membarrier: membarrier_test_single_thread # TAP version 13 # 1..2 @@ -12,3 +12,4 @@ ok 1 selftests: membarrier: membarrier_test_single_thread # ok 1 sys_membarrier available # ok 2 sys membarrier invalid command test: command = -1, flags = 0, errno = 22. Failed as expected ok 2 selftests: membarrier: membarrier_test_multi_thread +ok 3 # SKIP selftests: membarrier: membarrier_test_multi_thread
-- 2.50.0
On Tue, Jul 01, 2025 at 05:22:07PM -0400, Rae Moar wrote:
On Thu, Jun 26, 2025 at 2:10 AM Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
<snip>
-TEST_RESULT = re.compile(r'^\s*(ok|not ok) ([0-9]+) (- )?([^#]*)( # .*)?$') +TEST_RESULT = re.compile(r'^\s*(ok|not ok) ([0-9]+)? ?(- )?([^#]*)( # .*)?$')
-TEST_RESULT_SKIP = re.compile(r'^\s*(ok|not ok) ([0-9]+) (- )?(.*) # SKIP(.*)$') +TEST_RESULT_SKIP = re.compile(r'^\s*(ok|not ok) ([0-9]+)? ?(- )?(.*) # SKIP ?(.*)$')
My concern with this patch is that this makes the test number optional ([0-9]+) -> ([0-9]+)? for both the test result regex matches. But if I do pass in KTAP containing a line where the test number is missing, it causes the parser to crash because other lines of the code rely on the match with the test number.
I would prefer if we keep the test number as required. This is integral to the KTAP specification and seems to be largely shared by different frameworks test output.
This part of the patch was actually unnecessary, so I dropped it. Good catch, thanks.
<snip>
An upcoming change will add 'userprogs' to the kunit subdirectory. For kbuild to properly clean up these build artifacts the subdirectory needs to be always processed.
Pushing the special logic for hook.o into the kunit Makefile also makes the logic easier to understand.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com --- lib/Makefile | 4 ---- lib/kunit/Makefile | 2 +- 2 files changed, 1 insertion(+), 5 deletions(-)
diff --git a/lib/Makefile b/lib/Makefile index c38582f187dd81916113319072e5cfef26f26c84..698566135091cc3bf0054f1954b434dc3325364a 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -108,11 +108,7 @@ test_fpu-y := test_fpu_glue.o test_fpu_impl.o CFLAGS_test_fpu_impl.o += $(CC_FLAGS_FPU) CFLAGS_REMOVE_test_fpu_impl.o += $(CC_FLAGS_NO_FPU)
-# Some KUnit files (hooks.o) need to be built-in even when KUnit is a module, -# so we can't just use obj-$(CONFIG_KUNIT). -ifdef CONFIG_KUNIT obj-y += kunit/ -endif
ifeq ($(CONFIG_DEBUG_KOBJECT),y) CFLAGS_kobject.o += -DDEBUG diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile index 5aa51978e456ab3bb60c12071a26cf2bdcb1b508..656f1fa35abcc635e67d5b4cb1bc586b48415ac5 100644 --- a/lib/kunit/Makefile +++ b/lib/kunit/Makefile @@ -17,7 +17,7 @@ kunit-objs += debugfs.o endif
# KUnit 'hooks' are built-in even when KUnit is built as a module. -obj-y += hooks.o +obj-$(if $(CONFIG_KUNIT),y) += hooks.o
obj-$(CONFIG_KUNIT_TEST) += kunit-test.o obj-$(CONFIG_KUNIT_TEST) += platform-test.o
The upcoming kunit UAPI framework will run userspace executables as part of kunit. These may use the LSX or LASX instructions.
Make sure the kunit kernel can handle these instructions.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com --- tools/testing/kunit/qemu_configs/loongarch.py | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/tools/testing/kunit/qemu_configs/loongarch.py b/tools/testing/kunit/qemu_configs/loongarch.py index a92422967d1da9f1658ef1e80d0d7365ddbae307..1dba755284f11ffc94d8946105b0cfa49cb6f604 100644 --- a/tools/testing/kunit/qemu_configs/loongarch.py +++ b/tools/testing/kunit/qemu_configs/loongarch.py @@ -11,6 +11,8 @@ CONFIG_PVPANIC_PCI=y CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_OF_PLATFORM=y +CONFIG_CPU_HAS_LSX=y +CONFIG_CPU_HAS_LASX=y ''', qemu_arch='loongarch64', kernel_path='arch/loongarch/boot/vmlinux.elf',
Enable running UAPI tests as part of kunit. The selftests are embedded into the kernel image and their output is forwarded to kunit for unified reporting.
The implementation reuses parts of usermode drivers and usermode helpers. However these frameworks are not used directly as they make it impossible to retrieve a thread's exit code.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
--- To: Kees Cook kees@kernel.org To: Alexander Viro viro@zeniv.linux.org.uk To: Christian Brauner brauner@kernel.org To: Jan Kara jack@suse.cz Cc: linux-mm@kvack.org Cc: linux-fsdevel@vger.kernel.org --- Documentation/dev-tools/kunit/api/index.rst | 5 + Documentation/dev-tools/kunit/api/uapi.rst | 14 ++ MAINTAINERS | 6 + include/kunit/uapi.h | 77 +++++++ lib/kunit/Kconfig | 14 ++ lib/kunit/Makefile | 2 + lib/kunit/kunit-uapi.c | 300 ++++++++++++++++++++++++++++ 7 files changed, 418 insertions(+)
diff --git a/Documentation/dev-tools/kunit/api/index.rst b/Documentation/dev-tools/kunit/api/index.rst index 5cdb552a0808f25baeff5e47a9227b7b62c69e40..34d8fee9a97059d6da919a6fb1a7e359b5e0beef 100644 --- a/Documentation/dev-tools/kunit/api/index.rst +++ b/Documentation/dev-tools/kunit/api/index.rst @@ -9,6 +9,7 @@ API Reference test resource functionredirection + uapi clk of platformdevice @@ -32,6 +33,10 @@ Documentation/dev-tools/kunit/api/functionredirection.rst
- Documents the KUnit Function Redirection API
+Documentation/dev-tools/kunit/api/uapi.rst + + - Documents the KUnit Userspace testing API + Driver KUnit API ================
diff --git a/Documentation/dev-tools/kunit/api/uapi.rst b/Documentation/dev-tools/kunit/api/uapi.rst new file mode 100644 index 0000000000000000000000000000000000000000..1f01b5c6c9db42f603f6507f2f33ef388f5b91d7 --- /dev/null +++ b/Documentation/dev-tools/kunit/api/uapi.rst @@ -0,0 +1,14 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================== +Userspace Test API +================== + +This file documents all of the userspace testing API. +Userspace tests are built as :ref:`kbuild userprogs <kbuild_userprogs>`, +linked statically and without any external dependencies. + +For the widest platform compatibility they should use nolibc, as provided by `init/Makefile.nolibc`. + +.. kernel-doc:: include/kunit/uapi.h + :internal: diff --git a/MAINTAINERS b/MAINTAINERS index e806158cc6798cf97a4aab58c038fb5351d469aa..0f60501c6de570723123b24eb930d15f1bd956eb 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13540,6 +13540,12 @@ S: Maintained F: Documentation/devicetree/bindings/leds/backlight/kinetic,ktz8866.yaml F: drivers/video/backlight/ktz8866.c
+KUNIT UAPI TESTING FRAMEWORK (in addition to KERNEL UNIT TESTING FRAMEWORK) +M: Thomas Weißschuh thomas.weissschuh@linutronix.de +S: Maintained +F: include/kunit/uapi.h +F: lib/kunit/kunit-uapi.c + KVM PARAVIRT (KVM/paravirt) M: Paolo Bonzini pbonzini@redhat.com R: Vitaly Kuznetsov vkuznets@redhat.com diff --git a/include/kunit/uapi.h b/include/kunit/uapi.h new file mode 100644 index 0000000000000000000000000000000000000000..a5c923f5d82a91e0acd9dc17369f84f00b7d342f --- /dev/null +++ b/include/kunit/uapi.h @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KUnit Userspace testing API. + * + * Copyright (C) 2025, Linutronix GmbH. + * Author: Thomas Weißschuh thomas.weissschuh@linutronix.de + */ + +#ifndef _KUNIT_UAPI_H +#define _KUNIT_UAPI_H + +#include <linux/types.h> + +struct kunit; + +/** + * struct kunit_uapi_blob - Blob embedded build artifact + * @path: Path of the embedded artifact. + * @data: Start of the embedded data in memory. + * @end: End of the embedded data in memory. + */ +struct kunit_uapi_blob { + const char *const path; + const u8 *data; + const u8 *end; +}; + +#if IS_ENABLED(CONFIG_KUNIT_UAPI) + +/** + * KUNIT_UAPI_EMBED_BLOB() - Embed another build artifact into the kernel + * @_name: The name of symbol under which the artifact is embedded. + * @_path: Path to the artifact on disk. + * + * Embeds a build artifact like a userspace executable into the kernel or current module. + * The build artifact is read from disk and needs to be already built. + */ +#define KUNIT_UAPI_EMBED_BLOB(_name, _path) \ + asm ( \ + " .pushsection .rodata, "a" \n" \ + " .global " __stringify(CONCATENATE(_name, _data)) " \n" \ + __stringify(CONCATENATE(_name, _data)) ": \n" \ + " .incbin " __stringify(_path) " \n" \ + " .size " __stringify(CONCATENATE(_name, _data)) ", " \ + ". - " __stringify(CONCATENATE(_name, _data)) " \n" \ + " .global " __stringify(CONCATENATE(_name, _end)) " \n" \ + __stringify(CONCATENATE(_name, _end)) ": \n" \ + " .popsection \n" \ + ); \ + \ + extern const char CONCATENATE(_name, _data)[]; \ + extern const char CONCATENATE(_name, _end)[]; \ + \ + static const struct kunit_uapi_blob _name = { \ + .path = _path, \ + .data = CONCATENATE(_name, _data), \ + .end = CONCATENATE(_name, _end), \ + } \ + +#else /* !CONFIG_KUNIT_UAPI */ + +/* Unresolved external reference, to be optimized away */ +#define KUNIT_UAPI_EMBED_BLOB(_name, _path) \ + extern const struct kunit_uapi_blob _name + +#endif /* CONFIG_KUNIT_UAPI */ + +/** + * kunit_uapi_run_kselftest() - Run a userspace kselftest as part of kunit + * @test: The test context object. + * @executable: kselftest executable to run + * + * Runs the kselftest and forwards its TAP output and exit status to kunit. + */ +void kunit_uapi_run_kselftest(struct kunit *test, const struct kunit_uapi_blob *executable); + +#endif /* _KUNIT_UAPI_H */ diff --git a/lib/kunit/Kconfig b/lib/kunit/Kconfig index a97897edd9642f3e5df7fdd9dee26ee5cf00d6a4..c88e6cf5610bc8246c94d16b9025461f0cd3b2d2 100644 --- a/lib/kunit/Kconfig +++ b/lib/kunit/Kconfig @@ -93,4 +93,18 @@ config KUNIT_AUTORUN_ENABLED In most cases this should be left as Y. Only if additional opt-in behavior is needed should this be set to N.
+config KUNIT_UAPI + tristate "KUnit UAPI testing framework" + depends on KUNIT + depends on CC_CAN_LINK_STATIC || ARCH_HAS_NOLIBC + depends on !LTO_CLANG # https://github.com/llvm/llvm-project/issues/112920 + select HEADERS_INSTALL + default KUNIT + help + Enables support for building and running userspace selftests as part of kunit. + These tests should be statically linked and use kselftest.h or kselftest_harness.h + for status reporting. + + In most cases this should be left as Y. + endif # KUNIT diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile index 656f1fa35abcc635e67d5b4cb1bc586b48415ac5..6059621a2d32c8e7384acda59793f05826af8c81 100644 --- a/lib/kunit/Makefile +++ b/lib/kunit/Makefile @@ -12,6 +12,8 @@ kunit-objs += test.o \ device.o \ platform.o
+obj-$(CONFIG_KUNIT_UAPI) += kunit-uapi.o + ifeq ($(CONFIG_KUNIT_DEBUGFS),y) kunit-objs += debugfs.o endif diff --git a/lib/kunit/kunit-uapi.c b/lib/kunit/kunit-uapi.c new file mode 100644 index 0000000000000000000000000000000000000000..cfe8440e16fde942a5f0fa7ac9d8ab90a737215b --- /dev/null +++ b/lib/kunit/kunit-uapi.c @@ -0,0 +1,300 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit Userspace testing API. + * + * Copyright (C) 2025, Linutronix GmbH. + * Author: Thomas Weißschuh thomas.weissschuh@linutronix.de + */ + +#include <linux/binfmts.h> +#include <linux/export.h> +#include <linux/file.h> +#include <linux/fs.h> +#include <linux/fs_struct.h> +#include <linux/pid.h> +#include <linux/pipe_fs_i.h> +#include <linux/sched/task.h> +#include <linux/types.h> + +#include <kunit/test.h> +#include <kunit/uapi.h> + +#define KSFT_PASS 0 +#define KSFT_FAIL 1 +#define KSFT_XFAIL 2 +#define KSFT_XPASS 3 +#define KSFT_SKIP 4 + +static struct vfsmount *kunit_uapi_mount_ramfs(void) +{ + struct file_system_type *type; + struct vfsmount *mnt; + + type = get_fs_type("ramfs"); + if (!type) + return ERR_PTR(-ENODEV); + + /* FIXME + * The mount setup is supposed to look like this: + * kunit_uapi_mount_ramfs() sets up a private mount, + * with nothing visible except the new tmpfs. + * Then each executable execution gets a new namespace on top of that + * on which it can mount whatever it needs. + * However I didn't manage to set this up, so keep everything simple + * for now and let somebody familiar with the VFS figure this out. + */ + + mnt = kern_mount(type); + put_filesystem(type); + + return mnt; +} + +static int kunit_uapi_write_file(struct vfsmount *mnt, const char *name, mode_t mode, + const u8 *data, size_t size) +{ + struct file *file; + ssize_t written; + + file = file_open_root_mnt(mnt, name, O_CREAT | O_WRONLY, mode); + if (IS_ERR(file)) + return PTR_ERR(file); + + written = kernel_write(file, data, size, NULL); + filp_close(file, NULL); + if (written != size) { + if (written >= 0) + return -ENOMEM; + return written; + } + + /* Flush delayed fput so exec can open the file read-only */ + flush_delayed_fput(); + + return 0; +} + +static int kunit_uapi_write_executable(struct vfsmount *mnt, + const struct kunit_uapi_blob *executable) +{ + return kunit_uapi_write_file(mnt, kbasename(executable->path), 0755, + executable->data, executable->end - executable->data); +} + +struct kunit_uapi_user_mode_thread_ctx { + const char *executable; + + /* Signals mnt, out, pwd and tgid */ + struct completion setup_done; + struct vfsmount *mnt; + struct file *out; + struct path pwd; + pid_t tgid; + + /* Valid after wait(tgid) */ + int exec_err; +}; + +static int kunit_uapi_user_mode_thread_init(void *data) +{ + struct kunit_uapi_user_mode_thread_ctx *ctx = data; + const char *const argv[] = { + ctx->executable, + NULL + }; + struct file *out[2]; + int err; + + err = create_pipe_files(out, 0); + if (err) + return err; + + /* stdin, use the *write* end to the pipe to have an unreadable input */ + err = replace_fd(0, out[1], 0); + if (err < 0) { + fput(out[0]); + fput(out[1]); + return err; + } + + /* stdout */ + err = replace_fd(1, out[1], 0); + if (err < 0) { + replace_fd(0, NULL, 0); + fput(out[0]); + fput(out[1]); + return err; + } + + /* stderr */ + err = replace_fd(2, out[1], 0); + if (err < 0) { + replace_fd(0, NULL, 0); + replace_fd(1, NULL, 0); + fput(out[0]); + fput(out[1]); + return err; + } + + fput(out[1]); + + ctx->out = out[0]; + ctx->tgid = current->tgid; + + set_fs_pwd(current->fs, &ctx->pwd); + kernel_sigaction(SIGKILL, SIG_DFL); + kernel_sigaction(SIGABRT, SIG_DFL); + + complete(&ctx->setup_done); + ctx->exec_err = kernel_execve(ctx->executable, argv, NULL); + if (!ctx->exec_err) + return 0; + do_exit(0); +} + +static size_t kunit_uapi_printk_subtest_lines(struct kunit *test, char *buf, size_t s) +{ + const char *ptr = buf, *newline; + size_t n; + + while (s) { + newline = strnchr(ptr, s, '\n'); + if (!newline) + break; + + n = newline - ptr + 1; + + kunit_log(KERN_INFO, test, KUNIT_SUBSUBTEST_INDENT "%.*s", (int)n, ptr); + ptr += n; + s -= n; + } + + memmove(buf, ptr, s); + + return s; +} + +static int kunit_uapi_forward_to_printk(struct kunit *test, struct file *output) +{ + /* + * printk() automatically adds a newline after each message. + * Therefore only fully accumulated lines can be forwarded. + * Each line needs to fit into the buffer below. + */ + char buf[512]; + size_t s = 0; + ssize_t n; + + while (1) { + n = kernel_read(output, buf + s, sizeof(buf) - s, NULL); + if (n <= 0) + return n; + s = kunit_uapi_printk_subtest_lines(test, buf, s + n); + } +} + +static void kunit_uapi_kill_pid(pid_t pid) +{ + struct pid *p; + + p = find_get_pid(pid); + kill_pid(p, SIGKILL, 1); + put_pid(p); +} + +static int kunit_uapi_run_executable_in_mount(struct kunit *test, const char *executable, + struct vfsmount *mnt) +{ + struct kunit_uapi_user_mode_thread_ctx ctx = { + .setup_done = COMPLETION_INITIALIZER_ONSTACK(ctx.setup_done), + .executable = executable, + .pwd = { + .mnt = mnt, + .dentry = mnt->mnt_root, + }, + }; + int forward_err, wait_err, ret; + pid_t pid; + + /* If SIGCHLD is ignored do_wait won't populate the status. */ + kernel_sigaction(SIGCHLD, SIG_DFL); + pid = user_mode_thread(kunit_uapi_user_mode_thread_init, &ctx, SIGCHLD); + if (pid < 0) { + kernel_sigaction(SIGCHLD, SIG_IGN); + return pid; + } + + wait_for_completion(&ctx.setup_done); + + forward_err = kunit_uapi_forward_to_printk(test, ctx.out); + if (forward_err) + kunit_uapi_kill_pid(ctx.tgid); + + wait_err = kernel_wait(ctx.tgid, &ret); + + /* Restore default kernel sig handler */ + kernel_sigaction(SIGCHLD, SIG_IGN); + + if (ctx.exec_err) + return ctx.exec_err; + if (forward_err) + return forward_err; + if (wait_err < 0) + return wait_err; + return ret; +} + +static int kunit_uapi_run_executable(struct kunit *test, + const struct kunit_uapi_blob *executable) +{ + const char *exe_name = kbasename(executable->path); + struct vfsmount *mnt; + int err; + + mnt = kunit_uapi_mount_ramfs(); + if (IS_ERR(mnt)) + return PTR_ERR(mnt); + + err = kunit_uapi_write_executable(mnt, executable); + + if (!err) + err = kunit_uapi_run_executable_in_mount(test, exe_name, mnt); + + kern_unmount(mnt); + + return err; +} + +void kunit_uapi_run_kselftest(struct kunit *test, const struct kunit_uapi_blob *executable) +{ + u8 exit_code, exit_signal; + int err; + + err = kunit_uapi_run_executable(test, executable); + if (err < 0) + KUNIT_FAIL_AND_ABORT(test, "Could not run test executable: %pe\n", ERR_PTR(err)); + + exit_code = err >> 8; + exit_signal = err & 0xff; + + if (exit_signal) + KUNIT_FAIL_AND_ABORT(test, "kselftest exited with signal: %d\n", exit_signal); + else if (exit_code == KSFT_PASS) + ; /* Noop */ + else if (exit_code == KSFT_FAIL) + KUNIT_FAIL_AND_ABORT(test, "kselftest exited with code KSFT_FAIL\n"); + else if (exit_code == KSFT_XPASS) + KUNIT_FAIL_AND_ABORT(test, "kselftest exited with code KSFT_XPASS\n"); + else if (exit_code == KSFT_XFAIL) + ; /* Noop */ + else if (exit_code == KSFT_SKIP) + kunit_mark_skipped(test, "kselftest exited with code KSFT_SKIP\n"); + else + KUNIT_FAIL_AND_ABORT(test, "kselftest exited with unknown exit code: %d\n", + exit_code); +} +EXPORT_SYMBOL_GPL(kunit_uapi_run_kselftest); + +MODULE_DESCRIPTION("KUnit UAPI testing framework"); +MODULE_AUTHOR("Thomas Weißschuh <thomas.weissschuh@linutronix.de"); +MODULE_LICENSE("GPL");
Hi Thomas,
I ran into two minor issues trying out the patches, see inline.
On Thu, 2025-06-26 at 08:10 +0200, Thomas Weißschuh wrote:
Enable running UAPI tests as part of kunit. The selftests are embedded into the kernel image and their output is forwarded to kunit for unified reporting.
The implementation reuses parts of usermode drivers and usermode helpers. However these frameworks are not used directly as they make it impossible to retrieve a thread's exit code.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
[SNIP] +/**
- KUNIT_UAPI_EMBED_BLOB() - Embed another build artifact into the kernel
- @_name: The name of symbol under which the artifact is embedded.
- @_path: Path to the artifact on disk.
- Embeds a build artifact like a userspace executable into the kernel or current module.
- The build artifact is read from disk and needs to be already built.
- */
+#define KUNIT_UAPI_EMBED_BLOB(_name, _path) \
- asm ( \
- " .pushsection .rodata, "a" \n" \
- " .global " __stringify(CONCATENATE(_name, _data)) " \n" \
- __stringify(CONCATENATE(_name, _data)) ": \n" \
- " .incbin " __stringify(_path) " \n" \
- " .size " __stringify(CONCATENATE(_name, _data)) ", " \
". - " __stringify(CONCATENATE(_name, _data)) " \n" \- " .global " __stringify(CONCATENATE(_name, _end)) " \n" \
- __stringify(CONCATENATE(_name, _end)) ": \n" \
- " .popsection \n" \
- ); \
\- extern const char CONCATENATE(_name, _data)[]; \
- extern const char CONCATENATE(_name, _end)[]; \
\- static const struct kunit_uapi_blob _name = { \
.path = _path, \.data = CONCATENATE(_name, _data), \.end = CONCATENATE(_name, _end), \- } \
For me, the compiler could not find the files for the ".incbin" unless I added an include path. i.e. adding ccflags-y := -I$(obj) to lib/kunit/Makefile fixed the problem for me.
[SNIP] +static int kunit_uapi_run_executable_in_mount(struct kunit *test, const char *executable,
struct vfsmount *mnt)+{
- struct kunit_uapi_user_mode_thread_ctx ctx = {
.setup_done = COMPLETION_INITIALIZER_ONSTACK(ctx.setup_done),.executable = executable,.pwd = {.mnt = mnt,.dentry = mnt->mnt_root,},- };
- int forward_err, wait_err, ret;
ret needs to be initialized to zero here as the kernel_wait function will only set "ret" if wo.wo_stat is non-zero.
Benjamin
[SNIP]
Hi Benjamin,
On Thu, Jun 26, 2025 at 08:11:17PM +0200, Benjamin Berg wrote:
I ran into two minor issues trying out the patches, see inline.
Thanks for testing the series.
On Thu, 2025-06-26 at 08:10 +0200, Thomas Weißschuh wrote:
Enable running UAPI tests as part of kunit. The selftests are embedded into the kernel image and their output is forwarded to kunit for unified reporting.
The implementation reuses parts of usermode drivers and usermode helpers. However these frameworks are not used directly as they make it impossible to retrieve a thread's exit code.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
[SNIP] +/**
- KUNIT_UAPI_EMBED_BLOB() - Embed another build artifact into the kernel
- @_name: The name of symbol under which the artifact is embedded.
- @_path: Path to the artifact on disk.
- Embeds a build artifact like a userspace executable into the kernel or current module.
- The build artifact is read from disk and needs to be already built.
- */
+#define KUNIT_UAPI_EMBED_BLOB(_name, _path) \
- asm ( \
- " .pushsection .rodata, "a" \n" \
- " .global " __stringify(CONCATENATE(_name, _data)) " \n" \
- __stringify(CONCATENATE(_name, _data)) ": \n" \
- " .incbin " __stringify(_path) " \n" \
- " .size " __stringify(CONCATENATE(_name, _data)) ", " \
". - " __stringify(CONCATENATE(_name, _data)) " \n" \- " .global " __stringify(CONCATENATE(_name, _end)) " \n" \
- __stringify(CONCATENATE(_name, _end)) ": \n" \
- " .popsection \n" \
- ); \
\- extern const char CONCATENATE(_name, _data)[]; \
- extern const char CONCATENATE(_name, _end)[]; \
\- static const struct kunit_uapi_blob _name = { \
.path = _path, \.data = CONCATENATE(_name, _data), \.end = CONCATENATE(_name, _end), \- } \
For me, the compiler could not find the files for the ".incbin" unless I added an include path. i.e. adding ccflags-y := -I$(obj) to lib/kunit/Makefile fixed the problem for me.
Can you share some more details on your build setup? This worked for me as-is and also passed 0day build testing.
[SNIP] +static int kunit_uapi_run_executable_in_mount(struct kunit *test, const char *executable,
struct vfsmount *mnt)+{
- struct kunit_uapi_user_mode_thread_ctx ctx = {
.setup_done = COMPLETION_INITIALIZER_ONSTACK(ctx.setup_done),.executable = executable,.pwd = {.mnt = mnt,.dentry = mnt->mnt_root,},- };
- int forward_err, wait_err, ret;
ret needs to be initialized to zero here as the kernel_wait function will only set "ret" if wo.wo_stat is non-zero.
Ack.
[SNIP]
Thomas
Hi,
On Fri, 2025-06-27 at 06:20 +0200, Thomas Weißschuh wrote:
On Thu, Jun 26, 2025 at 08:11:17PM +0200, Benjamin Berg wrote:
I ran into two minor issues trying out the patches, see inline.
Thanks for testing the series.
On Thu, 2025-06-26 at 08:10 +0200, Thomas Weißschuh wrote:
Enable running UAPI tests as part of kunit. The selftests are embedded into the kernel image and their output is forwarded to kunit for unified reporting.
The implementation reuses parts of usermode drivers and usermode helpers. However these frameworks are not used directly as they make it impossible to retrieve a thread's exit code.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
[SNIP] +/**
- KUNIT_UAPI_EMBED_BLOB() - Embed another build artifact into the kernel
- @_name: The name of symbol under which the artifact is embedded.
- @_path: Path to the artifact on disk.
- Embeds a build artifact like a userspace executable into the kernel or current module.
- The build artifact is read from disk and needs to be already built.
- */
+#define KUNIT_UAPI_EMBED_BLOB(_name, _path) \
- asm ( \
- " .pushsection .rodata, "a" \n" \
- " .global " __stringify(CONCATENATE(_name, _data)) " \n" \
- __stringify(CONCATENATE(_name, _data)) ": \n" \
- " .incbin " __stringify(_path) " \n" \
- " .size " __stringify(CONCATENATE(_name, _data)) ", " \
". - " __stringify(CONCATENATE(_name, _data)) " \n" \- " .global " __stringify(CONCATENATE(_name, _end)) " \n" \
- __stringify(CONCATENATE(_name, _end)) ": \n" \
- " .popsection \n" \
- ); \
\- extern const char CONCATENATE(_name, _data)[]; \
- extern const char CONCATENATE(_name, _end)[]; \
\- static const struct kunit_uapi_blob _name = { \
.path = _path, \.data = CONCATENATE(_name, _data), \.end = CONCATENATE(_name, _end), \- } \
For me, the compiler could not find the files for the ".incbin" unless I added an include path. i.e. adding ccflags-y := -I$(obj) to lib/kunit/Makefile fixed the problem for me.
Can you share some more details on your build setup? This worked for me as-is and also passed 0day build testing.
Funny, I ran this on a Fedora 41 with gcc --version saying gcc (GCC) 14.3.1 20250523 (Red Hat 14.3.1-1)
I tried both 32 and 64 bit builds for ARCH=um.
Attaching my current kernel configuration and the last few lines of a V=1 build.
The kernel I used is a bit newer than what you had. Applied on top of ee88bddf7f2f5d1f1da87dd7bedc734048b70e88 (bpf-fixes merge).
Benjamin
[SNIP] +static int kunit_uapi_run_executable_in_mount(struct kunit *test, const char *executable,
struct vfsmount *mnt)+{
- struct kunit_uapi_user_mode_thread_ctx ctx = {
.setup_done = COMPLETION_INITIALIZER_ONSTACK(ctx.setup_done),.executable = executable,.pwd = {.mnt = mnt,.dentry = mnt->mnt_root,},- };
- int forward_err, wait_err, ret;
ret needs to be initialized to zero here as the kernel_wait function will only set "ret" if wo.wo_stat is non-zero.
Ack.
[SNIP]
Thomas
On Fri, Jun 27, 2025 at 08:58:11AM +0200, Benjamin Berg wrote:
On Fri, 2025-06-27 at 06:20 +0200, Thomas Weißschuh wrote:
On Thu, Jun 26, 2025 at 08:11:17PM +0200, Benjamin Berg wrote:
I ran into two minor issues trying out the patches, see inline.
Thanks for testing the series.
On Thu, 2025-06-26 at 08:10 +0200, Thomas Weißschuh wrote:
Enable running UAPI tests as part of kunit. The selftests are embedded into the kernel image and their output is forwarded to kunit for unified reporting.
The implementation reuses parts of usermode drivers and usermode helpers. However these frameworks are not used directly as they make it impossible to retrieve a thread's exit code.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
[SNIP] +/**
- KUNIT_UAPI_EMBED_BLOB() - Embed another build artifact into the kernel
- @_name: The name of symbol under which the artifact is embedded.
- @_path: Path to the artifact on disk.
- Embeds a build artifact like a userspace executable into the kernel or current module.
- The build artifact is read from disk and needs to be already built.
- */
+#define KUNIT_UAPI_EMBED_BLOB(_name, _path) \
- asm ( \
- " .pushsection .rodata, "a" \n" \
- " .global " __stringify(CONCATENATE(_name, _data)) " \n" \
- __stringify(CONCATENATE(_name, _data)) ": \n" \
- " .incbin " __stringify(_path) " \n" \
- " .size " __stringify(CONCATENATE(_name, _data)) ", " \
". - " __stringify(CONCATENATE(_name, _data)) " \n" \- " .global " __stringify(CONCATENATE(_name, _end)) " \n" \
- __stringify(CONCATENATE(_name, _end)) ": \n" \
- " .popsection \n" \
- ); \
\- extern const char CONCATENATE(_name, _data)[]; \
- extern const char CONCATENATE(_name, _end)[]; \
\- static const struct kunit_uapi_blob _name = { \
.path = _path, \.data = CONCATENATE(_name, _data), \.end = CONCATENATE(_name, _end), \- } \
For me, the compiler could not find the files for the ".incbin" unless I added an include path. i.e. adding ccflags-y := -I$(obj) to lib/kunit/Makefile fixed the problem for me.
Can you share some more details on your build setup? This worked for me as-is and also passed 0day build testing.
Funny, I ran this on a Fedora 41 with gcc --version saying gcc (GCC) 14.3.1 20250523 (Red Hat 14.3.1-1)
I tried both 32 and 64 bit builds for ARCH=um.
Attaching my current kernel configuration and the last few lines of a V=1 build.
The kernel I used is a bit newer than what you had. Applied on top of ee88bddf7f2f5d1f1da87dd7bedc734048b70e88 (bpf-fixes merge).
So this happens because you are building inside the source tree. scripts/Makefile.lib has this block:
# $(src) for including checkin headers from generated source files # $(obj) for including generated headers from checkin source files ifdef building_out_of_srctree _c_flags += $(addprefix -I, $(src) $(obj)) _a_flags += $(addprefix -I, $(src) $(obj)) _cpp_flags += $(addprefix -I, $(src) $(obj)) endif
Apparently GNU/clang assemblers don't look for .incbin/.include files next to the including files [0]. In contrast, the ARM compiler does (at least according to its docs) [1].
Maybe we can work around this in the macro, but I assume this will become even uglier. So for the next revision I'll use your proposal of explicit cflags. Or if Masahiro prefers to have a more global and generic solution, we can do that.
[0] https://sourceware.org/binutils/docs/as/I.html [1] https://developer.arm.com/documentation/dui0801/l/Directives-Reference/INCBI...
Thomas
Extend the example to show how to run a userspace executable.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com --- MAINTAINERS | 1 + lib/kunit/Makefile | 9 +++++++++ lib/kunit/kunit-example-test.c | 15 +++++++++++++++ lib/kunit/kunit-example-uapi.c | 22 ++++++++++++++++++++++ 4 files changed, 47 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS index 0f60501c6de570723123b24eb930d15f1bd956eb..b1405f0a0e638d1654d9dc9e51d784ddc838cf5b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13544,6 +13544,7 @@ KUNIT UAPI TESTING FRAMEWORK (in addition to KERNEL UNIT TESTING FRAMEWORK) M: Thomas Weißschuh thomas.weissschuh@linutronix.de S: Maintained F: include/kunit/uapi.h +F: lib/kunit/kunit-example-uapi.c F: lib/kunit/kunit-uapi.c
KVM PARAVIRT (KVM/paravirt) diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile index 6059621a2d32c8e7384acda59793f05826af8c81..1bba7965613e36e26939d6b31e1d65acf5bad0dc 100644 --- a/lib/kunit/Makefile +++ b/lib/kunit/Makefile @@ -1,3 +1,5 @@ +include $(srctree)/init/Makefile.nolibc + obj-$(CONFIG_KUNIT) += kunit.o
kunit-objs += test.o \ @@ -31,3 +33,10 @@ obj-$(CONFIG_KUNIT_TEST) += assert_test.o endif
obj-$(CONFIG_KUNIT_EXAMPLE_TEST) += kunit-example-test.o + +userprogs += kunit-example-uapi +kunit-example-uapi-userccflags := -static $(NOLIBC_USERCFLAGS) + +ifdef CONFIG_KUNIT_UAPI +$(obj)/kunit-example-test.o: $(obj)/kunit-example-uapi +endif diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c index 3056d6bc705d0a8f196f0f4412e679dbb0e03114..0c5e1e59f9358b84aee2621e342d824a2f99f9aa 100644 --- a/lib/kunit/kunit-example-test.c +++ b/lib/kunit/kunit-example-test.c @@ -8,6 +8,7 @@
#include <kunit/test.h> #include <kunit/static_stub.h> +#include <kunit/uapi.h>
/* * This is the most fundamental element of KUnit, the test case. A test case @@ -277,6 +278,19 @@ static void example_slow_test(struct kunit *test) KUNIT_EXPECT_EQ(test, 1 + 1, 2); }
+/* + * This test shows the usage of UAPI tests. + */ +static void example_uapi_test(struct kunit *test) +{ + KUNIT_UAPI_EMBED_BLOB(kunit_example_uapi, "kunit-example-uapi"); + + if (IS_ENABLED(CONFIG_KUNIT_UAPI)) + kunit_uapi_run_kselftest(test, &kunit_example_uapi); + else + kunit_skip(test, "CONFIG_KUNIT_UAPI is not enabled"); +} + /* * Here we make a list of all the test cases we want to add to the test suite * below. @@ -297,6 +311,7 @@ static struct kunit_case example_test_cases[] = { KUNIT_CASE(example_priv_test), KUNIT_CASE_PARAM(example_params_test, example_gen_params), KUNIT_CASE_SLOW(example_slow_test), + KUNIT_CASE(example_uapi_test), {} };
diff --git a/lib/kunit/kunit-example-uapi.c b/lib/kunit/kunit-example-uapi.c new file mode 100644 index 0000000000000000000000000000000000000000..4ce657050dd4a576632a41ca0309c4cb5134ce14 --- /dev/null +++ b/lib/kunit/kunit-example-uapi.c @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit Userspace example test. + * + * Copyright (C) 2025, Linutronix GmbH. + * Author: Thomas Weißschuh thomas.weissschuh@linutronix.de + * + * This is *userspace* code. + */ + +#include "../../tools/testing/selftests/kselftest.h" + +int main(void) +{ + ksft_print_header(); + ksft_set_plan(4); + ksft_test_result_pass("userspace test 1\n"); + ksft_test_result_pass("userspace test 2\n"); + ksft_test_result_skip("userspace test 3: some reason\n"); + ksft_test_result_pass("userspace test 4\n"); + ksft_finished(); +}
UAPI selftests may expect a "normal" userspace environment. For example the normal kernel API pseudo-filesystems should be mounted. This could be done from kernel code but it is non-idiomatic.
Add a preinit userspace executable which performs these setup steps before running the final test executable. This preinit executable is only ever run from the kernel. Give it access to autoconf.h and kconfig.h to adapt itself to the tested kernel.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com --- MAINTAINERS | 1 + lib/kunit/Makefile | 6 +++++ lib/kunit/kunit-uapi.c | 9 +++++-- lib/kunit/uapi-preinit.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS index b1405f0a0e638d1654d9dc9e51d784ddc838cf5b..e81dfa180ab374ef91c7a45e546e6e9a8f454fa7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13546,6 +13546,7 @@ S: Maintained F: include/kunit/uapi.h F: lib/kunit/kunit-example-uapi.c F: lib/kunit/kunit-uapi.c +F: lib/kunit/uapi-preinit.c
KVM PARAVIRT (KVM/paravirt) M: Paolo Bonzini pbonzini@redhat.com diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile index 1bba7965613e36e26939d6b31e1d65acf5bad0dc..b50f3bc8bc7f3ade03be4900d9163d7a0d96863c 100644 --- a/lib/kunit/Makefile +++ b/lib/kunit/Makefile @@ -14,8 +14,14 @@ kunit-objs += test.o \ device.o \ platform.o
+userprogs += uapi-preinit +uapi-preinit-userccflags += -static $(NOLIBC_USERCFLAGS) \ + -include include/generated/autoconf.h \ + -include $(srctree)/tools/include/linux/kconfig.h obj-$(CONFIG_KUNIT_UAPI) += kunit-uapi.o
+$(obj)/kunit-uapi.o: $(obj)/uapi-preinit + ifeq ($(CONFIG_KUNIT_DEBUGFS),y) kunit-objs += debugfs.o endif diff --git a/lib/kunit/kunit-uapi.c b/lib/kunit/kunit-uapi.c index cfe8440e16fde942a5f0fa7ac9d8ab90a737215b..7c87605b9ded9dbeb3968af8a8f4650ab5938887 100644 --- a/lib/kunit/kunit-uapi.c +++ b/lib/kunit/kunit-uapi.c @@ -25,6 +25,8 @@ #define KSFT_XPASS 3 #define KSFT_SKIP 4
+KUNIT_UAPI_EMBED_BLOB(kunit_uapi_preinit, "uapi-preinit"); + static struct vfsmount *kunit_uapi_mount_ramfs(void) { struct file_system_type *type; @@ -146,7 +148,7 @@ static int kunit_uapi_user_mode_thread_init(void *data) kernel_sigaction(SIGABRT, SIG_DFL);
complete(&ctx->setup_done); - ctx->exec_err = kernel_execve(ctx->executable, argv, NULL); + ctx->exec_err = kernel_execve(kbasename(kunit_uapi_preinit.path), argv, NULL); if (!ctx->exec_err) return 0; do_exit(0); @@ -255,7 +257,10 @@ static int kunit_uapi_run_executable(struct kunit *test, if (IS_ERR(mnt)) return PTR_ERR(mnt);
- err = kunit_uapi_write_executable(mnt, executable); + err = kunit_uapi_write_executable(mnt, &kunit_uapi_preinit); + + if (!err) + err = kunit_uapi_write_executable(mnt, executable);
if (!err) err = kunit_uapi_run_executable_in_mount(test, exe_name, mnt); diff --git a/lib/kunit/uapi-preinit.c b/lib/kunit/uapi-preinit.c new file mode 100644 index 0000000000000000000000000000000000000000..81182039965a8c93aebb2d5d76f4113bfef277a6 --- /dev/null +++ b/lib/kunit/uapi-preinit.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit Userspace environment setup. + * + * Copyright (C) 2025, Linutronix GmbH. + * Author: Thomas Weißschuh thomas.weissschuh@linutronix.de + * + * This is *userspace* code. + */ + +#include <sys/mount.h> +#include <sys/stat.h> + +#include "../../tools/testing/selftests/kselftest.h" + +static int setup_api_mount(const char *target, const char *fstype) +{ + int ret; + + ret = mkdir(target, 0755); + if (ret && errno != EEXIST) + return -errno; + + ret = mount("none", target, fstype, 0, NULL); + if (ret && errno != EBUSY) + return -errno; + + return 0; +} + +static void exit_failure(const char *stage, int err) +{ + /* If preinit fails synthesize a failed test report. */ + ksft_print_header(); + ksft_set_plan(1); + ksft_test_result_fail("Failed during test setup: %s: %s\n", stage, strerror(-err)); + ksft_finished(); +} + +int main(int argc, char **argv, char **envp) +{ + int ret; + + ret = setup_api_mount("/proc", "proc"); + if (ret) + exit_failure("mount /proc", ret); + + ret = setup_api_mount("/sys", "sysfs"); + if (ret) + exit_failure("mount /sys", ret); + + if (IS_ENABLED(CONFIG_DEVTMPFS)) { + ret = setup_api_mount("/dev", "devtmpfs"); + if (ret) + exit_failure("mount /dev", ret); + } + + ret = execve(argv[0], argv, envp); + if (ret) + exit_failure("execve", ret); + + return 0; +}
On 6/26/25 11:10 AM, Thomas WeiÃschuh wrote:
UAPI selftests may expect a "normal" userspace environment. For example the normal kernel API pseudo-filesystems should be mounted. This could be done from kernel code but it is non-idiomatic.
Add a preinit userspace executable which performs these setup steps before running the final test executable. This preinit executable is only ever run from the kernel. Give it access to autoconf.h and kconfig.h to adapt itself to the tested kernel.
From perspective of kselftests, I've liked the approach. Once we have an automated way to run all the tests by some simple stubs, it would solve the inherent problem of kselftests that it require different config options enabled before tests can run. Hopefully, they would be auto enabled / disabled as kernel config changes.
Acked-by: Muhammad Usama Anjum usama.anjum@collabora.com
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de Reviewed-by: David Gow davidgow@google.com
MAINTAINERS | 1 + lib/kunit/Makefile | 6 +++++ lib/kunit/kunit-uapi.c | 9 +++++-- lib/kunit/uapi-preinit.c | 63 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS index b1405f0a0e638d1654d9dc9e51d784ddc838cf5b..e81dfa180ab374ef91c7a45e546e6e9a8f454fa7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13546,6 +13546,7 @@ S: Maintained F: include/kunit/uapi.h F: lib/kunit/kunit-example-uapi.c F: lib/kunit/kunit-uapi.c +F: lib/kunit/uapi-preinit.c KVM PARAVIRT (KVM/paravirt) M: Paolo Bonzini pbonzini@redhat.com diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile index 1bba7965613e36e26939d6b31e1d65acf5bad0dc..b50f3bc8bc7f3ade03be4900d9163d7a0d96863c 100644 --- a/lib/kunit/Makefile +++ b/lib/kunit/Makefile @@ -14,8 +14,14 @@ kunit-objs += test.o \ device.o \ platform.o +userprogs += uapi-preinit +uapi-preinit-userccflags += -static $(NOLIBC_USERCFLAGS) \
-include include/generated/autoconf.h \-include $(srctree)/tools/include/linux/kconfig.hobj-$(CONFIG_KUNIT_UAPI) += kunit-uapi.o +$(obj)/kunit-uapi.o: $(obj)/uapi-preinit
ifeq ($(CONFIG_KUNIT_DEBUGFS),y) kunit-objs += debugfs.o endif diff --git a/lib/kunit/kunit-uapi.c b/lib/kunit/kunit-uapi.c index cfe8440e16fde942a5f0fa7ac9d8ab90a737215b..7c87605b9ded9dbeb3968af8a8f4650ab5938887 100644 --- a/lib/kunit/kunit-uapi.c +++ b/lib/kunit/kunit-uapi.c @@ -25,6 +25,8 @@ #define KSFT_XPASS 3 #define KSFT_SKIP 4 +KUNIT_UAPI_EMBED_BLOB(kunit_uapi_preinit, "uapi-preinit");
static struct vfsmount *kunit_uapi_mount_ramfs(void) { struct file_system_type *type; @@ -146,7 +148,7 @@ static int kunit_uapi_user_mode_thread_init(void *data) kernel_sigaction(SIGABRT, SIG_DFL); complete(&ctx->setup_done);
- ctx->exec_err = kernel_execve(ctx->executable, argv, NULL);
- ctx->exec_err = kernel_execve(kbasename(kunit_uapi_preinit.path), argv, NULL); if (!ctx->exec_err) return 0; do_exit(0);
@@ -255,7 +257,10 @@ static int kunit_uapi_run_executable(struct kunit *test, if (IS_ERR(mnt)) return PTR_ERR(mnt);
- err = kunit_uapi_write_executable(mnt, executable);
- err = kunit_uapi_write_executable(mnt, &kunit_uapi_preinit);
- if (!err)
err = kunit_uapi_write_executable(mnt, executable);if (!err) err = kunit_uapi_run_executable_in_mount(test, exe_name, mnt); diff --git a/lib/kunit/uapi-preinit.c b/lib/kunit/uapi-preinit.c new file mode 100644 index 0000000000000000000000000000000000000000..81182039965a8c93aebb2d5d76f4113bfef277a6 --- /dev/null +++ b/lib/kunit/uapi-preinit.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0 +/*
- KUnit Userspace environment setup.
- Copyright (C) 2025, Linutronix GmbH.
- Author: Thomas Weißschuh thomas.weissschuh@linutronix.de
- This is *userspace* code.
- */
+#include <sys/mount.h> +#include <sys/stat.h>
+#include "../../tools/testing/selftests/kselftest.h"
+static int setup_api_mount(const char *target, const char *fstype) +{
- int ret;
- ret = mkdir(target, 0755);
- if (ret && errno != EEXIST)
return -errno;- ret = mount("none", target, fstype, 0, NULL);
- if (ret && errno != EBUSY)
return -errno;- return 0;
+}
+static void exit_failure(const char *stage, int err) +{
- /* If preinit fails synthesize a failed test report. */
- ksft_print_header();
- ksft_set_plan(1);
- ksft_test_result_fail("Failed during test setup: %s: %s\n", stage, strerror(-err));
Positive error values are passed to strerror() without the - sign in userspace. Probably - needs to be removed from strerror() here.
- ksft_finished();
+}
+int main(int argc, char **argv, char **envp) +{
- int ret;
- ret = setup_api_mount("/proc", "proc");
- if (ret)
exit_failure("mount /proc", ret);- ret = setup_api_mount("/sys", "sysfs");
- if (ret)
exit_failure("mount /sys", ret);- if (IS_ENABLED(CONFIG_DEVTMPFS)) {
ret = setup_api_mount("/dev", "devtmpfs");if (ret)exit_failure("mount /dev", ret);- }
- ret = execve(argv[0], argv, envp);
- if (ret)
exit_failure("execve", ret);- return 0;
+}
Show that the selftests are executed from a fairly "normal" userspace context.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de --- MAINTAINERS | 1 + lib/kunit/Makefile | 8 +++++++ lib/kunit/kunit-test-uapi.c | 51 +++++++++++++++++++++++++++++++++++++++++++++ lib/kunit/kunit-test.c | 23 +++++++++++++++++++- 4 files changed, 82 insertions(+), 1 deletion(-)
diff --git a/MAINTAINERS b/MAINTAINERS index e81dfa180ab374ef91c7a45e546e6e9a8f454fa7..d86e8e0bfee75acecfeb9569d53ea8ea99727985 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13545,6 +13545,7 @@ M: Thomas Weißschuh thomas.weissschuh@linutronix.de S: Maintained F: include/kunit/uapi.h F: lib/kunit/kunit-example-uapi.c +F: lib/kunit/kunit-test-uapi.c F: lib/kunit/kunit-uapi.c F: lib/kunit/uapi-preinit.c
diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile index b50f3bc8bc7f3ade03be4900d9163d7a0d96863c..25384e5c3c4cd1f92e4ec2c3830e0be1c6732ac1 100644 --- a/lib/kunit/Makefile +++ b/lib/kunit/Makefile @@ -30,6 +30,14 @@ endif obj-$(if $(CONFIG_KUNIT),y) += hooks.o
obj-$(CONFIG_KUNIT_TEST) += kunit-test.o + +userprogs += kunit-test-uapi +kunit-test-uapi-userccflags := -static $(NOLIBC_USERCFLAGS) + +ifdef CONFIG_KUNIT_UAPI +$(obj)/kunit-test.o: $(obj)/kunit-test-uapi +endif + obj-$(CONFIG_KUNIT_TEST) += platform-test.o
# string-stream-test compiles built-in only. diff --git a/lib/kunit/kunit-test-uapi.c b/lib/kunit/kunit-test-uapi.c new file mode 100644 index 0000000000000000000000000000000000000000..ec5395d809ee2a4bd3c47a573a576e2f98ef5c1b --- /dev/null +++ b/lib/kunit/kunit-test-uapi.c @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit Userspace selftest. + * + * Copyright (C) 2025, Linutronix GmbH. + * Author: Thomas Weißschuh thomas.weissschuh@linutronix.de + * + * This is *userspace* code. + */ + +#include <fcntl.h> +#include <unistd.h> +#include <string.h> + +#include "../../tools/testing/selftests/kselftest.h" + +static void test_procfs(void) +{ + char buf[256]; + ssize_t r; + int fd; + + fd = open("/proc/self/comm", O_RDONLY); + if (fd == -1) { + ksft_test_result_fail("procfs: open() failed: %s\n", strerror(errno)); + return; + } + + r = read(fd, buf, sizeof(buf)); + close(fd); + + if (r == -1) { + ksft_test_result_fail("procfs: read() failed: %s\n", strerror(errno)); + return; + } + + if (r != 16 || strncmp("kunit-test-uapi\n", buf, 16) != 0) { + ksft_test_result_fail("procfs: incorrect comm\n"); + return; + } + + ksft_test_result_pass("procfs\n"); +} + +int main(void) +{ + ksft_print_header(); + ksft_set_plan(1); + test_procfs(); + ksft_finished(); +} diff --git a/lib/kunit/kunit-test.c b/lib/kunit/kunit-test.c index d9c781c859fde1f3623eb71b6829e82aa4803762..03972a3cb172d4af61bec17be800ddfb4bbd7268 100644 --- a/lib/kunit/kunit-test.c +++ b/lib/kunit/kunit-test.c @@ -8,6 +8,7 @@ #include "linux/gfp_types.h" #include <kunit/test.h> #include <kunit/test-bug.h> +#include <kunit/uapi.h>
#include <linux/device.h> #include <kunit/device.h> @@ -868,10 +869,30 @@ static struct kunit_suite kunit_current_test_suite = { .test_cases = kunit_current_test_cases, };
+static void kunit_uapi_test(struct kunit *test) +{ + KUNIT_UAPI_EMBED_BLOB(kunit_test_uapi, "kunit-test-uapi"); + + if (IS_ENABLED(CONFIG_KUNIT_UAPI)) + kunit_uapi_run_kselftest(test, &kunit_test_uapi); + else + kunit_skip(test, "CONFIG_KUNIT_UAPI is not enabled"); +} + +static struct kunit_case kunit_uapi_test_cases[] = { + KUNIT_CASE(kunit_uapi_test), + {} +}; + +static struct kunit_suite kunit_uapi_test_suite = { + .name = "kunit_uapi", + .test_cases = kunit_uapi_test_cases, +}; + kunit_test_suites(&kunit_try_catch_test_suite, &kunit_resource_test_suite, &kunit_log_test_suite, &kunit_status_test_suite, &kunit_current_test_suite, &kunit_device_test_suite, - &kunit_fault_test_suite); + &kunit_fault_test_suite, &kunit_uapi_test_suite);
MODULE_DESCRIPTION("KUnit test for core test infrastructure"); MODULE_LICENSE("GPL v2");
Thomas Weißschuh thomas.weissschuh@linutronix.de writes:
This series aims to combine kselftests and kunit, avoiding both their limitations. It works by compiling the userspace kselftests as part of the regular kernel build, embedding them into the kunit kernel or module and executing them from there.
Please forgive the possibly dumb question but ... this series sets up the framework, but doesn't actually integrate the kselftests, right? Will it be necessary to write a little KUnit glue function for each kselftest, or is there some other scheme in mind here?
Thanks,
jon
On Mon, Jul 07, 2025 at 12:18:01PM -0600, Jonathan Corbet wrote:
Thomas Weißschuh thomas.weissschuh@linutronix.de writes:
This series aims to combine kselftests and kunit, avoiding both their limitations. It works by compiling the userspace kselftests as part of the regular kernel build, embedding them into the kunit kernel or module and executing them from there.
Please forgive the possibly dumb question but ... this series sets up the framework, but doesn't actually integrate the kselftests, right?
Correct.
Will it be necessary to write a little KUnit glue function for each kselftest, or is there some other scheme in mind here?
With the current framework it is necessary to write some glue code: * A stub .c file which #includes the existing kselftest source * A kbuild userprog Makefile * A custom KUnit function which calls kunit_uapi_run_kselftest()
A more high-level scheme may come later, but so far I have not worked on that. It would be nice for example to build and run the tests for all ABIs supported by a kernel without a lot of manual code duplication. And maybe have some higher level helpers around declaring the tests.
Thomas
linux-kselftest-mirror@lists.linaro.org
-
Benjamin Berg -
Jonathan Corbet -
Masahiro Yamada -
Muhammad Usama Anjum -
Nicolas Schier -
Rae Moar -
Thomas Weißschuh