February 2020 - Linux-kselftest-mirror

[PATCH v3 0/7] kunit: create a centralized executor to dispatch all KUnit tests

by Brendan Higgins

## TL;DR This patchset adds a centralized executor to dispatch tests rather than relying on late_initcall to schedule each test suite separately along with a couple of new features that depend on it. Also, sorry for the delay in getting this new revision out. I have been really busy for the past couple weeks. ## What am I trying to do? Conceptually, I am trying to provide a mechanism by which test suites can be grouped together so that they can be reasoned about collectively. The last two of three patches in this series add features which depend on this: PATCH 5/7 Prints out a test plan[1] right before KUnit tests are run; this is valuable because it makes it possible for a test harness to detect whether the number of tests run matches the number of tests expected to be run, ensuring that no tests silently failed. The test plan includes a count of tests that will run. With the centralized executor, the tests are located in a single data structure and thus can be counted. PATCH 6/7 Add a new kernel command-line option which allows the user to specify that the kernel poweroff, halt, or reboot after completing all KUnit tests; this is very handy for running KUnit tests on UML or a VM so that the UML/VM process exits cleanly immediately after running all tests without needing a special initramfs. The centralized executor provides a definitive point when all tests have completed and the poweroff, halt, or reboot could occur. In addition, by dispatching tests from a single location, we can guarantee that all KUnit tests run after late_init is complete, which was a concern during the initial KUnit patchset review (this has not been a problem in practice, but resolving with certainty is nevertheless desirable). Other use cases for this exist, but the above features should provide an idea of the value that this could provide. ## Changes since last revision: - On patch 7/7, I added some additional wording around the kunit_shutdown command line option explaining that it runs after built-in tests as suggested by Frank. - On the coverletter, I improved some wording and added a missing link. I also specified the base-commit for the series. - Frank asked for some changes to the documentation; however, David is taking care of that in a separate patch[2], so I did not make those changes here. There will be some additional changes necessary after David's patch is applied. Alan Maguire (1): kunit: test: create a single centralized executor for all tests Brendan Higgins (5): vmlinux.lds.h: add linker section for KUnit test suites arch: um: add linker section for KUnit test suites init: main: add KUnit to kernel init kunit: test: add test plan to KUnit TAP format Documentation: Add kunit_shutdown to kernel-parameters.txt David Gow (1): kunit: Add 'kunit_shutdown' option .../admin-guide/kernel-parameters.txt | 8 ++ arch/um/include/asm/common.lds.S | 4 + include/asm-generic/vmlinux.lds.h | 8 ++ include/kunit/test.h | 82 ++++++++++++------- init/main.c | 4 + lib/kunit/Makefile | 3 +- lib/kunit/executor.c | 71 ++++++++++++++++ lib/kunit/test.c | 11 --- tools/testing/kunit/kunit_kernel.py | 2 +- tools/testing/kunit/kunit_parser.py | 76 ++++++++++++++--- .../test_is_test_passed-all_passed.log | 1 + .../test_data/test_is_test_passed-crash.log | 1 + .../test_data/test_is_test_passed-failure.log | 1 + 13 files changed, 218 insertions(+), 54 deletions(-) create mode 100644 lib/kunit/executor.c base-commit: a2f0b878c3ca531a1706cb2a8b079cea3b17bafc [1] https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-… [2] https://patchwork.kernel.org/patch/11383635/ -- 2.25.1.481.gfbce0eb801-goog

3 years, 10 months

7
19
0 0

[PATCH v2 0/2] ftrace/selftest: clean up failure cases

by Alan Maguire

When running the ftrace selftests, 2 failures and 6 unresolved cases were observed. The failures can be avoided by setting a sysctl prior to test execution (fixed in patch 1) and by having unresolved cases not return 0 from ftracetest by default since they indicate an absence of testing modules/programs rather than ftrace issues (patch 2). The latter are classified as "unresolved" tests, which operate differently from "unsupported" tests. For unsupported tests, we note the unsupported count but do not consider the tests as having failed, whereas with unresolved the test run is considered to have failed so returns "not ok" when run via kselftest ("make -C tools/testing/selftest/ftrace run_tests"). Patch 2 aligns the unresolved behaviour with the unsupported; by default, unresolved outcomes do not trigger overall failure, but they can if --fail-unresolved is specified. Changes since v1: - updated patch 1 to use /proc path instead of sysctl (Masami) - updated patch 2 to modify unresolved handling in ftracetest rather than change individual unresolved -> unsupported (Masami) Alan Maguire (2): ftrace/selftests: workaround cgroup RT scheduling issues ftrace/selftest: make unresolved cases cause failure if --fail-unresolved set tools/testing/selftests/ftrace/ftracetest | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) -- 1.8.3.1

3 years, 12 months

4
7
0 0

[PATCH v6 00/12] mm/gup: track FOLL_PIN pages

by John Hubbard

Hi, Jan and Kirill: I've tentatively removed your review and ACK, respectively, for patch 12 (the last dump_page patch), because even though they are logically the same as what you reviewed in v5, the base is Matthew's new patch instead of my earlier patch. (Trying to err on the side of caution with these tags.) There is a git repo and branch, for convenience in reviewing: git@github.com:johnhubbard/linux.git track_user_pages_v6 ============================================================ Changes since v5: * Rebased onto Linux 5.6.0-rc1. * Swapped in Matthew Wilcox's more comprehensive dump_page() patch, and moved it later in this series so that it immediately precedes my subsequent dump_page() patch, for slightly easier reviews and commit log history. * Fixed "the last bug!" in the /proc/vmstat patch, by moving the mod_node_page_state() call in put_compound_page() so that it only happens in the FOLL_PIN case. * Added a couple more ACKs from Kirill. * Tweaked the "Future steps" in this cover letter to add a little detail about what comes next. ============================================================ Changes since v4: * Added documentation about the huge page behavior of the new /proc/vmstat items. * Added a missing mode_node_page_state() call to put_compound_head(). * Fixed a tracepoint call in page_ref_sub_return(). * Added a trailing underscore to a URL in pin_user_pages.rst, to fix a broken generated link. * Added ACKs and reviewed-by's from Jan Kara and Kirill Shutemov. * Rebased onto today's linux.git, and * I am experimenting here with "git format-patch --base=<commit>". This generated the "base-commit:" tag you'll see at the end of this cover letter. I was inspired to do so after trying out a new get-lore-mbox.py tool (it's very nice), mentioned in a recent LWN article (https://lwn.net/Articles/811528/ ). That tool relies on the base-commit tag for some things. ============================================================ Changes since v3: * Rebased onto latest linux.git * Added ACKs and reviewed-by's from Kirill Shutemov and Jan Kara. * /proc/vmstat: * Renamed items, after realizing that I hate the previous names: nr_foll_pin_requested --> nr_foll_pin_acquired nr_foll_pin_returned --> nr_foll_pin_released * Removed the CONFIG_DEBUG_VM guard, and collapsed away a wrapper routine: now just calls mod_node_page_state() directly. * Tweaked the WARN_ON_ONCE() statements in mm/hugetlb.c to be more informative, and added comments above them as well. * Fixed gup_benchmark: signed int --> unsigned long. * One or two minor formatting changes. ============================================================ Changes since v2: * Rebased onto linux.git, because the akpm tree for 5.6 has been merged. * Split the tracking patch into even more patches, as requested. * Merged Matthew Wilcox's dump_page() changes into mine, as part of the first patch. * Renamed: page_dma_pinned() --> page_maybe_dma_pinned(), in response to Kirill Shutemov's review. * Moved a WARN to the top of a routine, and fixed a typo in the commit description of patch #7, also as suggested by Kirill. ============================================================ Changes since v1: * Split the tracking patch into 6 smaller patches * Rebased onto today's linux-next/akpm (there weren't any conflicts). * Fixed an "unsigned int" vs. "int" problem in gup_benchmark, reported by Nathan Chancellor. (I don't see it in my local builds, probably because they use gcc, but an LLVM test found the mismatch.) * Fixed a huge page pincount problem (add/subtract vs. increment/decrement), spotted by Jan Kara. ============================================================ There is a reasonable case to be made for merging two of the patches (patches 7 and 8), given that patch 7 provides tracking that has upper limits on the number of pins that can be done with huge pages. Let me know if anyone wants those merged, but unless there is some weird chance of someone grabbing patch 7 and not patch 8, I don't really see the need. Meanwhile, it's easier to review in this form. Also, patch 3 has been revived. Earlier reviewers asked for it to be merged into the tracking patch (one cannot please everyone, heh), but now it's back out on it's own. This activates tracking of FOLL_PIN pages. This is in support of fixing the get_user_pages()+DMA problem described in [1]-[4]. FOLL_PIN support is now in the main linux tree. However, the patch to use FOLL_PIN to track pages was *not* submitted, because Leon saw an RDMA test suite failure that involved (I think) page refcount overflows when huge pages were used. This patch definitively solves that kind of overflow problem, by adding an exact pincount, for compound pages (of order > 1), in the 3rd struct page of a compound page. If available, that form of pincounting is used, instead of the GUP_PIN_COUNTING_BIAS approach. Thanks again to Jan Kara for that idea. Other interesting changes: * dump_page(): added one, or two new things to report for compound pages: head refcount (for all compound pages), and map_pincount (for compound pages of order > 1). * Documentation/core-api/pin_user_pages.rst: removed the "TODO" for the huge page refcount upper limit problems, and added notes about how it works now. Also added a note about the dump_page() enhancements. * Added some comments in gup.c and mm.h, to explain that there are two ways to count pinned pages: exact (for compound pages of order > 1) and fuzzy (GUP_PIN_COUNTING_BIAS: for all other pages). ============================================================ General notes about the tracking patch: This is a prerequisite to solving the problem of proper interactions between file-backed pages, and [R]DMA activities, as discussed in [1], [2], [3], [4] and in a remarkable number of email threads since about 2017. :) In contrast to earlier approaches, the page tracking can be incrementally applied to the kernel call sites that, until now, have been simply calling get_user_pages() ("gup"). In other words, opt-in by changing from this: get_user_pages() (sets FOLL_GET) put_page() to this: pin_user_pages() (sets FOLL_PIN) unpin_user_page() ============================================================ Future steps: * Convert more subsystems from get_user_pages() to pin_user_pages(). The first probably needs to be bio/biovecs, because any filesystem testing is too difficult without those in place. * Change VFS and filesystems to respond appropriately when encountering dma-pinned pages. * Work with Ira and others to connect this all up with file system leases. [1] Some slow progress on get_user_pages() (Apr 2, 2019): https://lwn.net/Articles/784574/ [2] DMA and get_user_pages() (LPC: Dec 12, 2018): https://lwn.net/Articles/774411/ [3] The trouble with get_user_pages() (Apr 30, 2018): https://lwn.net/Articles/753027/ [4] LWN kernel index: get_user_pages() https://lwn.net/Kernel/Index/#Memory_management-get_user_pages John Hubbard (11): mm/gup: split get_user_pages_remote() into two routines mm/gup: pass a flags arg to __gup_device_* functions mm: introduce page_ref_sub_return() mm/gup: pass gup flags to two more routines mm/gup: require FOLL_GET for get_user_pages_fast() mm/gup: track FOLL_PIN pages mm/gup: page->hpage_pinned_refcount: exact pin counts for huge pages mm/gup: /proc/vmstat: pin_user_pages (FOLL_PIN) reporting mm/gup_benchmark: support pin_user_pages() and related calls selftests/vm: run_vmtests: invoke gup_benchmark with basic FOLL_PIN coverage mm: dump_page(): additional diagnostics for huge pinned pages Matthew Wilcox (Oracle) (1): mm: Improve dump_page() for compound pages Documentation/core-api/pin_user_pages.rst | 86 ++-- include/linux/mm.h | 108 ++++- include/linux/mm_types.h | 7 +- include/linux/mmzone.h | 2 + include/linux/page_ref.h | 9 + mm/debug.c | 44 +- mm/gup.c | 451 ++++++++++++++++----- mm/gup_benchmark.c | 71 +++- mm/huge_memory.c | 29 +- mm/hugetlb.c | 60 ++- mm/page_alloc.c | 2 + mm/rmap.c | 6 + mm/vmstat.c | 2 + tools/testing/selftests/vm/gup_benchmark.c | 15 +- tools/testing/selftests/vm/run_vmtests | 22 + 15 files changed, 734 insertions(+), 180 deletions(-) base-commit: bb6d3fb354c5ee8d6bde2d576eb7220ea09862b9 -- 2.25.0

3 years, 12 months

5
27
0 0

[PATCH 0/2] selftests: vm: Build fixes for powerpc64

by Sandipan Das

The second patch was already posted independently but because of the changes in the first patch, the second one now depends on it. Hence posting it now as a part of this series. The last version (v2) of the second patch can be found at: https://patchwork.ozlabs.org/patch/1225969/ Sandipan Das (2): selftests: vm: Do not override definition of ARCH selftests: vm: Fix 64-bit test builds for powerpc64le tools/testing/selftests/vm/Makefile | 4 ++-- tools/testing/selftests/vm/run_vmtests | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) -- 2.17.1

4 years

3
7
0 0

[PATCHv2] selftests/timers: Turn off timeout setting

by Po-Hsu Lin

The following 4 tests in timers can take longer than the default 45 seconds that added in commit 852c8cbf (selftests/kselftest/runner.sh: Add 45 second timeout per test) to run: * nsleep-lat - 2m7.350s * set-timer-lat - 2m0.66s * inconsistency-check - 1m45.074s * raw_skew - 2m0.013s Thus they will be marked as failed with the current 45s setting: not ok 3 selftests: timers: nsleep-lat # TIMEOUT not ok 4 selftests: timers: set-timer-lat # TIMEOUT not ok 6 selftests: timers: inconsistency-check # TIMEOUT not ok 7 selftests: timers: raw_skew # TIMEOUT Disable the timeout setting for timers can make these tests finish properly: ok 3 selftests: timers: nsleep-lat ok 4 selftests: timers: set-timer-lat ok 6 selftests: timers: inconsistency-check ok 7 selftests: timers: raw_skew https://bugs.launchpad.net/bugs/1864626 Signed-off-by: Po-Hsu Lin <po-hsu.lin(a)canonical.com> --- tools/testing/selftests/timers/Makefile | 1 + tools/testing/selftests/timers/settings | 1 + 2 files changed, 2 insertions(+) create mode 100644 tools/testing/selftests/timers/settings diff --git a/tools/testing/selftests/timers/Makefile b/tools/testing/selftests/timers/Makefile index 7656c7c..0e73a16 100644 --- a/tools/testing/selftests/timers/Makefile +++ b/tools/testing/selftests/timers/Makefile @@ -13,6 +13,7 @@ DESTRUCTIVE_TESTS = alarmtimer-suspend valid-adjtimex adjtick change_skew \ TEST_GEN_PROGS_EXTENDED = $(DESTRUCTIVE_TESTS) +TEST_FILES := settings include ../lib.mk diff --git a/tools/testing/selftests/timers/settings b/tools/testing/selftests/timers/settings new file mode 100644 index 0000000..e7b9417 --- /dev/null +++ b/tools/testing/selftests/timers/settings @@ -0,0 +1 @@ +timeout=0 -- 2.7.4

4 years

3
3
0 0

[RFC PATCH v14 00/10] Landlock LSM

by Mickaël Salaün

Hi, This new version of Landlock is a major revamp of the previous series [1], hence the RFC tag. The three main changes are the replacement of eBPF with a dedicated safe management of access rules, the replacement of the use of seccomp(2) with a dedicated syscall, and the management of filesystem access-control (back from the v10). As discussed in [2], eBPF may be too powerful and dangerous to be put in the hand of unprivileged and potentially malicious processes, especially because of side-channel attacks against access-controls or other parts of the kernel. Thanks to this new implementation (1540 SLOC), designed from the ground to be used by unprivileged processes, this series enables a process to sandbox itself without requiring CAP_SYS_ADMIN, but only the no_new_privs constraint (like seccomp). Not relying on eBPF also enables to improve performances, especially for stacked security policies thanks to mergeable rulesets. The compiled documentation is available here: https://landlock.io/linux-doc/landlock-v14/security/landlock/index.html This series can be applied on top of v5.6-rc3. This can be tested with CONFIG_SECURITY_LANDLOCK and CONFIG_SAMPLE_LANDLOCK. This patch series can be found in a Git repository here: https://github.com/landlock-lsm/linux/commits/landlock-v14 I would really appreciate constructive comments on the design and the code. # Landlock LSM The goal of Landlock is to enable to restrict ambient rights (e.g. global filesystem access) for a set of processes. Because Landlock is a stackable LSM [3], it makes possible to create safe security sandboxes as new security layers in addition to the existing system-wide access-controls. This kind of sandbox is expected to help mitigate the security impact of bugs or unexpected/malicious behaviors in user-space applications. Landlock empower any process, including unprivileged ones, to securely restrict themselves. Landlock is inspired by seccomp-bpf but instead of filtering syscalls and their raw arguments, a Landlock rule can restrict the use of kernel objects like file hierarchies, according to the kernel semantic. Landlock also takes inspiration from other OS sandbox mechanisms: XNU Sandbox, FreeBSD Capsicum or OpenBSD Pledge/Unveil. # Current limitations ## Path walk Landlock need to use dentries to identify a file hierarchy, which is needed for composable and unprivileged access-controls. This means that path resolution/walking (handled with inode_permission()) is not supported, yet. This could be filled with a future extension first of the LSM framework. The Landlock userspace ABI can handle such change with new option (e.g. to the struct landlock_ruleset). ## UnionFS An UnionFS super-block use a set of upper and lower directories. An access request to a file in one of these hierarchy trigger a call to ovl_path_real() which generate another access request according to the matching hierarchy. Because such super-block is not aware of its current mount point, OverlayFS can't create a dedicated mnt_parent for each of the upper and lower directories mount clones. It is then not currently possible to track the source of such indirect access-request, and then not possible to identify a unified OverlayFS hierarchy. ## Syscall Because it is only tested on x86_64, the syscall is only wired up for this architecture. The whole x86 family (and probably all the others) will be supported in the next patch series. ## Memory limits There is currently no limit on the memory usage. Any idea to leverage an existing mechanism (e.g. rlimit)? # Changes since v13 * Revamp of the LSM: remove the need for eBPF and seccomp(2). * Implement a full filesystem access-control. * Take care of the backward compatibility issues, especially for this security features. Previous version: https://lore.kernel.org/lkml/20191104172146.30797-1-mic@digikod.net/ [1] https://lore.kernel.org/lkml/20191104172146.30797-1-mic@digikod.net/ [2] https://lore.kernel.org/lkml/a6b61f33-82dc-0c1c-7a6c-1926343ef63e@digikod.n… [3] https://lore.kernel.org/lkml/50db058a-7dde-441b-a7f9-f6837fe8b69f@schaufler… Regards, Mickaël Salaün (10): landlock: Add object and rule management landlock: Add ruleset and domain management landlock: Set up the security framework and manage credentials landlock: Add ptrace restrictions fs,landlock: Support filesystem access-control landlock: Add syscall implementation arch: Wire up landlock() syscall selftests/landlock: Add initial tests samples/landlock: Add a sandbox manager example landlock: Add user and kernel documentation Documentation/security/index.rst | 1 + Documentation/security/landlock/index.rst | 18 + Documentation/security/landlock/kernel.rst | 44 ++ Documentation/security/landlock/user.rst | 233 +++++++ MAINTAINERS | 12 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + fs/super.c | 2 + include/linux/landlock.h | 22 + include/linux/syscalls.h | 3 + include/uapi/asm-generic/unistd.h | 4 +- include/uapi/linux/landlock.h | 315 +++++++++ samples/Kconfig | 7 + samples/Makefile | 1 + samples/landlock/.gitignore | 1 + samples/landlock/Makefile | 15 + samples/landlock/sandboxer.c | 226 +++++++ security/Kconfig | 11 +- security/Makefile | 2 + security/landlock/Kconfig | 16 + security/landlock/Makefile | 4 + security/landlock/cred.c | 47 ++ security/landlock/cred.h | 55 ++ security/landlock/fs.c | 591 +++++++++++++++++ security/landlock/fs.h | 42 ++ security/landlock/object.c | 341 ++++++++++ security/landlock/object.h | 134 ++++ security/landlock/ptrace.c | 118 ++++ security/landlock/ptrace.h | 14 + security/landlock/ruleset.c | 463 +++++++++++++ security/landlock/ruleset.h | 106 +++ security/landlock/setup.c | 38 ++ security/landlock/setup.h | 20 + security/landlock/syscall.c | 470 +++++++++++++ tools/testing/selftests/Makefile | 1 + tools/testing/selftests/landlock/.gitignore | 3 + tools/testing/selftests/landlock/Makefile | 13 + tools/testing/selftests/landlock/config | 4 + tools/testing/selftests/landlock/test.h | 40 ++ tools/testing/selftests/landlock/test_base.c | 80 +++ tools/testing/selftests/landlock/test_fs.c | 624 ++++++++++++++++++ .../testing/selftests/landlock/test_ptrace.c | 293 ++++++++ 41 files changed, 4429 insertions(+), 6 deletions(-) create mode 100644 Documentation/security/landlock/index.rst create mode 100644 Documentation/security/landlock/kernel.rst create mode 100644 Documentation/security/landlock/user.rst create mode 100644 include/linux/landlock.h create mode 100644 include/uapi/linux/landlock.h create mode 100644 samples/landlock/.gitignore create mode 100644 samples/landlock/Makefile create mode 100644 samples/landlock/sandboxer.c create mode 100644 security/landlock/Kconfig create mode 100644 security/landlock/Makefile create mode 100644 security/landlock/cred.c create mode 100644 security/landlock/cred.h create mode 100644 security/landlock/fs.c create mode 100644 security/landlock/fs.h create mode 100644 security/landlock/object.c create mode 100644 security/landlock/object.h create mode 100644 security/landlock/ptrace.c create mode 100644 security/landlock/ptrace.h create mode 100644 security/landlock/ruleset.c create mode 100644 security/landlock/ruleset.h create mode 100644 security/landlock/setup.c create mode 100644 security/landlock/setup.h create mode 100644 security/landlock/syscall.c create mode 100644 tools/testing/selftests/landlock/.gitignore create mode 100644 tools/testing/selftests/landlock/Makefile create mode 100644 tools/testing/selftests/landlock/config create mode 100644 tools/testing/selftests/landlock/test.h create mode 100644 tools/testing/selftests/landlock/test_base.c create mode 100644 tools/testing/selftests/landlock/test_fs.c create mode 100644 tools/testing/selftests/landlock/test_ptrace.c -- 2.25.0

4 years

5
33
0 0

WARNING in sk_stream_kill_queues (4)

by syzbot

Hello, syzbot found the following crash on: HEAD commit: 0c0ddd6a Merge tag 'linux-watchdog-5.6-rc3' of git://www.l.. git tree: net console output: https://syzkaller.appspot.com/x/log.txt?x=17c0fde9e00000 kernel config: https://syzkaller.appspot.com/x/.config?x=3b8906eb6a7d6028 dashboard link: https://syzkaller.appspot.com/bug?extid=fbe81b56f7df4c0fb21b compiler: gcc (GCC) 9.0.0 20181231 (experimental) syz repro: https://syzkaller.appspot.com/x/repro.syz?x=12025de9e00000 C reproducer: https://syzkaller.appspot.com/x/repro.c?x=16a4d245e00000 The bug was bisected to: commit fb99bce7120014307dde57b3d7def6977a9a62a1 Author: Dave Watson <davejwatson(a)fb.com> Date: Wed Jan 30 21:58:05 2019 +0000 net: tls: Support 256 bit keys bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=14898931e00000 final crash: https://syzkaller.appspot.com/x/report.txt?x=16898931e00000 console output: https://syzkaller.appspot.com/x/log.txt?x=12898931e00000 IMPORTANT: if you fix the bug, please add the following tag to the commit: Reported-by: syzbot+fbe81b56f7df4c0fb21b(a)syzkaller.appspotmail.com Fixes: fb99bce71200 ("net: tls: Support 256 bit keys") ------------[ cut here ]------------ WARNING: CPU: 1 PID: 9607 at net/core/stream.c:208 sk_stream_kill_queues+0x40d/0x590 net/core/stream.c:208 Kernel panic - not syncing: panic_on_warn set ... CPU: 1 PID: 9607 Comm: syz-executor386 Not tainted 5.6.0-rc2-syzkaller #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x197/0x210 lib/dump_stack.c:118 panic+0x2e3/0x75c kernel/panic.c:221 __warn.cold+0x2f/0x3e kernel/panic.c:582 report_bug+0x289/0x300 lib/bug.c:195 fixup_bug arch/x86/kernel/traps.c:174 [inline] fixup_bug arch/x86/kernel/traps.c:169 [inline] do_error_trap+0x11b/0x200 arch/x86/kernel/traps.c:267 do_invalid_op+0x37/0x50 arch/x86/kernel/traps.c:286 invalid_op+0x23/0x30 arch/x86/entry/entry_64.S:1027 RIP: 0010:sk_stream_kill_queues+0x40d/0x590 net/core/stream.c:208 Code: 48 c1 ea 03 0f b6 04 02 84 c0 74 04 3c 03 7e 41 44 8b a3 f8 01 00 00 e9 5f ff ff ff e8 0c 7f 40 fb 0f 0b eb 96 e8 03 7f 40 fb <0f> 0b eb a1 e8 fa 7e 40 fb 0f 0b e9 9f fe ff ff 4c 89 ef e8 7b 24 RSP: 0018:ffffc9000c6efb30 EFLAGS: 00010293 RAX: ffff8880a3102380 RBX: 0000000000000000 RCX: ffffffff8635085d RDX: 0000000000000000 RSI: ffffffff863508bd RDI: 0000000000000005 RBP: ffffc9000c6efb70 R08: ffff8880a3102380 R09: ffffed1015d2707c R10: ffffed1015d2707b R11: ffff8880ae9383db R12: 0000000000000fe3 R13: ffff88808f618238 R14: ffffffff8c406240 R15: ffff88808f618178 inet_csk_destroy_sock+0x1b7/0x4c0 net/ipv4/inet_connection_sock.c:846 tcp_close+0xe17/0x12b0 net/ipv4/tcp.c:2503 inet_release+0xed/0x200 net/ipv4/af_inet.c:427 __sock_release+0xce/0x280 net/socket.c:605 sock_close+0x1e/0x30 net/socket.c:1283 __fput+0x2ff/0x890 fs/file_table.c:280 ____fput+0x16/0x20 fs/file_table.c:313 task_work_run+0x145/0x1c0 kernel/task_work.c:113 exit_task_work include/linux/task_work.h:22 [inline] do_exit+0xba9/0x2f50 kernel/exit.c:801 do_group_exit+0x135/0x360 kernel/exit.c:899 __do_sys_exit_group kernel/exit.c:910 [inline] __se_sys_exit_group kernel/exit.c:908 [inline] __x64_sys_exit_group+0x44/0x50 kernel/exit.c:908 do_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x43eff8 Code: Bad RIP value. RSP: 002b:00007ffca386cb68 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 000000000043eff8 RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000 RBP: 00000000004be808 R08: 00000000000000e7 R09: ffffffffffffffd0 R10: 00000000200001c0 R11: 0000000000000246 R12: 0000000000000001 R13: 00000000006d0180 R14: 0000000000000000 R15: 0000000000000000 Kernel Offset: disabled Rebooting in 86400 seconds.. --- This bug is generated by a bot. It may contain errors. See https://goo.gl/tpsmEJ for more information about syzbot. syzbot engineers can be reached at syzkaller(a)googlegroups.com. syzbot will keep track of this bug report. See: https://goo.gl/tpsmEJ#status for how to communicate with syzbot. For information about bisection process see: https://goo.gl/tpsmEJ#bisection syzbot can test patches for this bug, for details see: https://goo.gl/tpsmEJ#testing-patches

4 years

1
1
0 0

[PATCH kselftest/test] kunit: Always print actual pointer values in asserts

by David Gow

KUnit assertions and expectations will print the values being tested. If these are pointers (e.g., KUNIT_EXPECT_PTR_EQ(test, a, b)), these pointers are currently printed with the %pK format specifier, which -- to prevent information leaks which may compromise, e.g., ASLR -- are often either hashed or replaced with ____ptrval____ or similar, making debugging tests difficult. By replacing %pK with %px as Documentation/core-api/printk-formats.rst suggests, we disable this security feature for KUnit assertions and expectations, allowing the actual pointer values to be printed. Given that KUnit is not intended for use in production kernels, and the pointers are only printed on failing tests, this seems like a worthwhile tradeoff. Signed-off-by: David Gow <davidgow(a)google.com> --- This seems like the best way of solving this problem to me, but if anyone has a better solution I'd love to hear it. Note also that this does trigger two checkpatch.pl warnings, which warn that the change will potentially cause the kernel memory layout to be exposed. Since that's the whole point of the change, they probably sohuld stay there. lib/kunit/assert.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c index 86013d4cf891..a87960409bd4 100644 --- a/lib/kunit/assert.c +++ b/lib/kunit/assert.c @@ -110,10 +110,10 @@ void kunit_binary_ptr_assert_format(const struct kunit_assert *assert, binary_assert->left_text, binary_assert->operation, binary_assert->right_text); - string_stream_add(stream, "\t\t%s == %pK\n", + string_stream_add(stream, "\t\t%s == %px\n", binary_assert->left_text, binary_assert->left_value); - string_stream_add(stream, "\t\t%s == %pK", + string_stream_add(stream, "\t\t%s == %px", binary_assert->right_text, binary_assert->right_value); kunit_assert_print_msg(assert, stream); -- 2.24.0.432.g9d3f5f5b63-goog

4 years, 1 month

3
4
0 0

[PATCH] kunit: add --make_options

by Greg Thelen

The kunit.py utility builds an ARCH=um kernel and then runs it. Add optional --make_options flag to kunit.py allowing for the operator to specify extra build options. This allows use of the clang compiler for kunit: tools/testing/kunit/kunit.py run --defconfig \ --make_options CC=clang --make_options HOSTCC=clang Signed-off-by: Greg Thelen <gthelen(a)google.com> --- tools/testing/kunit/kunit.py | 15 +++++++++++---- tools/testing/kunit/kunit_kernel.py | 24 ++++++++++++++---------- 2 files changed, 25 insertions(+), 14 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index e59eb9e7f923..34a7ec8d9594 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -22,7 +22,9 @@ import kunit_parser KunitResult = namedtuple('KunitResult', ['status','result']) -KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs', 'build_dir', 'defconfig']) +KunitRequest = namedtuple('KunitRequest', ['raw_output', 'timeout', 'jobs', + 'build_dir', 'defconfig', + 'make_options']) class KunitStatus(Enum): SUCCESS = auto() @@ -38,7 +40,7 @@ def create_default_kunitconfig(): def run_tests(linux: kunit_kernel.LinuxSourceTree, request: KunitRequest) -> KunitResult: config_start = time.time() - success = linux.build_reconfig(request.build_dir) + success = linux.build_reconfig(request.build_dir, request.make_options) config_end = time.time() if not success: return KunitResult(KunitStatus.CONFIG_FAILURE, 'could not configure kernel') @@ -46,7 +48,7 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, kunit_parser.print_with_timestamp('Building KUnit Kernel ...') build_start = time.time() - success = linux.build_um_kernel(request.jobs, request.build_dir) + success = linux.build_um_kernel(request.jobs, request.build_dir, request.make_options) build_end = time.time() if not success: return KunitResult(KunitStatus.BUILD_FAILURE, 'could not build kernel') @@ -111,6 +113,10 @@ def main(argv, linux=None): help='Uses a default .kunitconfig.', action='store_true') + run_parser.add_argument('--make_options', + help='X=Y make option, can be repeated.', + action='append') + cli_args = parser.parse_args(argv) if cli_args.subcommand == 'run': @@ -131,7 +137,8 @@ def main(argv, linux=None): cli_args.timeout, cli_args.jobs, cli_args.build_dir, - cli_args.defconfig) + cli_args.defconfig, + cli_args.make_options) result = run_tests(linux, request) if result.status != KunitStatus.SUCCESS: sys.exit(1) diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index cc5d844ecca1..2b9b3fdbc13f 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -35,8 +35,10 @@ class LinuxSourceTreeOperations(object): except subprocess.CalledProcessError as e: raise ConfigError(e.output) - def make_olddefconfig(self, build_dir): + def make_olddefconfig(self, build_dir, make_options): command = ['make', 'ARCH=um', 'olddefconfig'] + if make_options: + command.extend(make_options) if build_dir: command += ['O=' + build_dir] try: @@ -46,8 +48,10 @@ class LinuxSourceTreeOperations(object): except subprocess.CalledProcessError as e: raise ConfigError(e.output) - def make(self, jobs, build_dir): + def make(self, jobs, build_dir, make_options): command = ['make', 'ARCH=um', '--jobs=' + str(jobs)] + if make_options: + command.extend(make_options) if build_dir: command += ['O=' + build_dir] try: @@ -93,13 +97,13 @@ class LinuxSourceTree(object): return False return True - def build_config(self, build_dir): + def build_config(self, build_dir, make_options): kconfig_path = get_kconfig_path(build_dir) if build_dir and not os.path.exists(build_dir): os.mkdir(build_dir) self._kconfig.write_to_file(kconfig_path) try: - self._ops.make_olddefconfig(build_dir) + self._ops.make_olddefconfig(build_dir, make_options) except ConfigError as e: logging.error(e) return False @@ -110,7 +114,7 @@ class LinuxSourceTree(object): return False return True - def build_reconfig(self, build_dir): + def build_reconfig(self, build_dir, make_options): """Creates a new .config if it is not a subset of the .kunitconfig.""" kconfig_path = get_kconfig_path(build_dir) if os.path.exists(kconfig_path): @@ -119,17 +123,17 @@ class LinuxSourceTree(object): if not self._kconfig.is_subset_of(existing_kconfig): print('Regenerating .config ...') os.remove(kconfig_path) - return self.build_config(build_dir) + return self.build_config(build_dir, make_options) else: return True else: print('Generating .config ...') - return self.build_config(build_dir) + return self.build_config(build_dir, make_options) - def build_um_kernel(self, jobs, build_dir): + def build_um_kernel(self, jobs, build_dir, make_options): try: - self._ops.make_olddefconfig(build_dir) - self._ops.make(jobs, build_dir) + self._ops.make_olddefconfig(build_dir, make_options) + self._ops.make(jobs, build_dir, make_options) except (ConfigError, BuildError) as e: logging.error(e) return False -- 2.25.0.265.gbab2e86ba0-goog

4 years, 1 month

5
6
0 0

[PATCH v4] Documentation: kunit: Make the KUnit documentation less UML-specific

by David Gow

Remove some of the outmoded "Why KUnit" rationale, and move some UML-specific information to the kunit_tool page. Also update the Getting Started guide to mention running tests without the kunit_tool wrapper. Signed-off-by: David Gow <davidgow(a)google.com> Reviewed-by: Frank Rowand <frank.rowand(a)sony.com> --- Sorry: I missed a couple of issues in the last version. They're fixed here, and I think this should be ready to go. Changelog: v4: - Fix typo: s/offsers/offers - Talk about KUnit tests running on most "architectures" instead of "kernel configurations. v3: https://lore.kernel.org/linux-kselftest/20200214235723.254228-1-davidgow@go… - Added a note that KUnit can be used with UML, both with and without kunit_tool to replace the section moved to kunit_tool. v2: https://lore.kernel.org/linux-kselftest/f99a3d4d-ad65-5fd1-3407-db33f378b1f… - Reinstated the "Why Kunit?" section, minus the comparison with other testing frameworks (covered in the FAQ), and the description of UML. - Moved the description of UML into to kunit_tool page. - Tidied up the wording around how KUnit is built and run to make it work without the UML description. v1: https://lore.kernel.org/linux-kselftest/9c703dea-a9e1-94e2-c12d-3cb0a09e75a… - Initial patch Documentation/dev-tools/kunit/index.rst | 40 ++++++---- Documentation/dev-tools/kunit/kunit-tool.rst | 7 ++ Documentation/dev-tools/kunit/start.rst | 80 ++++++++++++++++---- 3 files changed, 99 insertions(+), 28 deletions(-) diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst index d16a4d2c3a41..e93606ecfb01 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -17,14 +17,23 @@ What is KUnit? ============== KUnit is a lightweight unit testing and mocking framework for the Linux kernel. -These tests are able to be run locally on a developer's workstation without a VM -or special hardware. KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, and much more. +KUnit consists of a kernel component, which provides a set of macros for easily +writing unit tests. Tests written against KUnit will run on kernel boot if +built-in, or when loaded if built as a module. These tests write out results to +the kernel log in `TAP <https://testanything.org/>`_ format. + +To make running these tests (and reading the results) easier, KUnit offers +:doc:`kunit_tool <kunit-tool>`, which builds a `User Mode Linux +<http://user-mode-linux.sourceforge.net>`_ kernel, runs it, and parses the test +results. This provides a quick way of running KUnit tests during development, +without requiring a virtual machine or separate hardware. + Get started now: :doc:`start` Why KUnit? @@ -36,21 +45,20 @@ allow all possible code paths to be tested in the code under test; this is only possible if the code under test is very small and does not have any external dependencies outside of the test's control like hardware. -Outside of KUnit, there are no testing frameworks currently -available for the kernel that do not require installing the kernel on a test -machine or in a VM and all require tests to be written in userspace running on -the kernel; this is true for Autotest, and kselftest, disqualifying -any of them from being considered unit testing frameworks. +KUnit provides a common framework for unit tests within the kernel. + +KUnit tests can be run on most architectures, and most tests are architecture +independent. All built-in KUnit tests run on kernel startup. Alternatively, +KUnit and KUnit tests can be built as modules and tests will run when the test +module is loaded. -KUnit addresses the problem of being able to run tests without needing a virtual -machine or actual hardware with User Mode Linux. User Mode Linux is a Linux -architecture, like ARM or x86; however, unlike other architectures it compiles -to a standalone program that can be run like any other program directly inside -of a host operating system; to be clear, it does not require any virtualization -support; it is just a regular program. +.. note:: -Alternatively, kunit and kunit tests can be built as modules and tests will -run when the test module is loaded. + KUnit can also run tests without needing a virtual machine or actual + hardware under User Mode Linux. User Mode Linux is a Linux architecture, + like ARM or x86, which compiles the kernel as a Linux executable. KUnit + can be used with UML either by building with ``ARCH=um`` (like any other + architecture), or by using :doc:`kunit_tool <kunit-tool>`. KUnit is fast. Excluding build time, from invocation to completion KUnit can run several dozen tests in only 10 to 20 seconds; this might not sound like a big @@ -81,3 +89,5 @@ How do I use it? * :doc:`start` - for new users of KUnit * :doc:`usage` - for a more detailed explanation of KUnit features * :doc:`api/index` - for the list of KUnit APIs used for testing +* :doc:`kunit-tool` - for more information on the kunit_tool helper script +* :doc:`faq` - for answers to some common questions about KUnit diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst index 50d46394e97e..949af2da81e5 100644 --- a/Documentation/dev-tools/kunit/kunit-tool.rst +++ b/Documentation/dev-tools/kunit/kunit-tool.rst @@ -12,6 +12,13 @@ the Linux kernel as UML (`User Mode Linux <http://user-mode-linux.sourceforge.net/>`_), running KUnit tests, parsing the test results and displaying them in a user friendly manner. +kunit_tool addresses the problem of being able to run tests without needing a +virtual machine or actual hardware with User Mode Linux. User Mode Linux is a +Linux architecture, like ARM or x86; however, unlike other architectures it +compiles the kernel as a standalone Linux executable that can be run like any +other program directly inside of a host operating system. To be clear, it does +not require any virtualization support: it is just a regular program. + What is a kunitconfig? ====================== diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 4e1d24db6b13..e1c5ce80ce12 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -9,11 +9,10 @@ Installing dependencies KUnit has the same dependencies as the Linux kernel. As long as you can build the kernel, you can run KUnit. -KUnit Wrapper -============= -Included with KUnit is a simple Python wrapper that helps format the output to -easily use and read KUnit output. It handles building and running the kernel, as -well as formatting the output. +Running tests with the KUnit Wrapper +==================================== +Included with KUnit is a simple Python wrapper which runs tests under User Mode +Linux, and formats the test results. The wrapper can be run with: @@ -21,22 +20,42 @@ The wrapper can be run with: ./tools/testing/kunit/kunit.py run --defconfig -For more information on this wrapper (also called kunit_tool) checkout the +For more information on this wrapper (also called kunit_tool) check out the :doc:`kunit-tool` page. Creating a .kunitconfig -======================= -The Python script is a thin wrapper around Kbuild. As such, it needs to be -configured with a ``.kunitconfig`` file. This file essentially contains the -regular Kernel config, with the specific test targets as well. - +----------------------- +If you want to run a specific set of tests (rather than those listed in the +KUnit defconfig), you can provide Kconfig options in the ``.kunitconfig`` file. +This file essentially contains the regular Kernel config, with the specific +test targets as well. The ``.kunitconfig`` should also contain any other config +options required by the tests. + +A good starting point for a ``.kunitconfig`` is the KUnit defconfig: .. code-block:: bash cd $PATH_TO_LINUX_REPO cp arch/um/configs/kunit_defconfig .kunitconfig -Verifying KUnit Works ---------------------- +You can then add any other Kconfig options you wish, e.g.: +.. code-block:: none + + CONFIG_LIST_KUNIT_TEST=y + +:doc:`kunit_tool <kunit-tool>` will ensure that all config options set in +``.kunitconfig`` are set in the kernel ``.config`` before running the tests. +It'll warn you if you haven't included the dependencies of the options you're +using. + +.. note:: + Note that removing something from the ``.kunitconfig`` will not trigger a + rebuild of the ``.config`` file: the configuration is only updated if the + ``.kunitconfig`` is not a subset of ``.config``. This means that you can use + other tools (such as make menuconfig) to adjust other config options. + + +Running the tests +----------------- To make sure that everything is set up correctly, simply invoke the Python wrapper from your kernel repo: @@ -62,6 +81,41 @@ followed by a list of tests that are run. All of them should be passing. Because it is building a lot of sources for the first time, the ``Building KUnit kernel`` step may take a while. +Running tests without the KUnit Wrapper +======================================= + +If you'd rather not use the KUnit Wrapper (if, for example, you need to +integrate with other systems, or use an architecture other than UML), KUnit can +be included in any kernel, and the results read out and parsed manually. + +.. note:: + KUnit is not designed for use in a production system, and it's possible that + tests may reduce the stability or security of the system. + + + +Configuring the kernel +---------------------- + +In order to enable KUnit itself, you simply need to enable the ``CONFIG_KUNIT`` +Kconfig option (it's under Kernel Hacking/Kernel Testing and Coverage in +menuconfig). From there, you can enable any KUnit tests you want: they usually +have config options ending in ``_KUNIT_TEST``. + +KUnit and KUnit tests can be compiled as modules: in this case the tests in a +module will be run when the module is loaded. + +Running the tests +----------------- + +Build and run your kernel as usual. Test output will be written to the kernel +log in `TAP <https://testanything.org/>`_ format. + +.. note:: + It's possible that there will be other lines and/or data interspersed in the + TAP output. + + Writing your first test ======================= -- 2.25.1.481.gfbce0eb801-goog

4 years, 1 month

4
4
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror February 2020