This patch set proposes KUnit, a lightweight unit testing and mocking
framework for the Linux kernel.
Unlike Autotest and kselftest, KUnit is a true unit testing framework;
it does not require installing the kernel on a test machine or in a VM
and does not require tests to be written in userspace running on a host
kernel. Additionally, KUnit is fast: From invocation to completion KUnit
can run several dozen tests in under a second. Currently, the entire
KUnit test suite for KUnit runs in under a second from the initial
invocation (build time excluded).
KUnit is heavily inspired by JUnit, Python's unittest.mock, and
Googletest/Googlemock for C++. KUnit provides facilities for defining
unit test cases, grouping related test cases into test suites, providing
common infrastructure for running tests, mocking, spying, and much more.
## What's so special about unit testing?
A unit test is supposed to test a single unit of code in isolation,
hence the name. There should be no dependencies outside the control of
the test; this means no external dependencies, which makes tests orders
of magnitudes faster. Likewise, since there are no external dependencies,
there are no hoops to jump through to run the tests. Additionally, this
makes unit tests deterministic: a failing unit test always indicates a
problem. Finally, because unit tests necessarily have finer granularity,
they are able to test all code paths easily solving the classic problem
of difficulty in exercising error handling code.
## Is KUnit trying to replace other testing frameworks for the kernel?
No. Most existing tests for the Linux kernel are end-to-end tests, which
have their place. A well tested system has lots of unit tests, a
reasonable number of integration tests, and some end-to-end tests. KUnit
is just trying to address the unit test space which is currently not
being addressed.
## More information on KUnit
There is a bunch of documentation near the end of this patch set that
describes how to use KUnit and best practices for writing unit tests.
For convenience I am hosting the compiled docs here:
https://google.github.io/kunit-docs/third_party/kernel/docs/
Additionally for convenience, I have applied these patches to a branch:
https://kunit.googlesource.com/linux/+/kunit/rfc/4.19/v3
The repo may be cloned with:
git clone https://kunit.googlesource.com/linux
This patchset is on the kunit/rfc/4.19/v3 branch.
## Changes Since Last Version
- Changed namespace prefix from `test_*` to `kunit_*` as requested by
Shuah.
- Started converting/cleaning up the device tree unittest to use KUnit.
- Started adding KUnit expectations with custom messages.
--
2.20.0.rc0.387.gc7a69e6b6c-goog
Use /bin/echo for console output with options like non
newline (-n) and/or backslash escape (-e).
Tom Zanussi reported that when he tested ftracetest, it
shows "-e" and "-n" options on the console, since a system
which uses dash as the alias of /bin/sh, uses dash built-in
echo command which doesn't accept "-e".
To avoid this issue, use /bin/echo instead of echo for
the output with options.
Fixes: 8f381ac4d321 ("selftests/ftrace: Add color to the PASS / FAIL results")
Link: http://lkml.kernel.org/r/cover.1542221862.git.tom.zanussi@linux.intel.com
Reported-by: Tom Zanussi <zanussi(a)kernel.org>
Suggested-by: Tom Zanussi <zanussi(a)kernel.org>
Signed-off-by: Masami Hiramatsu <mhiramat(a)kernel.org>
---
tools/testing/selftests/ftrace/ftracetest | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
index 75244db70331..ba670b452bdb 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -173,8 +173,8 @@ strip_esc() {
}
prlog() { # messages
- echo -e "$@"
- [ "$LOG_FILE" ] && echo -e "$@" | strip_esc >> $LOG_FILE
+ /bin/echo -e "$@"
+ [ "$LOG_FILE" ] && /bin/echo -e "$@" | strip_esc >> $LOG_FILE
}
catlog() { #file
cat $1
arm64 has a feature called Top Byte Ignore, which allows to embed pointer
tags into the top byte of each pointer. Userspace programs (such as
HWASan, a memory debugging tool [1]) might use this feature and pass
tagged user pointers to the kernel through syscalls or other interfaces.
Right now the kernel is already able to handle user faults with tagged
pointers, due to these patches:
1. 81cddd65 ("arm64: traps: fix userspace cache maintenance emulation on a
tagged pointer")
2. 7dcd9dd8 ("arm64: hw_breakpoint: fix watchpoint matching for tagged
pointers")
3. 276e9327 ("arm64: entry: improve data abort handling of tagged
pointers")
When passing tagged pointers to syscalls, there's a special case of such a
pointer being passed to one of the memory syscalls (mmap, mprotect, etc.).
These syscalls don't do memory accesses but rather deal with memory
ranges, hence an untagged pointer is better suited.
This patchset extends tagged pointer support to non-memory syscalls. This
is done by reusing the untagged_addr macro to untag user pointers when the
kernel performs pointer checking to find out whether the pointer comes
from userspace (most notably in access_ok). The untagging is done only
when the pointer is being checked, the tag is preserved as the pointer
makes its way through the kernel.
One of the alternative approaches to untagging that was considered is to
completely strip the pointer tag as the pointer enters the kernel with
some kind of a syscall wrapper, but that won't work with the countless
number of different ioctl calls. With this approach we would need a custom
wrapper for each ioctl variation, which doesn't seem practical.
The following testing approaches has been taken to find potential issues
with user pointer untagging:
1. Static testing (with sparse [2] and separately with a custom static
analyzer based on Clang) to track casts of __user pointers to integer
types to find places where untagging needs to be done.
2. Dynamic testing: adding BUG_ON(has_tag(addr)) to find_vma() and running
a modified syzkaller version that passes tagged pointers to the kernel.
Based on the results of the testing the requried patches have been added
to the patchset.
This patchset has been merged into the Pixel 2 kernel tree and is now
being used to enable testing of Pixel 2 phones with HWASan.
This patchset is a prerequisite for ARM's memory tagging hardware feature
support [3].
Thanks!
[1] http://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html
[2] https://github.com/lucvoo/sparse-dev/commit/5f960cb10f56ec2017c128ef9d16060…
[3] https://community.arm.com/processors/b/blog/posts/arm-a-profile-architectur…
Changes in v9:
- Rebased onto 4.20-rc6.
- Used u64 instead of __u64 in type casts in the untagged_addr macro for
arm64.
- Added braces around (addr) in the untagged_addr macro for other arches.
Changes in v8:
- Rebased onto 65102238 (4.20-rc1).
- Added a note to the cover letter on why syscall wrappers/shims that untag
user pointers won't work.
- Added a note to the cover letter that this patchset has been merged into
the Pixel 2 kernel tree.
- Documentation fixes, in particular added a list of syscalls that don't
support tagged user pointers.
Changes in v7:
- Rebased onto 17b57b18 (4.19-rc6).
- Dropped the "arm64: untag user address in __do_user_fault" patch, since
the existing patches already handle user faults properly.
- Dropped the "usb, arm64: untag user addresses in devio" patch, since the
passed pointer must come from a vma and therefore be untagged.
- Dropped the "arm64: annotate user pointers casts detected by sparse"
patch (see the discussion to the replies of the v6 of this patchset).
- Added more context to the cover letter.
- Updated Documentation/arm64/tagged-pointers.txt.
Changes in v6:
1 From 502466b9652c57a23af3bd72124144319212f30b Mon Sep 17 00:00:00 2001
- Added annotations for user pointer casts found by sparse.
1 From 502466b9652c57a23af3bd72124144319212f30b Mon Sep 17 00:00:00 2001
- Rebased onto 050cdc6c (4.19-rc1+).
1 From 502466b9652c57a23af3bd72124144319212f30b Mon Sep 17 00:00:00 2001
Changes in v5:
- Added 3 new patches that add untagging to places found with static
analysis.
- Rebased onto 44c929e1 (4.18-rc8).
Changes in v4:
- Added a selftest for checking that passing tagged pointers to the
kernel succeeds.
- Rebased onto 81e97f013 (4.18-rc1+).
Changes in v3:
- Rebased onto e5c51f30 (4.17-rc6+).
- Added linux-arch@ to the list of recipients.
Changes in v2:
- Rebased onto 2d618bdf (4.17-rc3+).
- Removed excessive untagging in gup.c.
- Removed untagging pointers returned from __uaccess_mask_ptr.
Changes in v1:
- Rebased onto 4.17-rc1.
Changes in RFC v2:
- Added "#ifndef untagged_addr..." fallback in linux/uaccess.h instead of
defining it for each arch individually.
- Updated Documentation/arm64/tagged-pointers.txt.
- Dropped "mm, arm64: untag user addresses in memory syscalls".
- Rebased onto 3eb2ce82 (4.16-rc7).
Reviewed-by: Luc Van Oostenryck <luc.vanoostenryck(a)gmail.com>
Signed-off-by: Andrey Konovalov <andreyknvl(a)google.com>
Andrey Konovalov (8):
arm64: add type casts to untagged_addr macro
uaccess: add untagged_addr definition for other arches
arm64: untag user addresses in access_ok and __uaccess_mask_ptr
mm, arm64: untag user addresses in mm/gup.c
lib, arm64: untag addrs passed to strncpy_from_user and strnlen_user
fs, arm64: untag user address in copy_mount_options
arm64: update Documentation/arm64/tagged-pointers.txt
selftests, arm64: add a selftest for passing tagged pointers to kernel
Documentation/arm64/tagged-pointers.txt | 25 +++++++++++--------
arch/arm64/include/asm/uaccess.h | 14 +++++++----
fs/namespace.c | 2 +-
include/linux/uaccess.h | 4 +++
lib/strncpy_from_user.c | 2 ++
lib/strnlen_user.c | 2 ++
mm/gup.c | 4 +++
tools/testing/selftests/arm64/.gitignore | 1 +
tools/testing/selftests/arm64/Makefile | 11 ++++++++
.../testing/selftests/arm64/run_tags_test.sh | 12 +++++++++
tools/testing/selftests/arm64/tags_test.c | 19 ++++++++++++++
11 files changed, 80 insertions(+), 16 deletions(-)
create mode 100644 tools/testing/selftests/arm64/.gitignore
create mode 100644 tools/testing/selftests/arm64/Makefile
create mode 100755 tools/testing/selftests/arm64/run_tags_test.sh
create mode 100644 tools/testing/selftests/arm64/tags_test.c
--
2.20.0.rc2.403.gdbc3b29805-goog
Fix fw_filesystem.sh to run in an automated environment under busybox.
After this change, fw_run_tests.sh still fails at some point in fw_fallback.sh,
with error "usermode helper disabled so ignoring test". This is coming from
fw_lib.sh:verify_reqs() because $HAS_FW_LOADER_USER_HELPER is set to no.
Dan Rue (2):
selftests: firmware: remove use of non-standard diff -Z option
selftests: firmware: add CONFIG_FW_LOADER_USER_HELPER_FALLBACK to
config
tools/testing/selftests/firmware/config | 1 +
tools/testing/selftests/firmware/fw_filesystem.sh | 9 +++------
2 files changed, 4 insertions(+), 6 deletions(-)
--
2.19.1
An expansion field was added to the kernel copy of this structure for
future use. See mm/gup_benchmark.c.
Add the same expansion field here, so that the IOCTL command decodes
correctly. Otherwise, it fails with EINVAL.
Signed-off-by: Alison Schofield <alison.schofield(a)intel.com>
---
tools/testing/selftests/vm/gup_benchmark.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/vm/gup_benchmark.c b/tools/testing/selftests/vm/gup_benchmark.c
index 880b96fc80d4..c0534e298b51 100644
--- a/tools/testing/selftests/vm/gup_benchmark.c
+++ b/tools/testing/selftests/vm/gup_benchmark.c
@@ -25,6 +25,7 @@ struct gup_benchmark {
__u64 size;
__u32 nr_pages_per_call;
__u32 flags;
+ __u64 expansion[10]; /* For future use */
};
int main(int argc, char **argv)
--
2.14.1
From: Ahmed Abd El Mawgood <ahmedsoliman(a)mena.vt.edu>
madvise() returns -1 without CONFIG_TRANSPARENT_HUGEPAGE=y. That would
trigger asserts when checking for return value of madvice. Following
similar decision to [1]. I thought it is ok to assume that madvise()
MADV_NOHUGEPAGE failures implies that THP is not supported by host kernel.
Other options was to check for Transparent Huge Page support in
/sys/kernel/mm/transparent_hugepage/enabled.
-- links --
[1] https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg04514.html
Signed-off-by: Ahmed Abd El Mawgood <ahmedsoliman(a)mena.vt.edu>
---
tools/testing/selftests/kvm/lib/kvm_util.c | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index 1b41e71283d5..437c5bb48061 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -586,14 +586,23 @@ void vm_userspace_mem_region_add(struct kvm_vm *vm,
src_type == VM_MEM_SRC_ANONYMOUS_THP ? huge_page_size : 1);
/* As needed perform madvise */
- if (src_type == VM_MEM_SRC_ANONYMOUS || src_type == VM_MEM_SRC_ANONYMOUS_THP) {
+ if (src_type == VM_MEM_SRC_ANONYMOUS) {
+ /*
+ * Neglect madvise error because it is ok to not have THP
+ * support in this case.
+ */
+ madvise(region->host_mem, npages * vm->page_size,
+ MADV_NOHUGEPAGE);
+ } else if (src_type == VM_MEM_SRC_ANONYMOUS_THP) {
ret = madvise(region->host_mem, npages * vm->page_size,
- src_type == VM_MEM_SRC_ANONYMOUS ? MADV_NOHUGEPAGE : MADV_HUGEPAGE);
+ MADV_HUGEPAGE);
TEST_ASSERT(ret == 0, "madvise failed,\n"
- " addr: %p\n"
- " length: 0x%lx\n"
- " src_type: %x",
- region->host_mem, npages * vm->page_size, src_type);
+ "Does the kernel have CONFIG_TRANSPARENT_HUGEPAGE=y\n"
+ " addr: %p\n"
+ " length: 0x%lx\n"
+ " src_type: %x\n",
+ region->host_mem, npages * vm->page_size,
+ src_type);
}
region->unused_phy_pages = sparsebit_alloc();
--
2.18.1