December 2019 - Linux-kselftest-mirror

[PATCH v15 00/24] selftests, powerpc, x86: Memory Protection Keys

by Sandipan Das

Memory protection keys enables an application to protect its address space from inadvertent access by its own code. This feature is now enabled on powerpc and has been available since 4.16-rc1. The patches move the selftests to arch neutral directory and enhance their test coverage. Testing ------- Verified for correctness on powerpc. Need help with x86 testing as I do not have access to a Skylake server. Client platforms like Coffee Lake do not have the required feature bits set in CPUID. Changelog --------- Link to previous version (v14): https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=55981&state=* v15: (1) Rebased on top of latest master. (2) Addressed review comments from Dave Hansen. (3) Moved code for getting or setting pkey bits to new helpers. These changes replace patch 7 of v14. (4) Added a fix which ensures that the correct count of reserved keys is used across different platforms. (5) Added a fix which ensures that the correct page size is used as powerpc supports both 4K and 64K pages. v14: (1) Incorporated another round of comments from Dave Hansen. v13: (1) Incorporated comments for Dave Hansen. (2) Added one more test for correct pkey-0 behavior. v12: (1) Fixed the offset of pkey field in the siginfo structure for x86_64 and powerpc. And tries to use the actual field if the headers have it defined. v11: (1) Fixed a deadlock in the ptrace testcase. v10 and prior: (1) Moved the testcase to arch neutral directory. (2) Split the changes into incremental patches. Desnes A. Nunes do Rosario (1): selftests/vm/pkeys: Fix number of reserved powerpc pkeys Ram Pai (17): selftests/x86/pkeys: Move selftests to arch-neutral directory selftests/vm/pkeys: Rename all references to pkru to a generic name selftests/vm/pkeys: Move generic definitions to header file selftests/vm/pkeys: Typecast the pkey register selftests/vm/pkeys: Fix pkey_disable_clear() selftests/vm/pkeys: Fix assertion in pkey_disable_set/clear() selftests/vm/pkeys: Fix alloc_random_pkey() to make it really random selftests/vm/pkeys: Introduce generic pkey abstractions selftests/vm/pkeys: Introduce powerpc support selftests/vm/pkeys: Fix assertion in test_pkey_alloc_exhaust() selftests/vm/pkeys: Improve checks to determine pkey support selftests/vm/pkeys: Associate key on a mapped page and detect access violation selftests/vm/pkeys: Associate key on a mapped page and detect write violation selftests/vm/pkeys: Detect write violation on a mapped access-denied-key page selftests/vm/pkeys: Introduce a sub-page allocator selftests/vm/pkeys: Test correct behaviour of pkey-0 selftests/vm/pkeys: Override access right definitions on powerpc Sandipan Das (3): selftests: vm: pkeys: Add helpers for pkey bits selftests: vm: pkeys: Use the correct huge page size selftests: vm: pkeys: Use the correct page size on powerpc Thiago Jung Bauermann (2): selftests/vm/pkeys: Move some definitions to arch-specific header selftests/vm/pkeys: Make gcc check arguments of sigsafe_printf() tools/testing/selftests/vm/.gitignore | 1 + tools/testing/selftests/vm/Makefile | 5 + tools/testing/selftests/vm/pkey-helpers.h | 226 ++++++ tools/testing/selftests/vm/pkey-powerpc.h | 138 ++++ tools/testing/selftests/vm/pkey-x86.h | 183 +++++ .../selftests/{x86 => vm}/protection_keys.c | 688 ++++++++++-------- tools/testing/selftests/x86/.gitignore | 1 - tools/testing/selftests/x86/pkey-helpers.h | 219 ------ 8 files changed, 931 insertions(+), 530 deletions(-) create mode 100644 tools/testing/selftests/vm/pkey-helpers.h create mode 100644 tools/testing/selftests/vm/pkey-powerpc.h create mode 100644 tools/testing/selftests/vm/pkey-x86.h rename tools/testing/selftests/{x86 => vm}/protection_keys.c (74%) delete mode 100644 tools/testing/selftests/x86/pkey-helpers.h -- 2.17.1

4 years, 11 months

5
35
0 0

[PATCH v2 0/8] vdso: Complete the conversion to 32bit syscalls

by Vincenzo Frascino

This patch series is a follow up to "lib/vdso, x86/vdso: Fix fallout from generic VDSO conversion" [1]. The main purpose is to complete the 32bit vDSOs conversion to use the legacy 32bit syscalls as a fallback. With the conversion of all the architectures present in -next complete, this patch series removes as well the conditional choice in between 32 and 64 bit for 32bit vDSOs. This series has been rebased on linux-next/master. [1] https://lkml.org/lkml/2019/7/28/86 Cc: Catalin Marinas <catalin.marinas(a)arm.com> Cc: Will Deacon <will(a)kernel.org> Cc: Paul Burton <paul.burton(a)mips.com> Cc: Thomas Gleixner <tglx(a)linutronix.de> Cc: Dmitry Safonov <0x7f454c46(a)gmail.com> Cc: Andy Lutomirski <luto(a)kernel.org> Signed-off-by: Vincenzo Frascino <vincenzo.frascino(a)arm.com> Vincenzo Frascino (8): arm64: compat: vdso: Expose BUILD_VDSO32 lib: vdso: Build 32 bit specific functions in the right context mips: compat: vdso: Use legacy syscalls as fallback lib: vdso: Remove VDSO_HAS_32BIT_FALLBACK lib: vdso: Remove checks on return value for 32 bit vDSO arm64: compat: vdso: Remove unused VDSO_HAS_32BIT_FALLBACK mips: vdso: Remove unused VDSO_HAS_32BIT_FALLBACK x86: vdso: Remove unused VDSO_HAS_32BIT_FALLBACK .../include/asm/vdso/compat_gettimeofday.h | 2 +- arch/mips/include/asm/vdso/gettimeofday.h | 43 +++++++++++++++++++ arch/mips/vdso/config-n32-o32-env.c | 1 + arch/x86/include/asm/vdso/gettimeofday.h | 2 - lib/vdso/gettimeofday.c | 30 ++++++------- 5 files changed, 57 insertions(+), 21 deletions(-) -- 2.23.0

4 years, 11 months

4
22
0 0

[PATCH v2] selftests/lkdtm: Add tests for LKDTM targets

by Kees Cook

This adds a basic framework for running all the "safe" LKDTM tests. This will allow easy introspection into any selftest logs to examine the results of most LKDTM tests. Signed-off-by: Kees Cook <keescook(a)chromium.org> --- v2: refreshed for v5.5, added new tests since v1 v1: https://lore.kernel.org/lkml/201905091013.E228F0F0BE@keescook/ --- MAINTAINERS | 1 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/lkdtm/Makefile | 12 ++++ tools/testing/selftests/lkdtm/config | 1 + tools/testing/selftests/lkdtm/run.sh | 91 +++++++++++++++++++++++++ tools/testing/selftests/lkdtm/tests.txt | 70 +++++++++++++++++++ 6 files changed, 176 insertions(+) create mode 100644 tools/testing/selftests/lkdtm/Makefile create mode 100644 tools/testing/selftests/lkdtm/config create mode 100755 tools/testing/selftests/lkdtm/run.sh create mode 100644 tools/testing/selftests/lkdtm/tests.txt diff --git a/MAINTAINERS b/MAINTAINERS index cc0a4a8ae06a..eacc00c6cfd5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9578,6 +9578,7 @@ LINUX KERNEL DUMP TEST MODULE (LKDTM) M: Kees Cook <keescook(a)chromium.org> S: Maintained F: drivers/misc/lkdtm/* +F: tools/testing/selftests/lkdtm/* LINUX KERNEL MEMORY CONSISTENCY MODEL (LKMM) M: Alan Stern <stern(a)rowland.harvard.edu> diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index b001c602414b..f0b02a12ba39 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -26,6 +26,7 @@ TARGETS += kexec TARGETS += kvm TARGETS += lib TARGETS += livepatch +TARGETS += lkdtm TARGETS += membarrier TARGETS += memfd TARGETS += memory-hotplug diff --git a/tools/testing/selftests/lkdtm/Makefile b/tools/testing/selftests/lkdtm/Makefile new file mode 100644 index 000000000000..1bcc9ee990eb --- /dev/null +++ b/tools/testing/selftests/lkdtm/Makefile @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0 +# Makefile for LKDTM regression tests + +include ../lib.mk + +# NOTE: $(OUTPUT) won't get default value if used before lib.mk +TEST_FILES := tests.txt +TEST_GEN_PROGS = $(patsubst %,$(OUTPUT)/%.sh,$(shell awk '{print $$1}' tests.txt | sed -e 's/\#//')) +all: $(TEST_GEN_PROGS) + +$(OUTPUT)/%: run.sh tests.txt + install -m 0744 run.sh $@ diff --git a/tools/testing/selftests/lkdtm/config b/tools/testing/selftests/lkdtm/config new file mode 100644 index 000000000000..d874990e442b --- /dev/null +++ b/tools/testing/selftests/lkdtm/config @@ -0,0 +1 @@ +CONFIG_LKDTM=y diff --git a/tools/testing/selftests/lkdtm/run.sh b/tools/testing/selftests/lkdtm/run.sh new file mode 100755 index 000000000000..793ee0d5d5a3 --- /dev/null +++ b/tools/testing/selftests/lkdtm/run.sh @@ -0,0 +1,91 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# This reads tests.txt for the list of LKDTM tests to invoke. Any marked +# with a leading "#" are skipped. The rest of the line after the +# test name is either the text to look for in dmesg for a "success", +# or the rationale for why a test is marked to be skipped. +# +set -e +TRIGGER=/sys/kernel/debug/provoke-crash/DIRECT + +# Verify we have LKDTM available in the kernel. +if [ ! -r $TRIGGER ] ; then + /sbin/modprobe -q lkdtm || true + if [ ! -r $TRIGGER ] ; then + echo "Cannot find $TRIGGER (missing CONFIG_LKDTM?)" + else + echo "Cannot write $TRIGGER (need to run as root?)" + fi + # Skip this test + exit 4 +fi + +# Figure out which test to run from our script name. +test=$(basename $0 .sh) +# Look up details about the test from master list of LKDTM tests. +line=$(egrep '^#?'"$test"'\b' tests.txt) +if [ -z "$line" ]; then + echo "Skipped: missing test '$test' in tests.txt" + exit 4 +fi +# Check that the test is known to LKDTM. +if ! egrep -q '^'"$test"'$' "$TRIGGER" ; then + echo "Skipped: test '$test' missing in $TRIGGER!" + exit 4 +fi + +# Extract notes/expected output from test list. +test=$(echo "$line" | cut -d" " -f1) +if echo "$line" | grep -q ' ' ; then + expect=$(echo "$line" | cut -d" " -f2-) +else + expect="" +fi + +# If the test is commented out, report a skip +if echo "$test" | grep -q '^#' ; then + test=$(echo "$test" | cut -c2-) + if [ -z "$expect" ]; then + expect="crashes entire system" + fi + echo "Skipping $test: $expect" + exit 4 +fi + +# If no expected output given, assume an Oops with back trace is success. +if [ -z "$expect" ]; then + expect="call trace:" +fi + +# Clear out dmesg for output reporting +dmesg -c >/dev/null + +# Prepare log for report checking +LOG=$(mktemp --tmpdir -t lkdtm-XXXXXX) +cleanup() { + rm -f "$LOG" +} +trap cleanup EXIT + +# Most shells yell about signals and we're expecting the "cat" process +# to usually be killed by the kernel. So we have to run it in a sub-shell +# and silence errors. +($SHELL -c 'cat <(echo '"$test"') >'"$TRIGGER" 2>/dev/null) || true + +# Record and dump the results +dmesg -c >"$LOG" +cat "$LOG" +# Check for expected output +if egrep -qi "$expect" "$LOG" ; then + echo "$test: saw '$expect': ok" + exit 0 +else + if egrep -qi XFAIL: "$LOG" ; then + echo "$test: saw 'XFAIL': [SKIP]" + exit 4 + else + echo "$test: missing '$expect': [FAIL]" + exit 1 + fi +fi diff --git a/tools/testing/selftests/lkdtm/tests.txt b/tools/testing/selftests/lkdtm/tests.txt new file mode 100644 index 000000000000..fc55f8ef8bee --- /dev/null +++ b/tools/testing/selftests/lkdtm/tests.txt @@ -0,0 +1,70 @@ +#PANIC +BUG kernel BUG at +WARNING WARNING: +WARNING_MESSAGE message trigger +EXCEPTION +#LOOP Hangs the system +#EXHAUST_STACK Corrupts memory on failure +#CORRUPT_STACK Crashes entire system on success +#CORRUPT_STACK_STRONG Crashes entire system on success +CORRUPT_LIST_ADD list_add corruption +CORRUPT_LIST_DEL list_del corruption +CORRUPT_USER_DS Invalid address limit on user-mode return +STACK_GUARD_PAGE_LEADING +STACK_GUARD_PAGE_TRAILING +UNSET_SMEP CR4 bits went missing +UNALIGNED_LOAD_STORE_WRITE +#OVERWRITE_ALLOCATION Corrupts memory on failure +#WRITE_AFTER_FREE Corrupts memory on failure +READ_AFTER_FREE +#WRITE_BUDDY_AFTER_FREE Corrupts memory on failure +READ_BUDDY_AFTER_FREE +SLAB_FREE_DOUBLE +SLAB_FREE_CROSS +SLAB_FREE_PAGE +#SOFTLOCKUP Hangs the system +#HARDLOCKUP Hangs the system +#SPINLOCKUP Hangs the system +#HUNG_TASK Hangs the system +EXEC_DATA +EXEC_STACK +EXEC_KMALLOC +EXEC_VMALLOC +EXEC_RODATA +EXEC_USERSPACE +EXEC_NULL +ACCESS_USERSPACE +ACCESS_NULL +WRITE_RO +WRITE_RO_AFTER_INIT +WRITE_KERN +REFCOUNT_INC_OVERFLOW +REFCOUNT_ADD_OVERFLOW +REFCOUNT_INC_NOT_ZERO_OVERFLOW +REFCOUNT_ADD_NOT_ZERO_OVERFLOW +REFCOUNT_DEC_ZERO +REFCOUNT_DEC_NEGATIVE Negative detected: saturated +REFCOUNT_DEC_AND_TEST_NEGATIVE Negative detected: saturated +REFCOUNT_SUB_AND_TEST_NEGATIVE Negative detected: saturated +REFCOUNT_INC_ZERO +REFCOUNT_ADD_ZERO +REFCOUNT_INC_SATURATED Saturation detected: still saturated +REFCOUNT_DEC_SATURATED Saturation detected: still saturated +REFCOUNT_ADD_SATURATED Saturation detected: still saturated +REFCOUNT_INC_NOT_ZERO_SATURATED +REFCOUNT_ADD_NOT_ZERO_SATURATED +REFCOUNT_DEC_AND_TEST_SATURATED Saturation detected: still saturated +REFCOUNT_SUB_AND_TEST_SATURATED Saturation detected: still saturated +#REFCOUNT_TIMING timing only +#ATOMIC_TIMING timing only +USERCOPY_HEAP_SIZE_TO +USERCOPY_HEAP_SIZE_FROM +USERCOPY_HEAP_WHITELIST_TO +USERCOPY_HEAP_WHITELIST_FROM +USERCOPY_STACK_FRAME_TO +USERCOPY_STACK_FRAME_FROM +USERCOPY_STACK_BEYOND +USERCOPY_KERNEL +USERCOPY_KERNEL_DS +STACKLEAK_ERASING OK: the rest of the thread stack is properly erased +CFI_FORWARD_PROTO -- 2.20.1 -- Kees Cook

4 years, 11 months

2
1
0 0

What is the best way to compare an unsigned and a constant?

by SeongJae Park

Hello, I have a function returning 'unsigned long', and would like to write a kunit test for the function, as below. unsigned long foo(void) { return 42; } static void foo_test(struct kunit *test) { KUNIT_EXPECT_EQ(test, 42, foo()); } However, this kunit gives me below warning for the above code: /.../linux/include/linux/kernel.h:842:29: warning: comparison of distinct pointer types lacks a cast (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) ^ /.../linux/include/kunit/test.h:493:9: note: in expansion of macro ‘__typecheck’ ((void)__typecheck(__left, __right)); \ ^~~~~~~~~~~ /.../linux/include/kunit/test.h:517:2: note: in expansion of macro ‘KUNIT_BASE_BINARY_ASSERTION’ KUNIT_BASE_BINARY_ASSERTION(test, \ ^~~~~~~~~~~~~~~~~~~~~~~~~~~ /.../linux/include/kunit/test.h:606:2: note: in expansion of macro ‘KUNIT_BASE_EQ_MSG_ASSERTION’ KUNIT_BASE_EQ_MSG_ASSERTION(test, \ ^~~~~~~~~~~~~~~~~~~~~~~~~~~ /.../linux/include/kunit/test.h:616:2: note: in expansion of macro ‘KUNIT_BINARY_EQ_MSG_ASSERTION’ KUNIT_BINARY_EQ_MSG_ASSERTION(test, \ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /.../linux/include/kunit/test.h:979:2: note: in expansion of macro ‘KUNIT_BINARY_EQ_ASSERTION’ KUNIT_BINARY_EQ_ASSERTION(test, KUNIT_EXPECTATION, left, right) ^~~~~~~~~~~~~~~~~~~~~~~~~ /.../linux/mm/foo-test.h:565:2: note: in expansion of macro ‘KUNIT_EXPECT_EQ’ KUNIT_EXPECT_EQ(test, 42, foo()); ^~~~~~~~~~~~~~~ I could remove the warning by explicitly type casting the constant as below: KUNIT_EXPECT_EQ(test, (unsigned long)42, foo()); However, now 'checkpatch.pl' complains about the type casting as below. WARNING: Unnecessary typecast of c90 int constant #565: FILE: mm/foo-test.h:565: + KUNIT_EXPECT_EQ(test, (unsigned long)42, foo()); Of course, there could be several work-arounds for these warnings, such as using 'EXPECT_TRUE(test, 42 == foo())' or casting the function's return value. Nonetheless, I'm not sure what is the right way. Could you please let me know what is the recommended way for this case? Thanks, SeongJae Park

4 years, 11 months

3
7
0 0

[PATCH v3 0/8] Create a userfaultfd demand paging test

by Ben Gardon

When handling page faults for many vCPUs during demand paging, KVM's MMU lock becomes highly contended. This series creates a test with a naive userfaultfd based demand paging implementation to demonstrate that contention. This test serves both as a functional test of userfaultfd and a microbenchmark of demand paging performance with a variable number of vCPUs and memory per vCPU. The test creates N userfaultfd threads, N vCPUs, and a region of memory with M pages per vCPU. The N userfaultfd polling threads are each set up to serve faults on a region of memory corresponding to one of the vCPUs. Each of the vCPUs is then started, and touches each page of its disjoint memory region, sequentially. In response to faults, the userfaultfd threads copy a static buffer into the guest's memory. This creates a worst case for MMU lock contention as we have removed most of the contention between the userfaultfd threads and there is no time required to fetch the contents of guest memory. This test was run successfully on Intel Haswell, Broadwell, and Cascadelake hosts with a variety of vCPU counts and memory sizes. This test was adapted from the dirty_log_test. The series can also be viewed in Gerrit here: https://linux-review.googlesource.com/c/virt/kvm/kvm/+/1464 (Thanks to Dmitry Vyukov <dvyukov(a)google.com> for setting up the Gerrit instance) Ben Gardon (9): KVM: selftests: Create a demand paging test KVM: selftests: Add demand paging content to the demand paging test KVM: selftests: Add memory size parameter to the demand paging test KVM: selftests: Pass args to vCPU instead of using globals KVM: selftests: Support multiple vCPUs in demand paging test KVM: selftests: Time guest demand paging KVM: selftests: Add parameter to _vm_create for memslot 0 base paddr KVM: selftests: Support large VMs in demand paging test Add static flag tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 4 +- .../selftests/kvm/demand_paging_test.c | 610 ++++++++++++++++++ tools/testing/selftests/kvm/dirty_log_test.c | 2 +- .../testing/selftests/kvm/include/kvm_util.h | 3 +- tools/testing/selftests/kvm/lib/kvm_util.c | 7 +- 6 files changed, 621 insertions(+), 6 deletions(-) create mode 100644 tools/testing/selftests/kvm/demand_paging_test.c -- 2.23.0.444.g18eeb5a265-goog

4 years, 11 months

3
25
0 0

[PATCH v11 00/25] mm/gup: track dma-pinned pages: FOLL_PIN

by John Hubbard

Hi, This implements an API naming change (put_user_page*() --> unpin_user_page*()), and also implements tracking of FOLL_PIN pages. It extends that tracking to a few select subsystems. More subsystems will be added in follow up work. Christoph Hellwig, a point of interest: a) I've moved the bulk of the code out of the inline functions, as requested, for the devmap changes (patch 4: "mm: devmap: refactor 1-based refcounting for ZONE_DEVICE pages"). Changes since v10: Remaining fixes resulting from Jan Kara's reviews: * Shifted to using the sign bit in page_dma_pinned() to allow accurate results even in the overflow case. See the comments in that routine for details. This allowed getting rid of the new page_ref_zero_or_close_to_bias_overflow(), in favor of a simple sign check via "page_ref_count() <= 0"). * Simplified some of the huge_memory.c changes, and simplified a gup.c WARN invocation. * Now using a standard -ENOMEM for most try_grab_page() failures. * Got rid of tabs in the comment headers (I had thought they were required there, but it's actually the reverse: they are not allowed there). * Rebased against 5.5-rc2 and retested. * Added Jan Kara's reviewed-by tag for patch 23 (the main patch of the series). Changes since v9: Fixes resulting from Jan Kara's and Jonathan Corbet's reviews: * Removed reviewed-by tags from the "mm/gup: track FOLL_PIN pages" (those were improperly inherited from the much smaller refactoring patch that was merged into it). * Made try_grab_compound_head() and try_grab_page() behavior similar in their behavior with flags, in order to avoid "gotchas" later. * follow_trans_huge_pmd(): moved the try_grab_page() to earlier in the routine, in order to avoid having to undo mlock_vma_page(). * follow_hugetlb_page(): removed a refcount overflow check that is now extraneous (and weaker than what try_grab_page() provides a few lines further down). * Fixed up two Documentation flaws, pointed out by Jonathan Corbet's review. Changes since v8: * Merged the "mm/gup: pass flags arg to __gup_device_* functions" patch into the "mm/gup: track FOLL_PIN pages" patch, as requested by Christoph and Jan. * Changed void grab_page() to bool try_grab_page(), and handled errors at the call sites. (From Jan's review comments.) try_grab_page() attempts to avoid page refcount overflows, even when counting up with GUP_PIN_COUNTING_BIAS increments. * Fixed a bug that I'd introduced, when changing a BUG() to a WARN(). * Added Jan's reviewed-by tag to the " mm/gup: allow FOLL_FORCE for get_user_pages_fast()" patch. * Documentation: pin_user_pages.rst: fixed an incorrect gup_benchmark invocation, left over from the pin_longterm days, spotted while preparing this version. * Rebased onto today's linux.git (-rc1), and re-tested. Changes since v7: * Rebased onto Linux 5.5-rc1 * Reworked the grab_page() and try_grab_compound_head(), for API consistency and less diffs (thanks to Jan Kara's reviews). * Added Leon Romanovsky's reviewed-by tags for two of the IB-related patches. * patch 4 refactoring changes, as mentioned above. There is a git repo and branch, for convenience: git@github.com:johnhubbard/linux.git pin_user_pages_tracking_v8 For the remaining list of "changes since version N", those are all in v7, which is here: https://lore.kernel.org/r/20191121071354.456618-1-jhubbard@nvidia.com ============================================================ Overview: This is a prerequisite to solving the problem of proper interactions between file-backed pages, and [R]DMA activities, as discussed in [1], [2], [3], and in a remarkable number of email threads since about 2017. :) A new internal gup flag, FOLL_PIN is introduced, and thoroughly documented in the last patch's Documentation/vm/pin_user_pages.rst. I believe that this will provide a good starting point for doing the layout lease work that Ira Weiny has been working on. That's because these new wrapper functions provide a clean, constrained, systematically named set of functionality that, again, is required in order to even know if a page is "dma-pinned". In contrast to earlier approaches, the page tracking can be incrementally applied to the kernel call sites that, until now, have been simply calling get_user_pages() ("gup"). In other words, opt-in by changing from this: get_user_pages() (sets FOLL_GET) put_page() to this: pin_user_pages() (sets FOLL_PIN) unpin_user_page() ============================================================ Testing: * I've done some overall kernel testing (LTP, and a few other goodies), and some directed testing to exercise some of the changes. And as you can see, gup_benchmark is enhanced to exercise this. Basically, I've been able to runtime test the core get_user_pages() and pin_user_pages() and related routines, but not so much on several of the call sites--but those are generally just a couple of lines changed, each. Not much of the kernel is actually using this, which on one hand reduces risk quite a lot. But on the other hand, testing coverage is low. So I'd love it if, in particular, the Infiniband and PowerPC folks could do a smoke test of this series for me. Runtime testing for the call sites so far is pretty light: * io_uring: Some directed tests from liburing exercise this, and they pass. * process_vm_access.c: A small directed test passes. * gup_benchmark: the enhanced version hits the new gup.c code, and passes. * infiniband: ran "ib_write_bw", which exercises the umem.c changes, but not the other changes. * VFIO: compiles (I'm vowing to set up a run time test soon, but it's not ready just yet) * powerpc: it compiles... * drm/via: compiles... * goldfish: compiles... * net/xdp: compiles... * media/v4l2: compiles... [1] Some slow progress on get_user_pages() (Apr 2, 2019): https://lwn.net/Articles/784574/ [2] DMA and get_user_pages() (LPC: Dec 12, 2018): https://lwn.net/Articles/774411/ [3] The trouble with get_user_pages() (Apr 30, 2018): https://lwn.net/Articles/753027/ Dan Williams (1): mm: Cleanup __put_devmap_managed_page() vs ->page_free() John Hubbard (24): mm/gup: factor out duplicate code from four routines mm/gup: move try_get_compound_head() to top, fix minor issues mm: devmap: refactor 1-based refcounting for ZONE_DEVICE pages goldish_pipe: rename local pin_user_pages() routine mm: fix get_user_pages_remote()'s handling of FOLL_LONGTERM vfio: fix FOLL_LONGTERM use, simplify get_user_pages_remote() call mm/gup: allow FOLL_FORCE for get_user_pages_fast() IB/umem: use get_user_pages_fast() to pin DMA pages mm/gup: introduce pin_user_pages*() and FOLL_PIN goldish_pipe: convert to pin_user_pages() and put_user_page() IB/{core,hw,umem}: set FOLL_PIN via pin_user_pages*(), fix up ODP mm/process_vm_access: set FOLL_PIN via pin_user_pages_remote() drm/via: set FOLL_PIN via pin_user_pages_fast() fs/io_uring: set FOLL_PIN via pin_user_pages() net/xdp: set FOLL_PIN via pin_user_pages() media/v4l2-core: set pages dirty upon releasing DMA buffers media/v4l2-core: pin_user_pages (FOLL_PIN) and put_user_page() conversion vfio, mm: pin_user_pages (FOLL_PIN) and put_user_page() conversion powerpc: book3s64: convert to pin_user_pages() and put_user_page() mm/gup_benchmark: use proper FOLL_WRITE flags instead of hard-coding "1" mm, tree-wide: rename put_user_page*() to unpin_user_page*() mm/gup: track FOLL_PIN pages mm/gup_benchmark: support pin_user_pages() and related calls selftests/vm: run_vmtests: invoke gup_benchmark with basic FOLL_PIN coverage Documentation/core-api/index.rst | 1 + Documentation/core-api/pin_user_pages.rst | 232 ++++++++ arch/powerpc/mm/book3s64/iommu_api.c | 10 +- drivers/gpu/drm/via/via_dmablit.c | 6 +- drivers/infiniband/core/umem.c | 19 +- drivers/infiniband/core/umem_odp.c | 13 +- drivers/infiniband/hw/hfi1/user_pages.c | 4 +- drivers/infiniband/hw/mthca/mthca_memfree.c | 8 +- drivers/infiniband/hw/qib/qib_user_pages.c | 4 +- drivers/infiniband/hw/qib/qib_user_sdma.c | 8 +- drivers/infiniband/hw/usnic/usnic_uiom.c | 4 +- drivers/infiniband/sw/siw/siw_mem.c | 4 +- drivers/media/v4l2-core/videobuf-dma-sg.c | 8 +- drivers/nvdimm/pmem.c | 6 - drivers/platform/goldfish/goldfish_pipe.c | 35 +- drivers/vfio/vfio_iommu_type1.c | 35 +- fs/io_uring.c | 6 +- include/linux/mm.h | 155 ++++- include/linux/mmzone.h | 2 + include/linux/page_ref.h | 10 + mm/gup.c | 626 +++++++++++++++----- mm/gup_benchmark.c | 74 ++- mm/huge_memory.c | 29 +- mm/hugetlb.c | 38 +- mm/memremap.c | 76 ++- mm/process_vm_access.c | 28 +- mm/swap.c | 24 + mm/vmstat.c | 2 + net/xdp/xdp_umem.c | 4 +- tools/testing/selftests/vm/gup_benchmark.c | 21 +- tools/testing/selftests/vm/run_vmtests | 22 + 31 files changed, 1145 insertions(+), 369 deletions(-) create mode 100644 Documentation/core-api/pin_user_pages.rst -- 2.24.1

4 years, 11 months

6
66
0 0

[PATCH v6 linux-kselftest-test 0/6] kunit: support building core/tests as modules

by Alan Maguire

The current kunit execution model is to provide base kunit functionality and tests built-in to the kernel. The aim of this series is to allow building kunit itself and tests as modules. This in turn allows a simple form of selective execution; load the module you wish to test. In doing so, kunit itself (if also built as a module) will be loaded as an implicit dependency. Because this requires a core API modification - if a module delivers multiple suites, they must be declared with the kunit_test_suites() macro - we're proposing this patch set as a candidate to be applied to the test tree before too many kunit consumers appear. We attempt to deal with existing consumers in patch 3. Changes since v5: - fixed fs/ext4/Makefile to remove unneeded conditional compilation (Iurii, patch 3) - added Reviewed-by, Acked-by to patches 3, 4, 5 and 6 Changes since v4: - fixed signoff chain to use Co-developed-by: prior to Knut's signoff (Stephen, all patches) - added Reviewed-by, Tested-by for patches 1, 2, 4 and 6 - updated comment describing try-catch-impl.h (Stephen, patch 2) - fixed MODULE_LICENSEs to be GPL v2 (Stephen, patches 3, 5) - added __init to kunit_init() (Stephen, patch 5) Changes since v3: - removed symbol lookup patch for separate submission later - removed use of sysctl_hung_task_timeout_seconds (patch 4, as discussed with Brendan and Stephen) - disabled build of string-stream-test when CONFIG_KUNIT_TEST=m; this is to avoid having to deal with symbol lookup issues - changed string-stream-impl.h back to string-stream.h (Brendan) - added module build support to new list, ext4 tests Changes since v2: - moved string-stream.h header to lib/kunit/string-stream-impl.h (Brendan) (patch 1) - split out non-exported interfaces in try-catch-impl.h (Brendan) (patch 2) - added kunit_find_symbol() and KUNIT_INIT_SYMBOL to lookup non-exported symbols (patches 3, 4) - removed #ifdef MODULE around module licenses (Randy, Brendan, Andy) (patch 4) - replaced kunit_test_suite() with kunit_test_suites() rather than supporting both (Brendan) (patch 4) - lookup sysctl_hung_task_timeout_secs as kunit may be built as a module and the symbol may not be available (patch 5) Alan Maguire (6): kunit: move string-stream.h to lib/kunit kunit: hide unexported try-catch interface in try-catch-impl.h kunit: allow kunit tests to be loaded as a module kunit: remove timeout dependence on sysctl_hung_task_timeout_seconds kunit: allow kunit to be loaded as a module kunit: update documentation to describe module-based build Documentation/dev-tools/kunit/faq.rst | 3 +- Documentation/dev-tools/kunit/index.rst | 3 ++ Documentation/dev-tools/kunit/usage.rst | 16 ++++++++++ fs/ext4/Kconfig | 2 +- fs/ext4/Makefile | 3 +- fs/ext4/inode-test.c | 4 ++- include/kunit/assert.h | 3 +- include/kunit/test.h | 35 ++++++++++++++------ include/kunit/try-catch.h | 10 ------ kernel/sysctl-test.c | 4 ++- lib/Kconfig.debug | 4 +-- lib/kunit/Kconfig | 6 ++-- lib/kunit/Makefile | 14 +++++--- lib/kunit/assert.c | 10 ++++++ lib/kunit/{example-test.c => kunit-example-test.c} | 4 ++- lib/kunit/{test-test.c => kunit-test.c} | 7 ++-- lib/kunit/string-stream-test.c | 5 +-- lib/kunit/string-stream.c | 3 +- {include => lib}/kunit/string-stream.h | 0 lib/kunit/test.c | 25 ++++++++++++++- lib/kunit/try-catch-impl.h | 27 ++++++++++++++++ lib/kunit/try-catch.c | 37 +++++----------------- lib/list-test.c | 4 ++- 23 files changed, 157 insertions(+), 72 deletions(-) rename lib/kunit/{example-test.c => kunit-example-test.c} (97%) rename lib/kunit/{test-test.c => kunit-test.c} (98%) rename {include => lib}/kunit/string-stream.h (100%) create mode 100644 lib/kunit/try-catch-impl.h -- 1.8.3.1

4 years, 11 months

5
12
0 0

[GIT PULL] Kselftest fixes update for Linux 5.5-rc4

by Shuah Khan

Hi Linus, Please pull the following Kselftest update for Linux 5.5-rc4. This Kselftest update for Linux 5.5-rc4 consists of: -- rseq build failures fixes related to glibc 2.30 compatibility from Mathieu Desnoyers -- Kunit fixes and cleanups from SeongJae Park -- Fixes to filesystems/epoll, firmware, and livepatch build failures and skip handling. diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit 46cf053efec6a3a5f343fead837777efe8252a46: Linux 5.5-rc3 (2019-12-22 17:02:23 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-5.5-rc4 for you to fetch changes up to 2a1f40adfb54ca65dc4c93bad444dd23b800a76e: rseq/selftests: Clarify rseq_prepare_unload() helper requirements (2019-12-23 10:52:41 -0700) ---------------------------------------------------------------- linux-kselftest-5.5-rc4 This Kselftest update for Linux 5.5-rc4 consists of: -- rseq build failures fixes related to glibc 2.30 compatibility from Mathieu Desnoyers -- Kunit fixes and cleanups from SeongJae Park -- Fixes to filesystems/epoll, firmware, and livepatch build failures and skip handling. ---------------------------------------------------------------- Mathieu Desnoyers (3): rseq/selftests: Turn off timeout setting rseq/selftests: Fix: Namespace gettid() for compatibility with glibc 2.30 rseq/selftests: Clarify rseq_prepare_unload() helper requirements SeongJae Park (6): docs/kunit/start: Use in-tree 'kunit_defconfig' kunit: Remove duplicated defconfig creation kunit: Create default config in '--build_dir' kunit: Place 'test.log' under the 'build_dir' kunit: Rename 'kunitconfig' to '.kunitconfig' kunit/kunit_tool_test: Test '--build_dir' option run Shuah Khan (3): selftests: filesystems/epoll: fix build error selftests: firmware: Fix it to do root uid check and skip selftests: livepatch: Fix it to do root uid check and skip Documentation/dev-tools/kunit/start.rst | 13 +++++-------- tools/testing/kunit/kunit.py | 18 +++++++++++------- tools/testing/kunit/kunit_kernel.py | 10 +++++----- tools/testing/kunit/kunit_tool_test.py | 10 +++++++++- tools/testing/selftests/filesystems/epoll/Makefile | 2 +- tools/testing/selftests/firmware/fw_lib.sh | 6 ++++++ tools/testing/selftests/livepatch/functions.sh | 15 ++++++++++++++- tools/testing/selftests/livepatch/test-state.sh | 3 +-- tools/testing/selftests/rseq/param_test.c | 18 ++++++++++-------- tools/testing/selftests/rseq/rseq.h | 12 +++++++----- tools/testing/selftests/rseq/settings | 1 + 11 files changed, 70 insertions(+), 38 deletions(-) create mode 100644 tools/testing/selftests/rseq/settings ----------------------------------------------------------------

4 years, 11 months

2
1
0 0

[PATCH AUTOSEL 4.19 26/84] selftests: forwarding: Delete IPv6 address at the end

by Sasha Levin

From: Ido Schimmel <idosch(a)mellanox.com> [ Upstream commit 65cb13986229cec02635a1ecbcd1e2dd18353201 ] When creating the second host in h2_create(), two addresses are assigned to the interface, but only one is deleted. When running the test twice in a row the following error is observed: $ ./router_bridge_vlan.sh TEST: ping [ OK ] TEST: ping6 [ OK ] TEST: vlan [ OK ] $ ./router_bridge_vlan.sh RTNETLINK answers: File exists TEST: ping [ OK ] TEST: ping6 [ OK ] TEST: vlan [ OK ] Fix this by deleting the address during cleanup. Fixes: 5b1e7f9ebd56 ("selftests: forwarding: Test routed bridge interface") Signed-off-by: Ido Schimmel <idosch(a)mellanox.com> Signed-off-by: David S. Miller <davem(a)davemloft.net> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/net/forwarding/router_bridge_vlan.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/net/forwarding/router_bridge_vlan.sh b/tools/testing/selftests/net/forwarding/router_bridge_vlan.sh index fef88eb4b873..fa6a88c50750 100755 --- a/tools/testing/selftests/net/forwarding/router_bridge_vlan.sh +++ b/tools/testing/selftests/net/forwarding/router_bridge_vlan.sh @@ -36,7 +36,7 @@ h2_destroy() { ip -6 route del 2001:db8:1::/64 vrf v$h2 ip -4 route del 192.0.2.0/28 vrf v$h2 - simple_if_fini $h2 192.0.2.130/28 + simple_if_fini $h2 192.0.2.130/28 2001:db8:2::2/64 } router_create() -- 2.20.1

4 years, 11 months

1
0
0 0

[PATCH AUTOSEL 4.19 24/84] selftests/ftrace: Fix multiple kprobe testcase

by Sasha Levin

From: Masami Hiramatsu <mhiramat(a)kernel.org> [ Upstream commit 5cc6c8d4a99d0ee4d5466498e258e593df1d3eb6 ] Fix multiple kprobe event testcase to work it correctly. There are 2 bugfixes. - Since `wc -l FILE` returns not only line number but also FILE filename, following "if" statement always failed. Fix this bug by replacing it with 'cat FILE | wc -l' - Since "while do-done loop" block with pipeline becomes a subshell, $N local variable is not update outside of the loop. Fix this bug by using actual target number (256) instead of $N. Signed-off-by: Masami Hiramatsu <mhiramat(a)kernel.org> Reviewed-by: Steven Rostedt (VMware) <rostedt(a)goodmis.org> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- .../selftests/ftrace/test.d/kprobe/multiple_kprobes.tc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc index ce361b9d62cf..da298f191086 100644 --- a/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc +++ b/tools/testing/selftests/ftrace/test.d/kprobe/multiple_kprobes.tc @@ -25,9 +25,9 @@ while read i; do test $N -eq 256 && break done -L=`wc -l kprobe_events` -if [ $L -ne $N ]; then - echo "The number of kprobes events ($L) is not $N" +L=`cat kprobe_events | wc -l` +if [ $L -ne 256 ]; then + echo "The number of kprobes events ($L) is not 256" exit_fail fi -- 2.20.1

4 years, 11 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror December 2019