- Linux-kselftest-mirror - lists.linaro.org

[PATCH] kasan: Enable KUnit integration whenever CONFIG_KUNIT is enabled

by David Gow

Enable the KASAN/KUnit integration even when the KASAN tests are disabled, as it's useful for testing other things under KASAN. Essentially, this reverts commit 49d9977ac909 ("kasan: check CONFIG_KASAN_KUNIT_TEST instead of CONFIG_KUNIT"). To mitigate the performance impact slightly, add a likely() to the check for a currently running test. There's more we can do for performance if/when it becomes more of a problem, such as only enabling the "expect a KASAN failure" support wif the KASAN tests are enabled, or putting the whole thing behind a "kunit tests are running" static branch (which I do plan to do eventually). Fixes: 49d9977ac909 ("kasan: check CONFIG_KASAN_KUNIT_TEST instead of CONFIG_KUNIT") Signed-off-by: David Gow <davidgow(a)google.com> --- Basically, hiding the KASAN/KUnit integration broke being able to just pass --kconfig_add CONFIG_KASAN=y to kunit_tool to enable KASAN integration. We didn't notice this, because usually CONFIG_KUNIT_ALL_TESTS is enabled, which in turn enables CONFIG_KASAN_KUNIT_TEST. However, using a separate .kunitconfig might result in failures being missed. Take, for example: ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_KASAN=y \ --kunitconfig drivers/gpu/drm/tests This should run the drm tests with KASAN enabled, but even if there's a KASAN failure (such as the one fixed by [1]), kunit_tool will report success. [1]: https://lore.kernel.org/dri-devel/20221019073239.3779180-1-davidgow@google.… --- mm/kasan/kasan.h | 2 +- mm/kasan/report.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index abbcc1b0eec5..afacef14c7f4 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -261,7 +261,7 @@ struct kasan_stack_ring { #endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ -#if IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) +#if IS_ENABLED(CONFIG_KUNIT) /* Used in KUnit-compatible KASAN tests. */ struct kunit_kasan_status { bool report_found; diff --git a/mm/kasan/report.c b/mm/kasan/report.c index df3602062bfd..efa063b9d093 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -114,7 +114,7 @@ EXPORT_SYMBOL_GPL(kasan_restore_multi_shot); #endif -#if IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) +#if IS_ENABLED(CONFIG_KUNIT) static void update_kunit_status(bool sync) { struct kunit *test; @@ -122,7 +122,7 @@ static void update_kunit_status(bool sync) struct kunit_kasan_status *status; test = current->kunit_test; - if (!test) + if (likely(!test)) return; resource = kunit_find_named_resource(test, "kasan_status"); -- 2.38.0.413.g74048e4d9e-goog

2 years

2
4
0 0

[PATCH v1] selftests/landlock: Build without static libraries

by Mickaël Salaün

The only (forced) static test binary doesn't depend on libcap. Because using -lcap on systems that don't have such static library would fail (e.g. on Arch Linux), let's be more specific and require only dynamic libcap linking. Fixes: a52540522c95 ("selftests/landlock: Fix out-of-tree builds") Cc: Anders Roxell <anders.roxell(a)linaro.org> Cc: Guillaume Tucker <guillaume.tucker(a)collabora.com> Cc: Mark Brown <broonie(a)kernel.org> Cc: Shuah Khan <skhan(a)linuxfoundation.org> Cc: stable(a)vger.kernel.org Signed-off-by: Mickaël Salaün <mic(a)digikod.net> Link: https://lore.kernel.org/r/20221019200536.2771316-1-mic@digikod.net --- tools/testing/selftests/landlock/Makefile | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/landlock/Makefile b/tools/testing/selftests/landlock/Makefile index 6632bfff486b..348e2dbdb4e0 100644 --- a/tools/testing/selftests/landlock/Makefile +++ b/tools/testing/selftests/landlock/Makefile @@ -3,7 +3,6 @@ # First run: make -C ../../../.. headers_install CFLAGS += -Wall -O2 $(KHDR_INCLUDES) -LDLIBS += -lcap LOCAL_HDRS += common.h @@ -13,10 +12,12 @@ TEST_GEN_PROGS := $(src_test:.c=) TEST_GEN_PROGS_EXTENDED := true -# Static linking for short targets: +# Short targets: +$(TEST_GEN_PROGS): LDLIBS += -lcap $(TEST_GEN_PROGS_EXTENDED): LDFLAGS += -static include ../lib.mk -# Static linking for targets with $(OUTPUT)/ prefix: +# Targets with $(OUTPUT)/ prefix: +$(TEST_GEN_PROGS): LDLIBS += -lcap $(TEST_GEN_PROGS_EXTENDED): LDFLAGS += -static base-commit: 9abf2313adc1ca1b6180c508c25f22f9395cc780 -- 2.37.2

2 years

1
0
0 0

[PATCH v11 0/2] SPDX: add copyleft-next-0.3.1

by Luis Chamberlain

As suggested by Thomas Gleixner, I'm following up to move on with the SPDX tag needed for copyleft-next-0.3.1. I've split this out from the test_sysfs selftest so to separate review from that. Changes on this v11: o Fixed a minor typo on patch #2 as noted by Kees Cook o Added Reviewed-by tags by Kees Cook Changes on this v10: o embraced paragraph from Thomas Gleixner which helps explain why the OR operator in the SPDX license name o dropped the GPL-2.0 and GPL-2.0+ tags as suggested by Thomas Gleixner as these are outdated (still valid) in the SPDX spec o trimmed the Cc list to remove the test_sysfs / block layer / fs folks as the test_sysfs stuff is now dropped from consideration in this series Prior to this the series was at v9 but it also had the test_sysfs and its changes, its history can be found here: https://lore.kernel.org/all/20211029184500.2821444-1-mcgrof@kernel.org/ Luis Chamberlain (2): LICENSES: Add the copyleft-next-0.3.1 license testing: use the copyleft-next-0.3.1 SPDX tag LICENSES/dual/copyleft-next-0.3.1 | 236 +++++++++++++++++++++++ lib/test_kmod.c | 12 +- lib/test_sysctl.c | 12 +- tools/testing/selftests/kmod/kmod.sh | 13 +- tools/testing/selftests/sysctl/sysctl.sh | 12 +- 5 files changed, 240 insertions(+), 45 deletions(-) create mode 100644 LICENSES/dual/copyleft-next-0.3.1 -- 2.35.1

2 years

3
6
0 0

[PATCH 6.0 658/862] selftest: tpm2: Add Client.__del__() to close /dev/tpm* handle

by Greg Kroah-Hartman

From: Stefan Berger <stefanb(a)linux.ibm.com> [ Upstream commit 2d869f0b458547386fbcd8cf3004b271b7347b7f ] The following output can bee seen when the test is executed: test_flush_context (tpm2_tests.SpaceTest) ... \ /usr/lib64/python3.6/unittest/case.py:605: ResourceWarning: \ unclosed file <_io.FileIO name='/dev/tpmrm0' mode='rb+' closefd=True> An instance of Client does not implicitly close /dev/tpm* handle, once it gets destroyed. Close the file handle in the class destructor Client.__del__(). Fixes: 6ea3dfe1e0732 ("selftests: add TPM 2.0 tests") Cc: Shuah Khan <shuah(a)kernel.org> Cc: linux-kselftest(a)vger.kernel.org Cc: Jarkko Sakkinen <jarkko(a)kernel.org> Signed-off-by: Stefan Berger <stefanb(a)linux.ibm.com> Reviewed-by: Jarkko Sakkinen <jarkko(a)kernel.org> Signed-off-by: Jarkko Sakkinen <jarkko(a)kernel.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/tpm2/tpm2.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/testing/selftests/tpm2/tpm2.py b/tools/testing/selftests/tpm2/tpm2.py index 057a4f49c79d..c7363c6764fc 100644 --- a/tools/testing/selftests/tpm2/tpm2.py +++ b/tools/testing/selftests/tpm2/tpm2.py @@ -371,6 +371,10 @@ class Client: fcntl.fcntl(self.tpm, fcntl.F_SETFL, flags) self.tpm_poll = select.poll() + def __del__(self): + if self.tpm: + self.tpm.close() + def close(self): self.tpm.close() -- 2.35.1

2 years

1
0
0 0

[PATCH v3 0/4] Implement IOCTL to get and clear soft dirty PTE

by Muhammad Usama Anjum

Hello, This patch series implements a new ioctl on the pagemap proc fs file to get, clear and perform both get and clear at the same time atomically on the specified range of the memory. Soft-dirty PTE bit of the memory pages can be viewed by using pagemap procfs file. The soft-dirty PTE bit for the whole memory range of the process can be cleared by writing to the clear_refs file. This series adds features that weren't present earlier. - There is no atomic get soft-dirty PTE bit status and clear operation present. - The soft-dirty PTE bit of only a part of memory cannot be cleared. Historically, soft-dirty PTE bit tracking has been used in the CRIU project. The proc fs interface is enough for that as I think the process is frozen. We have the use case where we need to track the soft-dirty PTE bit for the running processes. We need this tracking and clear mechanism of a region of memory while the process is running to emulate the getWriteWatch() syscall of Windows. This syscall is used by games to keep track of dirty pages and keep processing only the dirty pages. This new ioctl can be used by the CRIU project and other applications which require soft-dirty PTE bit information. As in the current kernel there is no way to clear a part of memory (instead of clearing the Soft-Dirty bits for the entire process) and get+clear operation cannot be performed atomically, there are other methods to mimic this information entirely in userspace with poor performance: - The mprotect syscall and SIGSEGV handler for bookkeeping - The userfaultfd syscall with the handler for bookkeeping Some benchmarks can be seen [1]. This ioctl can be used by the CRIU project and other applications which require soft-dirty PTE bit information. The following operations are supported in this ioctl: - Get the pages that are soft-dirty. - Clear the pages which are soft-dirty. - The optional flag to ignore the VM_SOFTDIRTY and only track per page soft-dirty PTE bit There are two decisions which have been taken about how to get the output from the syscall. - Return offsets of the pages from the start in the vec - Stop execution when vec is filled with dirty pages These two arguments doesn't follow the mincore() philosophy where the output array corresponds to the address range in one to one fashion, hence the output buffer length isn't passed and only a flag is set if the page is present. This makes mincore() easy to use with less control. We are passing the size of the output array and putting return data consecutively which is offset of dirty pages from the start. The user can convert these offsets back into the dirty page addresses easily. Suppose, the user want to get first 10 dirty pages from a total memory of 100 pages. He'll allocate output buffer of size 10 and the ioctl will abort after finding the 10 pages. This behaviour is needed to support Windows' getWriteWatch(). The behaviour like mincore() can be achieved by passing output buffer of 100 size. This interface can be used for any desired behaviour. [1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora… Regards, Muhammad Usama Anjum Muhammad Usama Anjum (4): fs/proc/task_mmu: update functions to clear the soft-dirty PTE bit fs/proc/task_mmu: Implement IOCTL to get and clear soft dirty PTE bit selftests: vm: add pagemap ioctl tests mm: add documentation of the new ioctl on pagemap Documentation/admin-guide/mm/soft-dirty.rst | 42 +- fs/proc/task_mmu.c | 342 ++++++++++- include/uapi/linux/fs.h | 23 + tools/include/uapi/linux/fs.h | 23 + tools/testing/selftests/vm/.gitignore | 1 + tools/testing/selftests/vm/Makefile | 2 + tools/testing/selftests/vm/pagemap_ioctl.c | 649 ++++++++++++++++++++ 7 files changed, 1050 insertions(+), 32 deletions(-) create mode 100644 tools/testing/selftests/vm/pagemap_ioctl.c -- 2.30.2

2 years

6
23
0 0

kselftest/fixes build: 7 builds: 0 failed, 7 passed (v6.1-rc1-5-gcb05c81ada76)

by kernelci.org bot

kselftest/fixes build: 7 builds: 0 failed, 7 passed (v6.1-rc1-5-gcb05c81ada76) Full Build Summary: https://kernelci.org/build/kselftest/branch/fixes/kernel/v6.1-rc1-5-gcb05c8… Tree: kselftest Branch: fixes Git Describe: v6.1-rc1-5-gcb05c81ada76 Git Commit: cb05c81ada76a30a25a5f79b249375e33473af33 Git URL: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git Built: 4 unique architectures --- For more info write to <info(a)kernelci.org>

2 years

1
0
0 0

[PATCH v6 0/3] Introduce KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ macros

by Maíra Canal

Currently, in order to compare memory blocks in KUnit, the KUNIT_EXPECT_EQ or KUNIT_EXPECT_FALSE macros are used in conjunction with the memcmp function, such as: KUNIT_EXPECT_EQ(test, memcmp(foo, bar, size), 0); Although this usage produces correct results for the test cases, if the expectation fails the error message is not very helpful, indicating only the return of the memcmp function. Therefore, create a new set of macros KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ that compare memory blocks until a determined size. In case of expectation failure, those macros print the hex dump of the memory blocks, making it easier to debug test failures for memory blocks. The v6 has some changes on the first patch, due to rebase on top of Linux 6.1, specially the renaming of KUNIT_ASSERTION macro to _KUNIT_FAILED (97d453bc4007d4ac148c2ba89904026612b91ec9). Moreover, the DRM KUnit tests were mainlined in 6.1. The first patch of the series introduces the KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ. The second patch adds an example of memory block expectations on the kunit-example-test.c. And the last patch replaces the KUNIT_EXPECT_EQ for KUNIT_EXPECT_MEMEQ on the existing occurrences. Best Regards, - Maíra Canal v1 -> v2: https://lore.kernel.org/linux-kselftest/2a0dcd75-5461-5266-2749-808f638f4c5… - Change "determinated" to "specified" (Daniel Latypov). - Change the macro KUNIT_EXPECT_ARREQ to KUNIT_EXPECT_MEMEQ, in order to make it easier for users to infer the right size unit (Daniel Latypov). - Mark the different bytes on the failure message with a <> (Daniel Latypov). - Replace a constant number of array elements for ARRAY_SIZE() (André Almeida). - Rename "array" and "expected" variables to "array1" and "array2" (Daniel Latypov). v2 -> v3: https://lore.kernel.org/linux-kselftest/20220802212621.420840-1-mairacanal@… - Make the bytes aligned at output. - Add KUNIT_SUBSUBTEST_INDENT to the output for the indentation (Daniel Latypov). - Line up the trailing \ at macros using tabs (Daniel Latypov). - Line up the params to the functions (Daniel Latypov). - Change "Increament" to "Augment" (Daniel Latypov). - Use sizeof() for array sizes (Daniel Latypov). - Add Daniel Latypov's tags. v3 -> v4: https://lore.kernel.org/linux-kselftest/CABVgOSm_59Yr82deQm2C=18jjSv_akmn66… - Fix wrapped lines by the mail client (David Gow). - Mention on documentation that KUNIT_EXPECT_MEMEQ is not recommended for structured data (David Gow). - Add Muhammad Usama Anjum's tag. v4 -> v5: https://lore.kernel.org/linux-kselftest/20220808125237.277126-1-mairacanal@… - Rebase on top of drm-misc-next. - Add David Gow's tags. v5 -> v6: https://lore.kernel.org/linux-kselftest/20220921014515.113062-1-mairacanal@… - Rebase on top of Linux 6.1. - Change KUNIT_ASSERTION macro to _KUNIT_FAILED. Maíra Canal (3): kunit: Introduce KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ macros kunit: Add KUnit memory block assertions to the example_all_expect_macros_test kunit: Use KUNIT_EXPECT_MEMEQ macro .../gpu/drm/tests/drm_format_helper_test.c | 12 +-- include/kunit/assert.h | 33 +++++++ include/kunit/test.h | 87 +++++++++++++++++++ lib/kunit/assert.c | 56 ++++++++++++ lib/kunit/kunit-example-test.c | 7 ++ net/core/dev_addr_lists_test.c | 4 +- 6 files changed, 191 insertions(+), 8 deletions(-) -- 2.37.3

2 years

3
6
0 0

[PATCH 0/4] kunit: more assertion reworking

by Daniel Latypov

RFC: https://lore.kernel.org/linux-kselftest/20220525154442.1438081-1-dlatypov@g… Changes since then: tweak commit messages, reformatting to make checkpatch.pl happy. Nothing substantial. Why send this out again now: the initial Rust patchset no longer contains the Kunit changes, so hopefully both series can go into 6.1 and later we can coordinate the update the kunit.rs wrapper. This is a follow up to these three series: https://lore.kernel.org/all/20220113165931.451305-1-dlatypov@google.com/ https://lore.kernel.org/all/20220118223506.1701553-1-dlatypov@google.com/ https://lore.kernel.org/all/20220125210011.3817742-1-dlatypov@google.com/ The two goals of those series were a) reduce the size of struct kunit_assert and friends. (struct kunit_assert went from 48 => 8 bytes on UML.) b) simplify the internal code, mostly by deleting macros This series goes further a) sizeof(struct kunit_assert) = 0 now b) e.g. we delete another class of macros (KUNIT_INIT_*_ASSERT_STRUCT) Note: this does change the function signature of kunit_do_failed_assertion, so we'd need to update the rust wrapper in https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/kunit.rs, but hopefully it's just a simple change, e.g. maybe just like: @@ -38,9 +38,7 @@ }); static CONDITION: &'static $crate::str::CStr = $crate::c_str!(stringify!($cond)); static ASSERTION: UnaryAssert = UnaryAssert($crate::bindings::kunit_unary_assert { - assert: $crate::bindings::kunit_assert { - format: Some($crate::bindings::kunit_unary_assert_format), - }, + assert: $crate::bindings::kunit_assert {}, condition: CONDITION.as_char_ptr(), expected_true: true, }); @@ -67,6 +65,7 @@ core::ptr::addr_of!(LOCATION.0), $crate::bindings::kunit_assert_type_KUNIT_ASSERTION, core::ptr::addr_of!(ASSERTION.0.assert), + Some($crate::bindings::kunit_unary_assert_format), core::ptr::null(), ); } Daniel Latypov (4): kunit: remove format func from struct kunit_assert, get it to 0 bytes kunit: rename base KUNIT_ASSERTION macro to _KUNIT_FAILED kunit: eliminate KUNIT_INIT_*_ASSERT_STRUCT macros kunit: declare kunit_assert structs as const include/kunit/assert.h | 74 ++---------------------- include/kunit/test.h | 127 +++++++++++++++++++++++------------------ lib/kunit/test.c | 7 ++- 3 files changed, 80 insertions(+), 128 deletions(-) base-commit: 511cce163b75bc3933fa3de769a82bb7e8663f2b -- 2.38.0.rc1.362.ged0d419d3c-goog

2 years

3
18
0 0

[PATCH v9] selftests/vm: enable running select groups of tests

by Joel Savitz

Our memory management kernel CI testing at Red Hat uses the VM selftests and we have run into two problems: First, our LTP tests overlap with the VM selftests. We want to avoid unhelpful redundancy in our testing practices. Second, we have observed the current run_vmtests.sh to report overall failure/ambiguous results in the case that a machine lacks the necessary hardware to perform one or more of the tests. E.g. ksm tests that require more than one numa node. We want to be able to run the vm selftests suitable to particular hardware. Add the ability to run one or more groups of vm tests via run_vmtests.sh instead of simply all-or-none in order to solve these problems. Preserve existing default behavior of running all tests when the script is invoked with no arguments. Documentation of test groups is included in the patch as follows: # ./run_vmtests.sh [ -h || --help ] usage: ./tools/testing/selftests/vm/run_vmtests.sh [ -h | -t "<categories>"] -t: specify specific categories to tests to run -h: display this message The default behavior is to run all tests. Alternatively, specific groups tests can be run by passing a string to the -t argument containing one or more of the following categories separated by spaces: - mmap tests for mmap(2) - gup_test tests for gup using gup_test interface - userfaultfd tests for userfaultfd(2) - compaction a test for the patch "Allow compaction of unevictable pages" - mlock tests for mlock(2) - mremap tests for mremap(2) - hugevm tests for very large virtual address space - vmalloc vmalloc smoke tests - hmm hmm smoke tests - madv_populate test memadvise(2) MADV_POPULATE_{READ,WRITE} options - memfd_secret test memfd_secret(2) - process_mrelease test process_mrelease(2) - ksm ksm tests that do not require >=2 NUMA nodes - ksm_numa ksm tests that require >=2 NUMA nodes - pkey memory protection key tests - soft_dirty test soft dirty page bit semantics - anon_cow test anonymous copy-on-write semantics example: ./run_vmtests.sh -t "hmm mmap ksm" Changes from v6: - rebase onto mm-unstable per akpm's request - add control and usage for soft_dirty and anon_cow tests Changes from v5: - rebase onto mainline master branch - integrate changes to pkey and userfaultfd invocation sites Changes from v4: - fix imprecise checking in test_selected - drop conditional setup/cleanup of hugetlb Changes from v3: - rename variable TEST_ITEMS as VM_TEST_ITEMS Changes from v2: - rebase onto the mm-everyting branch in https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git - integrate this functionality with new the tests Changes from v1: - use a command line argument to pass the test categories to the script instead of an environmet variable - remove novel prints to avoid messing with extant parsers of this script - update the usage text Signed-off-by: Joel Savitz <jsavitz(a)redhat.com> --- tools/testing/selftests/vm/run_vmtests.sh | 216 +++++++++++++++------- 1 file changed, 148 insertions(+), 68 deletions(-) diff --git a/tools/testing/selftests/vm/run_vmtests.sh b/tools/testing/selftests/vm/run_vmtests.sh index 1fa783732296..0eb16f427a61 100755 --- a/tools/testing/selftests/vm/run_vmtests.sh +++ b/tools/testing/selftests/vm/run_vmtests.sh @@ -1,21 +1,86 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0 -#please run as root +# Please run as root # Kselftest framework requirement - SKIP code is 4. ksft_skip=4 exitcode=0 -#get huge pagesize and freepages from /proc/meminfo -while read -r name size unit; do - if [ "$name" = "HugePages_Free:" ]; then - freepgs="$size" +usage() { + cat <<EOF +usage: ${BASH_SOURCE[0]:-$0} [ -h | -t "<categories>"] + -t: specify specific categories to tests to run + -h: display this message + +The default behavior is to run all tests. + +Alternatively, specific groups tests can be run by passing a string +to the -t argument containing one or more of the following categories +separated by spaces: +- mmap + tests for mmap(2) +- gup_test + tests for gup using gup_test interface +- userfaultfd + tests for userfaultfd(2) +- compaction + a test for the patch "Allow compaction of unevictable pages" +- mlock + tests for mlock(2) +- mremap + tests for mremap(2) +- hugevm + tests for very large virtual address space +- vmalloc + vmalloc smoke tests +- hmm + hmm smoke tests +- madv_populate + test memadvise(2) MADV_POPULATE_{READ,WRITE} options +- memfd_secret + test memfd_secret(2) +- process_mrelease + test process_mrelease(2) +- ksm + ksm tests that do not require >=2 NUMA nodes +- ksm_numa + ksm tests that require >=2 NUMA nodes +- pkey + memory protection key tests +- soft_dirty + test soft dirty page bit semantics +- anon_cow + test anonymous copy-on-write semantics +example: ./run_vmtests.sh -t "hmm mmap ksm" +EOF + exit 0 +} + + +while getopts "ht:" OPT; do + case ${OPT} in + "h") usage ;; + "t") VM_SELFTEST_ITEMS=${OPTARG} ;; + esac +done +shift $((OPTIND -1)) + +# default behavior: run all tests +VM_SELFTEST_ITEMS=${VM_SELFTEST_ITEMS:-default} + +test_selected() { + if [ "$VM_SELFTEST_ITEMS" == "default" ]; then + # If no VM_SELFTEST_ITEMS are specified, run all tests + return 0 fi - if [ "$name" = "Hugepagesize:" ]; then - hpgsize_KB="$size" + # If test selected argument is one of the test items + if [[ " ${VM_SELFTEST_ITEMS[*]} " =~ " ${1} " ]]; then + return 0 + else + return 1 fi -done < /proc/meminfo +} # Simple hugetlbfs tests have a hardcoded minimum requirement of # huge pages totaling 256MB (262144KB) in size. The userfaultfd @@ -27,7 +92,17 @@ hpgsize_MB=$((hpgsize_KB / 1024)) half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128)) needmem_KB=$((half_ufd_size_MB * 2 * 1024)) -#set proper nr_hugepages +# get huge pagesize and freepages from /proc/meminfo +while read -r name size unit; do + if [ "$name" = "HugePages_Free:" ]; then + freepgs="$size" + fi + if [ "$name" = "Hugepagesize:" ]; then + hpgsize_KB="$size" + fi +done < /proc/meminfo + +# set proper nr_hugepages if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages) needpgs=$((needmem_KB / hpgsize_KB)) @@ -56,136 +131,141 @@ else exit 1 fi -#filter 64bit architectures +# filter 64bit architectures ARCH64STR="arm64 ia64 mips64 parisc64 ppc64 ppc64le riscv64 s390x sh64 sparc64 x86_64" if [ -z "$ARCH" ]; then ARCH=$(uname -m 2>/dev/null | sed -e 's/aarch64.*/arm64/') fi VADDR64=0 -echo "$ARCH64STR" | grep "$ARCH" && VADDR64=1 +echo "$ARCH64STR" | grep "$ARCH" &>/dev/null && VADDR64=1 # Usage: run_test [test binary] [arbitrary test arguments...] run_test() { - local title="running $*" - local sep=$(echo -n "$title" | tr "[:graph:][:space:]" -) - printf "%s\n%s\n%s\n" "$sep" "$title" "$sep" - - "$@" - local ret=$? - if [ $ret -eq 0 ]; then - echo "[PASS]" - elif [ $ret -eq $ksft_skip ]; then - echo "[SKIP]" - exitcode=$ksft_skip - else - echo "[FAIL]" - exitcode=1 - fi + if test_selected ${CATEGORY}; then + echo "running: $1" + local title="running $*" + local sep=$(echo -n "$title" | tr "[:graph:][:space:]" -) + printf "%s\n%s\n%s\n" "$sep" "$title" "$sep" + + "$@" + local ret=$? + if [ $ret -eq 0 ]; then + echo "[PASS]" + elif [ $ret -eq $ksft_skip ]; then + echo "[SKIP]" + exitcode=$ksft_skip + else + echo "[FAIL]" + exitcode=1 + fi + fi # test_selected } -run_test ./hugepage-mmap +CATEGORY="hugetlb" run_test ./hugepage-mmap shmmax=$(cat /proc/sys/kernel/shmmax) shmall=$(cat /proc/sys/kernel/shmall) echo 268435456 > /proc/sys/kernel/shmmax echo 4194304 > /proc/sys/kernel/shmall -run_test ./hugepage-shm +CATEGORY="hugetlb" run_test ./hugepage-shm echo "$shmmax" > /proc/sys/kernel/shmmax echo "$shmall" > /proc/sys/kernel/shmall -run_test ./map_hugetlb -run_test ./hugepage-mremap -run_test ./hugepage-vmemmap -run_test ./hugetlb-madvise +CATEGORY="hugetlb" run_test ./map_hugetlb +CATEGORY="hugetlb" run_test ./hugepage-mremap +CATEGORY="hugetlb" run_test ./hugepage-vmemmap +CATEGORY="hugetlb" run_test ./hugetlb-madvise -echo "NOTE: The above hugetlb tests provide minimal coverage. Use" -echo " https://github.com/libhugetlbfs/libhugetlbfs.git for" -echo " hugetlb regression testing." +if test_selected "hugetlb"; then + echo "NOTE: These hugetlb tests provide minimal coverage. Use" + echo " https://github.com/libhugetlbfs/libhugetlbfs.git for" + echo " hugetlb regression testing." +fi -run_test ./map_fixed_noreplace +CATEGORY="mmap" run_test ./map_fixed_noreplace # get_user_pages_fast() benchmark -run_test ./gup_test -u +CATEGORY="gup_test" run_test ./gup_test -u # pin_user_pages_fast() benchmark -run_test ./gup_test -a +CATEGORY="gup_test" run_test ./gup_test -a # Dump pages 0, 19, and 4096, using pin_user_pages: -run_test ./gup_test -ct -F 0x1 0 19 0x1000 +CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000 uffd_mods=("" ":dev") for mod in "${uffd_mods[@]}"; do - run_test ./userfaultfd anon${mod} 20 16 + CATEGORY="userfaultfd" run_test ./userfaultfd anon${mod} 20 16 # Hugetlb tests require source and destination huge pages. Pass in half # the size ($half_ufd_size_MB), which is used for *each*. - run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32 - run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32 - run_test ./userfaultfd shmem${mod} 20 16 + CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32 + CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32 + CATEGORY="userfaultfd" run_test ./userfaultfd shmem${mod} 20 16 done #cleanup echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages -run_test ./compaction_test +CATEGORY="compaction" run_test ./compaction_test -run_test sudo -u nobody ./on-fault-limit +CATEGORY="mlock" run_test sudo -u nobody ./on-fault-limit -run_test ./map_populate +CATEGORY="mmap" run_test ./map_populate -run_test ./mlock-random-test +CATEGORY="mlock" run_test ./mlock-random-test -run_test ./mlock2-tests +CATEGORY="mlock" run_test ./mlock2-tests -run_test ./mrelease_test +CATEGORY="process_mrelease" run_test ./mrelease_test -run_test ./mremap_test +CATEGORY="mremap" run_test ./mremap_test -run_test ./thuge-gen +CATEGORY="hugetlb" run_test ./thuge-gen if [ $VADDR64 -ne 0 ]; then - run_test ./virtual_address_range + CATEGORY="hugevm" run_test ./virtual_address_range # virtual address 128TB switch test - run_test ./va_128TBswitch.sh + CATEGORY="hugevm" run_test ./va_128TBswitch.sh fi # VADDR64 # vmalloc stability smoke test -run_test ./test_vmalloc.sh smoke +CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke -run_test ./mremap_dontunmap +CATEGORY="mremap" run_test ./mremap_dontunmap -run_test ./test_hmm.sh smoke +CATEGORY="hmm" run_test ./test_hmm.sh smoke # MADV_POPULATE_READ and MADV_POPULATE_WRITE tests -run_test ./madv_populate +CATEGORY="madv_populate" run_test ./madv_populate -run_test ./memfd_secret +CATEGORY="memfd_secret" run_test ./memfd_secret # KSM MADV_MERGEABLE test with 10 identical pages -run_test ./ksm_tests -M -p 10 +CATEGORY="ksm" run_test ./ksm_tests -M -p 10 # KSM unmerge test -run_test ./ksm_tests -U +CATEGORY="ksm" run_test ./ksm_tests -U # KSM test with 10 zero pages and use_zero_pages = 0 -run_test ./ksm_tests -Z -p 10 -z 0 +CATEGORY="ksm" run_test ./ksm_tests -Z -p 10 -z 0 # KSM test with 10 zero pages and use_zero_pages = 1 -run_test ./ksm_tests -Z -p 10 -z 1 +CATEGORY="ksm" run_test ./ksm_tests -Z -p 10 -z 1 # KSM test with 2 NUMA nodes and merge_across_nodes = 1 -run_test ./ksm_tests -N -m 1 +CATEGORY="ksm_numa" run_test ./ksm_tests -N -m 1 # KSM test with 2 NUMA nodes and merge_across_nodes = 0 -run_test ./ksm_tests -N -m 0 +CATEGORY="ksm_numa" run_test ./ksm_tests -N -m 0 # protection_keys tests if [ -x ./protection_keys_32 ] then - run_test ./protection_keys_32 + CATEGORY="pkey" run_test ./protection_keys_32 fi if [ -x ./protection_keys_64 ] then - run_test ./protection_keys_64 + CATEGORY="pkey" run_test ./protection_keys_64 fi -run_test ./soft-dirty +CATEGORY="soft_dirty" run_test ./soft-dirty # COW tests for anonymous memory -run_test ./anon_cow +CATEGORY="anon_cow" run_test ./anon_cow exit $exitcode -- 2.31.1

2 years

1
0
0 0

[PATCH v6] selftests/vm: enable running select groups of tests

by Joel Savitz

Our memory management kernel CI testing at Red Hat uses the VM selftests and we have run into two problems: First, our LTP tests overlap with the VM selftests. We want to avoid unhelpful redundancy in our testing practices. Second, we have observed the current run_vmtests.sh to report overall failure/ambiguous results in the case that a machine lacks the necessary hardware to perform one or more of the tests. E.g. ksm tests that require more than one numa node. We want to be able to run the vm selftests suitable to particular hardware. Add the ability to run one or more groups of vm tests via run_vmtests.sh instead of simply all-or-none in order to solve these problems. Preserve existing default behavior of running all tests when the script is invoked with no arguments. Documentation of test groups is included in the patch as follows: # ./run_vmtests.sh [ -h || --help ] usage: ./tools/testing/selftests/vm/run_vmtests.sh [ -h | -t "<categories>"] -t: specify specific categories to tests to run -h: display this message The default behavior is to run all tests. Alternatively, specific groups tests can be run by passing a string to the -t argument containing one or more of the following categories separated by spaces: - mmap tests for mmap(2) - gup_test tests for gup using gup_test interface - userfaultfd tests for userfaultfd(2) - compaction a test for the patch "Allow compaction of unevictable pages" - mlock tests for mlock(2) - mremap tests for mremap(2) - hugevm tests for very large virtual address space - vmalloc vmalloc smoke tests - hmm hmm smoke tests - madv_populate test memadvise(2) MADV_POPULATE_{READ,WRITE} options - memfd_secret test memfd_secret(2) - process_mrelease test process_mrelease(2) - ksm ksm tests that do not require >=2 NUMA nodes - ksm_numa ksm tests that require >=2 NUMA nodes - pkey memory protection key tests example: ./run_vmtests.sh -t "hmm mmap ksm" Changes from v5: - rebase onto mainline master branch - integrate changes to pkey and userfaultfd invocation sites Changes from v4: - fix imprecise checking in test_selected - drop conditional setup/cleanup of hugetlb Changes from v3: - rename variable TEST_ITEMS as VM_TEST_ITEMS Changes from v2: - rebase onto the mm-everyting branch in https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git - integrate this functionality with new the tests Changes from v1: - use a command line argument to pass the test categories to the script instead of an environmet variable - remove novel prints to avoid messing with extant parsers of this script - update the usage text Signed-off-by: Joel Savitz <jsavitz(a)redhat.com> --- tools/testing/selftests/vm/run_vmtests.sh | 212 +++++++++++++++------- 1 file changed, 144 insertions(+), 68 deletions(-) diff --git a/tools/testing/selftests/vm/run_vmtests.sh b/tools/testing/selftests/vm/run_vmtests.sh index e780e76c26b8..4a2f2e331105 100755 --- a/tools/testing/selftests/vm/run_vmtests.sh +++ b/tools/testing/selftests/vm/run_vmtests.sh @@ -1,6 +1,6 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0 -#please run as root +# Please run as root # Kselftest framework requirement - SKIP code is 4. ksft_skip=4 @@ -8,15 +8,76 @@ ksft_skip=4 mnt=./huge exitcode=0 -#get huge pagesize and freepages from /proc/meminfo -while read -r name size unit; do - if [ "$name" = "HugePages_Free:" ]; then - freepgs="$size" +usage() { + cat <<EOF +usage: ${BASH_SOURCE[0]:-$0} [ -h | -t "<categories>"] + -t: specify specific categories to tests to run + -h: display this message + +The default behavior is to run all tests. + +Alternatively, specific groups tests can be run by passing a string +to the -t argument containing one or more of the following categories +separated by spaces: +- mmap + tests for mmap(2) +- gup_test + tests for gup using gup_test interface +- userfaultfd + tests for userfaultfd(2) +- compaction + a test for the patch "Allow compaction of unevictable pages" +- mlock + tests for mlock(2) +- mremap + tests for mremap(2) +- hugevm + tests for very large virtual address space +- vmalloc + vmalloc smoke tests +- hmm + hmm smoke tests +- madv_populate + test memadvise(2) MADV_POPULATE_{READ,WRITE} options +- memfd_secret + test memfd_secret(2) +- process_mrelease + test process_mrelease(2) +- ksm + ksm tests that do not require >=2 NUMA nodes +- ksm_numa + ksm tests that require >=2 NUMA nodes +- pkey + memory protection key tests +example: ./run_vmtests.sh -t "hmm mmap ksm" +EOF + exit 0 +} + + +while getopts "ht:" OPT; do + case ${OPT} in + "h") usage ;; + "t") VM_SELFTEST_ITEMS=${OPTARG} ;; + esac +done +shift $((OPTIND -1)) + +# default behavior: run all tests +VM_SELFTEST_ITEMS=${VM_SELFTEST_ITEMS:-default} + +test_selected() { + if [ "$VM_SELFTEST_ITEMS" == "default" ]; then + # If no VM_SELFTEST_ITEMS are specified, run all tests + return 0 fi - if [ "$name" = "Hugepagesize:" ]; then - hpgsize_KB="$size" + # If test selected argument is one of the test items + if [[ " ${VM_SELFTEST_ITEMS[*]} " =~ " ${1} " ]]; then + return 0 + else + return 1 fi -done < /proc/meminfo +} # Simple hugetlbfs tests have a hardcoded minimum requirement of # huge pages totaling 256MB (262144KB) in size. The userfaultfd @@ -28,7 +89,17 @@ hpgsize_MB=$((hpgsize_KB / 1024)) half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128)) needmem_KB=$((half_ufd_size_MB * 2 * 1024)) -#set proper nr_hugepages +# get huge pagesize and freepages from /proc/meminfo +while read -r name size unit; do + if [ "$name" = "HugePages_Free:" ]; then + freepgs="$size" + fi + if [ "$name" = "Hugepagesize:" ]; then + hpgsize_KB="$size" + fi +done < /proc/meminfo + +# set proper nr_hugepages if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then nr_hugepgs=$(cat /proc/sys/vm/nr_hugepages) needpgs=$((needmem_KB / hpgsize_KB)) @@ -57,78 +128,83 @@ else exit 1 fi -#filter 64bit architectures +# filter 64bit architectures ARCH64STR="arm64 ia64 mips64 parisc64 ppc64 ppc64le riscv64 s390x sh64 sparc64 x86_64" if [ -z "$ARCH" ]; then ARCH=$(uname -m 2>/dev/null | sed -e 's/aarch64.*/arm64/') fi VADDR64=0 -echo "$ARCH64STR" | grep "$ARCH" && VADDR64=1 +echo "$ARCH64STR" | grep "$ARCH" &>/dev/null && VADDR64=1 # Usage: run_test [test binary] [arbitrary test arguments...] run_test() { - local title="running $*" - local sep=$(echo -n "$title" | tr "[:graph:][:space:]" -) - printf "%s\n%s\n%s\n" "$sep" "$title" "$sep" - - "$@" - local ret=$? - if [ $ret -eq 0 ]; then - echo "[PASS]" - elif [ $ret -eq $ksft_skip ]; then - echo "[SKIP]" - exitcode=$ksft_skip - else - echo "[FAIL]" - exitcode=1 - fi + if test_selected ${CATEGORY}; then + echo "running: $1" + local title="running $*" + local sep=$(echo -n "$title" | tr "[:graph:][:space:]" -) + printf "%s\n%s\n%s\n" "$sep" "$title" "$sep" + + "$@" + local ret=$? + if [ $ret -eq 0 ]; then + echo "[PASS]" + elif [ $ret -eq $ksft_skip ]; then + echo "[SKIP]" + exitcode=$ksft_skip + else + echo "[FAIL]" + exitcode=1 + fi + fi # test_selected } mkdir "$mnt" mount -t hugetlbfs none "$mnt" -run_test ./hugepage-mmap +CATEGORY="hugetlb" run_test ./hugepage-mmap shmmax=$(cat /proc/sys/kernel/shmmax) shmall=$(cat /proc/sys/kernel/shmall) echo 268435456 > /proc/sys/kernel/shmmax echo 4194304 > /proc/sys/kernel/shmall -run_test ./hugepage-shm +CATEGORY="hugetlb" run_test ./hugepage-shm echo "$shmmax" > /proc/sys/kernel/shmmax echo "$shmall" > /proc/sys/kernel/shmall -run_test ./map_hugetlb +CATEGORY="hugetlb" run_test ./map_hugetlb -run_test ./hugepage-mremap "$mnt"/huge_mremap -rm -f "$mnt"/huge_mremap +CATEGORY="hugetlb" run_test ./hugepage-mremap "$mnt"/huge_mremap +test_selected "hugetlb" && rm -f "$mnt"/huge_mremap -run_test ./hugepage-vmemmap +CATEGORY="hugetlb" run_test ./hugepage-vmemmap -run_test ./hugetlb-madvise "$mnt"/madvise-test -rm -f "$mnt"/madvise-test +CATEGORY="hugetlb" run_test ./hugetlb-madvise "$mnt"/madvise-test +test_selected "hugetlb" && rm -f "$mnt"/madvise-test -echo "NOTE: The above hugetlb tests provide minimal coverage. Use" -echo " https://github.com/libhugetlbfs/libhugetlbfs.git for" -echo " hugetlb regression testing." +if test_selected "hugetlb"; then + echo "NOTE: These hugetlb tests provide minimal coverage. Use" + echo " https://github.com/libhugetlbfs/libhugetlbfs.git for" + echo " hugetlb regression testing." +fi -run_test ./map_fixed_noreplace +CATEGORY="mmap" run_test ./map_fixed_noreplace # get_user_pages_fast() benchmark -run_test ./gup_test -u +CATEGORY="gup_test" run_test ./gup_test -u # pin_user_pages_fast() benchmark -run_test ./gup_test -a +CATEGORY="gup_test" run_test ./gup_test -a # Dump pages 0, 19, and 4096, using pin_user_pages: -run_test ./gup_test -ct -F 0x1 0 19 0x1000 +CATEGORY="gup_test" run_test ./gup_test -ct -F 0x1 0 19 0x1000 uffd_mods=("" ":dev") for mod in "${uffd_mods[@]}"; do - run_test ./userfaultfd anon${mod} 20 16 + CATEGORY="userfaultfd" run_test ./userfaultfd anon${mod} 20 16 # Hugetlb tests require source and destination huge pages. Pass in half # the size ($half_ufd_size_MB), which is used for *each*. - run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32 - run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32 "$mnt"/uffd-test + CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb${mod} "$half_ufd_size_MB" 32 + CATEGORY="userfaultfd" run_test ./userfaultfd hugetlb_shared${mod} "$half_ufd_size_MB" 32 "$mnt"/uffd-test rm -f "$mnt"/uffd-test - run_test ./userfaultfd shmem${mod} 20 16 + CATEGORY="userfaultfd" run_test ./userfaultfd shmem${mod} 20 16 done #cleanup @@ -136,63 +212,63 @@ umount "$mnt" rm -rf "$mnt" echo "$nr_hugepgs" > /proc/sys/vm/nr_hugepages -run_test ./compaction_test +CATEGORY="compaction" run_test ./compaction_test -run_test sudo -u nobody ./on-fault-limit +CATEGORY="mlock" run_test sudo -u nobody ./on-fault-limit -run_test ./map_populate +CATEGORY="mmap" run_test ./map_populate -run_test ./mlock-random-test +CATEGORY="mlock" run_test ./mlock-random-test -run_test ./mlock2-tests +CATEGORY="mlock" run_test ./mlock2-tests -run_test ./mrelease_test +CATEGORY="process_mrelease" run_test ./mrelease_test -run_test ./mremap_test +CATEGORY="mremap" run_test ./mremap_test -run_test ./thuge-gen +CATEGORY="hugetlb" run_test ./thuge-gen if [ $VADDR64 -ne 0 ]; then - run_test ./virtual_address_range + CATEGORY="hugevm" run_test ./virtual_address_range # virtual address 128TB switch test - run_test ./va_128TBswitch.sh + CATEGORY="hugevm" run_test ./va_128TBswitch.sh fi # VADDR64 # vmalloc stability smoke test -run_test ./test_vmalloc.sh smoke +CATEGORY="vmalloc" run_test ./test_vmalloc.sh smoke -run_test ./mremap_dontunmap +CATEGORY="mremap" run_test ./mremap_dontunmap -run_test ./test_hmm.sh smoke +CATEGORY="hmm" run_test ./test_hmm.sh smoke # MADV_POPULATE_READ and MADV_POPULATE_WRITE tests -run_test ./madv_populate +CATEGORY="madv_populate" run_test ./madv_populate -run_test ./memfd_secret +CATEGORY="memfd_secret" run_test ./memfd_secret # KSM MADV_MERGEABLE test with 10 identical pages -run_test ./ksm_tests -M -p 10 +CATEGORY="ksm" run_test ./ksm_tests -M -p 10 # KSM unmerge test -run_test ./ksm_tests -U +CATEGORY="ksm" run_test ./ksm_tests -U # KSM test with 10 zero pages and use_zero_pages = 0 -run_test ./ksm_tests -Z -p 10 -z 0 +CATEGORY="ksm" run_test ./ksm_tests -Z -p 10 -z 0 # KSM test with 10 zero pages and use_zero_pages = 1 -run_test ./ksm_tests -Z -p 10 -z 1 +CATEGORY="ksm" run_test ./ksm_tests -Z -p 10 -z 1 # KSM test with 2 NUMA nodes and merge_across_nodes = 1 -run_test ./ksm_tests -N -m 1 +CATEGORY="ksm_numa" run_test ./ksm_tests -N -m 1 # KSM test with 2 NUMA nodes and merge_across_nodes = 0 -run_test ./ksm_tests -N -m 0 +CATEGORY="ksm_numa" run_test ./ksm_tests -N -m 0 # protection_keys tests if [ -x ./protection_keys_32 ] then - run_test ./protection_keys_32 + CATEGORY="pkey" run_test ./protection_keys_32 fi if [ -x ./protection_keys_64 ] then - run_test ./protection_keys_64 + CATEGORY="pkey" run_test ./protection_keys_64 fi run_test ./soft-dirty -- 2.31.1

2 years

2
2
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror