June 2024 - Linux-kselftest-mirror

[PATCH-cgroup v2 0/5] cgroup/cpuset: Fix miscellaneous issues

by Waiman Long

v2: - Fix test_cpuset_prs.sh problems reported by test robot - Relax restriction imposed between cpuset.cpus.exclusive and cpuset.cpus of sibling cpusets. - Make cpuset.cpus.exclusive independent of cpuset.cpus. - Update test_cpuset_prs.sh accordingly. [v1] https://lore.kernel.org/lkml/20240605171858.1323464-1-longman@redhat.com/ This patchset attempts to address the following cpuset issues. 1) While reviewing the generate_sched_domains() function, I found a bug in generating sched domains for remote non-isolating partitions. 2) Test robot had reported a test_cpuset_prs.sh test failure. 3) The current exclusivity test between cpuset.cpus.exclusive and cpuset.cpus and the restriction that the set effective exclusive CPUs has to be a subset of cpuset.cpus make it harder to preconfigure the cgroup hierarchy to enable remote partition. The test_cpuset_prs.sh script is updated to match changes made in this patchset and was run to verify that the new code did not cause any regression. Waiman Long (5): cgroup/cpuset: Fix remote root partition creation problem selftest/cgroup: Fix test_cpuset_prs.sh problems reported by test robot cgroup/cpuset: Delay setting of CS_CPU_EXCLUSIVE until valid partition cgroup/cpuset: Make cpuset.cpus.exclusive independent of cpuset.cpus selftest/cgroup: Update test_cpuset_prs.sh to match changes Documentation/admin-guide/cgroup-v2.rst | 12 +- kernel/cgroup/cpuset.c | 158 +++++++++++++----- .../selftests/cgroup/test_cpuset_prs.sh | 75 ++++++--- 3 files changed, 180 insertions(+), 65 deletions(-) -- 2.39.3

5 months, 1 week

3
8
0 0

[PATCH net-next v1 0/3] selftests: rds selftest

by allison.henderson＠oracle.com

From: Allison Henderson <allison.henderson(a)oracle.com> Hi All, This series is a new selftest that Vegard, Chuck and myself have been working on to provide some test coverage for rds. I've made quite a few updates since the rfc sent a few weeks ago: I've added several knobs to the script to tune network turbulance, and documented their usage in the README.txt. By default these options are left off. Added an extra flag to specify log location I've also added a flag to the config.sh to skip gcov configurations if the coverage report is not desired. run.sh has been adapted to skip the report if the required configs are not present, or if the required packages are not available A time out has been added to prevent the test from hanging indefinitely The previous gcov issues have been resolved with an appropriate gcov patch, as well as some extra logic to detect incompatible gcov and gcc versions. The shellcheck nits reported in the last review have been addressed In order to return an appropriate exit code, the run.sh script has been adapted to analyze the test.py strace, and determine if the test passed, failed or timed out. RDS specific GCOV configs have been documented under Documentation/dev-tools/gcov.rst Questions and comments appreciated. Thanks everyone! Allison Vegard Nossum (3): .gitignore: add .gcda files net: rds: add option for GCOV profiling selftests: rds: add testing infrastructure .gitignore | 1 + Documentation/dev-tools/gcov.rst | 11 + MAINTAINERS | 1 + net/rds/Kconfig | 9 + net/rds/Makefile | 5 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/net/rds/Makefile | 13 + tools/testing/selftests/net/rds/README.txt | 41 ++++ tools/testing/selftests/net/rds/config.sh | 56 +++++ tools/testing/selftests/net/rds/init.sh | 69 ++++++ tools/testing/selftests/net/rds/run.sh | 271 +++++++++++++++++++++ tools/testing/selftests/net/rds/test.py | 251 +++++++++++++++++++ 12 files changed, 729 insertions(+) create mode 100644 tools/testing/selftests/net/rds/Makefile create mode 100644 tools/testing/selftests/net/rds/README.txt create mode 100755 tools/testing/selftests/net/rds/config.sh create mode 100755 tools/testing/selftests/net/rds/init.sh create mode 100755 tools/testing/selftests/net/rds/run.sh create mode 100644 tools/testing/selftests/net/rds/test.py -- 2.25.1

5 months, 1 week

3
6
0 0

[PATCH v2] selftests: x86: conform test to TAP format output

by Muhammad Usama Anjum

Conform the layout, informational and status messages to TAP. No functional change is intended other than the layout of output messages. Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> --- Changes since v1: - No changes, sending it again as got no response on v1 even after weeks --- tools/testing/selftests/x86/vdso_restorer.c | 29 +++++++++------------ 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/tools/testing/selftests/x86/vdso_restorer.c b/tools/testing/selftests/x86/vdso_restorer.c index fe99f24341554..f621167424a9c 100644 --- a/tools/testing/selftests/x86/vdso_restorer.c +++ b/tools/testing/selftests/x86/vdso_restorer.c @@ -21,6 +21,7 @@ #include <unistd.h> #include <syscall.h> #include <sys/syscall.h> +#include "../kselftest.h" /* Open-code this -- the headers are too messy to easily use them. */ struct real_sigaction { @@ -44,17 +45,19 @@ static void handler_without_siginfo(int sig) int main() { - int nerrs = 0; struct real_sigaction sa; + ksft_print_header(); + ksft_set_plan(2); + void *vdso = dlopen("linux-vdso.so.1", RTLD_LAZY | RTLD_LOCAL | RTLD_NOLOAD); if (!vdso) vdso = dlopen("linux-gate.so.1", RTLD_LAZY | RTLD_LOCAL | RTLD_NOLOAD); if (!vdso) { - printf("[SKIP]\tFailed to find vDSO. Tests are not expected to work.\n"); - return 0; + ksft_print_msg("[SKIP]\tFailed to find vDSO. Tests are not expected to work.\n"); + return KSFT_SKIP; } memset(&sa, 0, sizeof(sa)); @@ -62,21 +65,16 @@ int main() sa.flags = SA_SIGINFO; sa.restorer = NULL; /* request kernel-provided restorer */ - printf("[RUN]\tRaise a signal, SA_SIGINFO, sa.restorer == NULL\n"); + ksft_print_msg("Raise a signal, SA_SIGINFO, sa.restorer == NULL\n"); if (syscall(SYS_rt_sigaction, SIGUSR1, &sa, NULL, 8) != 0) err(1, "raw rt_sigaction syscall"); raise(SIGUSR1); - if (handler_called) { - printf("[OK]\tSA_SIGINFO handler returned successfully\n"); - } else { - printf("[FAIL]\tSA_SIGINFO handler was not called\n"); - nerrs++; - } + ksft_test_result(handler_called, "SA_SIGINFO handler returned\n"); - printf("[RUN]\tRaise a signal, !SA_SIGINFO, sa.restorer == NULL\n"); + ksft_print_msg("Raise a signal, !SA_SIGINFO, sa.restorer == NULL\n"); sa.flags = 0; sa.handler = handler_without_siginfo; @@ -86,10 +84,7 @@ int main() raise(SIGUSR1); - if (handler_called) { - printf("[OK]\t!SA_SIGINFO handler returned successfully\n"); - } else { - printf("[FAIL]\t!SA_SIGINFO handler was not called\n"); - nerrs++; - } + ksft_test_result(handler_called, "SA_SIGINFO handler returned\n"); + + ksft_finished(); } -- 2.39.2

5 months, 1 week

2
7
0 0

[PATCH v2 0/4] add tests to verify IFS (In Field Scan) driver functionality

by Pengfei Xu

To verify IFS (In Field Scan [1]) driver functionality, add the following 6 test cases: 1. Verify that IFS sysfs entries are created after loading the IFS module 2. Check if loading an invalid IFS test image fails and loading a valid one succeeds 3. Perform IFS scan test on each CPU using all the available image files 4. Perform IFS scan with first test image file on a random CPU for 3 rounds 5. Perform IFS ARRAY BIST(Board Integrated System Test) test on each CPU 6. Perform IFS ARRAY BIST test on a random CPU for 3 rounds These are not exhaustive, but some minimal test runs to check various parts of the driver. Some negative tests are also included. [1] https://docs.kernel.org/arch/x86/ifs.html Pengfei Xu (4): selftests: ifs: verify test interfaces are created by the driver selftests: ifs: verify test image loading functionality selftests: ifs: verify IFS scan test functionality selftests: ifs: verify IFS ARRAY BIST functionality MAINTAINERS | 1 + tools/testing/selftests/Makefile | 1 + .../drivers/platform/x86/intel/ifs/Makefile | 6 + .../platform/x86/intel/ifs/test_ifs.sh | 494 ++++++++++++++++++ 4 files changed, 502 insertions(+) create mode 100644 tools/testing/selftests/drivers/platform/x86/intel/ifs/Makefile create mode 100755 tools/testing/selftests/drivers/platform/x86/intel/ifs/test_ifs.sh --- Changes: v1 to v2: - Rebase to v6.10 cycle kernel and resolve some code conflicts - Improved checking of IFS ARRAY_BIST support by leveraging sysfs entry methods (suggested by Ashok) -- 2.43.0

5 months, 1 week

3
7
0 0

[PATCH 1/2] selftests: arm64: tags_test: conform test to TAP output

by Muhammad Usama Anjum

Conform the layout, informational and status messages to TAP. No functional change is intended other than the layout of output messages. Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> --- tools/testing/selftests/arm64/tags/tags_test.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/arm64/tags/tags_test.c b/tools/testing/selftests/arm64/tags/tags_test.c index 955f87c1170d7..8ae26e496c89c 100644 --- a/tools/testing/selftests/arm64/tags/tags_test.c +++ b/tools/testing/selftests/arm64/tags/tags_test.c @@ -17,19 +17,21 @@ int main(void) static int tbi_enabled = 0; unsigned long tag = 0; struct utsname *ptr; - int err; + + ksft_print_header(); + ksft_set_plan(1); if (prctl(PR_SET_TAGGED_ADDR_CTRL, PR_TAGGED_ADDR_ENABLE, 0, 0, 0) == 0) tbi_enabled = 1; ptr = (struct utsname *)malloc(sizeof(*ptr)); if (!ptr) - ksft_exit_fail_msg("Failed to allocate utsname buffer\n"); + ksft_exit_fail_perror("Failed to allocate utsname buffer"); if (tbi_enabled) tag = 0x42; ptr = (struct utsname *)SET_TAG(ptr, tag); - err = uname(ptr); + ksft_test_result(!uname(ptr), "Syscall successful with tagged address\n"); free(ptr); - return err; + ksft_finished(); } -- 2.39.2

5 months, 1 week

3
7
0 0

[PATCH 0/4] selftest: x86: conform tests to TAP format output

by Muhammad Usama Anjum

In this series, 4 tests are being conformed to TAP. Muhammad Usama Anjum (4): selftests: x86: check_initial_reg_state: conform test to TAP format output selftests: x86: corrupt_xstate_header: conform test to TAP format output selftests: fsgsbase_restore: conform test to TAP format output selftests: entry_from_vm86: conform test to TAP format output .../selftests/x86/check_initial_reg_state.c | 24 ++-- .../selftests/x86/corrupt_xstate_header.c | 30 +++-- tools/testing/selftests/x86/entry_from_vm86.c | 109 ++++++++-------- .../testing/selftests/x86/fsgsbase_restore.c | 117 +++++++++--------- 4 files changed, 139 insertions(+), 141 deletions(-) -- 2.39.2

5 months, 1 week

2
13
0 0

[PATCH] selftests: openat2: don't print total number of tests and then skip

by Muhammad Usama Anjum

Don't print that 88 sub-tests are going to be executed. But then skip. The error is printed that executed test was only 1 while 88 should have run: Old output: TAP version 13 1..88 ok 2 # SKIP all tests require euid == 0 # Planned tests != run tests (88 != 1) # Totals: pass:0 fail:0 xfail:0 xpass:0 skip:1 error:0 New and correct output: TAP version 13 1..0 # SKIP all tests require euid == 0 Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> --- tools/testing/selftests/openat2/resolve_test.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/openat2/resolve_test.c b/tools/testing/selftests/openat2/resolve_test.c index bbafad440893c..5472ec478d227 100644 --- a/tools/testing/selftests/openat2/resolve_test.c +++ b/tools/testing/selftests/openat2/resolve_test.c @@ -508,12 +508,13 @@ void test_openat2_opath_tests(void) int main(int argc, char **argv) { ksft_print_header(); - ksft_set_plan(NUM_TESTS); /* NOTE: We should be checking for CAP_SYS_ADMIN here... */ - if (geteuid() != 0) + if (geteuid()) ksft_exit_skip("all tests require euid == 0\n"); + ksft_set_plan(NUM_TESTS); + test_openat2_opath_tests(); if (ksft_get_fail_cnt() + ksft_get_error_cnt() > 0) -- 2.39.2

5 months, 1 week

3
6
0 0

[PATCH 1/2] selftests/thermel/intel: conform the test to TAP output

by Muhammad Usama Anjum

Conform the layout, informational and status messages to TAP. No functional change is intended other than the layout of output messages. Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> --- .../intel/workload_hint/workload_hint_test.c | 98 ++++++++----------- 1 file changed, 39 insertions(+), 59 deletions(-) diff --git a/tools/testing/selftests/thermal/intel/workload_hint/workload_hint_test.c b/tools/testing/selftests/thermal/intel/workload_hint/workload_hint_test.c index 217c3a641c537..d107c2d6f3a22 100644 --- a/tools/testing/selftests/thermal/intel/workload_hint/workload_hint_test.c +++ b/tools/testing/selftests/thermal/intel/workload_hint/workload_hint_test.c @@ -9,6 +9,7 @@ #include <fcntl.h> #include <poll.h> #include <signal.h> +#include "../../../kselftest.h" #define WORKLOAD_NOTIFICATION_DELAY_ATTRIBUTE "/sys/bus/pci/devices/0000:00:04.0/workload_hint/notification_delay_ms" #define WORKLOAD_ENABLE_ATTRIBUTE "/sys/bus/pci/devices/0000:00:04.0/workload_hint/workload_hint_enable" @@ -31,17 +32,13 @@ void workload_hint_exit(int signum) /* Disable feature via sysfs knob */ fd = open(WORKLOAD_ENABLE_ATTRIBUTE, O_RDWR); - if (fd < 0) { - perror("Unable to open workload type feature enable file\n"); - exit(1); - } + if (fd < 0) + ksft_exit_fail_perror("Unable to open workload type feature enable file"); - if (write(fd, "0\n", 2) < 0) { - perror("Can' disable workload hints\n"); - exit(1); - } + if (write(fd, "0\n", 2) < 0) + ksft_exit_fail_perror("Can' disable workload hints"); - printf("Disabled workload type prediction\n"); + ksft_print_msg("Disabled workload type prediction\n"); close(fd); } @@ -54,32 +51,27 @@ int main(int argc, char **argv) char delay_str[64]; int delay = 0; - printf("Usage: workload_hint_test [notification delay in milli seconds]\n"); + ksft_print_header(); + ksft_set_plan(1); + + ksft_print_msg("Usage: workload_hint_test [notification delay in milli seconds]\n"); if (argc > 1) { ret = sscanf(argv[1], "%d", &delay); - if (ret < 0) { - printf("Invalid delay\n"); - exit(1); - } + if (ret < 0) + ksft_exit_fail_perror("Invalid delay"); - printf("Setting notification delay to %d ms\n", delay); + ksft_print_msg("Setting notification delay to %d ms\n", delay); if (delay < 0) - exit(1); - - sprintf(delay_str, "%s\n", argv[1]); + ksft_exit_fail_msg("delay can never be negative\n"); sprintf(delay_str, "%s\n", argv[1]); fd = open(WORKLOAD_NOTIFICATION_DELAY_ATTRIBUTE, O_RDWR); - if (fd < 0) { - perror("Unable to open workload notification delay\n"); - exit(1); - } + if (fd < 0) + ksft_exit_fail_perror("Unable to open workload notification delay"); - if (write(fd, delay_str, strlen(delay_str)) < 0) { - perror("Can't set delay\n"); - exit(1); - } + if (write(fd, delay_str, strlen(delay_str)) < 0) + ksft_exit_fail_perror("Can't set delay"); close(fd); } @@ -93,63 +85,51 @@ int main(int argc, char **argv) /* Enable feature via sysfs knob */ fd = open(WORKLOAD_ENABLE_ATTRIBUTE, O_RDWR); - if (fd < 0) { - perror("Unable to open workload type feature enable file\n"); - exit(1); - } + if (fd < 0) + ksft_exit_fail_perror("Unable to open workload type feature enable file"); - if (write(fd, "1\n", 2) < 0) { - perror("Can' enable workload hints\n"); - exit(1); - } + if (write(fd, "1\n", 2) < 0) + ksft_exit_fail_perror("Can' enable workload hints"); close(fd); - printf("Enabled workload type prediction\n"); + ksft_print_msg("Enabled workload type prediction\n"); while (1) { fd = open(WORKLOAD_TYPE_INDEX_ATTRIBUTE, O_RDONLY); - if (fd < 0) { - perror("Unable to open workload type file\n"); - exit(1); - } + if (fd < 0) + ksft_exit_fail_perror("Unable to open workload type file"); - if ((lseek(fd, 0L, SEEK_SET)) < 0) { - fprintf(stderr, "Failed to set pointer to beginning\n"); - exit(1); - } + if ((lseek(fd, 0L, SEEK_SET)) < 0) + ksft_exit_fail_perror("Failed to set pointer to beginning"); - if (read(fd, index_str, sizeof(index_str)) < 0) { - fprintf(stderr, "Failed to read from:%s\n", - WORKLOAD_TYPE_INDEX_ATTRIBUTE); - exit(1); - } + if (read(fd, index_str, sizeof(index_str)) < 0) + ksft_exit_fail_perror("Failed to read from: workload_type_index"); ufd.fd = fd; ufd.events = POLLPRI; ret = poll(&ufd, 1, -1); if (ret < 0) { - perror("poll error"); - exit(1); + ksft_exit_fail_perror("poll error"); } else if (ret == 0) { - printf("Poll Timeout\n"); + ksft_print_msg("Poll Timeout\n"); } else { - if ((lseek(fd, 0L, SEEK_SET)) < 0) { - fprintf(stderr, "Failed to set pointer to beginning\n"); - exit(1); - } + if ((lseek(fd, 0L, SEEK_SET)) < 0) + ksft_exit_fail_perror("Failed to set pointer to beginning"); - if (read(fd, index_str, sizeof(index_str)) < 0) - exit(0); + if (read(fd, index_str, sizeof(index_str)) < 0) { + ksft_test_result_pass("Successfully read\n"); + ksft_finished(); + } ret = sscanf(index_str, "%d", &index); if (ret < 0) break; if (index > WORKLOAD_TYPE_MAX_INDEX) - printf("Invalid workload type index\n"); + ksft_print_msg("Invalid workload type index\n"); else - printf("workload type:%s\n", workload_types[index]); + ksft_print_msg("workload type:%s\n", workload_types[index]); } close(fd); -- 2.39.2

5 months, 1 week

3
5
0 0

[PATCH] selftests: firmware: conform test to TAP

by Muhammad Usama Anjum

Conform the layout, informational and status messages to TAP. No functional change is intended other than the layout of output messages. Without using TAP messages, the passed/failed/skip test names cannot be found. This is a prepartory patch, more patches will be sent to comform the test completely. Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> --- .../selftests/firmware/fw_run_tests.sh | 63 ++++++++++++------- 1 file changed, 40 insertions(+), 23 deletions(-) diff --git a/tools/testing/selftests/firmware/fw_run_tests.sh b/tools/testing/selftests/firmware/fw_run_tests.sh index f6d95a2d51249..5063c75345680 100755 --- a/tools/testing/selftests/firmware/fw_run_tests.sh +++ b/tools/testing/selftests/firmware/fw_run_tests.sh @@ -6,6 +6,9 @@ set -e +DIR="$(dirname $(readlink -f "$0"))" +source "${DIR}"/../kselftest/ktap_helpers.sh + TEST_DIR=$(dirname $0) source $TEST_DIR/fw_lib.sh @@ -26,54 +29,68 @@ run_tests() proc_set_force_sysfs_fallback $1 proc_set_ignore_sysfs_fallback $2 $TEST_DIR/fw_upload.sh + + ktap_test_pass "Completed" } run_test_config_0001() { - echo "-----------------------------------------------------" - echo "Running kernel configuration test 1 -- rare" - echo "Emulates:" - echo "CONFIG_FW_LOADER=y" - echo "CONFIG_FW_LOADER_USER_HELPER=n" - echo "CONFIG_FW_LOADER_USER_HELPER_FALLBACK=n" + ktap_print_msg "-----------------------------------------------------" + ktap_print_msg "Running kernel configuration test 1 -- rare" + ktap_print_msg "Emulates:" + ktap_print_msg "CONFIG_FW_LOADER=y" + ktap_print_msg "CONFIG_FW_LOADER_USER_HELPER=n" + ktap_print_msg "CONFIG_FW_LOADER_USER_HELPER_FALLBACK=n" run_tests 0 1 } run_test_config_0002() { - echo "-----------------------------------------------------" - echo "Running kernel configuration test 2 -- distro" - echo "Emulates:" - echo "CONFIG_FW_LOADER=y" - echo "CONFIG_FW_LOADER_USER_HELPER=y" - echo "CONFIG_FW_LOADER_USER_HELPER_FALLBACK=n" + ktap_print_msg "-----------------------------------------------------" + ktap_print_msg "Running kernel configuration test 2 -- distro" + ktap_print_msg "Emulates:" + ktap_print_msg "CONFIG_FW_LOADER=y" + ktap_print_msg "CONFIG_FW_LOADER_USER_HELPER=y" + ktap_print_msg "CONFIG_FW_LOADER_USER_HELPER_FALLBACK=n" proc_set_ignore_sysfs_fallback 0 run_tests 0 0 } run_test_config_0003() { - echo "-----------------------------------------------------" - echo "Running kernel configuration test 3 -- android" - echo "Emulates:" - echo "CONFIG_FW_LOADER=y" - echo "CONFIG_FW_LOADER_USER_HELPER=y" - echo "CONFIG_FW_LOADER_USER_HELPER_FALLBACK=y" + ktap_print_msg "-----------------------------------------------------" + ktap_print_msg "Running kernel configuration test 3 -- android" + ktap_print_msg "Emulates:" + ktap_print_msg "CONFIG_FW_LOADER=y" + ktap_print_msg "CONFIG_FW_LOADER_USER_HELPER=y" + ktap_print_msg "CONFIG_FW_LOADER_USER_HELPER_FALLBACK=y" run_tests 1 0 } +ktap_print_header + check_mods check_setup -echo "Running namespace test: " -$TEST_DIR/fw_namespace $DIR/trigger_request -echo "OK" - if [ -f $FW_FORCE_SYSFS_FALLBACK ]; then + ktap_set_plan "4" + run_test_config_0001 run_test_config_0002 run_test_config_0003 else - echo "Running basic kernel configuration, working with your config" + ktap_set_plan "2" + + ktap_print_msg "Running basic kernel configuration, working with your config" run_tests fi + +ktap_print_msg "Running namespace test: " +$TEST_DIR/fw_namespace $DIR/trigger_request +if [ $? -eq 0 ]; then + ktap_test_pass "fw_namespace completed successfully" +else + ktap_test_fail "fw_namespace failed" +fi + +ktap_finished -- 2.39.2

5 months, 1 week

2
2
0 0

[PATCH v2 0/8] mm: workingset reporting

by Yuanchu Xie

Changes from PATCH v1 -> v2: - Updated selftest to use ksft_test_result_code instead of switch-case (Muhammad Usama Anjum) - Included more use cases in the cover letter (Huang, Ying) - Added documentation for sysfs and memcg interfaces - Added an aging-specific struct lru_gen_mm_walk in struct pglist_data to avoid allocating for each lruvec. Changes from RFC v3 -> PATCH v1: - Updated selftest to use ksft_print_msg instead of fprintf(stderr, ...) (Muhammad Usama Anjum) - Included more detail in patch skipping pmd_young with force_scan (Huang, Ying) - Deferred reaccess histogram as a followup - Removed per-memcg page age interval configs for simplicity Changes from RFC v2 -> RFC v3: - Update to v6.8 - Added an aging kernel thread (gated behind config) - Added basic selftests for sysfs interface files - Track swapped out pages for reaccesses - Refactoring and cleanup - Dropped the virtio-balloon extension to make things manageable Changes from RFC v1 -> RFC v2: - Refactored the patchs into smaller pieces - Renamed interfaces and functions from wss to wsr (Working Set Reporting) - Fixed build errors when CONFIG_WSR is not set - Changed working_set_num_bins to u8 for virtio-balloon - Added support for per-NUMA node reporting for virtio-balloon [rfc v1] https://lore.kernel.org/linux-mm/20230509185419.1088297-1-yuanchu@google.co… [rfc v2] https://lore.kernel.org/linux-mm/20230621180454.973862-1-yuanchu@google.com/ [rfc v3] https://lore.kernel.org/linux-mm/20240327213108.2384666-1-yuanchu@google.co… This patch series provides workingset reporting of user pages in lruvecs, of which coldness can be tracked by accessed bits and fd references. However, the concept of workingset applies generically to all types of memory, which could be kernel slab caches, discardable userspace caches (databases), or CXL.mem. Therefore, data sources might come from slab shrinkers, device drivers, or the userspace. IMO, the kernel should provide a set of workingset interfaces that should be generic enough to accommodate the various use cases, and be extensible to potential future use cases. The current proposed interfaces are not sufficient in that regard, but I would like to start somewhere, solicit feedback, and iterate. Use cases ========== Job scheduling On overcommitted hosts, workingset information allows the job scheduler to right-size each job and land more jobs on the same host or NUMA node, and in the case of a job with increasing workingset, policy decisions can be made to migrate other jobs off the host/NUMA node, or oom-kill the misbehaving job. If the job shape is very different from the machine shape, knowing the workingset per-node can also help inform page allocation policies. Proactive reclaim Workingset information allows the a container manager to proactively reclaim memory while not impacting a job's performance. While PSI may provide a reactive measure of when a proactive reclaim has reclaimed too much, workingset reporting allows the policy to be more accurate and flexible. Ballooning (similar to proactive reclaim) While this patch series does not extend the virtio-balloon device, balloon policies benefit from workingset to more precisely determine the size of the memory balloon. On desktops/laptops/mobile devices where memory is scarce and overcommitted, the balloon sizing in multiple VMs running on the same device can be orchestrated with workingset reports from each one. Promotion/Demotion If different mechanisms are used for promition and demotion, workingset information can help connect the two and avoid pages being migrated back and forth. For example, given a promotion hot page threshold defined in reaccess distance of N seconds (promote pages accessed more often than every N seconds). The threshold N should be set so that ~80% (e.g.) of pages on the fast memory node passes the threshold. This calculation can be done with workingset reports. To be directly useful for promotion policies, the workingset report interfaces need to be extended to report hotness and gather hotness information from the devices[1]. [1] https://www.opencompute.org/documents/ocp-cms-hotness-tracking-requirements… Sysfs and Cgroup Interfaces ========== The interfaces are detailed in the patches that introduce them. The main idea here is we break down the workingset per-node per-memcg into time intervals (ms), e.g. 1000 anon=137368 file=24530 20000 anon=34342 file=0 30000 anon=353232 file=333608 40000 anon=407198 file=206052 9223372036854775807 anon=4925624 file=892892 I realize this does not generalize well to hotness information, but I lack the intuition for an abstraction that presents hotness in a useful way. Based on a recent proposal for move_phys_pages[2], it seems like userspace tiering software would like to move specific physical pages, instead of informing the kernel "move x number of hot pages to y device". Please advise. [2] https://lore.kernel.org/lkml/20240319172609.332900-1-gregory.price@memverge… Implementation ========== Currently, the reporting of user pages is based off of MGLRU, and therefore requires CONFIG_LRU_GEN=y. We would benefit from more MGLRU generations for a more fine-grained workingset report. I will make the generation count configurable in the next version. The workingset reporting mechanism is gated behind CONFIG_WORKINGSET_REPORT, and the aging thread is behind CONFIG_WORKINGSET_REPORT_AGING. Yuanchu Xie (8): mm: multi-gen LRU: ignore non-leaf pmd_young for force_scan=true mm: aggregate working set information into histograms mm: use refresh interval to rate-limit workingset report aggregation mm: report workingset during memory pressure driven scanning mm: extend working set reporting to memcgs mm: add kernel aging thread for workingset reporting selftest: test system-wide workingset reporting Docs/admin-guide/mm/workingset_report: document sysfs and memcg interfaces Documentation/admin-guide/mm/index.rst | 1 + .../admin-guide/mm/workingset_report.rst | 105 ++++ drivers/base/node.c | 6 + include/linux/memcontrol.h | 5 + include/linux/mmzone.h | 9 + include/linux/workingset_report.h | 97 +++ mm/Kconfig | 15 + mm/Makefile | 2 + mm/internal.h | 18 + mm/memcontrol.c | 184 +++++- mm/mm_init.c | 2 + mm/mmzone.c | 2 + mm/vmscan.c | 58 +- mm/workingset_report.c | 561 ++++++++++++++++++ mm/workingset_report_aging.c | 127 ++++ tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 3 + tools/testing/selftests/mm/run_vmtests.sh | 5 + .../testing/selftests/mm/workingset_report.c | 306 ++++++++++ .../testing/selftests/mm/workingset_report.h | 39 ++ .../selftests/mm/workingset_report_test.c | 329 ++++++++++ 21 files changed, 1869 insertions(+), 6 deletions(-) create mode 100644 Documentation/admin-guide/mm/workingset_report.rst create mode 100644 include/linux/workingset_report.h create mode 100644 mm/workingset_report.c create mode 100644 mm/workingset_report_aging.c create mode 100644 tools/testing/selftests/mm/workingset_report.c create mode 100644 tools/testing/selftests/mm/workingset_report.h create mode 100644 tools/testing/selftests/mm/workingset_report_test.c -- 2.45.1.467.gbab1589fc0-goog

5 months, 1 week

4
11
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror June 2024