November 2023 - Linux-kselftest-mirror

[PATCHv2 net-next 00/14] Conver net selftests to run in unique namespace (Part 1)

by Hangbin Liu

As Guillaume pointed, many selftests create namespaces with very common names (like "client" or "server") or even (partially) run directly in init_net. This makes these tests prone to failure if another namespace with the same name already exists. It also makes it impossible to run several instances of these tests in parallel. This patch set intend to conver all the net selftests to run in unique namespace, so we can update the selftest freamwork to run all tests in it's own namespace in parallel. After update, we only need to wait for the test which need longest time. As the total patch set is too large. I break it to severl parts. This is the first part. v1 -> v2: - Split the large patch set to small parts for easy review (Paolo Abeni) - Move busywait from forwarding/lib.sh to net/lib.sh directly (Petr Machata) - Update setup_ns/cleanup_ns struct (Petr Machata) - Remove default trap in lib.sh (Petr Machata) Hangbin Liu (14): selftests/net: add lib.sh selftests/net: convert arp_ndisc_evict_nocarrier.sh to run it in unique namespace selftests/net: specify the interface when do arping selftests/net: convert arp_ndisc_untracked_subnets.sh to run it in unique namespace selftests/net: convert cmsg tests to make them run in unique namespace selftests/net: convert drop_monitor_tests.sh to run it in unique namespace selftests/net: convert traceroute.sh to run it in unique namespace selftests/net: convert icmp_redirect.sh to run it in unique namespace sleftests/net: convert icmp.sh to run it in unique namespace selftests/net: convert ioam6.sh to run it in unique namespace selftests/net: convert l2tp.sh to run it in unique namespace selftests/net: convert ndisc_unsolicited_na_test.sh to run it in unique namespace selftests/net: convert sctp_vrf.sh to run it in unique namespace selftests/net: convert unicast_extensions.sh to run it in unique namespace tools/testing/selftests/net/Makefile | 2 +- .../net/arp_ndisc_evict_nocarrier.sh | 46 ++-- .../net/arp_ndisc_untracked_subnets.sh | 20 +- tools/testing/selftests/net/cmsg_ipv6.sh | 10 +- tools/testing/selftests/net/cmsg_so_mark.sh | 7 +- tools/testing/selftests/net/cmsg_time.sh | 7 +- .../selftests/net/drop_monitor_tests.sh | 21 +- tools/testing/selftests/net/forwarding/lib.sh | 27 +- tools/testing/selftests/net/icmp.sh | 10 +- tools/testing/selftests/net/icmp_redirect.sh | 182 +++++++------ tools/testing/selftests/net/ioam6.sh | 247 +++++++++--------- tools/testing/selftests/net/l2tp.sh | 130 +++++---- tools/testing/selftests/net/lib.sh | 85 ++++++ .../net/ndisc_unsolicited_na_test.sh | 19 +- tools/testing/selftests/net/sctp_vrf.sh | 12 +- tools/testing/selftests/net/traceroute.sh | 82 +++--- .../selftests/net/unicast_extensions.sh | 99 ++++--- 17 files changed, 500 insertions(+), 506 deletions(-) create mode 100644 tools/testing/selftests/net/lib.sh -- 2.41.0

1 year, 3 months

6
22
0 0

[PATCH v2 RESEND] kunit: debugfs: Handle errors from alloc_string_stream()

by Richard Fitzgerald

In kunit_debugfs_create_suite() give up and skip creating the debugfs file if any of the alloc_string_stream() calls return an error or NULL. Only put a value in the log pointer of kunit_suite and kunit_test if it is a valid pointer to a log. This prevents the potential invalid dereference reported by smatch: lib/kunit/debugfs.c:115 kunit_debugfs_create_suite() error: 'suite->log' dereferencing possible ERR_PTR() lib/kunit/debugfs.c:119 kunit_debugfs_create_suite() error: 'test_case->log' dereferencing possible ERR_PTR() Signed-off-by: Richard Fitzgerald <rf(a)opensource.cirrus.com> Reported-by: Dan Carpenter <dan.carpenter(a)linaro.org> Fixes: 05e2006ce493 ("kunit: Use string_stream for test log") --- lib/kunit/debugfs.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/lib/kunit/debugfs.c b/lib/kunit/debugfs.c index 270d185737e6..9d167adfa746 100644 --- a/lib/kunit/debugfs.c +++ b/lib/kunit/debugfs.c @@ -109,14 +109,28 @@ static const struct file_operations debugfs_results_fops = { void kunit_debugfs_create_suite(struct kunit_suite *suite) { struct kunit_case *test_case; + struct string_stream *stream; - /* Allocate logs before creating debugfs representation. */ - suite->log = alloc_string_stream(GFP_KERNEL); - string_stream_set_append_newlines(suite->log, true); + /* + * Allocate logs before creating debugfs representation. + * The suite->log and test_case->log pointer are expected to be NULL + * if there isn't a log, so only set it if the log stream was created + * successfully. + */ + stream = alloc_string_stream(GFP_KERNEL); + if (IS_ERR_OR_NULL(stream)) + return; + + string_stream_set_append_newlines(stream, true); + suite->log = stream; kunit_suite_for_each_test_case(suite, test_case) { - test_case->log = alloc_string_stream(GFP_KERNEL); - string_stream_set_append_newlines(test_case->log, true); + stream = alloc_string_stream(GFP_KERNEL); + if (IS_ERR_OR_NULL(stream)) + goto err; + + string_stream_set_append_newlines(stream, true); + test_case->log = stream; } suite->debugfs = debugfs_create_dir(suite->name, debugfs_rootdir); @@ -124,6 +138,12 @@ void kunit_debugfs_create_suite(struct kunit_suite *suite) debugfs_create_file(KUNIT_DEBUGFS_RESULTS, S_IFREG | 0444, suite->debugfs, suite, &debugfs_results_fops); + return; + +err: + string_stream_destroy(suite->log); + kunit_suite_for_each_test_case(suite, test_case) + string_stream_destroy(test_case->log); } void kunit_debugfs_destroy_suite(struct kunit_suite *suite) -- 2.30.2

1 year, 3 months

4
3
0 0

[PATCH net-next 0/4] selftests: tc-testing: more tdc updates

by Pedro Tammela

Follow-up on a feedback from Jakub and random cleanups from related net/sched patches Pedro Tammela (4): selftests: tc-testing: remove spurious nsPlugin usage selftests: tc-testing: remove spurious './' from Makefile selftests: tc-testing: rename concurrency.json to flower.json selftests: tc-testing: remove filters/tests.json tools/testing/selftests/tc-testing/Makefile | 2 +- .../filters/{concurrency.json => flower.json} | 98 +++++++++++++ .../tc-testing/tc-tests/filters/matchall.json | 23 ++++ .../tc-testing/tc-tests/filters/tests.json | 129 ------------------ 4 files changed, 122 insertions(+), 130 deletions(-) rename tools/testing/selftests/tc-testing/tc-tests/filters/{concurrency.json => flower.json} (65%) delete mode 100644 tools/testing/selftests/tc-testing/tc-tests/filters/tests.json -- 2.40.1

1 year, 3 months

4
8
0 0

[PATCH 00/12] selftests/hid: tablets fixes

by Benjamin Tissoires

Hi, the main trigger of this series was the XP-Pen issue[0]. Basically, the tablets tests were good-ish but couldn't handle that tablet both in terms of emulation or in terms of detection of issues. So rework the tablets test a bit to be able to include the XP-Pen patch later, once I have a kernel fix for it (right now I only have a HID-BPF fix, meaning that the test will fail if I include them). Also, vmtest.sh needed a little bit of care, because boot2container moved, and I made it easier to reuse in a CI environment. Cheers, Benjamin [0] https://lore.kernel.org/all/nycvar.YFH.7.76.2311012033290.29220@cbobk.fhfr.… Signed-off-by: Benjamin Tissoires <bentiss(a)kernel.org> --- Benjamin Tissoires (12): selftests/hid: vmtest.sh: update vm2c and container selftests/hid: vmtest.sh: allow finer control on the build steps selftests/hid: base: allow for multiple skip_if_uhdev selftests/hid: tablets: remove unused class selftests/hid: tablets: move the transitions to PenState selftests/hid: tablets: move move_to function to PenDigitizer selftests/hid: tablets: do not set invert when the eraser is used selftests/hid: tablets: set initial data for tilt/twist selftests/hid: tablets: add variants of states with buttons selftests/hid: tablets: convert the primary button tests selftests/hid: tablets: add a secondary barrel switch test selftests/hid: tablets: be stricter for some transitions tools/testing/selftests/hid/tests/base.py | 3 +- tools/testing/selftests/hid/tests/test_tablet.py | 727 ++++++++++++++++------- tools/testing/selftests/hid/vmtest.sh | 46 +- 3 files changed, 525 insertions(+), 251 deletions(-) --- base-commit: 4ea4ed22b57846facd9cb4af5f67cb7bd2792cf3 change-id: 20231121-wip-selftests-001ac427e086 Best regards, -- Benjamin Tissoires <bentiss(a)kernel.org>

1 year, 3 months

2
15
0 0

[GIT PULL] KUnit fixes update for Linux 6.7-rc4

by Shuah Khan

Hi Linus, Please pull the following KUnit fixes update for Linux 6.7-rc4. This KUnit fixes update for Linux 6.7-rc4 consists of three fixes to warnings and run-time test behavior. With these fixes, test suite counter will be reset correctly before running tests, kunit will warn if tests are too slow, and eliminate warning when kfree() as an action. diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit b85ea95d086471afb4ad062012a4d73cd328fa86: Linux 6.7-rc1 (2023-11-12 16:19:07 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux_kselftest-kunit-fixes-6.7-rc4 for you to fetch changes up to 1bddcf77ce6668692fc15e968fd0870d5524d112: kunit: test: Avoid cast warning when adding kfree() as an action (2023-11-14 13:01:57 -0700) ---------------------------------------------------------------- linux_kselftest-kunit-fixes-6.7-rc4 This KUnit fixes update for Linux 6.7-rc4 consists of three fixes to warnings and run-time test behavior. With these fixes, test suite counter will be reset correctly before running tests, kunit will warn if tests are too slow, and eliminate warning when kfree() as an action. ---------------------------------------------------------------- Maxime Ripard (1): kunit: Warn if tests are slow Michal Wajdeczko (1): kunit: Reset suite counter right before running tests Richard Fitzgerald (1): kunit: test: Avoid cast warning when adding kfree() as an action lib/kunit/kunit-test.c | 2 +- lib/kunit/test.c | 42 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 41 insertions(+), 3 deletions(-) ----------------------------------------------------------------

1 year, 3 months

2
1
0 0

[PATCH RESEND] kunit: string-stream: Allow ERR_PTR to be passed to string_stream_destroy()

by Richard Fitzgerald

Check the stream pointer passed to string_stream_destroy() for IS_ERR_OR_NULL() instead of only NULL. Whatever alloc_string_stream() returns should be safe to pass to string_stream_destroy(), and that will be an ERR_PTR. It's obviously good practise and generally helpful to also check for NULL pointers so that client cleanup code can call string_stream_destroy() unconditionally - which could include pointers that have never been set to anything and so are NULL. Signed-off-by: Richard Fitzgerald <rf(a)opensource.cirrus.com> --- lib/kunit/string-stream.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/kunit/string-stream.c b/lib/kunit/string-stream.c index a6f3616c2048..54f4fdcbfac8 100644 --- a/lib/kunit/string-stream.c +++ b/lib/kunit/string-stream.c @@ -173,7 +173,7 @@ void string_stream_destroy(struct string_stream *stream) { KUNIT_STATIC_STUB_REDIRECT(string_stream_destroy, stream); - if (!stream) + if (IS_ERR_OR_NULL(stream)) return; string_stream_clear(stream); -- 2.30.2

1 year, 3 months

3
2
0 0

[PATCH v1] selftests: cgroup: Fixes code style errors

by Atul Kumar Pant

Fixes following checkpatch.pl issues: ERROR: do not use assignment in if condition ERROR: Macros starting with if should be enclosed by a do - while Signed-off-by: Atul Kumar Pant <atulpant.linux(a)gmail.com> --- tools/testing/selftests/cgroup/cgroup_util.c | 14 ++++++++------ tools/testing/selftests/cgroup/test_core.c | 3 ++- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c index 0340d4ca8f51..e165c4a703a6 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.c +++ b/tools/testing/selftests/cgroup/cgroup_util.c @@ -411,12 +411,14 @@ int dirfd_open_opath(const char *dir) return open(dir, O_DIRECTORY | O_CLOEXEC | O_NOFOLLOW | O_PATH); } -#define close_prot_errno(fd) \ - if (fd >= 0) { \ - int _e_ = errno; \ - close(fd); \ - errno = _e_; \ - } +#define close_prot_errno(fd) \ + do { \ + if (fd >= 0) { \ + int _e_ = errno; \ + close(fd); \ + errno = _e_; \ + } \ + } while (0); static int clone_into_cgroup_run_nowait(const char *cgroup, int (*fn)(const char *cgroup, void *arg), diff --git a/tools/testing/selftests/cgroup/test_core.c b/tools/testing/selftests/cgroup/test_core.c index 80aa6b2373b9..ad758e510174 100644 --- a/tools/testing/selftests/cgroup/test_core.c +++ b/tools/testing/selftests/cgroup/test_core.c @@ -817,7 +817,8 @@ static int test_cgcore_lesser_ns_open(const char *root) if (cg_enter_current(cg_test_a)) goto cleanup; - if ((status = write(cg_test_b_procs_fd, "0", 1)) >= 0 || errno != ENOENT) + status = write(cg_test_b_procs_fd, "0", 1); + if (status >= 0 || errno != ENOENT) goto cleanup; ret = KSFT_PASS; -- 2.25.1

1 year, 3 months

2
1
0 0

[PATCH] selftests: capabilities: namespace create varies for root and normal user

by Swarup Laxman Kotiaklapudi

Change namespace creation for root and non-root user differently in create_and_enter_ns() function Test result with root user: $sudo make TARGETS="capabilities" kselftest ... TAP version 13 1..1 timeout set to 45 selftests: capabilities: test_execve TAP version 13 1..12 [RUN] +++ Tests with uid == 0 +++ [NOTE] Using global UIDs for tests [RUN] Root => ep ... ok 12 Passed Totals: pass:12 fail:0 xfail:0 xpass:0 skip:0 error:0 ================================================== TAP version 13 1..9 [RUN] +++ Tests with uid != 0 +++ [NOTE] Using global UIDs for tests [RUN] Non-root => no caps ... ok 9 Passed Totals: pass:9 fail:0 xfail:0 xpass:0 skip:0 error:0 Test result without root or normal user: $make TARGETS="capabilities" kselftest ... timeout set to 45 selftests: capabilities: test_execve TAP version 13 1..12 [RUN] +++ Tests with uid == 0 +++ [NOTE] Using a user namespace for tests [RUN] Root => ep validate_cap:: Capabilities after execve were correct ok 1 Passed Check cap_ambient manipulation rules ok 2 PR_CAP_AMBIENT_RAISE failed on non-inheritable cap ok 3 PR_CAP_AMBIENT_RAISE failed on non-permitted cap ok 4 PR_CAP_AMBIENT_RAISE worked ok 5 Basic manipulation appears to work [RUN] Root +i => eip validate_cap:: Capabilities after execve were correct ok 6 Passed [RUN] UID 0 +ia => eipa validate_cap:: Capabilities after execve were correct ok 7 Passed ok 8 # SKIP SUID/SGID tests (needs privilege) Planned tests != run tests (12 != 8) Totals: pass:7 fail:0 xfail:0 xpass:0 skip:1 error:0 ================================================== TAP version 13 1..9 [RUN] +++ Tests with uid != 0 +++ [NOTE] Using a user namespace for tests [RUN] Non-root => no caps validate_cap:: Capabilities after execve were correct ok 1 Passed Check cap_ambient manipulation rules ok 2 PR_CAP_AMBIENT_RAISE failed on non-inheritable cap ok 3 PR_CAP_AMBIENT_RAISE failed on non-permitted cap ok 4 PR_CAP_AMBIENT_RAISE worked ok 5 Basic manipulation appears to work [RUN] Non-root +i => i validate_cap:: Capabilities after execve were correct ok 6 Passed [RUN] UID 1 +ia => eipa validate_cap:: Capabilities after execve were correct ok 7 Passed ok 8 # SKIP SUID/SGID tests (needs privilege) Planned tests != run tests (9 != 8) Totals: pass:7 fail:0 xfail:0 xpass:0 skip:1 error:0 Signed-off-by: Swarup Laxman Kotiaklapudi <swarupkotikalapudi(a)gmail.com> --- tools/testing/selftests/capabilities/test_execve.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/tools/testing/selftests/capabilities/test_execve.c b/tools/testing/selftests/capabilities/test_execve.c index df0ef02b4036..8236150d377e 100644 --- a/tools/testing/selftests/capabilities/test_execve.c +++ b/tools/testing/selftests/capabilities/test_execve.c @@ -96,11 +96,7 @@ static bool create_and_enter_ns(uid_t inner_uid) outer_uid = getuid(); outer_gid = getgid(); - /* - * TODO: If we're already root, we could skip creating the userns. - */ - - if (unshare(CLONE_NEWNS) == 0) { + if (outer_uid == 0 && unshare(CLONE_NEWNS) == 0) { ksft_print_msg("[NOTE]\tUsing global UIDs for tests\n"); if (prctl(PR_SET_KEEPCAPS, 1, 0, 0, 0) != 0) ksft_exit_fail_msg("PR_SET_KEEPCAPS - %s\n", -- 2.34.1

1 year, 3 months

3
5
0 0

[PATCH v2] selftests: prctl: Add prctl test for PR_GET_NAME

by Osama Muhammad

This patch covers the testing of PR_GET_NAME by reading it's value from proc/self/task/pid/comm and matching it with the value returned by PR_GET_NAME. If the values are matched then it's successful, otherwise it fails. changes since v1: - Handled fscanf,fopen error checking. - Defined MAX_PATH_LEN. Signed-off-by: Osama Muhammad <osmtendev(a)gmail.com> --- .../selftests/prctl/set-process-name.c | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/tools/testing/selftests/prctl/set-process-name.c b/tools/testing/selftests/prctl/set-process-name.c index 3bc5e0e09..562f707ba 100644 --- a/tools/testing/selftests/prctl/set-process-name.c +++ b/tools/testing/selftests/prctl/set-process-name.c @@ -12,6 +12,7 @@ #define CHANGE_NAME "changename" #define EMPTY_NAME "" #define TASK_COMM_LEN 16 +#define MAX_PATH_LEN 50 int set_name(char *name) { @@ -47,6 +48,35 @@ int check_null_pointer(char *check_name) return res; } +int check_name(void) +{ + + int pid; + + pid = getpid(); + FILE *fptr = NULL; + char path[MAX_PATH_LEN] = {}; + char name[TASK_COMM_LEN] = {}; + char output[TASK_COMM_LEN] = {}; + int j; + + j = snprintf(path, MAX_PATH_LEN, "/proc/self/task/%d/comm", pid); + fptr = fopen(path, "r"); + if (!fptr) + return -EIO; + + fscanf(fptr, "%s", output); + if (ferror(fptr)) + return -EIO; + + int res = prctl(PR_GET_NAME, name, NULL, NULL, NULL); + + if (res < 0) + return -errno; + + return !strcmp(output, name); +} + TEST(rename_process) { EXPECT_GE(set_name(CHANGE_NAME), 0); @@ -57,6 +87,8 @@ TEST(rename_process) { EXPECT_GE(set_name(CHANGE_NAME), 0); EXPECT_LT(check_null_pointer(CHANGE_NAME), 0); + + EXPECT_TRUE(check_name()); } TEST_HARNESS_MAIN -- 2.34.1

1 year, 3 months

2
3
0 0

[PATCH 0/9] mm/damon: let users feed and tame/auto-tune DAMOS

by SeongJae Park

Introduce Aim-oriented Feedback-driven DAMOS Aggressiveness Auto-tuning. It makes DAMOS self-tuned with periodic simple user feedback. Patchset Changelog ================== From RFC (https://lore.kernel.org/damon/20231112194607.61399-1-sj@kernel.org/) - Wordsmith commit messages and cover letter Background: DAMOS Control Difficulty ==================================== DAMOS helps users easily implement access pattern aware system operations. However, controlling DAMOS in the wild is not that easy. The basic way for DAMOS control is specifying the target access pattern. In this approach, the user is assumed to well understand the access pattern and the characteristics of the system and the workloads. Though there are useful tools for that, it takes time and effort depending on the complexity and the dynamicity of the system and the workloads. After all, the access pattern consists of three ranges, namely the size, the access rate, and the age of the regions. It means users need to tune six parameters, which is anyway not a simple task. One of the worst cases would be DAMOS being too aggressive like a berserker, and therefore consuming too much system resource and making unwanted radical system operations. To let users avoid such cases, DAMOS allows users to set the upper-limit of the schemes' aggressiveness, namely DAMOS quota. DAMOS further provides its best-effort under the limit by prioritizing regions based on the access pattern of the regions. For example, users can ask DAMOS to page out up to 100 MiB of memory regions per second. Then DAMOS pages out regions that are not accessed for a longer time (colder) first under the limit. This allows users to set the target access pattern a bit naive with wider ranges, and focus on tuning only one parameter, the quota. In other words, the number of parameters to tune can be reduced from six to one. Still, however, the optimum value for the quota depends on the system and the workloads' characteristics, so not that simple. The number of parameters to tune can also increase again if the user needs to run multiple schemes. Aim-oriented Feedback-driven DAMOS Aggressiveness Auto Tuning ============================================================= Users would use DAMOS since they want to achieve something with it. They will likely have measurable metrics representing the achievement and the target number of the metric like SLO, and continuously measure that anyway. While the additional cost of getting the information is nearly zero, it could be useful for DAMOS to understand how appropriate its current aggressiveness is set, and adjust it on its own to make the metric value more close to the target. Based on this idea, we introduce a new way of tuning DAMOS with nearly zero additional effort, namely Aim-oriented Feedback-driven DAMOS Aggressiveness Auto Tuning. It asks users to provide feedback representing how well DAMOS is doing relative to the users' aim. Then DAMOS adjusts its aggressiveness, specifically the quota that provides the best effort result under the limit, based on the current level of the aggressiveness and the users' feedback. Implementation -------------- The implementation asks users to represent the feedback with score numbers. The scores could be anything including user-space specific metrics including latency and throughput of special user-space workloads, and system metrics including free memory ratio, memory pressure stall time (PSI), and active to inactive LRU lists size ratio. The feedback scores and the aggressiveness of the given DAMOS scheme are assumed to be positively proportional, though. Selecting metrics of the assumption is the users' responsibility. The core logic uses the below simple feedback loop algorithm to calculate the next aggressiveness level of the scheme from the current aggressiveness level and the current feedback (target_score and current_score). It calculates the compensation for next aggressiveness as a proportion of current aggressiveness and distance to the target score. As a result, it arrives at the near-goal state in a short time using big steps when it's far from the goal, but avoids making unnecessarily radical changes that could turn out to be a bad decision using small steps when its near to the goal. f(n) = max(1, f(n - 1) * ((target_score - current_score) / target_score + 1)) Note that the compensation value becomes negative when it's over achieving the goal. That's why the feedback metric and the aggressiveness of the scheme should be positively proportional. The distance-adaptive speed manipulation is simply applied. Example Use Cases ----------------- If users want to reduce the memory footprint of the system as much as possible as long as the time spent for handling the resulting memory pressure is within a threshold, they could use DAMOS scheme that reclaims cold memory regions aiming for a little level of memory pressure stall time. If users want the active/inactive LRU lists well balanced to reduce the performance impact due to possible future memory pressure, they could use two schemes. The first one would be set to locate hot pages in the active LRU list, aiming for a specific active-to-inactive LRU list size ratio, say, 70%. The second one would be to locate cold pages in the inactive LRU list, aiming for a specific inactive-to-active LRU list size ratio, say, 30%. Then, DAMOS will balance the two schemes based on the goal and feedback. This aim-oriented auto tuning could also be useful for general balancing-required access aware system operations such as system memory auto scaling[3] and tiered memory management[4]. These two example usages are not what current DAMOS implementation is already supporting, but require additional DAMOS action developments, though. Evaluation: subtle memory pressure aiming proactive reclamation --------------------------------------------------------------- To show if the implementation works as expected, we prepare four different system configurations on AWS i3.metal instances. The first setup (original) runs the workload without any DAMOS scheme. The second setup (not-tuned) runs the workload with a virtual address space-based proactive reclamation scheme that pages out memory regions that are not accessed for five seconds or more. The third setup (offline-tuned) runs the same proactive reclamation DAMOS scheme, but after making it tuned for each workload offline, using our previous user-space driven automatic tuning approach, namely DAMOOS[1]. The fourth and final setup (AFDAA) runs the scheme that is the same as that of 'not-tuned' setup, but aims to keep 0.5% of 'some' memory pressure stall time (PSI) for the last 10 seconds using the aiming-oriented auto tuning. For each setup, we run realistic workloads from PARSEC3 and SPLASH-2X benchmark suites. For each run, we measure RSS and runtime of the workload, and 'some' memory pressure stall time (PSI) of the system. We repeat the runs five times and use averaged measurements. For simple comparison of the results, we normalize the measurements to those of 'original'. In the case of the PSI, though, the measurement for 'original' was zero, so we normalize the value to that of 'not-tuned' scheme's result. The normalized results are shown below. Not-tuned Offline-tuned AFDAA RSS 0.622688178226118 0.787950678944904 0.740093483278979 runtime 1.11767826657912 1.0564674983585 1.0910833880499 PSI 1 0.727521443794069 0.308498846350299 The 'not-tuned' scheme achieves about 38.7% memory saving but incur about 11.7% runtime slowdown. The 'offline-tuned' scheme achieves about 22.2% memory saving with about 5.5% runtime slowdown. It also achieves about 28.2% memory pressure stall time saving. AFDAA achieves about 26% memory saving with about 9.1% runtime slowdown. It also achieves about 69.1% memory pressure stall time saving. We repeat this test multiple times, and get consistent results. AFDAA is now integrated in our daily DAMON performance test setup. Apparently the aggressiveness of 'AFDAA' setup is somewhere between those of 'not-tuned' and 'offline-tuned' setup, since its memory saving and runtime overhead are between those of the other two setups. Actually we set the memory pressure stall time goal aiming for this middle aggressiveness. The difference in the two metrics are not significant, though. However, it shows significant saving of the memory pressure stall time, which was the goal of the auto-tuning, over the two variants. Hence, we conclude the automatic tuning is working as expected. Please note that the AFDAA setup is only for the evaluation, and therefore intentionally set a bit aggressive. It might not be appropriate for production environments. The test code is also available[2], so you could reproduce it on your system and workloads. Patches Sequence ================ The first four patches implement the core logic and user interfaces for the auto tuning. The first patch implements the core logic for the auto tuning, and the API for DAMOS users in the kernel space. The second patch implements basic file operations of DAMON sysfs directories and files that will be used for setting the goals and providing the feedback. The third patch connects the quota goals files inputs to the DAMOS core logic. Finally the fourth patch implements a dedicated DAMOS sysfs command for efficiently committing the quota goals feedback. Two patches for simple tests of the logic and interfaces follow. The fifth patch implements the core logic unit test. The sixth patch implements a selftest for the DAMON Sysfs interface for the goals. Finally, three patches for documentation follows. The seventh patch documents the design of the feature. The eighth patch updates the API doc for the new sysfs files. The final eighth patch updates the usage document for the features. References ========== [1] DAOS paper: https://www.amazon.science/publications/daos-data-access-aware-operating-sy… [2] Evaluation code: https://github.com/damonitor/damon-tests/commit/3f884e61193f0166b8724554b6d… [3] Memory auto scaling RFC idea: https://lore.kernel.org/damon/20231112195114.61474-1-sj@kernel.org/ [4] DAMON-based tiered memory management RFC idea: https://lore.kernel.org/damon/20231112195602.61525-1-sj@kernel.org/ SeongJae Park (9): mm/damon/core: implement goal-oriented feedback-driven quota auto-tuning mm/damon/sysfs-schemes: implement files for scheme quota goals setup mm/damon/sysfs-schemes: commit damos quota goals user input to DAMOS mm/damon/sysfs-schemes: implement a command for scheme quota goals only commit mm/damon/core-test: add a unit test for the feedback loop algorithm selftests/damon: test quota goals directory Docs/mm/damon/design: document DAMOS quota auto tuning Docs/ABI/damon: document DAMOS quota goals Docs/admin-guide/mm/damon/usage: document for quota goals .../ABI/testing/sysfs-kernel-mm-damon | 33 ++- Documentation/admin-guide/mm/damon/usage.rst | 48 +++- Documentation/mm/damon/design.rst | 13 + include/linux/damon.h | 19 ++ mm/damon/core-test.h | 32 +++ mm/damon/core.c | 68 ++++- mm/damon/sysfs-common.h | 3 + mm/damon/sysfs-schemes.c | 272 +++++++++++++++++- mm/damon/sysfs.c | 27 ++ tools/testing/selftests/damon/sysfs.sh | 27 ++ 10 files changed, 517 insertions(+), 25 deletions(-) base-commit: b4e0245a831a402cae8634a4dc277a04830ff07a -- 2.34.1

1 year, 3 months

2
3
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror November 2023