- Linux-kselftest-mirror - lists.linaro.org

[net] selftests: srv6: make srv6_end_dt46_l3vpn_test more robust

by Andrea Mayer

On some distributions, the rp_filter is automatically set (=1) by default on a netdev basis (also on VRFs). In an SRv6 End.DT46 behavior, decapsulated IPv4 packets are routed using the table associated with the VRF bound to that tunnel. During lookup operations, the rp_filter can lead to packet loss when activated on the VRF. Therefore, we chose to make this selftest more robust by explicitly disabling the rp_filter during tests (as it is automatically set by some Linux distributions). Fixes: 03a0b567a03d ("selftests: seg6: add selftest for SRv6 End.DT46 Behavior") Reported-by: Hangbin Liu <liuhangbin(a)gmail.com> Signed-off-by: Andrea Mayer <andrea.mayer(a)uniroma2.it> Tested-by: Hangbin Liu <liuhangbin(a)gmail.com> --- .../testing/selftests/net/srv6_end_dt46_l3vpn_test.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/net/srv6_end_dt46_l3vpn_test.sh b/tools/testing/selftests/net/srv6_end_dt46_l3vpn_test.sh index aebaab8ce44c..441eededa031 100755 --- a/tools/testing/selftests/net/srv6_end_dt46_l3vpn_test.sh +++ b/tools/testing/selftests/net/srv6_end_dt46_l3vpn_test.sh @@ -292,6 +292,11 @@ setup_hs() ip netns exec ${hsname} sysctl -wq net.ipv6.conf.all.accept_dad=0 ip netns exec ${hsname} sysctl -wq net.ipv6.conf.default.accept_dad=0 + # disable the rp_filter otherwise the kernel gets confused about how + # to route decap ipv4 packets. + ip netns exec ${rtname} sysctl -wq net.ipv4.conf.all.rp_filter=0 + ip netns exec ${rtname} sysctl -wq net.ipv4.conf.default.rp_filter=0 + ip -netns ${hsname} link add veth0 type veth peer name ${rtveth} ip -netns ${hsname} link set ${rtveth} netns ${rtname} ip -netns ${hsname} addr add ${IPv6_HS_NETWORK}::${hs}/64 dev veth0 nodad @@ -316,11 +321,6 @@ setup_hs() ip netns exec ${rtname} sysctl -wq net.ipv6.conf.${rtveth}.proxy_ndp=1 ip netns exec ${rtname} sysctl -wq net.ipv4.conf.${rtveth}.proxy_arp=1 - # disable the rp_filter otherwise the kernel gets confused about how - # to route decap ipv4 packets. - ip netns exec ${rtname} sysctl -wq net.ipv4.conf.all.rp_filter=0 - ip netns exec ${rtname} sysctl -wq net.ipv4.conf.${rtveth}.rp_filter=0 - ip netns exec ${rtname} sh -c "echo 1 > /proc/sys/net/vrf/strict_mode" } -- 2.20.1

2 years

3
2
0 0

[PATCH] selftests/powerpc: Replace obsolete memalign() with posix_memalign()

by Deming Wang

memalign() is obsolete according to its manpage. Replace memalign() with posix_memalign() and remove malloc.h include that was there for memalign(). As a pointer is passed into posix_memalign(), initialize *s to NULL to silence a warning about the function's return value being used as uninitialized (which is not valid anyway because the error is properly checked before s is returned). Signed-off-by: Deming Wang <wangdeming(a)inspur.com> --- tools/testing/selftests/powerpc/stringloops/strlen.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/powerpc/stringloops/strlen.c b/tools/testing/selftests/powerpc/stringloops/strlen.c index 9055ebc484d0..f9c1f9cc2d32 100644 --- a/tools/testing/selftests/powerpc/stringloops/strlen.c +++ b/tools/testing/selftests/powerpc/stringloops/strlen.c @@ -1,5 +1,4 @@ // SPDX-License-Identifier: GPL-2.0 -#include <malloc.h> #include <stdlib.h> #include <string.h> #include <time.h> @@ -51,10 +50,11 @@ static void bench_test(char *s) static int testcase(void) { char *s; + int ret; unsigned long i; - s = memalign(128, SIZE); - if (!s) { + ret = posix_memalign((void **)&s, 128, SIZE); + if (ret < 0) { perror("memalign"); exit(1); } -- 2.27.0

2 years

2
1
0 0

[PATCH tracing 0/3] tracing: support > 8 byte filter predicates

by Alan Maguire

For cases like IPv6 addresses, having a means to supply tracing predicates for fields with more than 8 bytes would be convenient. This series provides a simple way to support this by allowing simple ==, != memory comparison with the predicate supplied when the size of the field exceeds 8 bytes. For example, to trace ::1, the predicate "dst == 0x00000000000000000000000000000001" ..could be used. Patch 1 provides the support for > 8 byte fields via a memcmp()-style predicate. Patch 2 adds tests for filter predicates, and patch 3 documents the fact that for > 8 bytes. only == and != are supported. Changes since RFC [1]: - originally a fix was intermixed with the new functionality as patch 1 in series [1]; the fix landed separately - small tweaks to how filter predicates are defined via fn_num as opposed to via fn directly [1] https://lore.kernel.org/lkml/1659910883-18223-1-git-send-email-alan.maguire… Alan Maguire (3): tracing: support > 8 byte array filter predicates selftests/ftrace: add test coverage for filter predicates tracing: document > 8 byte numeric filtering support Documentation/trace/events.rst | 9 +++ kernel/trace/trace_events_filter.c | 55 +++++++++++++++- .../selftests/ftrace/test.d/event/filter.tc | 62 +++++++++++++++++++ 3 files changed, 125 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/ftrace/test.d/event/filter.tc -- 2.31.1

2 years

4
11
0 0

[PATCH bpf] selftests/bpf: fix pkg-config call building sign-file

by Jeremy Sowden

When building sign-file, the call to get the CFLAGS for libcrypto is missing white-space between `pkg-config` and `--cflags`: $(shell $(HOSTPKG_CONFIG)--cflags libcrypto 2> /dev/null) Removing the redirection of stderr, we see: $ make -C tools/testing/selftests/bpf sign-file make: Entering directory '[...]/tools/testing/selftests/bpf' make: pkg-config--cflags: No such file or directory SIGN-FILE sign-file make: Leaving directory '[...]/tools/testing/selftests/bpf' Add the missing space. Fixes: fc97590668ae ("selftests/bpf: Add test for bpf_verify_pkcs7_signature() kfunc") Signed-off-by: Jeremy Sowden <jeremy(a)azazel.net> --- tools/testing/selftests/bpf/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index b677dcd0b77a..ad01c9e1ff12 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -197,7 +197,7 @@ $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_r $(OUTPUT)/sign-file: ../../../../scripts/sign-file.c $(call msg,SIGN-FILE,,$@) - $(Q)$(CC) $(shell $(HOSTPKG_CONFIG)--cflags libcrypto 2> /dev/null) \ + $(Q)$(CC) $(shell $(HOSTPKG_CONFIG) --cflags libcrypto 2> /dev/null) \ $< -o $@ \ $(shell $(HOSTPKG_CONFIG) --libs libcrypto 2> /dev/null || echo -lcrypto) -- 2.39.2

2 years

3
3
0 0

[PATCH bpf,v3 0/4] Socket lookup BPF API from tc/xdp ingress does not respect VRF bindings.

by Gilad Sever

When calling socket lookup from L2 (tc, xdp), VRF boundaries aren't respected. This patchset fixes this by regarding the incoming device's VRF attachment when performing the socket lookups from tc/xdp. The first two patches are coding changes which factor out the tc helper's logic which was shared with cg/sk_skb (which operate correctly). This refactoring is needed in order to avoid affecting the cgroup/sk_skb flows as there does not seem to be a strict criteria for discerning which flow the helper is called from based on the net device or packet information. The third patch contains the actual bugfix. The fourth patch adds bpf tests for these lookup functions. --- v3: - Rename bpf_l2_sdif() to dev_sdif() as suggested by Stanislav Fomichev - Added xdp tests as suggested by Daniel Borkmann - Use start_server() to avoid duplicate code as suggested by Stanislav Fomichev v2: Fixed uninitialized var in test patch (4). Gilad Sever (4): bpf: factor out socket lookup functions for the TC hookpoint. bpf: Call __bpf_sk_lookup()/__bpf_skc_lookup() directly via TC hookpoint bpf: fix bpf socket lookup from tc/xdp to respect socket VRF bindings selftests/bpf: Add vrf_socket_lookup tests net/core/filter.c | 132 +++++-- .../bpf/prog_tests/vrf_socket_lookup.c | 327 ++++++++++++++++++ .../selftests/bpf/progs/vrf_socket_lookup.c | 88 +++++ 3 files changed, 526 insertions(+), 21 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/vrf_socket_lookup.c create mode 100644 tools/testing/selftests/bpf/progs/vrf_socket_lookup.c -- 2.34.1

2 years

2
8
0 0

[PATCH v2] selftests/kselftest/runner/run_one(): Allow running non-executable files

by SeongJae Park

From: SeongJae Park <sjpark(a)amazon.de> When running a test program, 'run_one()' checks if the program has the execution permission and fails if it doesn't. However, it's easy to mistakenly missing the permission, as some common tools like 'diff' don't support the permission change well[1]. Compared to that, making mistakes in the test program's path would only rare, as those are explicitly listed in 'TEST_PROGS'. Therefore, it might make more sense to resolve the situation on our own and run the program. For the reason, this commit makes the test program runner function to still print the warning message but try parsing the interpreter of the program and explicitly run it with the interpreter, in the case. [1] https://lore.kernel.org/mm-commits/YRJisBs9AunccCD4@kroah.com/ Suggested-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Signed-off-by: SeongJae Park <sjpark(a)amazon.de> --- Changes from v1 (https://lore.kernel.org/linux-kselftest/20210810140459.23990-1-sj38.park@gm…) - Parse and use the interpreter instead of changing the file tools/testing/selftests/kselftest/runner.sh | 28 +++++++++++++-------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh index cc9c846585f0..a9ba782d8ca0 100644 --- a/tools/testing/selftests/kselftest/runner.sh +++ b/tools/testing/selftests/kselftest/runner.sh @@ -33,9 +33,9 @@ tap_timeout() { # Make sure tests will time out if utility is available. if [ -x /usr/bin/timeout ] ; then - /usr/bin/timeout --foreground "$kselftest_timeout" "$1" + /usr/bin/timeout --foreground "$kselftest_timeout" $1 else - "$1" + $1 fi } @@ -65,17 +65,25 @@ run_one() TEST_HDR_MSG="selftests: $DIR: $BASENAME_TEST" echo "# $TEST_HDR_MSG" - if [ ! -x "$TEST" ]; then - echo -n "# Warning: file $TEST is " - if [ ! -e "$TEST" ]; then - echo "missing!" - else - echo "not executable, correct this." - fi + if [ ! -e "$TEST" ]; then + echo "# Warning: file $TEST is missing!" echo "not ok $test_num $TEST_HDR_MSG" else + cmd="./$BASENAME_TEST" + if [ ! -x "$TEST" ]; then + echo "# Warning: file $TEST is not executable" + + if [ $(head -n 1 "$TEST" | cut -c -2) = "#!" ] + then + interpreter=$(head -n 1 "$TEST" | cut -c 3-) + cmd="$interpreter ./$BASENAME_TEST" + else + echo "not ok $test_num $TEST_HDR_MSG" + return + fi + fi cd `dirname $TEST` > /dev/null - ((((( tap_timeout ./$BASENAME_TEST 2>&1; echo $? >&3) | + ((((( tap_timeout "$cmd" 2>&1; echo $? >&3) | tap_prefix >&4) 3>&1) | (read xs; exit $xs)) 4>>"$logfile" && echo "ok $test_num $TEST_HDR_MSG") || -- 2.17.1

2 years

3
6
0 0

[PATCH bpf-next v3 0/2] Introduce a new kfunc of bpf_task_under_cgroup

by Feng zhou

From: Feng Zhou <zhoufeng.zf(a)bytedance.com> Trace sched related functions, such as enqueue_task_fair, it is necessary to specify a task instead of the current task which within a given cgroup. Feng Zhou (2): bpf: Add bpf_task_under_cgroup() kfunc selftests/bpf: Add testcase for bpf_task_under_cgroup Changelog: v2->v3: Addressed comments from Alexei Starovoitov - Modify the comment information of the function. - Narrow down the testcase's hook point Details in here: https://lore.kernel.org/all/20230421090403.15515-1-zhoufeng.zf@bytedance.co… v1->v2: Addressed comments from Alexei Starovoitov - Add kfunc instead. Details in here: https://lore.kernel.org/all/20230420072657.80324-1-zhoufeng.zf@bytedance.co… kernel/bpf/helpers.c | 20 ++++++++ tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../bpf/prog_tests/task_under_cgroup.c | 47 +++++++++++++++++++ .../selftests/bpf/progs/cgrp_kfunc_common.h | 1 + .../bpf/progs/test_task_under_cgroup.c | 37 +++++++++++++++ 5 files changed, 106 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/task_under_cgroup.c create mode 100644 tools/testing/selftests/bpf/progs/test_task_under_cgroup.c -- 2.20.1

2 years

2
3
0 0

[RFC][PATCH 0/6] KEYS: Introduce user mode key and signature parsers

by Roberto Sassu

From: Roberto Sassu <roberto.sassu(a)huawei.com> Goals ===== Support new key and signature formats with the same kernel component. Verify the authenticity of system data with newly supported data formats. Mitigate the risk of parsing arbitrary data in the kernel. Motivation ========== Adding new functionality to the kernel comes with an increased risk of introducing new bugs which, once exploited, can lead to a partial or full system compromise. Parsing arbitrary data is particularly critical, since it allows an attacker to send a malicious sequence of bytes to exploit vulnerabilities in the parser code. The attacker might be able to overwrite kernel memory to bypass kernel protections, and obtain more privileges. User Mode Drivers (UMDs) can effectively mitigate this risk. If the parser runs in user space, even if it has a bug, it won't allow the attacker to overwrite kernel memory. The communication protocol between the UMD and the kernel should be simple enough, that the kernel can immediately recognize malformed data sent by an attacker controlling the UMD, and discard it. Solution ======== Register a new parser of the asymmetric key type which, instead of parsing the key blob, forwards it to a UMD, and populates the key fields from the UMD response. That response contains the data for each field of the public key structure, defined in the kernel, and possibly a key description. Supporting new data formats can be achieved by simply extending the UMD. As long as the UMD recognizes them, and provides the crypto material to the kernel Crypto API in the expected format, the kernel does not need to be aware of the UMD changes. Add a new API to verify the authenticity of system data, similar to the one for PKCS#7 signatures. As for the key parser, send the signature to a UMD, and fill the public_key_signature structure from the UMD response. The API still supports a very basic trust model, it accepts a key for signature verification if it is in the supplied keyring. The API can be extended later to support more sophisticated models. Use cases ========= eBPF ---- The eBPF infrastructure already offers to eBPF programs the ability to verify PKCS#7 signatures, through the bpf_verify_pkcs7_signature() kfunc. Add the new bpf_verify_umd_signature() kfunc, to allow eBPF programs verify signatures in a data format that is not PKCS#7 (for example PGP). IMA Appraisal ------------- An alternative to appraising each file with its signature (Fedora 38) is to build a repository of reference file digests from signed RPM headers, and lookup the calculated digest of files being accessed in that repository (DIGLIM[1]). With this patch set, the kernel can verify the authenticity of RPM headers from their PGP signature against the Linux distribution GPG keys. Once verified, RPM headers can be parsed with a UMD to build the repository of reference file digests. With DIGLIM, Linux distributions are not required to change anything in their building infrastructure (no extra data in the RPM header, no new PKI for IMA signatures). [1]: https://lore.kernel.org/linux-integrity/20210914163401.864635-1-roberto.sas… UMD development =============== The header file crypto/asymmetric_keys/umd_key_sig_umh.h contains the details of the communication protocol between the kernel and the UMD handler. The UMD handler should implement the commands defined, CMD_KEY and CMD_SIG, should set the result of the processing, and fill the key and signature-specific structures umd_key_msg_out and umd_sig_msg_out. The UMD handler should provide the key and signature blobs in a format that is understood by the kernel. For example, for RSA keys, it should provide them in ASN.1 format (SEQUENCE of INTEGER). The auth IDs of the keys and signatures should match, for signature verification. Auth ID matching can be partial. Patch set dependencies ====================== This patch set depends on 'usermode_driver: Add management library and API': https://lore.kernel.org/bpf/20230317145240.363908-1-roberto.sassu@huaweiclo… Patch set content ================= Patch 1 introduces the new parser for the asymmetric key type. Patch 2 introduces the parser for signatures and its API. Patch 3 introduces the system-level API for signature verification. Patch 4 extends eBPF to use the new system-level API. Patch 5 adds a test for UMD-parser signatures (not executed until the UMD supports PGP). Patch 6 introduces the skeleton of the UMD handler. PGP === A work in progress implementation of the PGP format (RFC 4880 and RFC 6637) in the UMD handler is available at: https://github.com/robertosassu/linux/commits/pgp-signatures-umd-v1-devel-v… It is based on a previous work of David Howells, available at: https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-modsign.git/… The patches have been adapted for use in user space. Roberto Sassu (6): KEYS: asymmetric: Introduce UMD-based asymmetric key parser KEYS: asymmetric: Introduce UMD-based asymmetric key signature parser verification: Introduce verify_umd_signature() and verify_umd_message_sig() bpf: Introduce bpf_verify_umd_signature() kfunc selftests/bpf: Prepare a test for UMD-parsed signatures KEYS: asymmetric: Add UMD handler .gitignore | 3 + MAINTAINERS | 1 + certs/system_keyring.c | 125 ++++++ crypto/asymmetric_keys/Kconfig | 32 ++ crypto/asymmetric_keys/Makefile | 23 + crypto/asymmetric_keys/asymmetric_type.c | 3 +- crypto/asymmetric_keys/umd_key.h | 28 ++ crypto/asymmetric_keys/umd_key_parser.c | 203 +++++++++ crypto/asymmetric_keys/umd_key_sig_loader.c | 32 ++ crypto/asymmetric_keys/umd_key_sig_umh.h | 71 +++ crypto/asymmetric_keys/umd_key_sig_umh_blob.S | 7 + crypto/asymmetric_keys/umd_key_sig_umh_user.c | 84 ++++ crypto/asymmetric_keys/umd_sig_parser.c | 416 ++++++++++++++++++ include/crypto/umd_sig.h | 71 +++ include/keys/asymmetric-type.h | 1 + include/linux/verification.h | 48 ++ kernel/trace/bpf_trace.c | 69 ++- ...ify_pkcs7_sig.c => verify_pkcs7_umd_sig.c} | 109 +++-- ...kcs7_sig.c => test_verify_pkcs7_umd_sig.c} | 18 +- .../testing/selftests/bpf/verify_sig_setup.sh | 82 +++- 20 files changed, 1378 insertions(+), 48 deletions(-) create mode 100644 crypto/asymmetric_keys/umd_key.h create mode 100644 crypto/asymmetric_keys/umd_key_parser.c create mode 100644 crypto/asymmetric_keys/umd_key_sig_loader.c create mode 100644 crypto/asymmetric_keys/umd_key_sig_umh.h create mode 100644 crypto/asymmetric_keys/umd_key_sig_umh_blob.S create mode 100644 crypto/asymmetric_keys/umd_key_sig_umh_user.c create mode 100644 crypto/asymmetric_keys/umd_sig_parser.c create mode 100644 include/crypto/umd_sig.h rename tools/testing/selftests/bpf/prog_tests/{verify_pkcs7_sig.c => verify_pkcs7_umd_sig.c} (75%) rename tools/testing/selftests/bpf/progs/{test_verify_pkcs7_sig.c => test_verify_pkcs7_umd_sig.c} (82%) -- 2.25.1

2 years

3
12
0 0

[PATCH] regmap: REGMAP_KUNIT must not select REGMAP

by Geert Uytterhoeven

Enabling a (modular) test must not silently enable additional kernel functionality, as that may increase the attack vector of a product. Fix this by making REGMAP_KUNIT depend on REGMAP instead. Fixes: 2238959b6ad27040 ("regmap: Add some basic kunit tests") Signed-off-by: Geert Uytterhoeven <geert(a)linux-m68k.org> --- drivers/base/regmap/Kconfig | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/base/regmap/Kconfig b/drivers/base/regmap/Kconfig index 33a8366e22a584a5..20327c15cbce5d8c 100644 --- a/drivers/base/regmap/Kconfig +++ b/drivers/base/regmap/Kconfig @@ -11,9 +11,8 @@ config REGMAP config REGMAP_KUNIT tristate "KUnit tests for regmap" - depends on KUNIT + depends on KUNIT && REGMAP default KUNIT_ALL_TESTS - select REGMAP select REGMAP_RAM config REGMAP_AC97 -- 2.34.1

2 years

2
2
0 0

[PATCH v5 1/3] test_firmware: prevent race conditions by a correct implementation of locking

by Mirsad Goran Todorovac

Dan Carpenter spotted a race condition in a couple of situations like these in the test_firmware driver: static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) { u8 val; int ret; ret = kstrtou8(buf, 10, &val); if (ret) return ret; mutex_lock(&test_fw_mutex); *(u8 *)cfg = val; mutex_unlock(&test_fw_mutex); /* Always return full write size even if we didn't consume all */ return size; } static ssize_t config_num_requests_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { int rc; mutex_lock(&test_fw_mutex); if (test_fw_config->reqs) { pr_err("Must call release_all_firmware prior to changing config\n"); rc = -EINVAL; mutex_unlock(&test_fw_mutex); goto out; } mutex_unlock(&test_fw_mutex); rc = test_dev_config_update_u8(buf, count, &test_fw_config->num_requests); out: return rc; } static ssize_t config_read_fw_idx_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t count) { return test_dev_config_update_u8(buf, count, &test_fw_config->read_fw_idx); } The function test_dev_config_update_u8() is called from both the locked and the unlocked context, function config_num_requests_store() and config_read_fw_idx_store() which can both be called asynchronously as they are driver's methods, while test_dev_config_update_u8() and siblings change their argument pointed to by u8 *cfg or similar pointer. To avoid deadlock on test_fw_mutex, the lock is dropped before calling test_dev_config_update_u8() and re-acquired within test_dev_config_update_u8() itself, but alas this creates a race condition. Having two locks wouldn't assure a race-proof mutual exclusion. This situation is best avoided by the introduction of a new, unlocked function __test_dev_config_update_u8() which can be called from the locked context and reducing test_dev_config_update_u8() to: static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) { int ret; mutex_lock(&test_fw_mutex); ret = __test_dev_config_update_u8(buf, size, cfg); mutex_unlock(&test_fw_mutex); return ret; } doing the locking and calling the unlocked primitive, which enables both locked and unlocked versions without duplication of code. The similar approach was applied to all functions called from the locked and the unlocked context, which safely mitigates both deadlocks and race conditions in the driver. __test_dev_config_update_bool(), __test_dev_config_update_u8() and __test_dev_config_update_size_t() unlocked versions of the functions were introduced to be called from the locked contexts as a workaround without releasing the main driver's lock and thereof causing a race condition. The test_dev_config_update_bool(), test_dev_config_update_u8() and test_dev_config_update_size_t() locked versions of the functions are being called from driver methods without the unnecessary multiplying of the locking and unlocking code for each method, and complicating the code with saving of the return value across lock. Fixes: 7feebfa487b92 ("test_firmware: add support for request_firmware_into_buf") Cc: Luis Chamberlain <mcgrof(a)kernel.org> Cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> Cc: Russ Weight <russell.h.weight(a)intel.com> Cc: Takashi Iwai <tiwai(a)suse.de> Cc: Tianfei Zhang <tianfei.zhang(a)intel.com> Cc: Shuah Khan <shuah(a)kernel.org> Cc: Colin Ian King <colin.i.king(a)gmail.com> Cc: Randy Dunlap <rdunlap(a)infradead.org> Cc: linux-kselftest(a)vger.kernel.org Cc: stable(a)vger.kernel.org # v5.4 Suggested-by: Dan Carpenter <error27(a)gmail.com> Signed-off-by: Mirsad Goran Todorovac <mirsad.todorovac(a)alu.unizg.hr> --- lib/test_firmware.c | 52 ++++++++++++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 17 deletions(-) diff --git a/lib/test_firmware.c b/lib/test_firmware.c index 05ed84c2fc4c..35417e0af3f4 100644 --- a/lib/test_firmware.c +++ b/lib/test_firmware.c @@ -353,16 +353,26 @@ static ssize_t config_test_show_str(char *dst, return len; } -static int test_dev_config_update_bool(const char *buf, size_t size, +static inline int __test_dev_config_update_bool(const char *buf, size_t size, bool *cfg) { int ret; - mutex_lock(&test_fw_mutex); if (kstrtobool(buf, cfg) < 0) ret = -EINVAL; else ret = size; + + return ret; +} + +static int test_dev_config_update_bool(const char *buf, size_t size, + bool *cfg) +{ + int ret; + + mutex_lock(&test_fw_mutex); + ret = __test_dev_config_update_bool(buf, size, cfg); mutex_unlock(&test_fw_mutex); return ret; @@ -373,7 +383,8 @@ static ssize_t test_dev_config_show_bool(char *buf, bool val) return snprintf(buf, PAGE_SIZE, "%d\n", val); } -static int test_dev_config_update_size_t(const char *buf, +static int __test_dev_config_update_size_t( + const char *buf, size_t size, size_t *cfg) { @@ -384,9 +395,7 @@ static int test_dev_config_update_size_t(const char *buf, if (ret) return ret; - mutex_lock(&test_fw_mutex); *(size_t *)cfg = new; - mutex_unlock(&test_fw_mutex); /* Always return full write size even if we didn't consume all */ return size; @@ -402,7 +411,7 @@ static ssize_t test_dev_config_show_int(char *buf, int val) return snprintf(buf, PAGE_SIZE, "%d\n", val); } -static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) +static int __test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) { u8 val; int ret; @@ -411,14 +420,23 @@ static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) if (ret) return ret; - mutex_lock(&test_fw_mutex); *(u8 *)cfg = val; - mutex_unlock(&test_fw_mutex); /* Always return full write size even if we didn't consume all */ return size; } +static int test_dev_config_update_u8(const char *buf, size_t size, u8 *cfg) +{ + int ret; + + mutex_lock(&test_fw_mutex); + ret = __test_dev_config_update_u8(buf, size, cfg); + mutex_unlock(&test_fw_mutex); + + return ret; +} + static ssize_t test_dev_config_show_u8(char *buf, u8 val) { return snprintf(buf, PAGE_SIZE, "%u\n", val); @@ -471,10 +489,10 @@ static ssize_t config_num_requests_store(struct device *dev, mutex_unlock(&test_fw_mutex); goto out; } - mutex_unlock(&test_fw_mutex); - rc = test_dev_config_update_u8(buf, count, - &test_fw_config->num_requests); + rc = __test_dev_config_update_u8(buf, count, + &test_fw_config->num_requests); + mutex_unlock(&test_fw_mutex); out: return rc; @@ -518,10 +536,10 @@ static ssize_t config_buf_size_store(struct device *dev, mutex_unlock(&test_fw_mutex); goto out; } - mutex_unlock(&test_fw_mutex); - rc = test_dev_config_update_size_t(buf, count, - &test_fw_config->buf_size); + rc = __test_dev_config_update_size_t(buf, count, + &test_fw_config->buf_size); + mutex_unlock(&test_fw_mutex); out: return rc; @@ -548,10 +566,10 @@ static ssize_t config_file_offset_store(struct device *dev, mutex_unlock(&test_fw_mutex); goto out; } - mutex_unlock(&test_fw_mutex); - rc = test_dev_config_update_size_t(buf, count, - &test_fw_config->file_offset); + rc = __test_dev_config_update_size_t(buf, count, + &test_fw_config->file_offset); + mutex_unlock(&test_fw_mutex); out: return rc; -- 2.30.2

2 years

1
2
0 0

[PATCH v2 0/2] selftests/ftrace: Add tests for kprobes and optimized probes

by Akanksha J N

This patchset adds a stress test for kprobes and a test for checking optimized probes. The two tests are being added based on the below discussion: https://lore.kernel.org/all/20230128101622.ce6f8e64d929e29d36b08b73@kernel.… Changelog: * Add an explicit fork after enabling the events ( echo "forked" ) * Remove the extended test from multiple_kprobe_types.tc which adds multiple consecutive probes in a function and add it as a separate test case. * Add new test case which checks for optimized probes. Akanksha J N (2): selftests/ftrace: Add new test case which adds multiple consecutive probes in a function selftests/ftrace: Add new test case which checks for optimized probes .../test.d/kprobe/kprobe_insn_boundary.tc | 19 +++++++++++ .../ftrace/test.d/kprobe/kprobe_opt_types.tc | 34 +++++++++++++++++++ 2 files changed, 53 insertions(+) create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_insn_boundary.tc create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kprobe_opt_types.tc -- 2.31.1

2 years

3
8
0 0

[PATCH v1 0/3] kunit: Deferred action helpers

by David Gow

This is v1 of the KUnit deferred actions API, which implements an equivalent of devm_add_action[1] on top of KUnit managed resources. This provides a simple way of scheduling a function to run when the test terminates (whether successfully, or with an error). It's therefore very useful for freeing resources, or otherwise cleaning up. The notable changes since RFCv2[2] are: - Got rid of the 'cancellation token' concept. It was overcomplicated, and we can add it back if we need to. - kunit_add_action() therefore now returns 0 on success, and an error otherwise (like devm_add_action()). Though you may wish to use: - Added kunit_add_action_or_reset(), which will call the deferred function if an error occurs. (See devm_add_action_or_reset()). This also returns an error on failure, which can be asserted safely. - Got rid of the function pointer typedef. Personally, I liked it, but it's more typedef-y than most kernel code. - Got rid of the 'internal_gfp' argument: all internal state is now allocated with GFP_KERNEL. The main KUnit resource API can be used instead if this doesn't work for your use-case. I'd love to hear any further thoughts! Cheers, -- David [1]: https://docs.kernel.org/driver-api/basics.html#c.devm_add_action [2]: https://patchwork.kernel.org/project/linux-kselftest/list/?series=735720 David Gow (3): kunit: Add kunit_add_action() to defer a call until test exit kunit: executor_test: Use kunit_add_action() kunit: kmalloc_array: Use kunit_add_action() include/kunit/resource.h | 76 +++++++++++++++++++++++++++++++ include/kunit/test.h | 10 ++++- lib/kunit/executor_test.c | 11 ++--- lib/kunit/kunit-test.c | 88 +++++++++++++++++++++++++++++++++++- lib/kunit/resource.c | 95 +++++++++++++++++++++++++++++++++++++++ lib/kunit/test.c | 48 ++++---------------- 6 files changed, 279 insertions(+), 49 deletions(-) -- 2.40.0.634.g4ca3ef3211-goog

2 years

4
10
0 0

[PATCH v2] selftests: netfilter: fix a build error on openSUSE

by Geliang Tang

This patch fixed the following build error on openSUSE Leap 15.3: ======================================================================= gcc nf-queue.c -lmnl -o tools/testing/selftests/netfilter/nf-queue nf-queue.c:13:10: fatal error: libmnl/libmnl.h: No such file or directory #include <libmnl/libmnl.h> ^~~~~~~~~~~~~~~~~ compilation terminated. ======================================================================= It is because libmnl.h is put in the directory of "/usr/include/libmnl/libmnl/" on openSUSE, not "/usr/include/libmnl/": > rpm -ql libmnl-devel /usr/include/libmnl /usr/include/libmnl/libmnl /usr/include/libmnl/libmnl/libmnl.h /usr/lib64/libmnl.so /usr/lib64/pkgconfig/libmnl.pc Suggested-by: Kai Liu <kai.liu(a)suse.com> Signed-off-by: Geliang Tang <geliang.tang(a)suse.com> --- v2: - use pkg-config --- tools/testing/selftests/netfilter/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/netfilter/Makefile b/tools/testing/selftests/netfilter/Makefile index e4f845dd942b..8136c1fab7ab 100644 --- a/tools/testing/selftests/netfilter/Makefile +++ b/tools/testing/selftests/netfilter/Makefile @@ -8,6 +8,7 @@ TEST_PROGS := nft_trans_stress.sh nft_fib.sh nft_nat.sh bridge_brouter.sh \ ipip-conntrack-mtu.sh conntrack_tcp_unreplied.sh \ conntrack_vrf.sh nft_synproxy.sh +CFLAGS += $(shell pkg-config --cflags libmnl 2>/dev/null || echo "-I/usr/include/libmnl") LDLIBS = -lmnl TEST_GEN_FILES = nf-queue -- 2.34.1

2 years

5
9
0 0

[PATCH v3 1/4] kunit: Always run cleanup from a test kthread

by David Gow

KUnit tests run in a kthread, with the current->kunit_test pointer set to the test's context. This allows the kunit_get_current_test() and kunit_fail_current_test() macros to work. Normally, this pointer is still valid during test shutdown (i.e., the suite->exit function, and any resource cleanup). However, if the test has exited early (e.g., due to a failed assertion), the cleanup is done in the parent KUnit thread, which does not have an active context. Instead, in the event test terminates early, run the test exit and cleanup from a new 'cleanup' kthread, which sets current->kunit_test, and better isolates the rest of KUnit from issues which arise in test cleanup. If a test cleanup function itself aborts (e.g., due to an assertion failing), there will be no further attempts to clean up: an error will be logged and the test failed. For example: # example_simple_test: test aborted during cleanup. continuing without cleaning up This should also make it easier to get access to the KUnit context, particularly from within resource cleanup functions, which may, for example, need access to data in test->priv. Signed-off-by: David Gow <davidgow(a)google.com> --- This is an updated version of / replacement of "kunit: Set the current KUnit context when cleaning up", which instead creates a new kthread for cleanup tasks if the original test kthread is aborted. This protects us from failed assertions during cleanup, if the test exited early. Changes since v2: https://lore.kernel.org/linux-kselftest/20230419085426.1671703-1-davidgow@g… - Always run cleanup in its own kthread - Therefore, never attempt to re-run it if it exits - Thanks, Benjamin. Changes since v1: https://lore.kernel.org/linux-kselftest/20230415091401.681395-1-davidgow@go… - Move cleanup execution to another kthread - (Thanks, Benjamin, for pointing out the assertion issues) --- lib/kunit/test.c | 55 ++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 48 insertions(+), 7 deletions(-) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index e2910b261112..2025e51941e6 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -419,10 +419,50 @@ static void kunit_try_run_case(void *data) * thread will resume control and handle any necessary clean up. */ kunit_run_case_internal(test, suite, test_case); - /* This line may never be reached. */ +} + +static void kunit_try_run_case_cleanup(void *data) +{ + struct kunit_try_catch_context *ctx = data; + struct kunit *test = ctx->test; + struct kunit_suite *suite = ctx->suite; + + current->kunit_test = test; + kunit_run_case_cleanup(test, suite); } +static void kunit_catch_run_case_cleanup(void *data) +{ + struct kunit_try_catch_context *ctx = data; + struct kunit *test = ctx->test; + int try_exit_code = kunit_try_catch_get_result(&test->try_catch); + + /* It is always a failure if cleanup aborts. */ + kunit_set_failure(test); + + if (try_exit_code) { + /* + * Test case could not finish, we have no idea what state it is + * in, so don't do clean up. + */ + if (try_exit_code == -ETIMEDOUT) { + kunit_err(test, "test case cleanup timed out\n"); + /* + * Unknown internal error occurred preventing test case from + * running, so there is nothing to clean up. + */ + } else { + kunit_err(test, "internal error occurred during test case cleanup: %d\n", + try_exit_code); + } + return; + } + + kunit_err(test, "test aborted during cleanup. continuing without cleaning up\n"); +} + + static void kunit_catch_run_case(void *data) { struct kunit_try_catch_context *ctx = data; @@ -448,12 +488,6 @@ static void kunit_catch_run_case(void *data) } return; } - - /* - * Test case was run, but aborted. It is the test case's business as to - * whether it failed or not, we just need to clean up. - */ - kunit_run_case_cleanup(test, suite); } /* @@ -478,6 +512,13 @@ static void kunit_run_case_catch_errors(struct kunit_suite *suite, context.test_case = test_case; kunit_try_catch_run(try_catch, &context); + /* Now run the cleanup */ + kunit_try_catch_init(try_catch, + test, + kunit_try_run_case_cleanup, + kunit_catch_run_case_cleanup); + kunit_try_catch_run(try_catch, &context); + /* Propagate the parameter result to the test case. */ if (test->status == KUNIT_FAILURE) test_case->status = KUNIT_FAILURE; -- 2.40.0.634.g4ca3ef3211-goog

2 years

6
9
0 0

[PATCH v2] selftests/ftrace: Improve integration with kselftest runner

by Mark Brown

The ftrace selftests do not currently produce KTAP output, they produce a custom format much nicer for human consumption. This means that when run in automated test systems we just get a single result for the suite as a whole rather than recording results for individual test cases, making it harder to look at the test data and masking things like inappropriate skips. Address this by adding support for KTAP output to the ftracetest script and providing a trivial wrapper which will be invoked by the kselftest runner to generate output in this format by default, users using ftracetest directly will continue to get the existing output. This is not the most elegant solution but it is simple and effective. I did consider implementing this by post processing the existing output format but that felt more complex and likely to result in all output being lost if something goes seriously wrong during the run which would not be helpful. I did also consider just writing a separate runner script but there's enough going on with things like the signal handling for that to seem like it would be duplicating too much. Acked-by: Steven Rostedt (Google) <rostedt(a)goodmis.org> Acked-by: Masami Hiramatsu (Google) <mhiramat(a)kernel.org> Tested-by: Masami Hiramatsu (Google) <mhiramat(a)kernel.org> Signed-off-by: Mark Brown <broonie(a)kernel.org> --- Changes in v2: - Update help text. - Link to v1: https://lore.kernel.org/r/20230302-ftrace-kselftest-ktap-v1-1-a84a0765b7ad@… --- tools/testing/selftests/ftrace/Makefile | 3 +- tools/testing/selftests/ftrace/ftracetest | 63 ++++++++++++++++++++++++-- tools/testing/selftests/ftrace/ftracetest-ktap | 8 ++++ 3 files changed, 70 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/ftrace/Makefile b/tools/testing/selftests/ftrace/Makefile index d6e106fbce11..a1e955d2de4c 100644 --- a/tools/testing/selftests/ftrace/Makefile +++ b/tools/testing/selftests/ftrace/Makefile @@ -1,7 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 all: -TEST_PROGS := ftracetest +TEST_PROGS_EXTENDED := ftracetest +TEST_PROGS := ftracetest-ktap TEST_FILES := test.d settings EXTRA_CLEAN := $(OUTPUT)/logs/* diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index c3311c8c4089..2506621e75df 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -13,6 +13,7 @@ echo "Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]" echo " Options:" echo " -h|--help Show help message" echo " -k|--keep Keep passed test logs" +echo " -K|--ktap Output in KTAP format" echo " -v|--verbose Increase verbosity of test messages" echo " -vv Alias of -v -v (Show all results in stdout)" echo " -vvv Alias of -v -v -v (Show all commands immediately)" @@ -85,6 +86,10 @@ parse_opts() { # opts KEEP_LOG=1 shift 1 ;; + --ktap|-K) + KTAP=1 + shift 1 + ;; --verbose|-v|-vv|-vvv) if [ $VERBOSE -eq -1 ]; then usage "--console can not use with --verbose" @@ -178,6 +183,7 @@ TEST_DIR=$TOP_DIR/test.d TEST_CASES=`find_testcases $TEST_DIR` LOG_DIR=$TOP_DIR/logs/`date +%Y%m%d-%H%M%S`/ KEEP_LOG=0 +KTAP=0 DEBUG=0 VERBOSE=0 UNSUPPORTED_RESULT=0 @@ -229,7 +235,7 @@ prlog() { # messages newline= shift fi - printf "$*$newline" + [ "$KTAP" != "1" ] && printf "$*$newline" [ "$LOG_FILE" ] && printf "$*$newline" | strip_esc >> $LOG_FILE } catlog() { #file @@ -260,11 +266,11 @@ TOTAL_RESULT=0 INSTANCE= CASENO=0 +CASENAME= testcase() { # testfile CASENO=$((CASENO+1)) - desc=`grep "^#[ \t]*description:" $1 | cut -f2- -d:` - prlog -n "[$CASENO]$INSTANCE$desc" + CASENAME=`grep "^#[ \t]*description:" $1 | cut -f2- -d:` } checkreq() { # testfile @@ -277,40 +283,68 @@ test_on_instance() { # testfile grep -q "^#[ \t]*flags:.*instance" $1 } +ktaptest() { # result comment + if [ "$KTAP" != "1" ]; then + return + fi + + local result= + if [ "$1" = "1" ]; then + result="ok" + else + result="not ok" + fi + shift + + local comment=$* + if [ "$comment" != "" ]; then + comment="# $comment" + fi + + echo $CASENO $result $INSTANCE$CASENAME $comment +} + eval_result() { # sigval case $1 in $PASS) prlog " [${color_green}PASS${color_reset}]" + ktaptest 1 PASSED_CASES="$PASSED_CASES $CASENO" return 0 ;; $FAIL) prlog " [${color_red}FAIL${color_reset}]" + ktaptest 0 FAILED_CASES="$FAILED_CASES $CASENO" return 1 # this is a bug. ;; $UNRESOLVED) prlog " [${color_blue}UNRESOLVED${color_reset}]" + ktaptest 0 UNRESOLVED UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO" return $UNRESOLVED_RESULT # depends on use case ;; $UNTESTED) prlog " [${color_blue}UNTESTED${color_reset}]" + ktaptest 1 SKIP UNTESTED_CASES="$UNTESTED_CASES $CASENO" return 0 ;; $UNSUPPORTED) prlog " [${color_blue}UNSUPPORTED${color_reset}]" + ktaptest 1 SKIP UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO" return $UNSUPPORTED_RESULT # depends on use case ;; $XFAIL) prlog " [${color_green}XFAIL${color_reset}]" + ktaptest 1 XFAIL XFAILED_CASES="$XFAILED_CASES $CASENO" return 0 ;; *) prlog " [${color_blue}UNDEFINED${color_reset}]" + ktaptest 0 error UNDEFINED_CASES="$UNDEFINED_CASES $CASENO" return 1 # this must be a test bug ;; @@ -371,6 +405,7 @@ __run_test() { # testfile run_test() { # testfile local testname=`basename $1` testcase $1 + prlog -n "[$CASENO]$INSTANCE$CASENAME" if [ ! -z "$LOG_FILE" ] ; then local testlog=`mktemp $LOG_DIR/${CASENO}-${testname}-log.XXXXXX` else @@ -405,6 +440,17 @@ run_test() { # testfile # load in the helper functions . $TEST_DIR/functions +if [ "$KTAP" = "1" ]; then + echo "TAP version 13" + + casecount=`echo $TEST_CASES | wc -w` + for t in $TEST_CASES; do + test_on_instance $t || continue + casecount=$((casecount+1)) + done + echo "1..${casecount}" +fi + # Main loop for t in $TEST_CASES; do run_test $t @@ -439,6 +485,17 @@ prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w` prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w` prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w` +if [ "$KTAP" = "1" ]; then + echo -n "# Totals:" + echo -n " pass:"`echo $PASSED_CASES | wc -w` + echo -n " faii:"`echo $FAILED_CASES | wc -w` + echo -n " xfail:"`echo $XFAILED_CASES | wc -w` + echo -n " xpass:0" + echo -n " skip:"`echo $UNTESTED_CASES $UNSUPPORTED_CASES | wc -w` + echo -n " error:"`echo $UNRESOLVED_CASES $UNDEFINED_CASES | wc -w` + echo +fi + cleanup # if no error, return 0 diff --git a/tools/testing/selftests/ftrace/ftracetest-ktap b/tools/testing/selftests/ftrace/ftracetest-ktap new file mode 100755 index 000000000000..b3284679ef3a --- /dev/null +++ b/tools/testing/selftests/ftrace/ftracetest-ktap @@ -0,0 +1,8 @@ +#!/bin/sh -e +# SPDX-License-Identifier: GPL-2.0-only +# +# ftracetest-ktap: Wrapper to integrate ftracetest with the kselftest runner +# +# Copyright (C) Arm Ltd., 2023 + +./ftracetest -K --- base-commit: 457391b0380335d5e9a5babdec90ac53928b23b4 change-id: 20230302-ftrace-kselftest-ktap-9d7878691557 Best regards, -- Mark Brown <broonie(a)kernel.org>

2 years

1
0
0 0

[PATCH RESEND] selftests/ftrace: Improve integration with kselftest runner

by Mark Brown

The ftrace selftests do not currently produce KTAP output, they produce a custom format much nicer for human consumption. This means that when run in automated test systems we just get a single result for the suite as a whole rather than recording results for individual test cases, making it harder to look at the test data and masking things like inappropriate skips. Address this by adding support for KTAP output to the ftracetest script and providing a trivial wrapper which will be invoked by the kselftest runner to generate output in this format by default, users using ftracetest directly will continue to get the existing output. This is not the most elegant solution but it is simple and effective. I did consider implementing this by post processing the existing output format but that felt more complex and likely to result in all output being lost if something goes seriously wrong during the run which would not be helpful. I did also consider just writing a separate runner script but there's enough going on with things like the signal handling for that to seem like it would be duplicating too much. Signed-off-by: Mark Brown <broonie(a)kernel.org> --- tools/testing/selftests/ftrace/Makefile | 3 +- tools/testing/selftests/ftrace/ftracetest | 63 ++++++++++++++++++++++++-- tools/testing/selftests/ftrace/ftracetest-ktap | 8 ++++ 3 files changed, 70 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/ftrace/Makefile b/tools/testing/selftests/ftrace/Makefile index d6e106fbce11..a1e955d2de4c 100644 --- a/tools/testing/selftests/ftrace/Makefile +++ b/tools/testing/selftests/ftrace/Makefile @@ -1,7 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 all: -TEST_PROGS := ftracetest +TEST_PROGS_EXTENDED := ftracetest +TEST_PROGS := ftracetest-ktap TEST_FILES := test.d settings EXTRA_CLEAN := $(OUTPUT)/logs/* diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index c3311c8c4089..539c8d6d5d71 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -13,6 +13,7 @@ echo "Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]" echo " Options:" echo " -h|--help Show help message" echo " -k|--keep Keep passed test logs" +echo " -K|--KTAP Output in KTAP format" echo " -v|--verbose Increase verbosity of test messages" echo " -vv Alias of -v -v (Show all results in stdout)" echo " -vvv Alias of -v -v -v (Show all commands immediately)" @@ -85,6 +86,10 @@ parse_opts() { # opts KEEP_LOG=1 shift 1 ;; + --ktap|-K) + KTAP=1 + shift 1 + ;; --verbose|-v|-vv|-vvv) if [ $VERBOSE -eq -1 ]; then usage "--console can not use with --verbose" @@ -178,6 +183,7 @@ TEST_DIR=$TOP_DIR/test.d TEST_CASES=`find_testcases $TEST_DIR` LOG_DIR=$TOP_DIR/logs/`date +%Y%m%d-%H%M%S`/ KEEP_LOG=0 +KTAP=0 DEBUG=0 VERBOSE=0 UNSUPPORTED_RESULT=0 @@ -229,7 +235,7 @@ prlog() { # messages newline= shift fi - printf "$*$newline" + [ "$KTAP" != "1" ] && printf "$*$newline" [ "$LOG_FILE" ] && printf "$*$newline" | strip_esc >> $LOG_FILE } catlog() { #file @@ -260,11 +266,11 @@ TOTAL_RESULT=0 INSTANCE= CASENO=0 +CASENAME= testcase() { # testfile CASENO=$((CASENO+1)) - desc=`grep "^#[ \t]*description:" $1 | cut -f2- -d:` - prlog -n "[$CASENO]$INSTANCE$desc" + CASENAME=`grep "^#[ \t]*description:" $1 | cut -f2- -d:` } checkreq() { # testfile @@ -277,40 +283,68 @@ test_on_instance() { # testfile grep -q "^#[ \t]*flags:.*instance" $1 } +ktaptest() { # result comment + if [ "$KTAP" != "1" ]; then + return + fi + + local result= + if [ "$1" = "1" ]; then + result="ok" + else + result="not ok" + fi + shift + + local comment=$* + if [ "$comment" != "" ]; then + comment="# $comment" + fi + + echo $CASENO $result $INSTANCE$CASENAME $comment +} + eval_result() { # sigval case $1 in $PASS) prlog " [${color_green}PASS${color_reset}]" + ktaptest 1 PASSED_CASES="$PASSED_CASES $CASENO" return 0 ;; $FAIL) prlog " [${color_red}FAIL${color_reset}]" + ktaptest 0 FAILED_CASES="$FAILED_CASES $CASENO" return 1 # this is a bug. ;; $UNRESOLVED) prlog " [${color_blue}UNRESOLVED${color_reset}]" + ktaptest 0 UNRESOLVED UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO" return $UNRESOLVED_RESULT # depends on use case ;; $UNTESTED) prlog " [${color_blue}UNTESTED${color_reset}]" + ktaptest 1 SKIP UNTESTED_CASES="$UNTESTED_CASES $CASENO" return 0 ;; $UNSUPPORTED) prlog " [${color_blue}UNSUPPORTED${color_reset}]" + ktaptest 1 SKIP UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO" return $UNSUPPORTED_RESULT # depends on use case ;; $XFAIL) prlog " [${color_green}XFAIL${color_reset}]" + ktaptest 1 XFAIL XFAILED_CASES="$XFAILED_CASES $CASENO" return 0 ;; *) prlog " [${color_blue}UNDEFINED${color_reset}]" + ktaptest 0 error UNDEFINED_CASES="$UNDEFINED_CASES $CASENO" return 1 # this must be a test bug ;; @@ -371,6 +405,7 @@ __run_test() { # testfile run_test() { # testfile local testname=`basename $1` testcase $1 + prlog -n "[$CASENO]$INSTANCE$CASENAME" if [ ! -z "$LOG_FILE" ] ; then local testlog=`mktemp $LOG_DIR/${CASENO}-${testname}-log.XXXXXX` else @@ -405,6 +440,17 @@ run_test() { # testfile # load in the helper functions . $TEST_DIR/functions +if [ "$KTAP" = "1" ]; then + echo "TAP version 13" + + casecount=`echo $TEST_CASES | wc -w` + for t in $TEST_CASES; do + test_on_instance $t || continue + casecount=$((casecount+1)) + done + echo "1..${casecount}" +fi + # Main loop for t in $TEST_CASES; do run_test $t @@ -439,6 +485,17 @@ prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w` prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w` prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w` +if [ "$KTAP" = "1" ]; then + echo -n "# Totals:" + echo -n " pass:"`echo $PASSED_CASES | wc -w` + echo -n " faii:"`echo $FAILED_CASES | wc -w` + echo -n " xfail:"`echo $XFAILED_CASES | wc -w` + echo -n " xpass:0" + echo -n " skip:"`echo $UNTESTED_CASES $UNSUPPORTED_CASES | wc -w` + echo -n " error:"`echo $UNRESOLVED_CASES $UNDEFINED_CASES | wc -w` + echo +fi + cleanup # if no error, return 0 diff --git a/tools/testing/selftests/ftrace/ftracetest-ktap b/tools/testing/selftests/ftrace/ftracetest-ktap new file mode 100755 index 000000000000..b3284679ef3a --- /dev/null +++ b/tools/testing/selftests/ftrace/ftracetest-ktap @@ -0,0 +1,8 @@ +#!/bin/sh -e +# SPDX-License-Identifier: GPL-2.0-only +# +# ftracetest-ktap: Wrapper to integrate ftracetest with the kselftest runner +# +# Copyright (C) Arm Ltd., 2023 + +./ftracetest -K --- base-commit: fe15c26ee26efa11741a7b632e9f23b01aca4cc6 change-id: 20230302-ftrace-kselftest-ktap-9d7878691557 Best regards, -- Mark Brown <broonie(a)kernel.org>

2 years

3
2
0 0

biznesowy angielski

by Krzysztof Maj

Dzień dobry, czy rozważali Państwo rozwój kwalifikacji językowych swoich pracowników? Opracowaliśmy kursy językowe dla różnych branż, w których koncentrujemy się na podniesieniu poziomu słownictwa i jakości komunikacji wykorzystując autorską metodę, stworzoną specjalnie dla wymagającego biznesu. Niestandardowy kurs on-line, dopasowany do profilu firmy i obszarów świadczonych usług, w szybkim czasie przyniesie efekty, które zwiększą komfort i jakość pracy, rozwijając możliwości biznesowe. Zdalne szkolenie językowe to m.in. zajęcia z native speakerami, które w szybkim czasie nauczą pracowników rozmawiać za pomocą jasnego i zwięzłego języka Business English. Czy mógłbym przedstawić więcej szczegółów i opowiedzieć jak działamy? Pozdrawiam Krzysztof Maj

2 years

1
0
0 0

[PATCH bpf,v2 0/4] Socket lookup BPF API from tc/xdp ingress does not respect VRF bindings.

by Gilad Sever

When calling socket lookup from L2 (tc, xdp), VRF boundaries aren't respected. This patchset fixes this by regarding the incoming device's VRF attachment when performing the socket lookups from tc/xdp. The first two patches are coding changes which facilitate this fix by factoring out the tc helper's logic which was shared with cg/sk_skb (which operate correctly). The third patch contains the actual bugfix. The fourth patch adds bpf tests for these lookup functions. --- v2: Fixed uninitialized var in test patch (4). Gilad Sever (4): bpf: factor out socket lookup functions for the TC hookpoint. bpf: Call __bpf_sk_lookup()/__bpf_skc_lookup() directly via TC hookpoint bpf: fix bpf socket lookup from tc/xdp to respect socket VRF bindings selftests/bpf: Add tc_socket_lookup tests net/core/filter.c | 132 +++++-- .../bpf/prog_tests/tc_socket_lookup.c | 341 ++++++++++++++++++ .../selftests/bpf/progs/tc_socket_lookup.c | 73 ++++ 3 files changed, 525 insertions(+), 21 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/tc_socket_lookup.c create mode 100644 tools/testing/selftests/bpf/progs/tc_socket_lookup.c -- 2.34.1

2 years

4
12
0 0

[PATCH mm-stable v2 0/3] mm/ksm: improve PR_SET_MEMORY_MERGE=0 handling and cleanup disabling KSM

by David Hildenbrand

This is a follow-up to [1]: [PATCH v9 0/3] mm: process/cgroup ksm support which is now in mm-stable. Ideally we'd get at least patch #1 into the same kernel release as [1], so the semantics of setting PR_SET_MEMORY_MERGE=0 are unchanged between kernel versions. (1) Make PR_SET_MEMORY_MERGE=0 unmerge pages like setting MADV_UNMERGEABLE does, (2) add a selftest for it and (3) factor out disabling of KSM from s390/gmap code. v1 -> v2: - "mm/ksm: unmerge and clear VM_MERGEABLE when setting PR_SET_MEMORY_MERGE=0" -> Cleanup one if/else -> Add doc for ksm_disable_merge_any() - Added ACKs [1] https://lkml.kernel.org/r/20230418051342.1919757-1-shr@devkernel.io Cc: Andrew Morton <akpm(a)linux-foundation.org> Cc: Stefan Roesch <shr(a)devkernel.io> Cc: Rik van Riel <riel(a)surriel.com> Cc: Johannes Weiner <hannes(a)cmpxchg.org> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Christian Borntraeger <borntraeger(a)linux.ibm.com> Cc: Janosch Frank <frankja(a)linux.ibm.com> Cc: Claudio Imbrenda <imbrenda(a)linux.ibm.com> Cc: Heiko Carstens <hca(a)linux.ibm.com> Cc: Vasily Gorbik <gor(a)linux.ibm.com> Cc: Sven Schnelle <svens(a)linux.ibm.com> Cc: Shuah Khan <shuah(a)kernel.org> David Hildenbrand (3): mm/ksm: unmerge and clear VM_MERGEABLE when setting PR_SET_MEMORY_MERGE=0 selftests/ksm: ksm_functional_tests: add prctl unmerge test mm/ksm: move disabling KSM from s390/gmap code to KSM code arch/s390/mm/gmap.c | 20 +----- include/linux/ksm.h | 7 ++ kernel/sys.c | 12 +--- mm/ksm.c | 70 +++++++++++++++++++ .../selftests/mm/ksm_functional_tests.c | 46 ++++++++++-- 5 files changed, 121 insertions(+), 34 deletions(-) -- 2.40.0

2 years

2
4
0 0

[GIT PULL] KUnit next update for Linux 6.4-rc1

by Shuah Khan

Hi Linus, Please pull the following KUnit next update for Linux 6.4-rc1. linux-kselftest-kunit-6.4-rc1 This KUnit update Linux 6.4-rc1 consists of: - several fixes to kunit tool - new klist structure test - support for m68k under QEMU - support for overriding the QEMU serial port - support for SH under QEMU diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit fe15c26ee26efa11741a7b632e9f23b01aca4cc6: Linux 6.3-rc1 (2023-03-05 14:52:03 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-kunit-6.4-rc1 for you to fetch changes up to a42077b787680cbc365a96446b30f32399fa3f6f: kunit: add tests for using current KUnit test field (2023-04-05 12:51:30 -0600) ---------------------------------------------------------------- linux-kselftest-kunit-6.4-rc1 This KUnit update Linux 6.4-rc1 consists of: - several fixes to kunit tool - new klist structure test - support for m68k under QEMU - support for overriding the QEMU serial port - support for SH under QEMU ---------------------------------------------------------------- Andy Shevchenko (1): .gitignore: Unignore .kunitconfig Daniel Latypov (3): kunit: tool: add subscripts for type annotations where appropriate kunit: tool: remove unused imports and variables kunit: tool: fix pre-existing `mypy --strict` errors and update run_checks.py Geert Uytterhoeven (3): kunit: tool: Add support for m68k under QEMU kunit: tool: Add support for overriding the QEMU serial port kunit: tool: Add support for SH under QEMU Heiko Carstens (1): kunit: increase KUNIT_LOG_SIZE to 2048 bytes Rae Moar (4): kunit: fix bug in debugfs logs of parameterized tests kunit: fix bug in the order of lines in debugfs logs kunit: fix bug of extra newline characters in debugfs logs kunit: add tests for using current KUnit test field Sadiya Kazi (1): list: test: Test the klist structure Stephen Boyd (1): kunit: Use gfp in kunit_alloc_resource() kernel-doc .gitignore | 1 + include/kunit/resource.h | 2 +- include/kunit/test.h | 4 +- lib/kunit/debugfs.c | 14 +- lib/kunit/kunit-test.c | 77 ++++++-- lib/kunit/test.c | 57 ++++-- lib/list-test.c | 300 ++++++++++++++++++++++++++++++- tools/testing/kunit/kunit.py | 26 +-- tools/testing/kunit/kunit_config.py | 4 +- tools/testing/kunit/kunit_kernel.py | 39 ++-- tools/testing/kunit/kunit_parser.py | 1 - tools/testing/kunit/kunit_printer.py | 2 +- tools/testing/kunit/kunit_tool_test.py | 2 +- tools/testing/kunit/qemu_config.py | 1 + tools/testing/kunit/qemu_configs/m68k.py | 10 ++ tools/testing/kunit/qemu_configs/sh.py | 17 ++ tools/testing/kunit/run_checks.py | 6 +- 17 files changed, 491 insertions(+), 72 deletions(-) create mode 100644 tools/testing/kunit/qemu_configs/m68k.py create mode 100644 tools/testing/kunit/qemu_configs/sh.py ----------------------------------------------------------------

2 years

2
1
0 0

[GIT PULL] Kselftest update for Linux 6.4-rc1

by Shuah Khan

Hi Linus, Please pull the following Kselftest update for Linux 6.4-rc1. This Kselftest update for Linux 6.4-rc1 consists of: - several patches to enhance and fix resctrl test - nolibc support for kselftest with an addition to vprintf() to tools/nolibc/stdio and related test changes - Refactor 'peeksiginfo' ptrace test part - add 'malloc' failures checks in cgroup test_memcontrol - a new prctl test - enhancements sched test with additional ore schedule prctl calls diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit fe15c26ee26efa11741a7b632e9f23b01aca4cc6: Linux 6.3-rc1 (2023-03-05 14:52:03 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-next-6.4-rc1 for you to fetch changes up to 50ad2fb7ec2b18186b8a4fa1c0e00f78b3de5119: selftests/resctrl: Fix incorrect error return on test complete (2023-04-14 11:13:18 -0600) ---------------------------------------------------------------- linux-kselftest-next-6.4-rc1 linux-kselftest-next-6.4-rc1 This Kselftest update for Linux 6.4-rc1 consists of: - several patches to enhance and fix resctrl test - nolibc support for kselftest with an addition to vprintf() to tools/nolibc/stdio and related test changes - Refactor 'peeksiginfo' ptrace test part - add 'malloc' failures checks in cgroup test_memcontrol - a new prctl test - enhancements sched test with additional ore schedule prctl calls ---------------------------------------------------------------- Fenghua Yu (1): selftests/resctrl: Change name from CBM_MASK_PATH to INFO_PATH Ilpo Järvinen (8): selftests/resctrl: Return NULL if malloc_and_init_memory() did not alloc mem selftests/resctrl: Move ->setup() call outside of test specific branches selftests/resctrl: Allow ->setup() to return errors selftests/resctrl: Check for return value after write_schemata() selftests/resctrl: Replace obsolete memalign() with posix_memalign() selftests/resctrl: Change initialize_llc_perf() return type to void selftests/resctrl: Use remount_resctrlfs() consistently with boolean selftests/resctrl: Correct get_llc_perf() param in function comment Ivan Orlov (4): selftests: Refactor 'peeksiginfo' ptrace test part selftests: cgroup: Add 'malloc' failures checks in test_memcontrol selftests: sched: Add more core schedule prctl calls selftests: prctl: Add new prctl test for PR_SET_VMA action Mark Brown (3): tools/nolibc/stdio: Implement vprintf() kselftest: Support nolibc kselftest/arm64: Convert za-fork to use kselftest.h Peter Newman (1): selftests/resctrl: Use correct exit code when tests fail Reinette Chatre (1): selftests/resctrl: Fix incorrect error return on test complete Shaopeng Tan (6): selftests/resctrl: Fix set up schemata with 100% allocation on first run in MBM test selftests/resctrl: Return MBA check result and make it to output message selftests/resctrl: Flush stdout file buffer before executing fork() selftests/resctrl: Cleanup properly when an error occurs in CAT test selftests/resctrl: Commonize the signal handler register/unregister for all tests selftests/resctrl: Remove duplicate codes that clear each test result file Sukrut Bellary (1): kselftest: amd-pstate: Fix spelling mistakes tools/include/nolibc/stdio.h | 6 ++ tools/testing/selftests/Makefile | 1 + tools/testing/selftests/amd-pstate/gitsource.sh | 4 +- tools/testing/selftests/amd-pstate/run.sh | 4 +- tools/testing/selftests/arm64/fp/Makefile | 2 +- tools/testing/selftests/arm64/fp/za-fork.c | 88 ++++------------- tools/testing/selftests/cgroup/test_memcontrol.c | 15 +++ tools/testing/selftests/kselftest.h | 2 + tools/testing/selftests/prctl/.gitignore | 1 + tools/testing/selftests/prctl/Makefile | 2 +- tools/testing/selftests/prctl/config | 1 + .../selftests/prctl/set-anon-vma-name-test.c | 104 +++++++++++++++++++++ tools/testing/selftests/ptrace/peeksiginfo.c | 14 +-- tools/testing/selftests/resctrl/cache.c | 17 ++-- tools/testing/selftests/resctrl/cat_test.c | 33 ++++--- tools/testing/selftests/resctrl/cmt_test.c | 16 ++-- tools/testing/selftests/resctrl/fill_buf.c | 21 +---- tools/testing/selftests/resctrl/mba_test.c | 34 ++++--- tools/testing/selftests/resctrl/mbm_test.c | 22 ++--- tools/testing/selftests/resctrl/resctrl.h | 8 +- tools/testing/selftests/resctrl/resctrl_tests.c | 14 +-- tools/testing/selftests/resctrl/resctrl_val.c | 88 +++++++++++------ tools/testing/selftests/resctrl/resctrlfs.c | 7 +- tools/testing/selftests/sched/cs_prctl_test.c | 6 ++ 24 files changed, 306 insertions(+), 204 deletions(-) create mode 100644 tools/testing/selftests/prctl/config create mode 100644 tools/testing/selftests/prctl/set-anon-vma-name-test.c ----------------------------------------------------------------

2 years

2
1
0 0

[PATCH linux-next v3] selftests: net: udpgso_bench_rx: Fix verifty exceptions

by yang.yang29＠zte.com.cn

From: Zhang Yunkai (CGEL ZTE) <zhang.yunkai(a)zte.com.cn> The verification function of this test case is likely to encounter the following error, which may confuse users. The problem is easily reproducible in the latest kernel. Environment A, the sender: bash# udpgso_bench_tx -l 4 -4 -D "$IP_B" udpgso_bench_tx: write: Connection refused Environment B, the receiver: bash# udpgso_bench_rx -4 -G -S 1472 -v udpgso_bench_rx: data[1472]: len 17664, a(97) != q(113) If the packet is captured, you will see: Environment A, the sender: bash# tcpdump -i eth0 host "$IP_B" & IP $IP_A.41025 > $IP_B.8000: UDP, length 1472 IP $IP_A.41025 > $IP_B.8000: UDP, length 1472 IP $IP_B > $IP_A: ICMP $IP_B udp port 8000 unreachable, length 556 Environment B, the receiver: bash# tcpdump -i eth0 host "$IP_B" & IP $IP_A.41025 > $IP_B.8000: UDP, length 7360 IP $IP_A.41025 > $IP_B.8000: UDP, length 14720 IP $IP_B > $IP_A: ICMP $IP_B udp port 8000 unreachable, length 556 In one test, the verification data is printed as follows: abcd...xyz | 1... .. | abcd...xyz | abcd...opabcd...xyz | ...1472... Not xyzabcd, messages are merged .. | The issue is that the test on receive for expected payload pattern {AB..Z}+ fail for GRO packets if segment payload does not end on a Z. The issue still exists when using the GRO with -G, but not using the -S to obtain gsosize. Therefore, a print has been added to remind users. Changes in v3: - Simplify description and adjust judgment order. Changes in v2: - Fix confusing descriptions. Signed-off-by: Zhang Yunkai (CGEL ZTE) <zhang.yunkai(a)zte.com.cn> Reviewed-by: Xu Xin (CGEL ZTE) <xu.xin16(a)zte.com.cn> Reviewed-by: Yang Yang (CGEL ZTE) <yang.yang29(a)zte.com.cn> Cc: Xuexin Jiang (CGEL ZTE) <jiang.xuexin(a)zte.com.cn> --- tools/testing/selftests/net/udpgso_bench_rx.c | 34 +++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/net/udpgso_bench_rx.c b/tools/testing/selftests/net/udpgso_bench_rx.c index f35a924d4a30..3ad18cbc570d 100644 --- a/tools/testing/selftests/net/udpgso_bench_rx.c +++ b/tools/testing/selftests/net/udpgso_bench_rx.c @@ -189,26 +189,45 @@ static char sanitized_char(char val) return (val >= 'a' && val <= 'z') ? val : '.'; } -static void do_verify_udp(const char *data, int len) +static void do_verify_udp(const char *data, int start, int len) { - char cur = data[0]; + char cur = data[start]; int i; /* verify contents */ if (cur < 'a' || cur > 'z') error(1, 0, "data initial byte out of range"); - for (i = 1; i < len; i++) { + for (i = start + 1; i < start + len; i++) { if (cur == 'z') cur = 'a'; else cur++; - if (data[i] != cur) + if (data[i] != cur) { + if (cfg_gro_segment && !cfg_expected_gso_size) + error(0, 0, "Use -S to obtain gsosize to guide " + "splitting and verification."); + error(1, 0, "data[%d]: len %d, %c(%hhu) != %c(%hhu)\n", i, len, sanitized_char(data[i]), data[i], sanitized_char(cur), cur); + } + } +} + +static void do_verify_udp_gro(const char *data, int len, int segment_size) +{ + int start = 0; + + while (len - start > 0) { + if (len - start > segment_size) + do_verify_udp(data, start, segment_size); + else + do_verify_udp(data, start, len - start); + + start += segment_size; } } @@ -268,7 +287,12 @@ static void do_flush_udp(int fd) if (ret == 0) error(1, errno, "recv: 0 byte datagram\n"); - do_verify_udp(rbuf, ret); + if (!cfg_gro_segment) + do_verify_udp(rbuf, 0, ret); + else if (gso_size > 0) + do_verify_udp_gro(rbuf, ret, gso_size); + else + do_verify_udp_gro(rbuf, ret, ret); } if (cfg_expected_gso_size && cfg_expected_gso_size != gso_size) error(1, 0, "recv: bad gso size, got %d, expected %d " -- 2.15.2

2 years

2
1
0 0

Re: [External] Re: [PATCH bpf-next v2 2/2] selftests/bpf: Add testcase for bpf_task_under_cgroup

by Alexei Starovoitov

On Sat, Apr 22, 2023 at 8:51 PM Feng Zhou <zhoufeng.zf(a)bytedance.com> wrote: > > 在 2023/4/22 02:24, Alexei Starovoitov 写道: > > On Fri, Apr 21, 2023 at 05:04:03PM +0800, Feng zhou wrote: > > From: Feng Zhou <zhoufeng.zf(a)bytedance.com> > > test_progs: > Tests new kfunc bpf_task_under_cgroup(). > > The bpf program saves the pid which call the getuid syscall within a > given cgroup to the remote_pid, which is convenient for the user-mode > program to verify the test correctness. > > The user-mode program creates its own mount namespace, and mounts the > cgroupsv2 hierarchy in there, call the getuid syscall, then check if > remote_pid and local_pid are equal. > > Signed-off-by: Feng Zhou <zhoufeng.zf(a)bytedance.com> > --- > .../bpf/prog_tests/task_under_cgroup.c | 46 +++++++++++++++++++ > .../selftests/bpf/progs/cgrp_kfunc_common.h | 1 + > .../bpf/progs/test_task_under_cgroup.c | 40 ++++++++++++++++ > 3 files changed, 87 insertions(+) > create mode 100644 tools/testing/selftests/bpf/prog_tests/task_under_cgroup.c > create mode 100644 tools/testing/selftests/bpf/progs/test_task_under_cgroup.c > > diff --git a/tools/testing/selftests/bpf/prog_tests/task_under_cgroup.c b/tools/testing/selftests/bpf/prog_tests/task_under_cgroup.c > new file mode 100644 > index 000000000000..bd3deb469938 > --- /dev/null > +++ b/tools/testing/selftests/bpf/prog_tests/task_under_cgroup.c > @@ -0,0 +1,46 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* Copyright (c) 2023 Bytedance */ > + > +#include <test_progs.h> > +#include <cgroup_helpers.h> > +#include "test_task_under_cgroup.skel.h" > + > +#define FOO "/foo" > + > +void test_task_under_cgroup(void) > +{ > + struct test_task_under_cgroup *skel; > + int ret, foo = -1; > + > + foo = test__join_cgroup(FOO); > + if (!ASSERT_OK(foo < 0, "cgroup_join_foo")) > + return; > + > + skel = test_task_under_cgroup__open(); > + if (!ASSERT_OK_PTR(skel, "test_task_under_cgroup__open")) > + goto cleanup; > + > + skel->rodata->local_pid = getpid(); > + skel->rodata->cgid = get_cgroup_id(FOO); > + > + ret = test_task_under_cgroup__load(skel); > + if (!ASSERT_OK(ret, "test_task_under_cgroup__load")) > + goto cleanup; > + > + ret = test_task_under_cgroup__attach(skel); > + if (!ASSERT_OK(ret, "test_task_under_cgroup__attach")) > + goto cleanup; > + > + syscall(__NR_getuid); > + > + test_task_under_cgroup__detach(skel); > + > + ASSERT_EQ(skel->bss->remote_pid, skel->rodata->local_pid, > + "test task_under_cgroup"); > + > +cleanup: > + if (foo) > + close(foo); > > Looks wrong. should be if (foo >= 0) ? > > Yes. > > + > + test_task_under_cgroup__destroy(skel); > +} > diff --git a/tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h b/tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h > index 22914a70db54..41b3ea231698 100644 > --- a/tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h > +++ b/tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h > @@ -26,6 +26,7 @@ struct cgroup *bpf_cgroup_ancestor(struct cgroup *cgrp, int level) __ksym; > struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; > void bpf_rcu_read_lock(void) __ksym; > void bpf_rcu_read_unlock(void) __ksym; > +int bpf_task_under_cgroup(struct cgroup *cgrp, struct task_struct *task) __ksym; > > static inline struct __cgrps_kfunc_map_value *cgrps_kfunc_map_value_lookup(struct cgroup *cgrp) > { > diff --git a/tools/testing/selftests/bpf/progs/test_task_under_cgroup.c b/tools/testing/selftests/bpf/progs/test_task_under_cgroup.c > new file mode 100644 > index 000000000000..e2740f9b029d > --- /dev/null > +++ b/tools/testing/selftests/bpf/progs/test_task_under_cgroup.c > @@ -0,0 +1,40 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* Copyright (c) 2023 Bytedance */ > + > +#include <vmlinux.h> > +#include <asm/unistd.h> > +#include <bpf/bpf_tracing.h> > +#include <bpf/bpf_helpers.h> > + > +#include "cgrp_kfunc_common.h" > + > +const volatile int local_pid; > +const volatile long cgid; > +int remote_pid; > + > +SEC("tp_btf/sys_enter") > > pls narrow down to specific syscall. Like you use in user space part: getuid > > Also add this test to denylist.s390. See BPF CI failure. > > bpf_task_under_cgroup is placed in generic_btf_ids, belongs to BPF_PROG_TYPE_TRACING, > > if narrow down to specific syscall and uses SEC ("tp/syscalls/sys_enter_getuid"), > > bpf prog type is TRACEPOINT, kfunc cannot be used, and reports > > "calls kernel function bpf_cgroup_from_id is not allowed". sure, pls narrow it down. sys_enter is too broad. In parallel test_progs the bpf prog will be way too many times.

2 years

1
0
0 0

[PATCH v1 0/3] mm/ksm: improve PR_SET_MEMORY_MERGE=0 handling and cleanup disabling KSM

by David Hildenbrand

This is a follow-up to [1]: [PATCH v9 0/3] mm: process/cgroup ksm support which is not in mm-unstable yet (but soon? :) ). I'll be on vacation for ~2 weeks, so sending it out now as reply to [1]. (1) Make PR_SET_MEMORY_MERGE=0 unmerge pages like setting MADV_UNMERGEABLE does, (2) add a selftest for it and (3) factor out disabling of KSM from s390/gmap code. Cc: Andrew Morton <akpm(a)linux-foundation.org> Cc: Stefan Roesch <shr(a)devkernel.io> Cc: Rik van Riel <riel(a)surriel.com> Cc: Johannes Weiner <hannes(a)cmpxchg.org> Cc: Michal Hocko <mhocko(a)suse.com> Cc: Christian Borntraeger <borntraeger(a)linux.ibm.com> Cc: Janosch Frank <frankja(a)linux.ibm.com> Cc: Claudio Imbrenda <imbrenda(a)linux.ibm.com> Cc: Heiko Carstens <hca(a)linux.ibm.com> Cc: Vasily Gorbik <gor(a)linux.ibm.com> Cc: Sven Schnelle <svens(a)linux.ibm.com> Cc: Andrew Morton <akpm(a)linux-foundation.org> Cc: Shuah Khan <shuah(a)kernel.org> [1] https://lkml.kernel.org/r/20230418051342.1919757-1-shr@devkernel.io David Hildenbrand (3): mm/ksm: unmerge and clear VM_MERGEABLE when setting PR_SET_MEMORY_MERGE=0 selftests/ksm: ksm_functional_tests: add prctl unmerge test mm/ksm: move disabling KSM from s390/gmap code to KSM code arch/s390/mm/gmap.c | 20 +------ include/linux/ksm.h | 7 +++ kernel/sys.c | 7 +-- mm/ksm.c | 58 +++++++++++++++++++ .../selftests/mm/ksm_functional_tests.c | 46 +++++++++++++-- 5 files changed, 107 insertions(+), 31 deletions(-) -- 2.39.2

2 years

3
13
0 0

[PATCH bpf-next v2 0/2] Introduce a new kfunc of bpf_task_under_cgroup

by Feng zhou

From: Feng Zhou <zhoufeng.zf(a)bytedance.com> Trace sched related functions, such as enqueue_task_fair, it is necessary to specify a task instead of the current task which within a given cgroup. Feng Zhou (2): bpf: Add bpf_task_under_cgroup() kfunc selftests/bpf: Add testcase for bpf_task_under_cgroup Changelog: v1->v2: Addressed comments from Alexei Starovoitov - Add kfunc instead. Details in here: https://lore.kernel.org/all/20230420072657.80324-1-zhoufeng.zf@bytedance.co… kernel/bpf/helpers.c | 19 ++++++++ .../bpf/prog_tests/task_under_cgroup.c | 46 +++++++++++++++++++ .../selftests/bpf/progs/cgrp_kfunc_common.h | 1 + .../bpf/progs/test_task_under_cgroup.c | 40 ++++++++++++++++ 4 files changed, 106 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/task_under_cgroup.c create mode 100644 tools/testing/selftests/bpf/progs/test_task_under_cgroup.c -- 2.20.1

2 years

2
4
0 0

[PATCH v2 net-next 0/9] ethtool mm API consolidation

by Vladimir Oltean

This series consolidates the behavior of the 2 drivers that implement the ethtool MAC Merge layer by making NXP ENETC commit its preemptible traffic classes to hardware only when MM TX is active (same as Ocelot). Then, after resolving an issue with the ENETC driver, it restricts user space from entering 2 states which don't make sense: - pmac-enabled off tx-enabled on verify-enabled * - pmac-enabled * tx-enabled off verify-enabled on Then, it introduces a selftest (ethtool_mm.sh) which puts everything together and tests all valid configurations known to me. This is simultaneously the v2 of "[PATCH net-next 0/2] ethtool mm API improvements": https://lore.kernel.org/netdev/20230415173454.3970647-1-vladimir.oltean@nxp… which had caused some problems to openlldp. Those were solved in the meantime, see: https://github.com/intel/openlldp/commit/11171b474f6f3cbccac5d608b7f26b32ff… and of "[RFC PATCH net-next] selftests: forwarding: add a test for MAC Merge layer": https://lore.kernel.org/netdev/20230210221243.228932-1-vladimir.oltean@nxp.… Petr Machata (2): selftests: forwarding: sch_tbf_*: Add a pre-run hook selftests: forwarding: generalize bail_on_lldpad from mlxsw Vladimir Oltean (7): net: enetc: fix MAC Merge layer remaining enabled until a link down event net: enetc: report mm tx-active based on tx-enabled and verify-status net: enetc: only commit preemptible TCs to hardware when MM TX is active net: enetc: include MAC Merge / FP registers in register dump net: ethtool: mm: sanitize some UAPI configurations selftests: forwarding: introduce helper for standard ethtool counters selftests: forwarding: add a test for MAC Merge layer drivers/net/ethernet/freescale/enetc/enetc.c | 23 +- drivers/net/ethernet/freescale/enetc/enetc.h | 5 +- .../ethernet/freescale/enetc/enetc_ethtool.c | 94 +++++- .../net/ethernet/freescale/enetc/enetc_hw.h | 3 + net/ethtool/mm.c | 10 + .../drivers/net/mlxsw/qos_headroom.sh | 3 +- .../selftests/drivers/net/mlxsw/qos_lib.sh | 28 -- .../selftests/drivers/net/mlxsw/qos_pfc.sh | 3 +- .../selftests/drivers/net/mlxsw/sch_ets.sh | 3 +- .../drivers/net/mlxsw/sch_red_core.sh | 1 - .../drivers/net/mlxsw/sch_red_ets.sh | 2 +- .../drivers/net/mlxsw/sch_red_root.sh | 2 +- .../drivers/net/mlxsw/sch_tbf_ets.sh | 6 +- .../drivers/net/mlxsw/sch_tbf_prio.sh | 6 +- .../drivers/net/mlxsw/sch_tbf_root.sh | 6 +- .../testing/selftests/net/forwarding/Makefile | 1 + .../selftests/net/forwarding/ethtool_mm.sh | 288 ++++++++++++++++++ tools/testing/selftests/net/forwarding/lib.sh | 60 ++++ .../net/forwarding/sch_tbf_etsprio.sh | 4 + .../selftests/net/forwarding/sch_tbf_root.sh | 4 + 20 files changed, 486 insertions(+), 66 deletions(-) create mode 100755 tools/testing/selftests/net/forwarding/ethtool_mm.sh -- 2.34.1

2 years

3
21
0 0

[PATCH v6 00/19] Add iommufd physical device operations for replace and alloc hwpt

by Jason Gunthorpe

This is the basic functionality for iommufd to support iommufd_device_replace() and IOMMU_HWPT_ALLOC for physical devices. iommufd_device_replace() allows changing the HWPT associated with the device to a new IOAS or HWPT. Replace does this in way that failure leaves things unchanged, and utilizes the iommu iommu_group_replace_domain() API to allow the iommu driver to perform an optional non-disruptive change. IOMMU_HWPT_ALLOC allows HWPTs to be explicitly allocated by the user and used by attach or replace. At this point it isn't very useful since the HWPT is the same as the automatically managed HWPT from the IOAS. However a following series will allow userspace to customize the created HWPT. The implementation is complicated because we have to introduce some per-iommu_group memory in iommufd and redo how we think about multi-device groups to be more explicit. This solves all the locking problems in the prior attempts. This series is infrastructure work for the following series which: - Add replace for attach - Expose replace through VFIO APIs - Implement driver parameters for HWPT creation (nesting) Once review of this is complete I will keep it on a side branch and accumulate the following series when they are ready so we can have a stable base and make more incremental progress. When we have all the parts together to get a full implementation it can go to Linus. This is on github: https://github.com/jgunthorpe/linux/commits/iommufd_hwpt v6: - Go back to the v4 locking arragnment with now both the attach/detach igroup->locks inside the functions, Kevin says he needs this for a followup series. This still fixes the syzkaller bug - Fix two more error unwind locking bugs where iommufd_object_abort_and_destroy(hwpt) would deadlock or be mislocked. Make sure fail_nth will catch these mistakes - Add a patch allowing objects to have different abort than destroy function, it allows hwpt abort to require the caller to continue to hold the lock and enforces this with lockdep. v5: https://lore.kernel.org/r/0-v5-6716da355392+c5-iommufd_alloc_jgg@nvidia.com - Go back to the v3 version of the code, keep the comment changes from v4. Syzkaller says the group lock change in v4 didn't work. - Adjust the fail_nth test to cover the path syzkaller found. We need to have an ioas with a mapped page installed to inject a failure during domain attachment. v4: https://lore.kernel.org/r/0-v4-9cd79ad52ee8+13f5-iommufd_alloc_jgg@nvidia.c… - Refine comments and commit messages - Move the group lock into iommufd_hw_pagetable_attach() - Fix error unwind in iommufd_device_do_replace() v3: https://lore.kernel.org/r/0-v3-61d41fd9e13e+1f5-iommufd_alloc_jgg@nvidia.com - Refine comments and commit messages - Adjust the flow in iommufd_device_auto_get_domain() so pt_id is only set on success - Reject replace on non-attached devices - Add missing __reserved check for IOMMU_HWPT_ALLOC v2: https://lore.kernel.org/r/0-v2-51b9896e7862+8a8c-iommufd_alloc_jgg@nvidia.c… - Use WARN_ON for the igroup->group test and move that logic to a function iommufd_group_try_get() - Change igroup->devices to igroup->device list Replace will need to iterate over all attached idevs - Rename to iommufd_group_setup_msi() - New patch to export iommu_get_resv_regions() - New patch to use per-device reserved regions instead of per-group regions - Split out the reorganizing of iommufd_device_change_pt() from the replace patch - Replace uses the per-dev reserved regions - Use stdev_id in a few more places in the selftest - Fix error handling in IOMMU_HWPT_ALLOC - Clarify comments - Rebase on v6.3-rc1 v1: https://lore.kernel.org/all/0-v1-7612f88c19f5+2f21-iommufd_alloc_jgg@nvidia… Compared to v3 the diff for the whole series looks like: diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c index 024ed8ee9939cd..2770087059ba73 100644 --- a/drivers/iommu/iommufd/device.c +++ b/drivers/iommu/iommufd/device.c @@ -341,14 +341,15 @@ iommufd_hw_pagetable_detach(struct iommufd_device *idev) { struct iommufd_hw_pagetable *hwpt = idev->igroup->hwpt; - lockdep_assert_held(&idev->igroup->lock); - + mutex_lock(&idev->igroup->lock); list_del(&idev->group_item); if (list_empty(&idev->igroup->device_list)) { iommu_detach_group(hwpt->domain, idev->igroup->group); idev->igroup->hwpt = NULL; } iopt_remove_reserved_iova(&hwpt->ioas->iopt, idev->dev); + mutex_unlock(&idev->igroup->lock); + /* Caller must destroy hwpt */ return hwpt; } @@ -515,8 +516,8 @@ iommufd_device_auto_get_domain(struct iommufd_device *idev, hwpt->auto_domain = true; *pt_id = hwpt->obj.id; - mutex_unlock(&ioas->mutex); iommufd_object_finalize(idev->ictx, &hwpt->obj); + mutex_unlock(&ioas->mutex); return destroy_hwpt; out_abort: @@ -610,7 +611,6 @@ EXPORT_SYMBOL_NS_GPL(iommufd_device_attach, IOMMUFD); * This is the same as * iommufd_device_detach(); * iommufd_device_attach(); - * * If it fails then no change is made to the attachment. The iommu driver may * implement this so there is no disruption in translation. This can only be * called if iommufd_device_attach() has already succeeded. @@ -633,10 +633,7 @@ void iommufd_device_detach(struct iommufd_device *idev) { struct iommufd_hw_pagetable *hwpt; - mutex_lock(&idev->igroup->lock); hwpt = iommufd_hw_pagetable_detach(idev); - mutex_unlock(&idev->igroup->lock); - iommufd_hw_pagetable_put(idev->ictx, hwpt); refcount_dec(&idev->obj.users); } diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index 8aa9ac130b5960..655ed32144f62e 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -26,6 +26,21 @@ void iommufd_hw_pagetable_destroy(struct iommufd_object *obj) refcount_dec(&hwpt->ioas->obj.users); } +void iommufd_hw_pagetable_abort(struct iommufd_object *obj) +{ + struct iommufd_hw_pagetable *hwpt = + container_of(obj, struct iommufd_hw_pagetable, obj); + + /* The ioas->mutex must be held until finalize is called. */ + lockdep_assert_held(&hwpt->ioas->mutex); + + if (!list_empty(&hwpt->hwpt_item)) { + list_del_init(&hwpt->hwpt_item); + iopt_table_remove_domain(&hwpt->ioas->iopt, hwpt->domain); + } + iommufd_hw_pagetable_destroy(obj); +} + int iommufd_hw_pagetable_enforce_cc(struct iommufd_hw_pagetable *hwpt) { if (hwpt->enforce_cache_coherency) @@ -50,6 +65,10 @@ int iommufd_hw_pagetable_enforce_cc(struct iommufd_hw_pagetable *hwpt) * Allocate a new iommu_domain and return it as a hw_pagetable. The HWPT * will be linked to the given ioas and upon return the underlying iommu_domain * is fully popoulated. + * + * The caller must hold the ioas->mutex until after + * iommufd_object_abort_and_destroy() or iommufd_object_finalize() is called on + * the returned hwpt. */ struct iommufd_hw_pagetable * iommufd_hw_pagetable_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, @@ -93,9 +112,6 @@ iommufd_hw_pagetable_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, * directly allocate a domain. These drivers do not finish creating the * domain until attach is completed. Thus we must have this call * sequence. Once those drivers are fixed this should be removed. - * - * Note we hold the igroup->lock here which prevents any other thread - * from observing igroup->hwpt until we finish setting it up. */ if (immediate_attach) { rc = iommufd_hw_pagetable_attach(hwpt, idev); @@ -140,10 +156,9 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) mutex_lock(&ioas->mutex); hwpt = iommufd_hw_pagetable_alloc(ucmd->ictx, ioas, idev, false); - mutex_unlock(&ioas->mutex); if (IS_ERR(hwpt)) { rc = PTR_ERR(hwpt); - goto out_put_ioas; + goto out_unlock; } cmd->out_hwpt_id = hwpt->obj.id; @@ -151,11 +166,12 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) if (rc) goto out_hwpt; iommufd_object_finalize(ucmd->ictx, &hwpt->obj); - goto out_put_ioas; + goto out_unlock; out_hwpt: iommufd_object_abort_and_destroy(ucmd->ictx, &hwpt->obj); -out_put_ioas: +out_unlock: + mutex_unlock(&ioas->mutex); iommufd_put_object(&ioas->obj); out_put_idev: iommufd_put_object(&idev->obj); diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c index f842768b2e250b..21052f64f95649 100644 --- a/drivers/iommu/iommufd/io_pagetable.c +++ b/drivers/iommu/iommufd/io_pagetable.c @@ -1172,6 +1172,9 @@ int iopt_table_enforce_dev_resv_regions(struct io_pagetable *iopt, unsigned int num_sw_msi = 0; int rc; + if (iommufd_should_fail()) + return -EINVAL; + down_write(&iopt->iova_rwsem); /* FIXME: drivers allocate memory but there is no failure propogated */ iommu_get_resv_regions(dev, &resv_regions); diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index cb693190bf51c5..ba50eb4661e217 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -261,6 +261,7 @@ int iommufd_hw_pagetable_attach(struct iommufd_hw_pagetable *hwpt, struct iommufd_hw_pagetable * iommufd_hw_pagetable_detach(struct iommufd_device *idev); void iommufd_hw_pagetable_destroy(struct iommufd_object *obj); +void iommufd_hw_pagetable_abort(struct iommufd_object *obj); int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd); static inline void iommufd_hw_pagetable_put(struct iommufd_ctx *ictx, diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 694da191e4b155..73a91e96896252 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -24,6 +24,7 @@ struct iommufd_object_ops { void (*destroy)(struct iommufd_object *obj); + void (*abort)(struct iommufd_object *obj); }; static const struct iommufd_object_ops iommufd_object_ops[]; static struct miscdevice vfio_misc_dev; @@ -104,7 +105,10 @@ void iommufd_object_abort(struct iommufd_ctx *ictx, struct iommufd_object *obj) void iommufd_object_abort_and_destroy(struct iommufd_ctx *ictx, struct iommufd_object *obj) { - iommufd_object_ops[obj->type].destroy(obj); + if (iommufd_object_ops[obj->type].abort) + iommufd_object_ops[obj->type].abort(obj); + else + iommufd_object_ops[obj->type].destroy(obj); iommufd_object_abort(ictx, obj); } @@ -413,6 +417,7 @@ static const struct iommufd_object_ops iommufd_object_ops[] = { }, [IOMMUFD_OBJ_HW_PAGETABLE] = { .destroy = iommufd_hw_pagetable_destroy, + .abort = iommufd_hw_pagetable_abort, }, #ifdef CONFIG_IOMMUFD_TEST [IOMMUFD_OBJ_SELFTEST] = { diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index c07252dbf62d72..8b2c18ac6a2864 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -9,9 +9,6 @@ #include "iommufd_utils.h" -static void *buffer; - -static unsigned long PAGE_SIZE; static unsigned long HUGEPAGE_SIZE; #define MOCK_PAGE_SIZE (PAGE_SIZE / 2) diff --git a/tools/testing/selftests/iommu/iommufd_fail_nth.c b/tools/testing/selftests/iommu/iommufd_fail_nth.c index 7e1afb6ff9bd8d..d4c552e5694812 100644 --- a/tools/testing/selftests/iommu/iommufd_fail_nth.c +++ b/tools/testing/selftests/iommu/iommufd_fail_nth.c @@ -41,6 +41,8 @@ static int writeat(int dfd, const char *fn, const char *val) static __attribute__((constructor)) void setup_buffer(void) { + PAGE_SIZE = sysconf(_SC_PAGE_SIZE); + BUFFER_SIZE = 2*1024*1024; buffer = mmap(0, BUFFER_SIZE, PROT_READ | PROT_WRITE, @@ -579,6 +581,7 @@ TEST_FAIL_NTH(basic_fail_nth, device) uint32_t stdev_id; uint32_t idev_id; uint32_t hwpt_id; + __u64 iova; self->fd = open("/dev/iommu", O_RDWR); if (self->fd == -1) @@ -590,6 +593,18 @@ TEST_FAIL_NTH(basic_fail_nth, device) if (_test_ioctl_ioas_alloc(self->fd, &ioas_id2)) return -1; + iova = MOCK_APERTURE_START; + if (_test_ioctl_ioas_map(self->fd, ioas_id, buffer, PAGE_SIZE, &iova, + IOMMU_IOAS_MAP_FIXED_IOVA | + IOMMU_IOAS_MAP_WRITEABLE | + IOMMU_IOAS_MAP_READABLE)) + return -1; + if (_test_ioctl_ioas_map(self->fd, ioas_id2, buffer, PAGE_SIZE, &iova, + IOMMU_IOAS_MAP_FIXED_IOVA | + IOMMU_IOAS_MAP_WRITEABLE | + IOMMU_IOAS_MAP_READABLE)) + return -1; + fail_nth_enable(); if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id, NULL, diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index 9b6dcb921750b6..53b4d3f2d9fc6c 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -19,6 +19,8 @@ static void *buffer; static unsigned long BUFFER_SIZE; +static unsigned long PAGE_SIZE; + /* * Have the kernel check the refcount on pages. I don't know why a freshly * mmap'd anon non-compound page starts out with a ref of 3 Jason Gunthorpe (17): iommufd: Move isolated msi enforcement to iommufd_device_bind() iommufd: Add iommufd_group iommufd: Replace the hwpt->devices list with iommufd_group iommu: Export iommu_get_resv_regions() iommufd: Keep track of each device's reserved regions instead of groups iommufd: Use the iommufd_group to avoid duplicate MSI setup iommufd: Make sw_msi_start a group global iommufd: Move putting a hwpt to a helper function iommufd: Add enforced_cache_coherency to iommufd_hw_pagetable_alloc() iommufd: Allow a hwpt to be aborted after allocation iommufd: Fix locking around hwpt allocation iommufd: Reorganize iommufd_device_attach into iommufd_device_change_pt iommufd: Add iommufd_device_replace() iommufd: Make destroy_rwsem use a lock class per object type iommufd: Add IOMMU_HWPT_ALLOC iommufd/selftest: Return the real idev id from selftest mock_domain iommufd/selftest: Add a selftest for IOMMU_HWPT_ALLOC Nicolin Chen (2): iommu: Introduce a new iommu_group_replace_domain() API iommufd/selftest: Test iommufd_device_replace() drivers/iommu/iommu-priv.h | 10 + drivers/iommu/iommu.c | 41 +- drivers/iommu/iommufd/device.c | 525 +++++++++++++----- drivers/iommu/iommufd/hw_pagetable.c | 112 +++- drivers/iommu/iommufd/io_pagetable.c | 30 +- drivers/iommu/iommufd/iommufd_private.h | 52 +- drivers/iommu/iommufd/iommufd_test.h | 6 + drivers/iommu/iommufd/main.c | 24 +- drivers/iommu/iommufd/selftest.c | 40 ++ include/linux/iommufd.h | 1 + include/uapi/linux/iommufd.h | 26 + tools/testing/selftests/iommu/iommufd.c | 67 ++- .../selftests/iommu/iommufd_fail_nth.c | 67 ++- tools/testing/selftests/iommu/iommufd_utils.h | 63 ++- 14 files changed, 853 insertions(+), 211 deletions(-) create mode 100644 drivers/iommu/iommu-priv.h base-commit: fd8c1a4aee973e87d890a5861e106625a33b2c4e -- 2.40.0

2 years

3
22
0 0

[PATCH bpf-next 0/2] Introduce a new bpf helper of bpf_task_under_cgroup

by Feng zhou

From: Feng Zhou <zhoufeng.zf(a)bytedance.com> Trace sched related functions, such as enqueue_task_fair, it is necessary to specify a task instead of the current task which within a given cgroup to a map. Feng Zhou (2): bpf: Add bpf_task_under_cgroup helper selftests/bpf: Add testcase for bpf_task_under_cgroup include/uapi/linux/bpf.h | 13 +++++ kernel/bpf/verifier.c | 4 +- kernel/trace/bpf_trace.c | 31 ++++++++++++ tools/include/uapi/linux/bpf.h | 13 +++++ .../bpf/prog_tests/task_under_cgroup.c | 49 +++++++++++++++++++ .../bpf/progs/test_task_under_cgroup.c | 31 ++++++++++++ 6 files changed, 140 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/task_under_cgroup.c create mode 100644 tools/testing/selftests/bpf/progs/test_task_under_cgroup.c -- 2.20.1

2 years

3
4
0 0

[PATCH v6 0/6] RISC-V Hardware Probing User Interface

by Evan Green

There's been a bunch of off-list discussions about this, including at Plumbers. The original plan was to do something involving providing an ISA string to userspace, but ISA strings just aren't sufficient for a stable ABI any more: in order to parse an ISA string users need the version of the specifications that the string is written to, the version of each extension (sometimes at a finer granularity than the RISC-V releases/versions encode), and the expected use case for the ISA string (ie, is it a U-mode or M-mode string). That's a lot of complexity to try and keep ABI compatible and it's probably going to continue to grow, as even if there's no more complexity in the specifications we'll have to deal with the various ISA string parsing oddities that end up all over userspace. Instead this patch set takes a very different approach and provides a set of key/value pairs that encode various bits about the system. The big advantage here is that we can clearly define what these mean so we can ensure ABI stability, but it also allows us to encode information that's unlikely to ever appear in an ISA string (see the misaligned access performance, for example). The resulting interface looks a lot like what arm64 and x86 do, and will hopefully fit well into something like ACPI in the future. The actual user interface is a syscall, with a vDSO function in front of it. The vDSO function can answer some queries without a syscall at all, and falls back to the syscall for cases it doesn't have answers to. Currently we prepopulate it with an array of answers for all keys and a CPU set of "all CPUs". This can be adjusted as necessary to provide fast answers to the most common queries. An example series in glibc exposing this syscall and using it in an ifunc selector for memcpy can be found at [1]. I was asked about the performance delta between this and something like sysfs. I created a small test program [2] and ran it on a Nezha D1 Allwinner board. Doing each operation 100000 times and dividing, these operations take the following amount of time: - open()+read()+close() of /sys/kernel/cpu_byteorder: 3.8us - access("/sys/kernel/cpu_byteorder", R_OK): 1.3us - riscv_hwprobe() vDSO and syscall: .0094us - riscv_hwprobe() vDSO with no syscall: 0.0091us These numbers get farther apart if we query multiple keys, as sysfs will scale linearly with the number of keys, where the dedicated syscall stays the same. To frame these numbers, I also did a tight fork/exec/wait loop, which I measured as 4.8ms. So doing 4 open/read/close operations is a delta of about 0.3%, versus a single vDSO call is a delta of essentially zero. [1] https://patchwork.ozlabs.org/project/glibc/list/?series=343050 [2] https://pastebin.com/x84NEKaS Changes in v6: - Remove spurious blank line (Conorbot) - Update copyrights (Paul) - Update copyrights (Paul) - Wrap init_hwprobe_vdso_data() in CONFIG_MMU to fix nommu build break (Conorbot) - Update copyrights (Paul) Changes in v5: - Added tags - Fixed misuse of ISA_EXT_c as bitmap, changed to use riscv_isa_extension_available() (Heiko, Conor) - Document the alternatives approach in the commit message (Conor and Heiko). - Fix __init call warnings by making probe_vendor_features() and thead_feature_probe_func() __init_or_module. - Fixed compat vdso compilation failure (lkp). Changes in v4: - Used real types in syscall prototypes (Arnd) - Fixed static line break in do_riscv_hwprobe() (Conor) - Added newlines between documentation lists (Conor) - Crispen up size types to size_t, and cpu indices to int (Joe) - Fix copy_from_user() return logic bug (found via kselftests!) - Add __user to SYSCALL_DEFINE() to fix warning - More newlines in BASE_BEHAVIOR_IMA documentation (Conor) - Add newlines to CPUPERF_0 documentation (Conor) - Add UNSUPPORTED value (Conor) - Switched from DT to alternatives-based probing (Rob) - Crispen up cpu index type to always be int (Conor) - Fixed selftests commit description, no more tiny libc (Mark Brown) - Fixed selftest syscall prototype types to match v4. - Added a prototype to fix -Wmissing-prototype warning (lkp(a)intel.com) - Fixed rv32 build failure (lkp(a)intel.com) - Make vdso prototype match syscall types update Changes in v3: - Updated copyright date in cpufeature.h - Fixed typo in cpufeature.h comment (Conor) - Refactored functions so that kernel mode can query too, in preparation for the vDSO data population. - Changed the vendor/arch/imp IDs to return a value of -1 on mismatch rather than failing the whole call. - Const cpumask pointer in hwprobe_mid() - Embellished documentation WRT cpu_set and the returned values. - Renamed hwprobe_mid() to hwprobe_arch_id() (Conor) - Fixed machine ID doc warnings, changed elements to c:macro:. - Completed dangling unistd.h comment (Conor) - Fixed line breaks and minor logic optimization (Conor). - Use riscv_cached_mxxxid() (Conor) - Refactored base ISA behavior probe to allow kernel probing as well, in prep for vDSO data initialization. - Fixed doc warnings in IMA text list, use :c:macro:. - Have hwprobe_misaligned return int instead of long. - Constify cpumask pointer in hwprobe_misaligned() - Fix warnings in _PERF_O list documentation, use :c:macro:. - Move include cpufeature.h to misaligned patch. - Fix documentation mismatch for RISCV_HWPROBE_KEY_CPUPERF_0 (Conor) - Use for_each_possible_cpu() instead of NR_CPUS (Conor) - Break early in misaligned access iteration (Conor) - Increase MISALIGNED_MASK from 2 bits to 3 for possible UNSUPPORTED future value (Conor) - Introduced vDSO function Changes in v2: - Factored the move of struct riscv_cpuinfo to its own header - Changed the interface to look more like poll(). Rather than supplying key_offset and getting back an array of values with numerically contiguous keys, have the user pre-fill the key members of the array, and the kernel will fill in the corresponding values. For any key it doesn't recognize, it will set the key of that element to -1. This allows usermode to quickly ask for exactly the elements it cares about, and not get bogged down in a back and forth about newer keys that older kernels might not recognize. In other words, the kernel can communicate that it doesn't recognize some of the keys while still providing the data for the keys it does know. - Added a shortcut to the cpuset parameters that if a size of 0 and NULL is provided for the CPU set, the kernel will use a cpu mask of all online CPUs. This is convenient because I suspect most callers will only want to act on a feature if it's supported on all CPUs, and it's a headache to dynamically allocate an array of all 1s, not to mention a waste to have the kernel loop over all of the offline bits. - Fixed logic error in if(of_property_read_string...) that caused crash - Include cpufeature.h in cpufeature.h to avoid undeclared variable warning. - Added a _MASK define - Fix random checkpatch complaints - Updated the selftests to the new API and added some more. - Fixed indentation, comments in .S, and general checkpatch complaints. Evan Green (6): RISC-V: Move struct riscv_cpuinfo to new header RISC-V: Add a syscall for HW probing RISC-V: hwprobe: Add support for RISCV_HWPROBE_BASE_BEHAVIOR_IMA RISC-V: hwprobe: Support probing of misaligned access performance selftests: Test the new RISC-V hwprobe interface RISC-V: Add hwprobe vDSO function and data Documentation/riscv/hwprobe.rst | 86 +++++++ Documentation/riscv/index.rst | 1 + arch/riscv/Kconfig | 1 + arch/riscv/errata/thead/errata.c | 10 + arch/riscv/include/asm/alternative.h | 5 + arch/riscv/include/asm/cpufeature.h | 23 ++ arch/riscv/include/asm/hwprobe.h | 13 + arch/riscv/include/asm/syscall.h | 4 + arch/riscv/include/asm/vdso/data.h | 17 ++ arch/riscv/include/asm/vdso/gettimeofday.h | 8 + arch/riscv/include/uapi/asm/hwprobe.h | 37 +++ arch/riscv/include/uapi/asm/unistd.h | 9 + arch/riscv/kernel/alternative.c | 19 ++ arch/riscv/kernel/compat_vdso/Makefile | 2 +- arch/riscv/kernel/cpu.c | 8 +- arch/riscv/kernel/cpufeature.c | 3 + arch/riscv/kernel/smpboot.c | 1 + arch/riscv/kernel/sys_riscv.c | 228 +++++++++++++++++- arch/riscv/kernel/vdso.c | 6 - arch/riscv/kernel/vdso/Makefile | 4 + arch/riscv/kernel/vdso/hwprobe.c | 52 ++++ arch/riscv/kernel/vdso/sys_hwprobe.S | 15 ++ arch/riscv/kernel/vdso/vdso.lds.S | 3 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/riscv/Makefile | 58 +++++ .../testing/selftests/riscv/hwprobe/Makefile | 10 + .../testing/selftests/riscv/hwprobe/hwprobe.c | 90 +++++++ .../selftests/riscv/hwprobe/sys_hwprobe.S | 12 + 28 files changed, 712 insertions(+), 14 deletions(-) create mode 100644 Documentation/riscv/hwprobe.rst create mode 100644 arch/riscv/include/asm/cpufeature.h create mode 100644 arch/riscv/include/asm/hwprobe.h create mode 100644 arch/riscv/include/asm/vdso/data.h create mode 100644 arch/riscv/include/uapi/asm/hwprobe.h create mode 100644 arch/riscv/kernel/vdso/hwprobe.c create mode 100644 arch/riscv/kernel/vdso/sys_hwprobe.S create mode 100644 tools/testing/selftests/riscv/Makefile create mode 100644 tools/testing/selftests/riscv/hwprobe/Makefile create mode 100644 tools/testing/selftests/riscv/hwprobe/hwprobe.c create mode 100644 tools/testing/selftests/riscv/hwprobe/sys_hwprobe.S -- 2.25.1

2 years

3
3
0 0

[PATCH v3 00/17] Add iommufd physical device operations for replace and alloc hwpt

by Jason Gunthorpe

This is the basic functionality for iommufd to support iommufd_device_replace() and IOMMU_HWPT_ALLOC for physical devices. iommufd_device_replace() allows changing the HWPT associated with the device to a new IOAS or HWPT. Replace does this in way that failure leaves things unchanged, and utilizes the iommu iommu_group_replace_domain() API to allow the iommu driver to perform an optional non-disruptive change. IOMMU_HWPT_ALLOC allows HWPTs to be explicitly allocated by the user and used by attach or replace. At this point it isn't very useful since the HWPT is the same as the automatically managed HWPT from the IOAS. However a following series will allow userspace to customize the created HWPT. The implementation is complicated because we have to introduce some per-iommu_group memory in iommufd and redo how we think about multi-device groups to be more explicit. This solves all the locking problems in the prior attempts. This series is infrastructure work for the following series which: - Add replace for attach - Expose replace through VFIO APIs - Implement driver parameters for HWPT creation (nesting) Once review of this is complete I will keep it on a side branch and accumulate the following series when they are ready so we can have a stable base and make more incremental progress. When we have all the parts together to get a full implementation it can go to Linus. I have this on github: https://github.com/jgunthorpe/linux/commits/iommufd_hwpt v3: - Refine comments and commit messages - Adjust the flow in iommufd_device_auto_get_domain() so pt_id is only set on success - Reject replace on non-attached devices - Add missing __reserved check for IOMMU_HWPT_ALLOC v2: https://lore.kernel.org/r/0-v2-51b9896e7862+8a8c-iommufd_alloc_jgg@nvidia.c… - Use WARN_ON for the igroup->group test and move that logic to a function iommufd_group_try_get() - Change igroup->devices to igroup->device list Replace will need to iterate over all attached idevs - Rename to iommufd_group_setup_msi() - New patch to export iommu_get_resv_regions() - New patch to use per-device reserved regions instead of per-group regions - Split out the reorganizing of iommufd_device_change_pt() from the replace patch - Replace uses the per-dev reserved regions - Use stdev_id in a few more places in the selftest - Fix error handling in IOMMU_HWPT_ALLOC - Clarify comments - Rebase on v6.3-rc1 v1: https://lore.kernel.org/all/0-v1-7612f88c19f5+2f21-iommufd_alloc_jgg@nvidia… Jason Gunthorpe (15): iommufd: Move isolated msi enforcement to iommufd_device_bind() iommufd: Add iommufd_group iommufd: Replace the hwpt->devices list with iommufd_group iommu: Export iommu_get_resv_regions() iommufd: Keep track of each device's reserved regions instead of groups iommufd: Use the iommufd_group to avoid duplicate MSI setup iommufd: Make sw_msi_start a group global iommufd: Move putting a hwpt to a helper function iommufd: Add enforced_cache_coherency to iommufd_hw_pagetable_alloc() iommufd: Reorganize iommufd_device_attach into iommufd_device_change_pt iommufd: Add iommufd_device_replace() iommufd: Make destroy_rwsem use a lock class per object type iommufd: Add IOMMU_HWPT_ALLOC iommufd/selftest: Return the real idev id from selftest mock_domain iommufd/selftest: Add a selftest for IOMMU_HWPT_ALLOC Nicolin Chen (2): iommu: Introduce a new iommu_group_replace_domain() API iommufd/selftest: Test iommufd_device_replace() drivers/iommu/iommu-priv.h | 10 + drivers/iommu/iommu.c | 41 +- drivers/iommu/iommufd/device.c | 512 +++++++++++++----- drivers/iommu/iommufd/hw_pagetable.c | 96 +++- drivers/iommu/iommufd/io_pagetable.c | 27 +- drivers/iommu/iommufd/iommufd_private.h | 51 +- drivers/iommu/iommufd/iommufd_test.h | 6 + drivers/iommu/iommufd/main.c | 17 +- drivers/iommu/iommufd/selftest.c | 40 ++ include/linux/iommufd.h | 1 + include/uapi/linux/iommufd.h | 26 + tools/testing/selftests/iommu/iommufd.c | 64 ++- .../selftests/iommu/iommufd_fail_nth.c | 52 +- tools/testing/selftests/iommu/iommufd_utils.h | 61 ++- 14 files changed, 804 insertions(+), 200 deletions(-) create mode 100644 drivers/iommu/iommu-priv.h base-commit: fd8c1a4aee973e87d890a5861e106625a33b2c4e -- 2.40.0

2 years

2
45
0 0

[PATCH RESEND] .gitignore: Do not ignore .kunitconfig files

by Chuck Lever

From: Chuck Lever <chuck.lever(a)oracle.com> Circumvent the .gitignore wildcard to avoid warnings about ignored .kunitconfig files. As far as I can tell, the warnings are harmless and these files are not actually ignored. Reported-by: kernel test robot <lkp(a)intel.com> Link: https://lore.kernel.org/oe-kbuild-all/202304142337.jc4oUrov-lkp@intel.com/ Signed-off-by: Chuck Lever <chuck.lever(a)oracle.com> --- .gitignore | 1 + 1 file changed, 1 insertion(+) Resending... It was not clear to me if this file has a specific maintainer. I chose to send it to the most recent committer. diff --git a/.gitignore b/.gitignore index 70ec6037fa7a..51117ba29c88 100644 --- a/.gitignore +++ b/.gitignore @@ -105,6 +105,7 @@ modules.order !.gitignore !.mailmap !.rustfmt.toml +!.kunitconfig # # Generated include files

2 years

3
2
0 0

[PATCH bpf 0/4] Socket lookup BPF API from tc/xdp ingress does not respect VRF bindings.

by Gilad Sever

When calling socket lookup from L2 (tc, xdp), VRF boundaries aren't respected. This patchset fixes this by regarding the incoming device's VRF attachment when performing the socket lookups from tc/xdp. The first two patches are coding changes which facilitate this fix by factoring out the tc helper's logic which was shared with cg/sk_skb (which operate correctly). The third patch contains the actual bugfix. The fourth patch adds bpf tests for these lookup functions. Gilad Sever (4): bpf: factor out socket lookup functions for the TC hookpoint. bpf: Call __bpf_sk_lookup()/__bpf_skc_lookup() directly via TC hookpoint bpf: fix bpf socket lookup from tc/xdp to respect socket VRF bindings selftests/bpf: Add tc_socket_lookup tests net/core/filter.c | 132 +++++-- .../bpf/prog_tests/tc_socket_lookup.c | 341 ++++++++++++++++++ .../selftests/bpf/progs/tc_socket_lookup.c | 73 ++++ 3 files changed, 525 insertions(+), 21 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/tc_socket_lookup.c create mode 100644 tools/testing/selftests/bpf/progs/tc_socket_lookup.c -- 2.34.1

2 years

1
4
0 0

[PATCH AUTOSEL 5.15 6/7] selftests mount: Fix mount_setattr_test builds failed

by Sasha Levin

From: Anh Tuan Phan <tuananhlfc(a)gmail.com> [ Upstream commit f1594bc676579133a3cd906d7d27733289edfb86 ] When compiling selftests with target mount_setattr I encountered some errors with the below messages: mount_setattr_test.c: In function ‘mount_setattr_thread’: mount_setattr_test.c:343:16: error: variable ‘attr’ has initializer but incomplete type 343 | struct mount_attr attr = { | ^~~~~~~~~~ These errors might be because of linux/mount.h is not included. This patch resolves that issue. Signed-off-by: Anh Tuan Phan <tuananhlfc(a)gmail.com> Acked-by: Christian Brauner <brauner(a)kernel.org> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/mount_setattr/mount_setattr_test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/mount_setattr/mount_setattr_test.c b/tools/testing/selftests/mount_setattr/mount_setattr_test.c index 8c5fea68ae677..969647228817b 100644 --- a/tools/testing/selftests/mount_setattr/mount_setattr_test.c +++ b/tools/testing/selftests/mount_setattr/mount_setattr_test.c @@ -18,6 +18,7 @@ #include <grp.h> #include <stdbool.h> #include <stdarg.h> +#include <linux/mount.h> #include "../kselftest_harness.h" -- 2.39.2

2 years

1
0
0 0

[PATCH AUTOSEL 6.1 10/15] selftests mount: Fix mount_setattr_test builds failed

by Sasha Levin

From: Anh Tuan Phan <tuananhlfc(a)gmail.com> [ Upstream commit f1594bc676579133a3cd906d7d27733289edfb86 ] When compiling selftests with target mount_setattr I encountered some errors with the below messages: mount_setattr_test.c: In function ‘mount_setattr_thread’: mount_setattr_test.c:343:16: error: variable ‘attr’ has initializer but incomplete type 343 | struct mount_attr attr = { | ^~~~~~~~~~ These errors might be because of linux/mount.h is not included. This patch resolves that issue. Signed-off-by: Anh Tuan Phan <tuananhlfc(a)gmail.com> Acked-by: Christian Brauner <brauner(a)kernel.org> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/mount_setattr/mount_setattr_test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/mount_setattr/mount_setattr_test.c b/tools/testing/selftests/mount_setattr/mount_setattr_test.c index 8c5fea68ae677..969647228817b 100644 --- a/tools/testing/selftests/mount_setattr/mount_setattr_test.c +++ b/tools/testing/selftests/mount_setattr/mount_setattr_test.c @@ -18,6 +18,7 @@ #include <grp.h> #include <stdbool.h> #include <stdarg.h> +#include <linux/mount.h> #include "../kselftest_harness.h" -- 2.39.2

2 years

1
0
0 0

[PATCH AUTOSEL 6.2 12/17] selftests mount: Fix mount_setattr_test builds failed

by Sasha Levin

From: Anh Tuan Phan <tuananhlfc(a)gmail.com> [ Upstream commit f1594bc676579133a3cd906d7d27733289edfb86 ] When compiling selftests with target mount_setattr I encountered some errors with the below messages: mount_setattr_test.c: In function ‘mount_setattr_thread’: mount_setattr_test.c:343:16: error: variable ‘attr’ has initializer but incomplete type 343 | struct mount_attr attr = { | ^~~~~~~~~~ These errors might be because of linux/mount.h is not included. This patch resolves that issue. Signed-off-by: Anh Tuan Phan <tuananhlfc(a)gmail.com> Acked-by: Christian Brauner <brauner(a)kernel.org> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/mount_setattr/mount_setattr_test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/mount_setattr/mount_setattr_test.c b/tools/testing/selftests/mount_setattr/mount_setattr_test.c index 8c5fea68ae677..969647228817b 100644 --- a/tools/testing/selftests/mount_setattr/mount_setattr_test.c +++ b/tools/testing/selftests/mount_setattr/mount_setattr_test.c @@ -18,6 +18,7 @@ #include <grp.h> #include <stdbool.h> #include <stdarg.h> +#include <linux/mount.h> #include "../kselftest_harness.h" -- 2.39.2

2 years

1
0
0 0

[PATCH v2 1/3] kunit: Always run cleanup from a test kthread

by David Gow

KUnit tests run in a kthread, with the current->kunit_test pointer set to the test's context. This allows the kunit_get_current_test() and kunit_fail_current_test() macros to work. Normally, this pointer is still valid during test shutdown (i.e., the suite->exit function, and any resource cleanup). However, if the test has exited early (e.g., due to a failed assertion), the cleanup is done in the parent KUnit thread, which does not have an active context. Instead, in the event test terminates early, run the test exit and cleanup from a new 'cleanup' kthread, which sets current->kunit_test, and better isolates the rest of KUnit from issues which arise in test cleanup. If a test cleanup function itself aborts (e.g., due to an assertion failing), there will be no further attempts to clean up: an error will be logged and the test failed. This should also make it easier to get access to the KUnit context, particularly from within resource cleanup functions, which may, for example, need access to data in test->priv. Signed-off-by: David Gow <davidgow(a)google.com> --- This is an updated version of / replacement of "kunit: Set the current KUnit context when cleaning up", which instead creates a new kthread for cleanup tasks if the original test kthread is aborted. This protects us from failed assertions during cleanup, if the test exited early. Changes since v1: https://lore.kernel.org/linux-kselftest/20230415091401.681395-1-davidgow@go… - Move cleanup execution to another kthread - (Thanks, Benjamin, for pointing out the assertion issues) --- lib/kunit/test.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 52 insertions(+), 2 deletions(-) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index e2910b261112..caeae0dfd82b 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -423,8 +423,51 @@ static void kunit_try_run_case(void *data) kunit_run_case_cleanup(test, suite); } +static void kunit_try_run_case_cleanup(void *data) +{ + struct kunit_try_catch_context *ctx = data; + struct kunit *test = ctx->test; + struct kunit_suite *suite = ctx->suite; + + current->kunit_test = test; + + kunit_run_case_cleanup(test, suite); +} + +static void kunit_catch_run_case_cleanup(void *data) +{ + struct kunit_try_catch_context *ctx = data; + struct kunit *test = ctx->test; + int try_exit_code = kunit_try_catch_get_result(&test->try_catch); + + /* It is always a failure if cleanup aborts. */ + kunit_set_failure(test); + + if (try_exit_code) { + /* + * Test case could not finish, we have no idea what state it is + * in, so don't do clean up. + */ + if (try_exit_code == -ETIMEDOUT) { + kunit_err(test, "test case cleanup timed out\n"); + /* + * Unknown internal error occurred preventing test case from + * running, so there is nothing to clean up. + */ + } else { + kunit_err(test, "internal error occurred during test case cleanup: %d\n", + try_exit_code); + } + return; + } + + kunit_err(test, "test aborted during cleanup. continuing without cleaning up\n"); +} + + static void kunit_catch_run_case(void *data) { + struct kunit_try_catch cleanup; struct kunit_try_catch_context *ctx = data; struct kunit *test = ctx->test; struct kunit_suite *suite = ctx->suite; @@ -451,9 +494,16 @@ static void kunit_catch_run_case(void *data) /* * Test case was run, but aborted. It is the test case's business as to - * whether it failed or not, we just need to clean up. + * whether it failed or not, we just need to clean up. Do this in a new + * try / catch context, in case it asserts, too. */ - kunit_run_case_cleanup(test, suite); + kunit_try_catch_init(&cleanup, + test, + kunit_try_run_case_cleanup, + kunit_catch_run_case_cleanup); + ctx->test = test; + ctx->suite = suite; + kunit_try_catch_run(&cleanup, ctx); } /* -- 2.40.0.634.g4ca3ef3211-goog

2 years

2
3
0 0

[PATCH bpf-next v2 0/2] Access variable length array relaxed for integer type

by Feng zhou

From: Feng Zhou <zhoufeng.zf(a)bytedance.com> Add support for integer type of accessing variable length array. Add a selftest to check it. Feng Zhou (2): bpf: support access variable length array of integer type selftests/bpf: Add test to access integer type of variable array Changelog: v1->v2: Addressed comments from Alexei Starovoitov - Add one more use case. Details in here: https://lore.kernel.org/bpf/20230417080749.39074-1-zhoufeng.zf@bytedance.co… kernel/bpf/btf.c | 8 +++++--- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 20 +++++++++++++++++++ .../bpf/prog_tests/access_variable_array.c | 16 +++++++++++++++ .../selftests/bpf/prog_tests/tracing_struct.c | 2 ++ .../bpf/progs/test_access_variable_array.c | 19 ++++++++++++++++++ .../selftests/bpf/progs/tracing_struct.c | 13 ++++++++++++ 6 files changed, 75 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/access_variable_array.c create mode 100644 tools/testing/selftests/bpf/progs/test_access_variable_array.c -- 2.20.1

2 years

2
3
0 0

[PATCH v15 0/5] Implement IOCTL to get and optionally clear info about PTEs

by Muhammad Usama Anjum

*Changes in v15* - Build fix *Changes in v14* - Fix build error caused by #ifdef added at last minute in some configs *Changes in v13* - Rebase on top of next-20230414 - Give-up on using uffd_wp_range() and write new helpers, flush tlb only once *Changes in v12* - Update and other memory types to UFFD_FEATURE_WP_ASYNC - Rebaase on top of next-20230406 - Review updates *Changes in v11* - Rebase on top of next-20230307 - Base patches on UFFD_FEATURE_WP_UNPOPULATED - Do a lot of cosmetic changes and review updates - Remove ENGAGE_WP + !GET operation as it can be performed with UFFDIO_WRITEPROTECT *Changes in v10* - Add specific condition to return error if hugetlb is used with wp async - Move changes in tools/include/uapi/linux/fs.h to separate patch - Add documentation *Changes in v9:* - Correct fault resolution for userfaultfd wp async - Fix build warnings and errors which were happening on some configs - Simplify pagemap ioctl's code *Changes in v8:* - Update uffd async wp implementation - Improve PAGEMAP_IOCTL implementation *Changes in v7:* - Add uffd wp async - Update the IOCTL to use uffd under the hood instead of soft-dirty flags *Motivation* The real motivation for adding PAGEMAP_SCAN IOCTL is to emulate Windows GetWriteWatch() syscall [1]. The GetWriteWatch{} retrieves the addresses of the pages that are written to in a region of virtual memory. This syscall is used in Windows applications and games etc. This syscall is being emulated in pretty slow manner in userspace. Our purpose is to enhance the kernel such that we translate it efficiently in a better way. Currently some out of tree hack patches are being used to efficiently emulate it in some kernels. We intend to replace those with these patches. So the whole gaming on Linux can effectively get benefit from this. It means there would be tons of users of this code. CRIU use case [2] was mentioned by Andrei and Danylo: > Use cases for migrating sparse VMAs are binaries sanitized with ASAN, > MSAN or TSAN [3]. All of these sanitizers produce sparse mappings of > shadow memory [4]. Being able to migrate such binaries allows to highly > reduce the amount of work needed to identify and fix post-migration > crashes, which happen constantly. Andrei's defines the following uses of this code: * it is more granular and allows us to track changed pages more effectively. The current interface can clear dirty bits for the entire process only. In addition, reading info about pages is a separate operation. It means we must freeze the process to read information about all its pages, reset dirty bits, only then we can start dumping pages. The information about pages becomes more and more outdated, while we are processing pages. The new interface solves both these downsides. First, it allows us to read pte bits and clear the soft-dirty bit atomically. It means that CRIU will not need to freeze processes to pre-dump their memory. Second, it clears soft-dirty bits for a specified region of memory. It means CRIU will have actual info about pages to the moment of dumping them. * The new interface has to be much faster because basic page filtering is happening in the kernel. With the old interface, we have to read pagemap for each page. *Implementation Evolution (Short Summary)* From the definition of GetWriteWatch(), we feel like kernel's soft-dirty feature can be used under the hood with some additions like: * reset soft-dirty flag for only a specific region of memory instead of clearing the flag for the entire process * get and clear soft-dirty flag for a specific region atomically So we decided to use ioctl on pagemap file to read or/and reset soft-dirty flag. But using soft-dirty flag, sometimes we get extra pages which weren't even written. They had become soft-dirty because of VMA merging and VM_SOFTDIRTY flag. This breaks the definition of GetWriteWatch(). We were able to by-pass this short coming by ignoring VM_SOFTDIRTY until David reported that mprotect etc messes up the soft-dirty flag while ignoring VM_SOFTDIRTY [5]. This wasn't happening until [6] got introduced. We discussed if we can revert these patches. But we could not reach to any conclusion. So at this point, I made couple of tries to solve this whole VM_SOFTDIRTY issue by correcting the soft-dirty implementation: * [7] Correct the bug fixed wrongly back in 2014. It had potential to cause regression. We left it behind. * [8] Keep a list of soft-dirty part of a VMA across splits and merges. I got the reply don't increase the size of the VMA by 8 bytes. At this point, we left soft-dirty considering it is too much delicate and userfaultfd [9] seemed like the only way forward. From there onward, we have been basing soft-dirty emulation on userfaultfd wp feature where kernel resolves the faults itself when WP_ASYNC feature is used. It was straight forward to add WP_ASYNC feature in userfautlfd. Now we get only those pages dirty or written-to which are really written in reality. (PS There is another WP_UNPOPULATED userfautfd feature is required which is needed to avoid pre-faulting memory before write-protecting [9].) All the different masks were added on the request of CRIU devs to create interface more generic and better. [1] https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-… [2] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com [3] https://github.com/google/sanitizers [4] https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#64-bit [5] https://lore.kernel.org/all/bfcae708-db21-04b4-0bbe-712badd03071@redhat.com [6] https://lore.kernel.org/all/20220725142048.30450-1-peterx@redhat.com/ [7] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [8] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [9] https://lore.kernel.org/all/20230306213925.617814-1-peterx@redhat.com [10] https://lore.kernel.org/all/20230125144529.1630917-1-mdanylo@google.com * Original Cover letter from v8* Hello, Note: Soft-dirty pages and pages which have been written-to are synonyms. As kernel already has soft-dirty feature inside which we have given up to use, we are using written-to terminology while using UFFD async WP under the hood. This IOCTL, PAGEMAP_SCAN on pagemap file can be used to get and/or clear the info about page table entries. The following operations are supported in this ioctl: - Get the information if the pages have been written-to (PAGE_IS_WRITTEN), file mapped (PAGE_IS_FILE), present (PAGE_IS_PRESENT) or swapped (PAGE_IS_SWAPPED). - Write-protect the pages (PAGEMAP_WP_ENGAGE) to start finding which pages have been written-to. - Find pages which have been written-to and write protect the pages (atomic PAGE_IS_WRITTEN + PAGEMAP_WP_ENGAGE) It is possible to find and clear soft-dirty pages entirely in userspace. But it isn't efficient: - The mprotect and SIGSEGV handler for bookkeeping - The userfaultfd wp (synchronous) with the handler for bookkeeping Some benchmarks can be seen here[1]. This series adds features that weren't present earlier: - There is no atomic get soft-dirty/Written-to status and clear present in the kernel. - The pages which have been written-to can not be found in accurate way. (Kernel's soft-dirty PTE bit + sof_dirty VMA bit shows more soft-dirty pages than there actually are.) Historically, soft-dirty PTE bit tracking has been used in the CRIU project. The procfs interface is enough for finding the soft-dirty bit status and clearing the soft-dirty bit of all the pages of a process. We have the use case where we need to track the soft-dirty PTE bit for only specific pages on-demand. We need this tracking and clear mechanism of a region of memory while the process is running to emulate the getWriteWatch() syscall of Windows. *(Moved to using UFFD instead of soft-dirtyi feature to find pages which have been written-to from v7 patch series)*: Stop using the soft-dirty flags for finding which pages have been written to. It is too delicate and wrong as it shows more soft-dirty pages than the actual soft-dirty pages. There is no interest in correcting it [2][3] as this is how the feature was written years ago. It shouldn't be updated to changed behaviour. Peter Xu has suggested using the async version of the UFFD WP [4] as it is based inherently on the PTEs. So in this patch series, I've added a new mode to the UFFD which is asynchronous version of the write protect. When this variant of the UFFD WP is used, the page faults are resolved automatically by the kernel. The pages which have been written-to can be found by reading pagemap file (!PM_UFFD_WP). This feature can be used successfully to find which pages have been written to from the time the pages were write protected. This works just like the soft-dirty flag without showing any extra pages which aren't soft-dirty in reality. The information related to pages if the page is file mapped, present and swapped is required for the CRIU project [5][6]. The addition of the required mask, any mask, excluded mask and return masks are also required for the CRIU project [5]. The IOCTL returns the addresses of the pages which match the specific masks. The page addresses are returned in struct page_region in a compact form. The max_pages is needed to support a use case where user only wants to get a specific number of pages. So there is no need to find all the pages of interest in the range when max_pages is specified. The IOCTL returns when the maximum number of the pages are found. The max_pages is optional. If max_pages is specified, it must be equal or greater than the vec_size. This restriction is needed to handle worse case when one page_region only contains info of one page and it cannot be compacted. This is needed to emulate the Windows getWriteWatch() syscall. The patch series include the detailed selftest which can be used as an example for the uffd async wp test and PAGEMAP_IOCTL. It shows the interface usages as well. [1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora… [2] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [3] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [4] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n [5] https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com/ [6] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com/ Regards, Muhammad Usama Anjum Muhammad Usama Anjum (4): fs/proc/task_mmu: Implement IOCTL to get and optionally clear info about PTEs tools headers UAPI: Update linux/fs.h with the kernel sources mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL selftests: mm: add pagemap ioctl tests Peter Xu (1): userfaultfd: UFFD_FEATURE_WP_ASYNC Documentation/admin-guide/mm/pagemap.rst | 56 + Documentation/admin-guide/mm/userfaultfd.rst | 35 + fs/proc/task_mmu.c | 481 +++++++ fs/userfaultfd.c | 26 +- include/linux/userfaultfd_k.h | 21 +- include/uapi/linux/fs.h | 53 + include/uapi/linux/userfaultfd.h | 9 +- mm/hugetlb.c | 32 +- mm/memory.c | 27 +- tools/include/uapi/linux/fs.h | 53 + tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 3 +- tools/testing/selftests/mm/config | 1 + tools/testing/selftests/mm/pagemap_ioctl.c | 1326 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 4 + 15 files changed, 2105 insertions(+), 23 deletions(-) create mode 100644 tools/testing/selftests/mm/pagemap_ioctl.c mode change 100644 => 100755 tools/testing/selftests/mm/run_vmtests.sh -- 2.39.2

2 years

2
7
0 0

[PATCH linux-next v2] selftests: net: udpgso_bench_rx: Fix verifty exceptions

by yang.yang29＠zte.com.cn

From: Zhang Yunkai (CGEL ZTE) <zhang.yunkai(a)zte.com.cn> The verification function of this test case is likely to encounter the following error, which may confuse users. The problem is easily reproducible in the latest kernel. Environment A, the sender: bash# udpgso_bench_tx -l 4 -4 -D "$IP_B" udpgso_bench_tx: write: Connection refused Environment B, the receiver: bash# udpgso_bench_rx -4 -G -S 1472 -v udpgso_bench_rx: data[1472]: len 17664, a(97) != q(113) If the packet is captured, you will see: Environment A, the sender: bash# tcpdump -i eth0 host "$IP_B" & IP $IP_A.41025 > $IP_B.8000: UDP, length 1472 IP $IP_A.41025 > $IP_B.8000: UDP, length 1472 IP $IP_B > $IP_A: ICMP $IP_B udp port 8000 unreachable, length 556 Environment B, the receiver: bash# tcpdump -i eth0 host "$IP_B" & IP $IP_A.41025 > $IP_B.8000: UDP, length 7360 IP $IP_A.41025 > $IP_B.8000: UDP, length 14720 IP $IP_B > $IP_A: ICMP $IP_B udp port 8000 unreachable, length 556 In one test, the verification data is printed as follows: abcd...xyz | 1... .. | abcd...xyz | abcd...opabcd...xyz | ...1472... Not xyzabcd, messages are merged .. | This is because the sending buffer is buf[64K], and its content is a loop of A-Z. But maybe only 1472 bytes per send, or more if UDP GSO is used. The message content does not necessarily end with XYZ, but GRO will merge these packets, and the -v parameter directly verifies the entire GRO receive buffer. So we do the validation after the data is split at the receiving end, just as the application actually uses this feature. If the sender does not use GSO, each individual segment starts at A, end at somewhere. Using GSO also has the same problem, and. The data between each segment during transmission is continuous, but GRO is merged in the order received, which is not necessarily the order of transmission. Execution in the same environment does not cause problems, because the lo device is not NAPI, and does not perform GRO processing. Perhaps it could be worth supporting to reduce system calls. bash# tcpdump -i lo host "$IP_self" & bash# echo udp_gro_receive > /sys/kernel/debug/tracing/set_ftrace_filter bash# echo function > /sys/kernel/debug/tracing/current_tracer bash# udpgso_bench_rx -4 -G -S 1472 -v & bash# udpgso_bench_tx -l 4 -4 -D "$IP_self" The issue still exists when using the GRO with -G, but not using the -S to obtain gsosize. Therefore, a print has been added to remind users. After this issue is resolved, another issue will be encountered and will be resolved in the next patch. Environment A, the sender: bash# udpgso_bench_tx -l 4 -4 -D "$DST" udpgso_bench_tx: write: Connection refused Environment B, the receiver: bash# udpgso_bench_rx -4 -G -S 1472 udp rx: 15 MB/s 256 calls/s udp rx: 30 MB/s 512 calls/s udpgso_bench_rx: recv: bad gso size, got -1, expected 1472 (-1 == no gso cmsg)) v2: - Fix confusing descriptions Signed-off-by: Zhang Yunkai (CGEL ZTE) <zhang.yunkai(a)zte.com.cn> Reviewed-by: Xu Xin (CGEL ZTE) <xu.xin16(a)zte.com.cn> Reviewed-by: Yang Yang (CGEL ZTE) <yang.yang29(a)zte.com.cn> Cc: Xuexin Jiang (CGEL ZTE) <jiang.xuexin(a)zte.com.cn> --- tools/testing/selftests/net/udpgso_bench_rx.c | 40 +++++++++++++++++++++------ 1 file changed, 31 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/net/udpgso_bench_rx.c b/tools/testing/selftests/net/udpgso_bench_rx.c index f35a924d4a30..6a2026494cdb 100644 --- a/tools/testing/selftests/net/udpgso_bench_rx.c +++ b/tools/testing/selftests/net/udpgso_bench_rx.c @@ -189,26 +189,44 @@ static char sanitized_char(char val) return (val >= 'a' && val <= 'z') ? val : '.'; } -static void do_verify_udp(const char *data, int len) +static void do_verify_udp(const char *data, int start, int len) { - char cur = data[0]; + char cur = data[start]; int i; /* verify contents */ if (cur < 'a' || cur > 'z') error(1, 0, "data initial byte out of range"); - for (i = 1; i < len; i++) { + for (i = start + 1; i < start + len; i++) { if (cur == 'z') cur = 'a'; else cur++; - if (data[i] != cur) + if (data[i] != cur) { + if (cfg_gro_segment && !cfg_expected_gso_size) + error(0, 0, "Use -S to obtain gsosize, to %s" + , "help guide split and verification."); + error(1, 0, "data[%d]: len %d, %c(%hhu) != %c(%hhu)\n", i, len, sanitized_char(data[i]), data[i], sanitized_char(cur), cur); + } + } +} + +static void do_verify_udp_gro(const char *data, int len, int gso_size) +{ + int start = 0; + + while (len - start > 0) { + if (len - start > gso_size) + do_verify_udp(data, start, gso_size); + else + do_verify_udp(data, start, len - start); + start += gso_size; } } @@ -264,16 +282,20 @@ static void do_flush_udp(int fd) if (cfg_expected_pkt_len && ret != cfg_expected_pkt_len) error(1, 0, "recv: bad packet len, got %d," " expected %d\n", ret, cfg_expected_pkt_len); + if (cfg_expected_gso_size && cfg_expected_gso_size != gso_size) + error(1, 0, "recv: bad gso size, got %d, expected %d %s", + gso_size, cfg_expected_gso_size, "(-1 == no gso cmsg))\n"); if (len && cfg_verify) { if (ret == 0) error(1, errno, "recv: 0 byte datagram\n"); - do_verify_udp(rbuf, ret); + if (!cfg_gro_segment) + do_verify_udp(rbuf, 0, ret); + else if (gso_size > 0) + do_verify_udp_gro(rbuf, ret, gso_size); + else + do_verify_udp_gro(rbuf, ret, ret); } - if (cfg_expected_gso_size && cfg_expected_gso_size != gso_size) - error(1, 0, "recv: bad gso size, got %d, expected %d " - "(-1 == no gso cmsg))\n", gso_size, - cfg_expected_gso_size); packets++; bytes += ret; -- 2.15.2

2 years

2
1
0 0

[PATCH v14 0/5] Implement IOCTL to get and optionally clear info about PTEs

by Muhammad Usama Anjum

*Changes in v14* - Fix build error caused by #ifdef added at last minute in some configs *Changes in v13* - Rebase on top of next-20230414 - Give-up on using uffd_wp_range() and write new helpers, flush tlb only once *Changes in v12* - Update and other memory types to UFFD_FEATURE_WP_ASYNC - Rebaase on top of next-20230406 - Review updates *Changes in v11* - Rebase on top of next-20230307 - Base patches on UFFD_FEATURE_WP_UNPOPULATED - Do a lot of cosmetic changes and review updates - Remove ENGAGE_WP + !GET operation as it can be performed with UFFDIO_WRITEPROTECT *Changes in v10* - Add specific condition to return error if hugetlb is used with wp async - Move changes in tools/include/uapi/linux/fs.h to separate patch - Add documentation *Changes in v9:* - Correct fault resolution for userfaultfd wp async - Fix build warnings and errors which were happening on some configs - Simplify pagemap ioctl's code *Changes in v8:* - Update uffd async wp implementation - Improve PAGEMAP_IOCTL implementation *Changes in v7:* - Add uffd wp async - Update the IOCTL to use uffd under the hood instead of soft-dirty flags *Motivation* The real motivation for adding PAGEMAP_SCAN IOCTL is to emulate Windows GetWriteWatch() syscall [1]. The GetWriteWatch{} retrieves the addresses of the pages that are written to in a region of virtual memory. This syscall is used in Windows applications and games etc. This syscall is being emulated in pretty slow manner in userspace. Our purpose is to enhance the kernel such that we translate it efficiently in a better way. Currently some out of tree hack patches are being used to efficiently emulate it in some kernels. We intend to replace those with these patches. So the whole gaming on Linux can effectively get benefit from this. It means there would be tons of users of this code. CRIU use case [2] was mentioned by Andrei and Danylo: > Use cases for migrating sparse VMAs are binaries sanitized with ASAN, > MSAN or TSAN [3]. All of these sanitizers produce sparse mappings of > shadow memory [4]. Being able to migrate such binaries allows to highly > reduce the amount of work needed to identify and fix post-migration > crashes, which happen constantly. Andrei's defines the following uses of this code: * it is more granular and allows us to track changed pages more effectively. The current interface can clear dirty bits for the entire process only. In addition, reading info about pages is a separate operation. It means we must freeze the process to read information about all its pages, reset dirty bits, only then we can start dumping pages. The information about pages becomes more and more outdated, while we are processing pages. The new interface solves both these downsides. First, it allows us to read pte bits and clear the soft-dirty bit atomically. It means that CRIU will not need to freeze processes to pre-dump their memory. Second, it clears soft-dirty bits for a specified region of memory. It means CRIU will have actual info about pages to the moment of dumping them. * The new interface has to be much faster because basic page filtering is happening in the kernel. With the old interface, we have to read pagemap for each page. *Implementation Evolution (Short Summary)* From the definition of GetWriteWatch(), we feel like kernel's soft-dirty feature can be used under the hood with some additions like: * reset soft-dirty flag for only a specific region of memory instead of clearing the flag for the entire process * get and clear soft-dirty flag for a specific region atomically So we decided to use ioctl on pagemap file to read or/and reset soft-dirty flag. But using soft-dirty flag, sometimes we get extra pages which weren't even written. They had become soft-dirty because of VMA merging and VM_SOFTDIRTY flag. This breaks the definition of GetWriteWatch(). We were able to by-pass this short coming by ignoring VM_SOFTDIRTY until David reported that mprotect etc messes up the soft-dirty flag while ignoring VM_SOFTDIRTY [5]. This wasn't happening until [6] got introduced. We discussed if we can revert these patches. But we could not reach to any conclusion. So at this point, I made couple of tries to solve this whole VM_SOFTDIRTY issue by correcting the soft-dirty implementation: * [7] Correct the bug fixed wrongly back in 2014. It had potential to cause regression. We left it behind. * [8] Keep a list of soft-dirty part of a VMA across splits and merges. I got the reply don't increase the size of the VMA by 8 bytes. At this point, we left soft-dirty considering it is too much delicate and userfaultfd [9] seemed like the only way forward. From there onward, we have been basing soft-dirty emulation on userfaultfd wp feature where kernel resolves the faults itself when WP_ASYNC feature is used. It was straight forward to add WP_ASYNC feature in userfautlfd. Now we get only those pages dirty or written-to which are really written in reality. (PS There is another WP_UNPOPULATED userfautfd feature is required which is needed to avoid pre-faulting memory before write-protecting [9].) All the different masks were added on the request of CRIU devs to create interface more generic and better. [1] https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-… [2] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com [3] https://github.com/google/sanitizers [4] https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#64-bit [5] https://lore.kernel.org/all/bfcae708-db21-04b4-0bbe-712badd03071@redhat.com [6] https://lore.kernel.org/all/20220725142048.30450-1-peterx@redhat.com/ [7] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [8] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [9] https://lore.kernel.org/all/20230306213925.617814-1-peterx@redhat.com [10] https://lore.kernel.org/all/20230125144529.1630917-1-mdanylo@google.com * Original Cover letter from v8* Hello, Note: Soft-dirty pages and pages which have been written-to are synonyms. As kernel already has soft-dirty feature inside which we have given up to use, we are using written-to terminology while using UFFD async WP under the hood. This IOCTL, PAGEMAP_SCAN on pagemap file can be used to get and/or clear the info about page table entries. The following operations are supported in this ioctl: - Get the information if the pages have been written-to (PAGE_IS_WRITTEN), file mapped (PAGE_IS_FILE), present (PAGE_IS_PRESENT) or swapped (PAGE_IS_SWAPPED). - Write-protect the pages (PAGEMAP_WP_ENGAGE) to start finding which pages have been written-to. - Find pages which have been written-to and write protect the pages (atomic PAGE_IS_WRITTEN + PAGEMAP_WP_ENGAGE) It is possible to find and clear soft-dirty pages entirely in userspace. But it isn't efficient: - The mprotect and SIGSEGV handler for bookkeeping - The userfaultfd wp (synchronous) with the handler for bookkeeping Some benchmarks can be seen here[1]. This series adds features that weren't present earlier: - There is no atomic get soft-dirty/Written-to status and clear present in the kernel. - The pages which have been written-to can not be found in accurate way. (Kernel's soft-dirty PTE bit + sof_dirty VMA bit shows more soft-dirty pages than there actually are.) Historically, soft-dirty PTE bit tracking has been used in the CRIU project. The procfs interface is enough for finding the soft-dirty bit status and clearing the soft-dirty bit of all the pages of a process. We have the use case where we need to track the soft-dirty PTE bit for only specific pages on-demand. We need this tracking and clear mechanism of a region of memory while the process is running to emulate the getWriteWatch() syscall of Windows. *(Moved to using UFFD instead of soft-dirtyi feature to find pages which have been written-to from v7 patch series)*: Stop using the soft-dirty flags for finding which pages have been written to. It is too delicate and wrong as it shows more soft-dirty pages than the actual soft-dirty pages. There is no interest in correcting it [2][3] as this is how the feature was written years ago. It shouldn't be updated to changed behaviour. Peter Xu has suggested using the async version of the UFFD WP [4] as it is based inherently on the PTEs. So in this patch series, I've added a new mode to the UFFD which is asynchronous version of the write protect. When this variant of the UFFD WP is used, the page faults are resolved automatically by the kernel. The pages which have been written-to can be found by reading pagemap file (!PM_UFFD_WP). This feature can be used successfully to find which pages have been written to from the time the pages were write protected. This works just like the soft-dirty flag without showing any extra pages which aren't soft-dirty in reality. The information related to pages if the page is file mapped, present and swapped is required for the CRIU project [5][6]. The addition of the required mask, any mask, excluded mask and return masks are also required for the CRIU project [5]. The IOCTL returns the addresses of the pages which match the specific masks. The page addresses are returned in struct page_region in a compact form. The max_pages is needed to support a use case where user only wants to get a specific number of pages. So there is no need to find all the pages of interest in the range when max_pages is specified. The IOCTL returns when the maximum number of the pages are found. The max_pages is optional. If max_pages is specified, it must be equal or greater than the vec_size. This restriction is needed to handle worse case when one page_region only contains info of one page and it cannot be compacted. This is needed to emulate the Windows getWriteWatch() syscall. The patch series include the detailed selftest which can be used as an example for the uffd async wp test and PAGEMAP_IOCTL. It shows the interface usages as well. [1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora… [2] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [3] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [4] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n [5] https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com/ [6] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com/ Regards, Muhammad Usama Anjum Muhammad Usama Anjum (4): fs/proc/task_mmu: Implement IOCTL to get and optionally clear info about PTEs tools headers UAPI: Update linux/fs.h with the kernel sources mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL selftests: mm: add pagemap ioctl tests Peter Xu (1): userfaultfd: UFFD_FEATURE_WP_ASYNC Documentation/admin-guide/mm/pagemap.rst | 56 + Documentation/admin-guide/mm/userfaultfd.rst | 35 + fs/proc/task_mmu.c | 478 +++++++ fs/userfaultfd.c | 26 +- include/linux/userfaultfd_k.h | 21 +- include/uapi/linux/fs.h | 53 + include/uapi/linux/userfaultfd.h | 9 +- mm/hugetlb.c | 32 +- mm/memory.c | 27 +- tools/include/uapi/linux/fs.h | 53 + tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 3 +- tools/testing/selftests/mm/config | 1 + tools/testing/selftests/mm/pagemap_ioctl.c | 1326 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 4 + 15 files changed, 2102 insertions(+), 23 deletions(-) create mode 100644 tools/testing/selftests/mm/pagemap_ioctl.c mode change 100644 => 100755 tools/testing/selftests/mm/run_vmtests.sh -- 2.39.2

2 years

2
6
0 0

[PATCH linux-next 0/3] selftests: net: udpgso_bench_rx: Fix verifty, gsosize, and packet number exceptions

by yang.yang29＠zte.com.cn

From: zhang yunkai (CGEL ZTE) <zhang.yunkai(a)zte.com.cn> 1.Fix verifty exception Executing the following command fails: bash# udpgso_bench_tx -l 4 -4 -D "$DST" bash# udpgso_bench_tx -l 4 -4 -D "$DST" -S 0 bash# udpgso_bench_rx -4 -G -S 1472 -v udpgso_bench_rx: data[1472]: len 2944, a(97) != q(113) This is because the sending buffers are not aligned by 26 bytes, and the GRO is not merged sequentially, and the receiver does not judge this situation. We think of the receiver to split the data and then validate it, just as the application actually uses this feature. 2.Fix gsosize exception Executing the following command fails: bash# udpgso_bench_tx -l 4 -4 -D "$DST" bash# udpgso_bench_tx -l 4 -4 -D "$DST" -S 0 bash# udpgso_bench_rx -4 -G -S 1472 udp rx: 15 MB/s 256 calls/s udp rx: 30 MB/s 512 calls/s udpgso_bench_rx: recv: bad gso size, got -1, expected 1472 (-1 == no gso cmsg)) IP 192.168.2.199.55238 > 192.168.2.203.8000: UDP, length 7360 IP 192.168.2.199.55238 > 192.168.2.203.8000: UDP, length 1472 IP 192.168.2.199.55238 > 192.168.2.203.8000: UDP, length 1472 IP 192.168.2.199.55238 > 192.168.2.203.8000: UDP, length 4416 IP 192.168.2.199.55238 > 192.168.2.203.8000: UDP, length 11776 IP 192.168.2.199.55238 > 192.168.2.203.8000: UDP, length 20608 recv: got one message len:1472, probably not an error. recv: got one message len:1472, probably not an error. This is due to network, NAPI, timer, etc., only one message being received. We believe that this situation should be normal. 3.Fix packet number exception bash# udpgso_bench_rx -4 -n 100 bash# udpgso_bench_tx -l 1 -4 -D "$DST" udpgso_bench_rx: wrong packet number! got 0, expected 100 This is because the packets is cleared after print. Zhang Yunkai (3): selftests: net: udpgso_bench_rx: Fix verifty exceptions selftests: net: udpgso_bench_rx: Fix gsosize exceptions selftests: net: udpgso_bench_rx: Fix packet number exceptions tools/testing/selftests/net/udpgso_bench_rx.c | 45 +++++++++++++++++++++------ 1 file changed, 35 insertions(+), 10 deletions(-) -- 2.15.2

2 years

4
8
0 0

[PATCH] kunit: Set the current KUnit context when cleaning up

by David Gow

KUnit tests run in a kthread, with the current->kunit_test pointer set to the test's context. This allows the kunit_get_current_test() and kunit_fail_current_test() macros to work. Normally, this pointer is still valid during test shutdown (i.e., the suite->exit function, and any resource cleanup). However, if the test has exited early (e.g., due to a failed assertion), the cleanup is done in the parent KUnit thread, which does not have an active context. Fix this by setting the active KUnit context for the duration of the test shutdown procedure. When the test exits normally, this does nothing. When run from the KUits previous value (probably NULL) afterwards. This should make it easier to get access to the KUnit context, particularly from within resource cleanup functions, which may, for example, need access to data in test->priv. Signed-off-by: David Gow <davidgow(a)google.com> --- This becomes useful with the current kunit_add_action() implementation, as actions do not get the KUnit context passed in by default: https://lore.kernel.org/linux-kselftest/CABVgOSmjs0wLUa4=ErkB9tH8p6A1P6N33b… I think it's probably correct anyway, though, so we should either do this, or totally rule out using kunit_get_current_test() here at all, by resetting current->kunit_test to NULL before running cleanup even in the normal case. I've only given this the most cursory testing so far (I'm not sure how much of the executor innards I want to expose to be able to actually write a proper test for it), so more eyes and/or suggestions are welcome. Cheers, -- David --- lib/kunit/test.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index e2910b261112..2d7cad249863 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -392,10 +392,21 @@ static void kunit_case_internal_cleanup(struct kunit *test) static void kunit_run_case_cleanup(struct kunit *test, struct kunit_suite *suite) { + /* + * If we're no-longer running from within the test kthread() because it failed + * or timed out, we still need the context to be okay when running exit and + * cleanup functions. + */ + struct kunit *old_current = current->kunit_test; + + current->kunit_test = test; if (suite->exit) suite->exit(test); kunit_case_internal_cleanup(test); + + /* Restore the thread's previous test context (probably NULL or test). */ + current->kunit_test = old_current; } struct kunit_try_catch_context { -- 2.40.0.634.g4ca3ef3211-goog

2 years

2
3
0 0

[PATCH 0/2] Access variable length array relaxed for integer type

by Feng zhou

From: Feng Zhou <zhoufeng.zf(a)bytedance.com> Add support for integer type of accessing variable length array. Add a selftest to check it. Feng Zhou (2): bpf: support access variable length array of integer type selftests/bpf: Add test to access integer type of variable array kernel/bpf/btf.c | 8 +++++--- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 20 +++++++++++++++++++ .../selftests/bpf/prog_tests/tracing_struct.c | 2 ++ .../selftests/bpf/progs/tracing_struct.c | 13 ++++++++++++ 4 files changed, 40 insertions(+), 3 deletions(-) -- 2.20.1

2 years

3
4
0 0

[PATCH] selftests/futex: Fix pthread_create() error handling

by tavianator＠tavianator.com

From: Tavian Barnes <tavianator(a)tavianator.com> The pthread APIs return error codes, rather than setting errno. Signed-off-by: Tavian Barnes <tavianator(a)tavianator.com> --- .../selftests/futex/functional/futex_requeue.c | 10 ++++++---- .../functional/futex_requeue_pi_mismatched_ops.c | 5 +++-- .../selftests/futex/functional/futex_wait.c | 15 +++++++++------ .../functional/futex_wait_private_mapped_file.c | 2 +- .../functional/futex_wait_uninitialized_heap.c | 2 +- .../selftests/futex/functional/futex_waitv.c | 10 ++++++---- 6 files changed, 26 insertions(+), 18 deletions(-) diff --git a/tools/testing/selftests/futex/functional/futex_requeue.c b/tools/testing/selftests/futex/functional/futex_requeue.c index 51485be6eb2f..76f672ad5263 100644 --- a/tools/testing/selftests/futex/functional/futex_requeue.c +++ b/tools/testing/selftests/futex/functional/futex_requeue.c @@ -73,8 +73,9 @@ int main(int argc, char *argv[]) /* * Requeue a waiter from f1 to f2, and wake f2. */ - if (pthread_create(&waiter[0], NULL, waiterfn, NULL)) - error("pthread_create failed\n", errno); + res = pthread_create(&waiter[0], NULL, waiterfn, NULL); + if (res) + error("pthread_create failed\n", res); usleep(WAKE_WAIT_US); @@ -105,8 +106,9 @@ int main(int argc, char *argv[]) * At futex_wake, wake INT_MAX (should be exactly 7). */ for (i = 0; i < 10; i++) { - if (pthread_create(&waiter[i], NULL, waiterfn, NULL)) - error("pthread_create failed\n", errno); + res = pthread_create(&waiter[i], NULL, waiterfn, NULL); + if (res) + error("pthread_create failed\n", res); } usleep(WAKE_WAIT_US); diff --git a/tools/testing/selftests/futex/functional/futex_requeue_pi_mismatched_ops.c b/tools/testing/selftests/futex/functional/futex_requeue_pi_mismatched_ops.c index d0a4d332ea44..e28de62bc03a 100644 --- a/tools/testing/selftests/futex/functional/futex_requeue_pi_mismatched_ops.c +++ b/tools/testing/selftests/futex/functional/futex_requeue_pi_mismatched_ops.c @@ -79,8 +79,9 @@ int main(int argc, char *argv[]) ksft_print_msg("%s: Detect mismatched requeue_pi operations\n", basename(argv[0])); - if (pthread_create(&child, NULL, blocking_child, NULL)) { - error("pthread_create\n", errno); + ret = pthread_create(&child, NULL, blocking_child, NULL); + if (ret) { + error("pthread_create\n", ret); ret = RET_ERROR; goto out; } diff --git a/tools/testing/selftests/futex/functional/futex_wait.c b/tools/testing/selftests/futex/functional/futex_wait.c index 685140d9b93d..2e06539e5bb7 100644 --- a/tools/testing/selftests/futex/functional/futex_wait.c +++ b/tools/testing/selftests/futex/functional/futex_wait.c @@ -78,8 +78,9 @@ int main(int argc, char *argv[]) /* Testing a private futex */ info("Calling private futex_wait on futex: %p\n", futex); - if (pthread_create(&waiter, NULL, waiterfn, (void *) &flags)) - error("pthread_create failed\n", errno); + res = pthread_create(&waiter, NULL, waiterfn, (void *) &flags); + if (res) + error("pthread_create failed\n", res); usleep(WAKE_WAIT_US); @@ -106,8 +107,9 @@ int main(int argc, char *argv[]) futex = shared_data; info("Calling shared (page anon) futex_wait on futex: %p\n", futex); - if (pthread_create(&waiter, NULL, waiterfn, NULL)) - error("pthread_create failed\n", errno); + res = pthread_create(&waiter, NULL, waiterfn, NULL); + if (res) + error("pthread_create failed\n", res); usleep(WAKE_WAIT_US); @@ -145,8 +147,9 @@ int main(int argc, char *argv[]) futex = shm; info("Calling shared (file backed) futex_wait on futex: %p\n", futex); - if (pthread_create(&waiter, NULL, waiterfn, NULL)) - error("pthread_create failed\n", errno); + res = pthread_create(&waiter, NULL, waiterfn, NULL); + if (res) + error("pthread_create failed\n", res); usleep(WAKE_WAIT_US); diff --git a/tools/testing/selftests/futex/functional/futex_wait_private_mapped_file.c b/tools/testing/selftests/futex/functional/futex_wait_private_mapped_file.c index fb4148f23fa3..7b4a28e8dc1a 100644 --- a/tools/testing/selftests/futex/functional/futex_wait_private_mapped_file.c +++ b/tools/testing/selftests/futex/functional/futex_wait_private_mapped_file.c @@ -100,7 +100,7 @@ int main(int argc, char **argv) basename(argv[0])); ret = pthread_create(&thr, NULL, thr_futex_wait, NULL); - if (ret < 0) { + if (ret) { fprintf(stderr, "pthread_create error\n"); ret = RET_ERROR; goto out; diff --git a/tools/testing/selftests/futex/functional/futex_wait_uninitialized_heap.c b/tools/testing/selftests/futex/functional/futex_wait_uninitialized_heap.c index ed9cd07e31c1..bf7529bd7f0d 100644 --- a/tools/testing/selftests/futex/functional/futex_wait_uninitialized_heap.c +++ b/tools/testing/selftests/futex/functional/futex_wait_uninitialized_heap.c @@ -103,7 +103,7 @@ int main(int argc, char **argv) ret = pthread_create(&thr, NULL, wait_thread, NULL); if (ret) { - error("pthread_create\n", errno); + error("pthread_create\n", ret); ret = RET_ERROR; goto out; } diff --git a/tools/testing/selftests/futex/functional/futex_waitv.c b/tools/testing/selftests/futex/functional/futex_waitv.c index a94337f677e1..d2400b3c7d0a 100644 --- a/tools/testing/selftests/futex/functional/futex_waitv.c +++ b/tools/testing/selftests/futex/functional/futex_waitv.c @@ -94,8 +94,9 @@ int main(int argc, char *argv[]) } /* Private waitv */ - if (pthread_create(&waiter, NULL, waiterfn, NULL)) - error("pthread_create failed\n", errno); + res = pthread_create(&waiter, NULL, waiterfn, NULL); + if (res) + error("pthread_create failed\n", res); usleep(WAKE_WAIT_US); @@ -127,8 +128,9 @@ int main(int argc, char *argv[]) waitv[i].__reserved = 0; } - if (pthread_create(&waiter, NULL, waiterfn, NULL)) - error("pthread_create failed\n", errno); + res = pthread_create(&waiter, NULL, waiterfn, NULL); + if (res) + error("pthread_create failed\n", res); usleep(WAKE_WAIT_US); -- 2.40.0

2 years

2
1
0 0

[PATCH v13 0/5] Implement IOCTL to get and optionally clear info about PTEs

by Muhammad Usama Anjum

*Changes in v13* - Rebase on top of next-20230414 - Give-up on using uffd_wp_range() and write new helpers, flush tlb only once *Changes in v12* - Update and other memory types to UFFD_FEATURE_WP_ASYNC - Rebaase on top of next-20230406 - Review updates *Changes in v11* - Rebase on top of next-20230307 - Base patches on UFFD_FEATURE_WP_UNPOPULATED - Do a lot of cosmetic changes and review updates - Remove ENGAGE_WP + !GET operation as it can be performed with UFFDIO_WRITEPROTECT *Changes in v10* - Add specific condition to return error if hugetlb is used with wp async - Move changes in tools/include/uapi/linux/fs.h to separate patch - Add documentation *Changes in v9:* - Correct fault resolution for userfaultfd wp async - Fix build warnings and errors which were happening on some configs - Simplify pagemap ioctl's code *Changes in v8:* - Update uffd async wp implementation - Improve PAGEMAP_IOCTL implementation *Changes in v7:* - Add uffd wp async - Update the IOCTL to use uffd under the hood instead of soft-dirty flags *Motivation* The real motivation for adding PAGEMAP_SCAN IOCTL is to emulate Windows GetWriteWatch() syscall [1]. The GetWriteWatch{} retrieves the addresses of the pages that are written to in a region of virtual memory. This syscall is used in Windows applications and games etc. This syscall is being emulated in pretty slow manner in userspace. Our purpose is to enhance the kernel such that we translate it efficiently in a better way. Currently some out of tree hack patches are being used to efficiently emulate it in some kernels. We intend to replace those with these patches. So the whole gaming on Linux can effectively get benefit from this. It means there would be tons of users of this code. CRIU use case [2] was mentioned by Andrei and Danylo: > Use cases for migrating sparse VMAs are binaries sanitized with ASAN, > MSAN or TSAN [3]. All of these sanitizers produce sparse mappings of > shadow memory [4]. Being able to migrate such binaries allows to highly > reduce the amount of work needed to identify and fix post-migration > crashes, which happen constantly. Andrei's defines the following uses of this code: * it is more granular and allows us to track changed pages more effectively. The current interface can clear dirty bits for the entire process only. In addition, reading info about pages is a separate operation. It means we must freeze the process to read information about all its pages, reset dirty bits, only then we can start dumping pages. The information about pages becomes more and more outdated, while we are processing pages. The new interface solves both these downsides. First, it allows us to read pte bits and clear the soft-dirty bit atomically. It means that CRIU will not need to freeze processes to pre-dump their memory. Second, it clears soft-dirty bits for a specified region of memory. It means CRIU will have actual info about pages to the moment of dumping them. * The new interface has to be much faster because basic page filtering is happening in the kernel. With the old interface, we have to read pagemap for each page. *Implementation Evolution (Short Summary)* From the definition of GetWriteWatch(), we feel like kernel's soft-dirty feature can be used under the hood with some additions like: * reset soft-dirty flag for only a specific region of memory instead of clearing the flag for the entire process * get and clear soft-dirty flag for a specific region atomically So we decided to use ioctl on pagemap file to read or/and reset soft-dirty flag. But using soft-dirty flag, sometimes we get extra pages which weren't even written. They had become soft-dirty because of VMA merging and VM_SOFTDIRTY flag. This breaks the definition of GetWriteWatch(). We were able to by-pass this short coming by ignoring VM_SOFTDIRTY until David reported that mprotect etc messes up the soft-dirty flag while ignoring VM_SOFTDIRTY [5]. This wasn't happening until [6] got introduced. We discussed if we can revert these patches. But we could not reach to any conclusion. So at this point, I made couple of tries to solve this whole VM_SOFTDIRTY issue by correcting the soft-dirty implementation: * [7] Correct the bug fixed wrongly back in 2014. It had potential to cause regression. We left it behind. * [8] Keep a list of soft-dirty part of a VMA across splits and merges. I got the reply don't increase the size of the VMA by 8 bytes. At this point, we left soft-dirty considering it is too much delicate and userfaultfd [9] seemed like the only way forward. From there onward, we have been basing soft-dirty emulation on userfaultfd wp feature where kernel resolves the faults itself when WP_ASYNC feature is used. It was straight forward to add WP_ASYNC feature in userfautlfd. Now we get only those pages dirty or written-to which are really written in reality. (PS There is another WP_UNPOPULATED userfautfd feature is required which is needed to avoid pre-faulting memory before write-protecting [9].) All the different masks were added on the request of CRIU devs to create interface more generic and better. [1] https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-… [2] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com [3] https://github.com/google/sanitizers [4] https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#64-bit [5] https://lore.kernel.org/all/bfcae708-db21-04b4-0bbe-712badd03071@redhat.com [6] https://lore.kernel.org/all/20220725142048.30450-1-peterx@redhat.com/ [7] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [8] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [9] https://lore.kernel.org/all/20230306213925.617814-1-peterx@redhat.com [10] https://lore.kernel.org/all/20230125144529.1630917-1-mdanylo@google.com * Original Cover letter from v8* Hello, Note: Soft-dirty pages and pages which have been written-to are synonyms. As kernel already has soft-dirty feature inside which we have given up to use, we are using written-to terminology while using UFFD async WP under the hood. This IOCTL, PAGEMAP_SCAN on pagemap file can be used to get and/or clear the info about page table entries. The following operations are supported in this ioctl: - Get the information if the pages have been written-to (PAGE_IS_WRITTEN), file mapped (PAGE_IS_FILE), present (PAGE_IS_PRESENT) or swapped (PAGE_IS_SWAPPED). - Write-protect the pages (PAGEMAP_WP_ENGAGE) to start finding which pages have been written-to. - Find pages which have been written-to and write protect the pages (atomic PAGE_IS_WRITTEN + PAGEMAP_WP_ENGAGE) It is possible to find and clear soft-dirty pages entirely in userspace. But it isn't efficient: - The mprotect and SIGSEGV handler for bookkeeping - The userfaultfd wp (synchronous) with the handler for bookkeeping Some benchmarks can be seen here[1]. This series adds features that weren't present earlier: - There is no atomic get soft-dirty/Written-to status and clear present in the kernel. - The pages which have been written-to can not be found in accurate way. (Kernel's soft-dirty PTE bit + sof_dirty VMA bit shows more soft-dirty pages than there actually are.) Historically, soft-dirty PTE bit tracking has been used in the CRIU project. The procfs interface is enough for finding the soft-dirty bit status and clearing the soft-dirty bit of all the pages of a process. We have the use case where we need to track the soft-dirty PTE bit for only specific pages on-demand. We need this tracking and clear mechanism of a region of memory while the process is running to emulate the getWriteWatch() syscall of Windows. *(Moved to using UFFD instead of soft-dirtyi feature to find pages which have been written-to from v7 patch series)*: Stop using the soft-dirty flags for finding which pages have been written to. It is too delicate and wrong as it shows more soft-dirty pages than the actual soft-dirty pages. There is no interest in correcting it [2][3] as this is how the feature was written years ago. It shouldn't be updated to changed behaviour. Peter Xu has suggested using the async version of the UFFD WP [4] as it is based inherently on the PTEs. So in this patch series, I've added a new mode to the UFFD which is asynchronous version of the write protect. When this variant of the UFFD WP is used, the page faults are resolved automatically by the kernel. The pages which have been written-to can be found by reading pagemap file (!PM_UFFD_WP). This feature can be used successfully to find which pages have been written to from the time the pages were write protected. This works just like the soft-dirty flag without showing any extra pages which aren't soft-dirty in reality. The information related to pages if the page is file mapped, present and swapped is required for the CRIU project [5][6]. The addition of the required mask, any mask, excluded mask and return masks are also required for the CRIU project [5]. The IOCTL returns the addresses of the pages which match the specific masks. The page addresses are returned in struct page_region in a compact form. The max_pages is needed to support a use case where user only wants to get a specific number of pages. So there is no need to find all the pages of interest in the range when max_pages is specified. The IOCTL returns when the maximum number of the pages are found. The max_pages is optional. If max_pages is specified, it must be equal or greater than the vec_size. This restriction is needed to handle worse case when one page_region only contains info of one page and it cannot be compacted. This is needed to emulate the Windows getWriteWatch() syscall. The patch series include the detailed selftest which can be used as an example for the uffd async wp test and PAGEMAP_IOCTL. It shows the interface usages as well. [1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora… [2] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [3] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [4] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n [5] https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com/ [6] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com/ Regards, Muhammad Usama Anjum Muhammad Usama Anjum (4): fs/proc/task_mmu: Implement IOCTL to get and optionally clear info about PTEs tools headers UAPI: Update linux/fs.h with the kernel sources mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL selftests: mm: add pagemap ioctl tests Peter Xu (1): userfaultfd: UFFD_FEATURE_WP_ASYNC Documentation/admin-guide/mm/pagemap.rst | 56 + Documentation/admin-guide/mm/userfaultfd.rst | 35 + fs/proc/task_mmu.c | 478 +++++++ fs/userfaultfd.c | 26 +- include/linux/userfaultfd_k.h | 21 +- include/uapi/linux/fs.h | 53 + include/uapi/linux/userfaultfd.h | 9 +- mm/hugetlb.c | 32 +- mm/memory.c | 27 +- tools/include/uapi/linux/fs.h | 53 + tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 3 +- tools/testing/selftests/mm/config | 1 + tools/testing/selftests/mm/pagemap_ioctl.c | 1326 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 4 + 15 files changed, 2102 insertions(+), 23 deletions(-) create mode 100644 tools/testing/selftests/mm/pagemap_ioctl.c mode change 100644 => 100755 tools/testing/selftests/mm/run_vmtests.sh -- 2.39.2

2 years

3
8
0 0

[PATCH 0/6] tools/nolibc: add stackprotector support for more architectures

by Thomas Weißschuh

Add stackprotector support for all remaining architectures, except s390. On s390 the stackprotectors are not supported in "global" mode; only "sysreg" mode which is not suppored in nolibc. The series also contains a small optimization to strace output during execution of nolibc-test. This series is based on the 20230415-nolibc-updates-4a branch of the nolibc tree. Signed-off-by: Thomas Weißschuh <linux(a)weissschuh.net> --- Thomas Weißschuh (6): selftests/nolibc: reduce syscalls during space padding tools/nolibc: riscv: add stackprotector support tools/nolibc: aarch64: add stackprotector support tools/nolibc: arm: add stackprotector support tools/nolibc: loongarch: add stackprotector support tools/nolibc: mips: add stackprotector support tools/include/nolibc/arch-aarch64.h | 7 ++++++- tools/include/nolibc/arch-arm.h | 7 ++++++- tools/include/nolibc/arch-loongarch.h | 7 ++++++- tools/include/nolibc/arch-mips.h | 8 +++++++- tools/include/nolibc/arch-riscv.h | 7 ++++++- tools/testing/selftests/nolibc/Makefile | 5 +++++ tools/testing/selftests/nolibc/nolibc-test.c | 15 +++++++++++---- 7 files changed, 47 insertions(+), 9 deletions(-) --- base-commit: e35214ea9db7477a02e67a8b412e8046534bb97c change-id: 20230408-nolibc-stackprotector-archs-42244674616e Best regards, -- Thomas Weißschuh <linux(a)weissschuh.net>

2 years

2
7
0 0

[PATCH] kselftest/arm64: Add a smoke test for ptracing hardware break/watch points

by Mark Brown

There was a report that the hardware breakpoints and watch points weren't reporting the debug architecture version as expected, they were reporting a version of 0 which is not defined in the architecture. This happens when running in a KVM guest if the host has a debug architecture version not supported by KVM, it in turn confuses GDB which rejects any debug architecture version it does not know about. Add a test that covers that situation and while we're at it reports the debug architecture version and number of slots available to aid with figuring out problems that may arise. Signed-off-by: Mark Brown <broonie(a)kernel.org> --- tools/testing/selftests/arm64/abi/ptrace.c | 32 +++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/arm64/abi/ptrace.c b/tools/testing/selftests/arm64/abi/ptrace.c index be952511af22..abe4d58d731d 100644 --- a/tools/testing/selftests/arm64/abi/ptrace.c +++ b/tools/testing/selftests/arm64/abi/ptrace.c @@ -20,7 +20,7 @@ #include "../../kselftest.h" -#define EXPECTED_TESTS 7 +#define EXPECTED_TESTS 11 #define MAX_TPIDRS 2 @@ -132,6 +132,34 @@ static void test_tpidr(pid_t child) } } +static void test_hw_debug(pid_t child, int type, const char *type_name) +{ + struct user_hwdebug_state state; + struct iovec iov; + int slots, arch, ret; + + iov.iov_len = sizeof(state); + iov.iov_base = &state; + + /* Should be able to read the values */ + ret = ptrace(PTRACE_GETREGSET, child, type, &iov); + ksft_test_result(ret == 0, "read_%s\n", type_name); + + if (ret == 0) { + /* Low 8 bits is the number of slots, next 4 bits the arch */ + slots = state.dbg_info & 0xff; + arch = (state.dbg_info >> 8) & 0xf; + + ksft_print_msg("%s version %d with %d slots\n", type_name, + arch, slots); + + /* Zero is not currently architecturally valid */ + ksft_test_result(arch, "%s_arch_set\n", type_name); + } else { + ksft_test_result_skip("%s_arch_set\n"); + } +} + static int do_child(void) { if (ptrace(PTRACE_TRACEME, -1, NULL, NULL)) @@ -207,6 +235,8 @@ static int do_parent(pid_t child) ksft_print_msg("Parent is %d, child is %d\n", getpid(), child); test_tpidr(child); + test_hw_debug(child, NT_ARM_HW_WATCH, "NT_ARM_HW_WATCH"); + test_hw_debug(child, NT_ARM_HW_BREAK, "NT_ARM_HW_BREAK"); ret = EXIT_SUCCESS; --- base-commit: e8d018dd0257f744ca50a729e3d042cf2ec9da65 change-id: 20230414-arm64-test-hw-breakpoint-83fe02f607fc Best regards, -- Mark Brown <broonie(a)kernel.org>

2 years

1
0
0 0

Re: [PATCH v8 3/3] selftests/mm: add new selftests for KSM

by David Hildenbrand

> +/* Verify that KSM can be enabled / queried with prctl. */ > +static void test_prctl(void) > +{ > + int ret; > + > + ksft_print_msg("[RUN] %s\n", __func__); > + > + ret = prctl(PR_SET_MEMORY_MERGE, 1, 0, 0, 0); > + if (ret < 0 && errno == EINVAL) { > + ksft_test_result_skip("PR_SET_MEMORY_MERGE not supported\n"); > + return; > + } else if (ret) { > + ksft_test_result_fail("PR_SET_MEMORY_MERGE=1 failed\n"); > + } Just realized we're missing a "return;" in case of the failure here. -- Thanks, David / dhildenb

2 years

2
3
0 0

Re: [PATCH v8 1/3] mm: add new api to enable ksm per process

by David Hildenbrand

> +static void __ksm_add_vma(struct vm_area_struct *vma) > +{ > + unsigned long vm_flags = vma->vm_flags; > + > + if (vm_flags & VM_MERGEABLE) > + return; > + > + if (vma_ksm_compatible(vma)) { > + vm_flags |= VM_MERGEABLE; > + vm_flags_reset(vma, vm_flags); > + } > +} We can do the following simplification on top: diff --git a/mm/ksm.c b/mm/ksm.c index 905c8edce5cf..26e7f585d65d 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2511,15 +2511,11 @@ static int ksm_scan_thread(void *nothing) static void __ksm_add_vma(struct vm_area_struct *vma) { - unsigned long vm_flags = vma->vm_flags; - - if (vm_flags & VM_MERGEABLE) + if (vma->vm_flags & VM_MERGEABLE) return; - if (vma_ksm_compatible(vma)) { - vm_flags |= VM_MERGEABLE; - vm_flags_reset(vma, vm_flags); - } + if (vma_ksm_compatible(vma)) + vm_flags_set(vma, VM_MERGEABLE); } /** -- 2.39.2 I have some patches based on your patch set (handling/testing unmerging whens setting PR_SET_MEMORY_MERGE=0. Will send out once you changes are queued. -- Thanks, David / dhildenb

2 years

2
1
0 0

Re: new order

by FASTWAYBASE.US94＠consultant.com

Dea Sir. Please see attached new order and revert to us the delivery date/ FOB also inform us payment terms. Best Regards Susan Purchase Manager.

2 years

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror