Hi!
When running selftests for our subsystem in our CI we'd like all
tests to pass. Currently some tests use SKIP for cases they
expect to fail, because the kselftest_harness limits the return
codes to pass/fail/skip. XFAIL which would be a great match
here cannot be used.
Remove the no_print handling and use vfork() to run the test in
a different process than the setup. This way we don't need to
pass "failing step" via the exit code. Further clean up the exit
codes so that we can use all KSFT_* values. Rewrite the result
printing to make handling XFAIL/XPASS easier. Support tests
declaring combinations of fixture + variant they expect to fail.
Merge plan is to put it on top of -rc6 and merge into net-next.
That way others should be able to pull the patches without
any networking changes.
v4:
- rebase on top of Mickael's vfork() changes
v3: https://lore.kernel.org/all/20240220192235.2953484-1-kuba@kernel.org/
- combine multiple series
- change to "list of expected failures" rather than SKIP()-like handling
v2: https://lore.kernel.org/all/20240216002619.1999225-1-kuba@kernel.org/
- fix alignment
follow up RFC: https://lore.kernel.org/all/20240216004122.2004689-1-kuba@kernel.org/
v1: https://lore.kernel.org/all/20240213154416.422739-1-kuba@kernel.org/
Jakub Kicinski (10):
selftests: kselftest_harness: use KSFT_* exit codes
selftests: kselftest_harness: generate test name once
selftests: kselftest_harness: save full exit code in metadata
selftests: kselftest_harness: use exit code to store skip
selftests: kselftest: add ksft_test_result_code(), handling all exit
codes
selftests: kselftest_harness: print test name for SKIP
selftests: kselftest_harness: separate diagnostic message with # in
ksft_test_result_code()
selftests: kselftest_harness: let PASS / FAIL provide diagnostic
selftests: kselftest_harness: support using xfail
selftests: ip_local_port_range: use XFAIL instead of SKIP
Mickaël Salaün (2):
selftests/landlock: Redefine TEST_F() as TEST_F_FORK()
selftests/harness: Merge TEST_F_FORK() into TEST_F()
tools/testing/selftests/kselftest.h | 45 +++++
tools/testing/selftests/kselftest_harness.h | 182 +++++++++++-------
tools/testing/selftests/landlock/base_test.c | 2 +-
tools/testing/selftests/landlock/common.h | 58 +-----
tools/testing/selftests/landlock/fs_test.c | 4 +-
tools/testing/selftests/landlock/net_test.c | 4 +-
.../testing/selftests/landlock/ptrace_test.c | 7 +-
.../selftests/net/ip_local_port_range.c | 6 +-
tools/testing/selftests/net/tls.c | 2 +-
tools/testing/selftests/seccomp/seccomp_bpf.c | 9 +-
10 files changed, 178 insertions(+), 141 deletions(-)
--
2.43.2
Extend DAMON selftest-purpose sysfs wrapper to support DAMOS quota goal,
and implement a simple selftest for the feature using it.
SeongJae Park (2):
selftests/damon/_damon_sysfs: support quota goals
selftests/damon: add a test for DAMOS quota goal
tools/testing/selftests/damon/Makefile | 2 +-
tools/testing/selftests/damon/_damon_sysfs.py | 84 ++++++++++++++++++-
.../selftests/damon/damos_quota_goal.py | 77 +++++++++++++++++
3 files changed, 161 insertions(+), 2 deletions(-)
create mode 100755 tools/testing/selftests/damon/damos_quota_goal.py
base-commit: ff0a7c4126d225e56aa3e0164c53e82aabf61921
--
2.39.2
The failing hugetlb vmsplice() COW tests keep confusing people, and
having tests that have been failing for years and likely will keep failing
for years to come because nobody cares enough is rather suboptimal. Let's
mark them as XFAIL and document why fixing them is not that easy as
it would appear at first sight.
More details can be found in [1], especially around how hugetlb pages
cannot really be overcommitted, and why we don't particularly care about
these vmsplice() leaks for hugetlb -- in contrast to ordinary memory.
[1] https://lore.kernel.org/all/8b42a24d-caf0-46ef-9e15-0f88d47d2f21@redhat.com/
Cc: Andrew Morton <akpm(a)linux-foundation.org>
Cc: Muchun Song <muchun.song(a)linux.dev>
Cc: Shuah Khan <shuah(a)kernel.org>
David Hildenbrand (2):
selftests: mm: cow: flag vmsplice() hugetlb tests as XFAIL
mm/hugetlb: document why hugetlb uses folio_mapcount() for COW reuse
decisions
mm/hugetlb.c | 7 ++
tools/testing/selftests/mm/cow.c | 106 +++++++++++++++++++++----------
2 files changed, 78 insertions(+), 35 deletions(-)
--
2.44.0
Hi,
As reported by Kernel Test Robot [1], some pidfd tests fail. This is
due to the use of vfork() which introduced some side effects.
Similarly, while making it more generic, a previous commit made some
Landlock file system tests flaky, and subject to the host's file system
mount configuration.
This series fixes all these side effects by replacing vfork() with
clone3() and CLONE_VFORK, which is cleaner (no arbitrary shared memory)
and makes the Kselftest framework more robust.
I tried different approaches and I found this one to be the cleaner and
less invasive for current test cases.
This third series replace improve the clone3_vfork() helper and add
Reviewed-by tags.
I successfully ran the following tests (using TEST_F and
fork/clone/clone3) with this series:
- landlock:fs_test
- landlock:net_test
- landlock:ptrace_test
- move_mount_set_group:move_mount_set_group_test
- net/af_unix:scm_pidfd
- perf_events:remove_on_exec
- pidfd:pidfd_getfd_test
- pidfd:pidfd_setns_test
- seccomp:seccomp_bpf
- user_events:abi_test
[1] https://lore.kernel.org/oe-lkp/202403291015.1fcfa957-oliver.sang@intel.com
Previous versions:
v1: https://lore.kernel.org/r/20240426172252.1862930-1-mic@digikod.net
v2: https://lore.kernel.org/r/20240429130931.2394118-1-mic@digikod.net
Regards,
Mickaël Salaün (9):
selftests/pidfd: Fix config for pidfd_setns_test
selftests/landlock: Fix FS tests when run on a private mount point
selftests/harness: Fix fixture teardown
selftests/harness: Fix interleaved scheduling leading to race
conditions
selftests/landlock: Do not allocate memory in fixture data
selftests/harness: Constify fixture variants
selftests/pidfd: Fix wrong expectation
selftests/harness: Share _metadata between forked processes
selftests/harness: Fix vfork() side effects
tools/testing/selftests/kselftest_harness.h | 113 +++++++++++++-----
tools/testing/selftests/landlock/fs_test.c | 83 ++++++++-----
tools/testing/selftests/pidfd/config | 2 +
.../selftests/pidfd/pidfd_setns_test.c | 2 +-
4 files changed, 135 insertions(+), 65 deletions(-)
base-commit: e67572cd2204894179d89bd7b984072f19313b03
--
2.44.0
From: Willem de Bruijn <willemb(a)google.com>
Run tools/testing/selftest/net/csum.c as part of drv-net.
This binary covers multiple scenarios, based on arguments given,
for both IPv4 and IPv6:
- Accept UDP correct checksum
- Detect UDP invalid checksum
- Accept TCP correct checksum
- Detect TCP invalid checksum
- Transmit UDP: basic checksum offload
- Transmit UDP: zero checksum conversion
The test direction is reversed between receive and transmit tests, so
that the NIC under test is always the local machine.
In total this adds up to 12 testcases, with more to follow. For
conciseness, I replaced individual functions with a function factory.
It saves a lot of boilerplate, but is a little harder to follow, so
partially here as a point for discussion.
Also detect hardware offload feature availability using Ethtool
netlink and skip tests when either feature is off. This need may be
common for offload feature tests and eventually deserving of a thin
wrapper in lib.py.
Missing are the PF_PACKET based send tests ('-P'). These use
virtio_net_hdr to program hardware checksum offload. Which requires
looking up the local MAC address and (harder) the MAC of the next hop.
I'll have to give it some though how to do that robustly and where
that code would belong.
Tested on Google cloud:
make -C tools/testing/selftests/ \
TARGETS="drivers/net drivers/net/hw net" \
install INSTALL_PATH=/tmp/ksft
cd /tmp/ksft
sudo NETIF=ens4 REMOTE_TYPE=ssh \
REMOTE_ARGS="root(a)10.40.0.2" \
LOCAL_V4="10.40.0.1"
REMOTE_V4="10.40.0.2" \
LOCAL_V6="<REDACTED>" \
REMOTE_V6="<REDACTED>" \
./run_kselftest.sh -t drivers/net/hw:csum.py
TAP version 13
1..1
# timeout set to 0
# selftests: drivers/net/hw: csum.py
# KTAP version 1
# 1..12
# ok 1 csum.ipv4_rx_tcp
# ok 2 csum.ipv4_rx_tcp_invalid
# ok 3 csum.ipv4_rx_udp
# ok 4 csum.ipv4_rx_udp_invalid
# ok 5 csum.ipv4_tx_udp_csum_offload
# ok 6 csum.ipv4_tx_udp_zero_checksum
# ok 7 csum.ipv6_rx_tcp
# ok 8 csum.ipv6_rx_tcp_invalid
# ok 9 csum.ipv6_rx_udp
# ok 10 csum.ipv6_rx_udp_invalid
# ok 11 csum.ipv6_tx_udp_csum_offload
# ok 12 csum.ipv6_tx_udp_zero_checksum
# # Totals: pass:12 fail:0 xfail:0 xpass:0 skip:0 error:0
ok 1 selftests: drivers/net/hw: csum.py
Warning that for now transmit errors are not detected, as for those
the receiver runs remotely and failures with bkg are ignored.
Signed-off-by: Willem de Bruijn <willemb(a)google.com>
---
.../testing/selftests/drivers/net/hw/Makefile | 1 +
.../testing/selftests/drivers/net/hw/csum.py | 114 ++++++++++++++++++
2 files changed, 115 insertions(+)
create mode 100755 tools/testing/selftests/drivers/net/hw/csum.py
diff --git a/tools/testing/selftests/drivers/net/hw/Makefile b/tools/testing/selftests/drivers/net/hw/Makefile
index 1dd732855d76..4933d045ab66 100644
--- a/tools/testing/selftests/drivers/net/hw/Makefile
+++ b/tools/testing/selftests/drivers/net/hw/Makefile
@@ -1,6 +1,7 @@
# SPDX-License-Identifier: GPL-2.0+ OR MIT
TEST_PROGS = \
+ csum.py \
devlink_port_split.py \
ethtool.sh \
ethtool_extended_state.sh \
diff --git a/tools/testing/selftests/drivers/net/hw/csum.py b/tools/testing/selftests/drivers/net/hw/csum.py
new file mode 100755
index 000000000000..e40c510f303d
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/hw/csum.py
@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+"""Run the tools/testing/selftests/net/csum testsuite."""
+
+from os import path
+
+from lib.py import ksft_run, ksft_exit, KsftSkipEx
+from lib.py import EthtoolFamily, NetDrvEpEnv
+from lib.py import bkg, cmd, wait_port_listen
+
+def test_receive(cfg, ipv4=False, extra_args=None):
+ """Test local nic checksum receive. Remote host sends crafted packets."""
+ if not cfg.have_rx_csum:
+ raise KsftSkipEx(f"Test requires rx checksum offload on {cfg.ifname}")
+
+ if ipv4:
+ ip_args = f"-4 -S {cfg.remote_v4} -D {cfg.v4}"
+ else:
+ ip_args = f"-6 -S {cfg.remote_v6} -D {cfg.v6}"
+
+ rx_cmd = f"{cfg.bin_local} -i {cfg.ifname} -n 100 {ip_args} -r 1 -R {extra_args}"
+ tx_cmd = f"{cfg.bin_remote} -i {cfg.ifname} -n 100 {ip_args} -r 1 -T {extra_args}"
+
+ with bkg(rx_cmd, exit_wait=True):
+ wait_port_listen(34000, proto='udp')
+ cmd(tx_cmd, host=cfg.remote)
+
+
+def test_transmit(cfg, ipv4=False, extra_args=None):
+ """Test local nic checksum transmit. Remote host verifies packets."""
+ if not cfg.have_tx_csum:
+ raise KsftSkipEx(f"Test requires tx checksum offload on {cfg.ifname}")
+
+ if ipv4:
+ ip_args = f"-4 -S {cfg.v4} -D {cfg.remote_v4}"
+ else:
+ ip_args = f"-6 -S {cfg.v6} -D {cfg.remote_v6}"
+
+ # Cannot randomize input when calculating zero checksum
+ if extra_args != "-U -Z":
+ extra_args += " -r 1"
+
+ rx_cmd = f"{cfg.bin_remote} -i {cfg.ifname} -L 1 -n 100 {ip_args} -R {extra_args}"
+ tx_cmd = f"{cfg.bin_local} -i {cfg.ifname} -L 1 -n 100 {ip_args} -T {extra_args}"
+
+ with bkg(rx_cmd, host=cfg.remote, exit_wait=True):
+ wait_port_listen(34000, proto='udp', host=cfg.remote)
+ cmd(tx_cmd)
+
+
+def test_builder(name, cfg, ipv4=False, tx=False, extra_args=""):
+ """Construct specific tests from the common template.
+
+ Most tests follow the same basic pattern, differing only in
+ Direction of the test and optional flags passed to csum."""
+ def f(cfg):
+ if ipv4:
+ cfg.require_v4()
+ else:
+ cfg.require_v6()
+
+ if tx:
+ test_transmit(cfg, ipv4, extra_args)
+ else:
+ test_receive(cfg, ipv4, extra_args)
+
+ if ipv4:
+ f.__name__ = "ipv4_" + name
+ else:
+ f.__name__ = "ipv6_" + name
+ return f
+
+
+def check_nic_features(cfg) -> None:
+ """Test whether Tx and Rx checksum offload are enabled.
+
+ If the device under test has either off, then skip the relevant tests."""
+ cfg.have_tx_csum = False
+ cfg.have_rx_csum = False
+
+ ethnl = EthtoolFamily()
+ features = ethnl.features_get({"header": {"dev-index": cfg.ifindex}})
+ for f in features["active"]["bits"]["bit"]:
+ if f["name"] == "tx-checksum-ip-generic":
+ cfg.have_tx_csum = True
+ elif f["name"] == "rx-checksum":
+ cfg.have_rx_csum = True
+
+
+def main() -> None:
+ with NetDrvEpEnv(__file__, nsim_test=False) as cfg:
+ check_nic_features(cfg)
+
+ cfg.bin_local = path.abspath(path.dirname(__file__) + "/../../../net/csum")
+ cfg.bin_remote = cfg.remote.deploy(cfg.bin_local)
+
+ cases = []
+ for ipv4 in [True, False]:
+ cases.append(test_builder("rx_tcp", cfg, ipv4, False, "-t"))
+ cases.append(test_builder("rx_tcp_invalid", cfg, ipv4, False, "-t -E"))
+
+ cases.append(test_builder("rx_udp", cfg, ipv4, False, ""))
+ cases.append(test_builder("rx_udp_invalid", cfg, ipv4, False, "-E"))
+
+ cases.append(test_builder("tx_udp_csum_offload", cfg, ipv4, True, "-U"))
+ cases.append(test_builder("tx_udp_zero_checksum", cfg, ipv4, True, "-U -Z"))
+
+ ksft_run(cases=cases, args=(cfg, ))
+ ksft_exit()
+
+
+if __name__ == "__main__":
+ main()
--
2.45.0.rc1.225.g2a3ae87e7f-goog
Commit c72a870926c2 added a mutex to prevent kunit tests from running
concurrently. Unfortunately that mutex gets locked during module load
regardless of whether the module actually has any kunit tests. This
causes a problem for kunit tests that might need to load other kernel
modules (e.g. gss_krb5_test loading the camellia module).
So check to see if there are actually any tests to run before locking
the kunit_run_lock mutex.
Fixes: c72a870926c2 ("kunit: add ability to run tests after boot using debugfs")
Reported-by: Nico Pache <npache(a)redhat.com>
Signed-off-by: Scott Mayhew <smayhew(a)redhat.com>
---
lib/kunit/test.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index 1d1475578515..b8514dbb337c 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -712,6 +712,9 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_
{
unsigned int i;
+ if (num_suites == 0)
+ return 0;
+
if (!kunit_enabled() && num_suites > 0) {
pr_info("kunit: disabled\n");
return 0;
--
2.43.0
Fix spelling mistakes in the comments.
Signed-off-by: Saurav Shah <sauravshah.31(a)gmail.com>
---
tools/testing/selftests/memfd/fuse_test.c | 2 +-
tools/testing/selftests/memfd/memfd_test.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/memfd/fuse_test.c b/tools/testing/selftests/memfd/fuse_test.c
index 93798c8c5d54..dbc171a3806d 100644
--- a/tools/testing/selftests/memfd/fuse_test.c
+++ b/tools/testing/selftests/memfd/fuse_test.c
@@ -306,7 +306,7 @@ int main(int argc, char **argv)
* then the kernel did a page-replacement or canceled the read() (or
* whatever magic it did..). In that case, the memfd object is still
* all zero.
- * In case the memfd-object was *not* sealed, the read() was successfull
+ * In case the memfd-object was *not* sealed, the read() was successful
* and the memfd object must *not* be all zero.
* Note that in real scenarios, there might be a mixture of both, but
* in this test-cases, we have explicit 200ms delays which should be
diff --git a/tools/testing/selftests/memfd/memfd_test.c b/tools/testing/selftests/memfd/memfd_test.c
index 18f585684e20..95af2d78fd31 100644
--- a/tools/testing/selftests/memfd/memfd_test.c
+++ b/tools/testing/selftests/memfd/memfd_test.c
@@ -1528,7 +1528,7 @@ static void test_share_open(char *banner, char *b_suffix)
/*
* Test sharing via fork()
- * Test whether seal-modifications work as expected with forked childs.
+ * Test whether seal-modifications work as expected with forked children.
*/
static void test_share_fork(char *banner, char *b_suffix)
{
--
2.34.1
This patch series adds test coverage for BPF sockaddr hooks and their
interactions with kernel socket functions (i.e. kernel_bind(),
kernel_connect(), kernel_sendmsg(), sock_sendmsg(),
kernel_getpeername(), and kernel_getsockname()) while also rounding out
IPv4 and IPv6 sockaddr hook coverage in prog_tests/sock_addr.c.
As with v1 of this patch series, we add regression coverage for the
issues addressed by these patches,
- commit 0bdf399342c5("net: Avoid address overwrite in kernel_connect")
- commit 86a7e0b69bd5("net: prevent rewrite of msg_name in sock_sendmsg()")
- commit c889a99a21bf("net: prevent address rewrite in kernel_bind()")
- commit 01b2885d9415("net: Save and restore msg_namelen in sock_sendmsg")
but broaden the focus a bit.
In order to extend prog_tests/sock_addr.c to test these kernel
functions, we add a set of new kfuncs that wrap individual socket
operations to bpf_testmod and invoke them through set of corresponding
SYSCALL programs (progs/sock_addr_kern.c). Each test case can be
configured to use a different set of "sock_ops" depending on whether it
is testing kernel calls (kernel_bind(), kernel_connect(), etc.) or
system calls (bind(), connect(), etc.).
=======
Patches
=======
* Patch 1 fixes the sock_addr bind test program to work for big endian
architectures such as s390x.
* Patch 2 introduces the new kfuncs to bpf_testmod.
* Patch 3 introduces the BPF program which allows us to invoke these
kfuncs invividually from the test program.
* Patch 4 lays the groundwork for IPv4 and IPv6 sockaddr hook coverage
by migrating much of the environment setup logic from
bpf/test_sock_addr.sh into prog_tests/sock_addr.c and adds test cases
to cover bind4/6, connect4/6, sendmsg4/6 and recvmsg4/6 hooks.
* Patch 5 makes the set of socket operations for each test case
configurable, laying the groundwork for Patch 6.
* Patch 6 introduces two sets of sock_ops that invoke the kernel
equivalents of connect(), bind(), etc. and uses these to add coverage
for the kernel socket functions.
=======
Changes
=======
v1->v2
------
* Dropped test_progs/sock_addr_kern.c and the sock_addr_kern test module
in favor of simply expanding bpf_testmod and test_progs/sock_addr.c.
* Migrated environment setup logic from bpf/test_sock_addr.sh into
prog_tests/sock_addr.c rather than invoking the script from the test
program.
* Added kfuncs to bpf_testmod as well as the sock_addr_kern BPF program
to enable us to invoke kernel socket functions from
test_progs/sock_addr.c.
* Added test coverage for kernel socket functions to
test_progs/sock_addr.c.
Link: https://lore.kernel.org/bpf/20240329191907.1808635-1-jrife@google.com/T/#u
Jordan Rife (6):
selftests/bpf: Fix bind program for big endian systems
selftests/bpf: Implement socket kfuncs for bpf_testmod
selftests/bpf: Implement BPF programs for kernel socket operations
selftests/bpf: Add IPv4 and IPv6 sockaddr test cases
selftests/bpf: Make sock configurable for each test case
selftests/bpf: Add kernel socket operation tests
.../selftests/bpf/bpf_testmod/bpf_testmod.c | 139 +++
.../bpf/bpf_testmod/bpf_testmod_kfunc.h | 27 +
.../selftests/bpf/prog_tests/sock_addr.c | 940 +++++++++++++++---
.../testing/selftests/bpf/progs/bind4_prog.c | 18 +-
.../testing/selftests/bpf/progs/bind6_prog.c | 18 +-
tools/testing/selftests/bpf/progs/bind_prog.h | 19 +
.../selftests/bpf/progs/sock_addr_kern.c | 65 ++
7 files changed, 1077 insertions(+), 149 deletions(-)
create mode 100644 tools/testing/selftests/bpf/progs/bind_prog.h
create mode 100644 tools/testing/selftests/bpf/progs/sock_addr_kern.c
--
2.44.0.478.gd926399ef9-goog
In order to build with clang at all, in order to see these symptoms, one
must first apply Valentin Obst's build fix for LLVM [1]. Once that is
done, then when building with clang, via:
make LLVM=1 -C tools/testing/selftests
...clang emits a "format string is empty" warning. (gcc also emits a
similar warning.)
Fix by passing NULL, instead of "", for the msg argument to
ksft_test_result_code(). This removes dozens of warnings and a few
errors (some tests have -Werror set).
[1] https://lore.kernel.org/all/20240329-selftests-libmk-llvm-rfc-v1-1-2f9ed7d1…
Cc: Valentin Obst <kernel(a)valentinobst.de>
Cc: Kees Cook <keescook(a)chromium.org>
Cc: Nick Desaulniers <ndesaulniers(a)google.com>
Cc: Nathan Chancellor <nathan(a)kernel.org>
Cc: Shuah Khan <shuah(a)kernel.org>
Cc: Justin Stitt <justinstitt(a)google.com>
Cc: Bill Wendling <morbo(a)google.com>
Signed-off-by: John Hubbard <jhubbard(a)nvidia.com>
---
tools/testing/selftests/kselftest_harness.h | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h
index d98702b6955d..456b8694e678 100644
--- a/tools/testing/selftests/kselftest_harness.h
+++ b/tools/testing/selftests/kselftest_harness.h
@@ -1207,8 +1207,10 @@ void __run_test(struct __fixture_metadata *f,
else
diagnostic = "unknown";
- ksft_test_result_code(t->exit_code, test_name,
- diagnostic ? "%s" : NULL, diagnostic);
+ if (diagnostic)
+ ksft_test_result_code(t->exit_code, test_name, "%s", diagnostic);
+ else
+ ksft_test_result_code(t->exit_code, test_name, NULL);
free(test_name);
}
base-commit: 18daea77cca626f590fb140fc11e3a43c5d41354
prerequisite-patch-id: b901ece2a5b78503e2fb5480f20e304d36a0ea27
--
2.45.0