After a few years of increasing test coverage in the MPTCP selftests, we realised [1] the last version of the selftests is supposed to run on old kernels without issues.
Supporting older versions is not that easy for this MPTCP case: these selftests are often validating the internals by checking packets that are exchanged, when some MIB counters are incremented after some actions, how connections are getting opened and closed in some cases, etc. In other words, it is not limited to the socket interface between the userspace and the kernelspace. In addition, the current selftests run a lot of different sub-tests but the TAP13 protocol used in the selftests don't support sub-tests: in other words, one failure in sub-tests implies that the whole selftest is seen as failed at the end because sub-tests are not tracked. It is then important to skip sub-tests not supported by old kernels.
To minimise the modifications and reduce the complexity to support old versions, the idea is to look at external signs and skip the whole selftests or just some sub-tests before starting them.
This first part focuses on marking the different selftests as skipped if MPTCP is not even supported. That's what is done in patches 2 to 8. Patch 2/8 introduces a new file (mptcp_lib.sh) to be able to re-use some helpers in the different selftests. The first MPTCP selftest has been introduced in v5.6.
Patch 1/8 is a bit different but still linked: it modifies mptcp_join.sh selftest not to use 'cmp --bytes' which is not supported by the BusyBox implementation. It is apparently quite common to use BusyBox in CI environments. This tool is needed for a subtest introduced in v6.1.
Link: https://lore.kernel.org/stable/CA+G9fYtDGpgT4dckXD-y-N92nqUxuvue_7AtDdBcHrbO... [1] Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- Matthieu Baerts (8): selftests: mptcp: join: avoid using 'cmp --bytes' selftests: mptcp: connect: skip if MPTCP is not supported selftests: mptcp: pm nl: skip if MPTCP is not supported selftests: mptcp: join: skip if MPTCP is not supported selftests: mptcp: diag: skip if MPTCP is not supported selftests: mptcp: simult flows: skip if MPTCP is not supported selftests: mptcp: sockopt: skip if MPTCP is not supported selftests: mptcp: userspace pm: skip if MPTCP is not supported
tools/testing/selftests/net/mptcp/Makefile | 2 +- tools/testing/selftests/net/mptcp/diag.sh | 4 +++ tools/testing/selftests/net/mptcp/mptcp_connect.sh | 4 +++ tools/testing/selftests/net/mptcp/mptcp_join.sh | 17 +++++++-- tools/testing/selftests/net/mptcp/mptcp_lib.sh | 40 ++++++++++++++++++++++ tools/testing/selftests/net/mptcp/mptcp_sockopt.sh | 4 +++ tools/testing/selftests/net/mptcp/pm_netlink.sh | 4 +++ tools/testing/selftests/net/mptcp/simult_flows.sh | 4 +++ tools/testing/selftests/net/mptcp/userspace_pm.sh | 4 +++ 9 files changed, 80 insertions(+), 3 deletions(-) --- base-commit: 9b9e46aa07273ceb96866b2e812b46f1ee0b8d2f change-id: 20230528-upstream-net-20230528-mptcp-selftests-support-old-kernels-part-1-305638f4dbc0
Best regards,
BusyBox's 'cmp' command doesn't support the '--bytes' parameter.
Some CIs -- i.e. LKFT -- use BusyBox and have the mptcp_join.sh test failing [1] because their 'cmp' command doesn't support this '--bytes' option:
cmp: unrecognized option '--bytes=1024' BusyBox v1.35.0 () multi-call binary.
Usage: cmp [-ls] [-n NUM] FILE1 [FILE2]
Instead, 'head --bytes' can be used as this option is supported by BusyBox. A temporary file is needed for this operation.
Because it is apparently quite common to use BusyBox, it is certainly better to backport this fix to impacted kernels.
Fixes: 6bf41020b72b ("selftests: mptcp: update and extend fastclose test-cases") Cc: stable@vger.kernel.org Link: https://qa-reports.linaro.org/lkft/linux-mainline-master/build/v6.3-rc5-5-g1... [1] Suggested-by: Paolo Abeni pabeni@redhat.com Reviewed-by: Mat Martineau martineau@kernel.org Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- tools/testing/selftests/net/mptcp/mptcp_join.sh | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh index 26310c17b4c6..4f3fe45f8f71 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -17,6 +17,7 @@ sout="" cin="" cinfail="" cinsent="" +tmpfile="" cout="" capout="" ns1="" @@ -175,6 +176,7 @@ cleanup() { rm -f "$cin" "$cout" "$sinfail" rm -f "$sin" "$sout" "$cinsent" "$cinfail" + rm -f "$tmpfile" rm -rf $evts_ns1 $evts_ns2 cleanup_partial } @@ -383,9 +385,16 @@ check_transfer() fail_test return 1 fi - bytes="--bytes=${bytes}" + + # note: BusyBox's "cmp" command doesn't support --bytes + tmpfile=$(mktemp) + head --bytes="$bytes" "$in" > "$tmpfile" + mv "$tmpfile" "$in" + head --bytes="$bytes" "$out" > "$tmpfile" + mv "$tmpfile" "$out" + tmpfile="" fi - cmp -l "$in" "$out" ${bytes} | while read -r i a b; do + cmp -l "$in" "$out" | while read -r i a b; do local sum=$((0${a} + 0${b})) if [ $check_invert -eq 0 ] || [ $sum -ne $((0xff)) ]; then echo "[ FAIL ] $what does not match (in, out):"
Selftests are supposed to run on any kernels, including the old ones not supporting MPTCP.
A new check is then added to make sure MPTCP is supported. If not, the test stops and is marked as "skipped". Note that this check can also mark the test as failed if 'SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES' env var is set to 1: by doing that, we can make sure a test is not being skipped by mistake.
A new shared file is added here to be able to re-used the same check in the different selftests we have.
Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Fixes: 048d19d444be ("mptcp: add basic kselftest for mptcp") Cc: stable@vger.kernel.org Acked-by: Paolo Abeni pabeni@redhat.com Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- tools/testing/selftests/net/mptcp/Makefile | 2 +- tools/testing/selftests/net/mptcp/mptcp_connect.sh | 4 +++ tools/testing/selftests/net/mptcp/mptcp_lib.sh | 40 ++++++++++++++++++++++ 3 files changed, 45 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/net/mptcp/Makefile b/tools/testing/selftests/net/mptcp/Makefile index 43a723626126..7b936a926859 100644 --- a/tools/testing/selftests/net/mptcp/Makefile +++ b/tools/testing/selftests/net/mptcp/Makefile @@ -9,7 +9,7 @@ TEST_PROGS := mptcp_connect.sh pm_netlink.sh mptcp_join.sh diag.sh \
TEST_GEN_FILES = mptcp_connect pm_nl_ctl mptcp_sockopt mptcp_inq
-TEST_FILES := settings +TEST_FILES := mptcp_lib.sh settings
EXTRA_CLEAN := *.pcap
diff --git a/tools/testing/selftests/net/mptcp/mptcp_connect.sh b/tools/testing/selftests/net/mptcp/mptcp_connect.sh index a43d3e2f59bb..c1f7bac19942 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_connect.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_connect.sh @@ -1,6 +1,8 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0
+. "$(dirname "${0}")/mptcp_lib.sh" + time_start=$(date +%s)
optstring="S:R:d:e:l:r:h4cm:f:tC" @@ -141,6 +143,8 @@ cleanup() done }
+mptcp_lib_check_mptcp + ip -Version > /dev/null 2>&1 if [ $? -ne 0 ];then echo "SKIP: Could not run test without ip tool" diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing/selftests/net/mptcp/mptcp_lib.sh new file mode 100644 index 000000000000..3286536b79d5 --- /dev/null +++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh @@ -0,0 +1,40 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 + +readonly KSFT_FAIL=1 +readonly KSFT_SKIP=4 + +# SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES env var can be set when validating all +# features using the last version of the kernel and the selftests to make sure +# a test is not being skipped by mistake. +mptcp_lib_expect_all_features() { + [ "${SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES:-}" = "1" ] +} + +# $1: msg +mptcp_lib_fail_if_expected_feature() { + if mptcp_lib_expect_all_features; then + echo "ERROR: missing feature: ${*}" + exit ${KSFT_FAIL} + fi + + return 1 +} + +# $1: file +mptcp_lib_has_file() { + local f="${1}" + + if [ -f "${f}" ]; then + return 0 + fi + + mptcp_lib_fail_if_expected_feature "${f} file not found" +} + +mptcp_lib_check_mptcp() { + if ! mptcp_lib_has_file "/proc/sys/net/mptcp/enabled"; then + echo "SKIP: MPTCP support is not available" + exit ${KSFT_SKIP} + fi +}
Selftests are supposed to run on any kernels, including the old ones not supporting MPTCP.
A new check is then added to make sure MPTCP is supported. If not, the test stops and is marked as "skipped".
Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Fixes: eedbc685321b ("selftests: add PM netlink functional tests") Cc: stable@vger.kernel.org Acked-by: Paolo Abeni pabeni@redhat.com Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- tools/testing/selftests/net/mptcp/pm_netlink.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/net/mptcp/pm_netlink.sh b/tools/testing/selftests/net/mptcp/pm_netlink.sh index 89839d1ff9d8..32f7533e0919 100755 --- a/tools/testing/selftests/net/mptcp/pm_netlink.sh +++ b/tools/testing/selftests/net/mptcp/pm_netlink.sh @@ -1,6 +1,8 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0
+. "$(dirname "${0}")/mptcp_lib.sh" + ksft_skip=4 ret=0
@@ -34,6 +36,8 @@ cleanup() ip netns del $ns1 }
+mptcp_lib_check_mptcp + ip -Version > /dev/null 2>&1 if [ $? -ne 0 ];then echo "SKIP: Could not run test without ip tool"
Selftests are supposed to run on any kernels, including the old ones not supporting MPTCP.
A new check is then added to make sure MPTCP is supported. If not, the test stops and is marked as "skipped".
Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Fixes: b08fbf241064 ("selftests: add test-cases for MPTCP MP_JOIN") Cc: stable@vger.kernel.org Acked-by: Paolo Abeni pabeni@redhat.com Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- tools/testing/selftests/net/mptcp/mptcp_join.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh index 4f3fe45f8f71..96f63172b8fe 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -10,6 +10,8 @@ # because it's invoked by variable name, see how the "tests" array is used #shellcheck disable=SC2317
+. "$(dirname "${0}")/mptcp_lib.sh" + ret=0 sin="" sinfail="" @@ -137,6 +139,8 @@ cleanup_partial()
check_tools() { + mptcp_lib_check_mptcp + if ! ip -Version &> /dev/null; then echo "SKIP: Could not run test without ip tool" exit $ksft_skip
Selftests are supposed to run on any kernels, including the old ones not supporting MPTCP.
A new check is then added to make sure MPTCP is supported. If not, the test stops and is marked as "skipped".
Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Fixes: df62f2ec3df6 ("selftests/mptcp: add diag interface tests") Cc: stable@vger.kernel.org Acked-by: Paolo Abeni pabeni@redhat.com Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- tools/testing/selftests/net/mptcp/diag.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/net/mptcp/diag.sh b/tools/testing/selftests/net/mptcp/diag.sh index ef628b16fe9b..4eacdb1ab962 100755 --- a/tools/testing/selftests/net/mptcp/diag.sh +++ b/tools/testing/selftests/net/mptcp/diag.sh @@ -1,6 +1,8 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0
+. "$(dirname "${0}")/mptcp_lib.sh" + sec=$(date +%s) rndh=$(printf %x $sec)-$(mktemp -u XXXXXX) ns="ns1-$rndh" @@ -31,6 +33,8 @@ cleanup() ip netns del $ns }
+mptcp_lib_check_mptcp + ip -Version > /dev/null 2>&1 if [ $? -ne 0 ];then echo "SKIP: Could not run test without ip tool"
Selftests are supposed to run on any kernels, including the old ones not supporting MPTCP.
A new check is then added to make sure MPTCP is supported. If not, the test stops and is marked as "skipped".
Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Fixes: 1a418cb8e888 ("mptcp: simult flow self-tests") Cc: stable@vger.kernel.org Acked-by: Paolo Abeni pabeni@redhat.com Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- tools/testing/selftests/net/mptcp/simult_flows.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/testing/selftests/net/mptcp/simult_flows.sh index 9f22f7e5027d..36a3c9d92e20 100755 --- a/tools/testing/selftests/net/mptcp/simult_flows.sh +++ b/tools/testing/selftests/net/mptcp/simult_flows.sh @@ -1,6 +1,8 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0
+. "$(dirname "${0}")/mptcp_lib.sh" + sec=$(date +%s) rndh=$(printf %x $sec)-$(mktemp -u XXXXXX) ns1="ns1-$rndh" @@ -34,6 +36,8 @@ cleanup() done }
+mptcp_lib_check_mptcp + ip -Version > /dev/null 2>&1 if [ $? -ne 0 ];then echo "SKIP: Could not run test without ip tool"
Selftests are supposed to run on any kernels, including the old ones not supporting MPTCP.
A new check is then added to make sure MPTCP is supported. If not, the test stops and is marked as "skipped".
Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Fixes: dc65fe82fb07 ("selftests: mptcp: add packet mark test case") Cc: stable@vger.kernel.org Acked-by: Paolo Abeni pabeni@redhat.com Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- tools/testing/selftests/net/mptcp/mptcp_sockopt.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/net/mptcp/mptcp_sockopt.sh b/tools/testing/selftests/net/mptcp/mptcp_sockopt.sh index 1b70c0a304ce..ff5adbb9c7f2 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_sockopt.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_sockopt.sh @@ -1,6 +1,8 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0
+. "$(dirname "${0}")/mptcp_lib.sh" + ret=0 sin="" sout="" @@ -84,6 +86,8 @@ cleanup() rm -f "$sin" "$sout" }
+mptcp_lib_check_mptcp + ip -Version > /dev/null 2>&1 if [ $? -ne 0 ];then echo "SKIP: Could not run test without ip tool"
Selftests are supposed to run on any kernels, including the old ones not supporting MPTCP.
A new check is then added to make sure MPTCP is supported. If not, the test stops and is marked as "skipped".
Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368 Fixes: 259a834fadda ("selftests: mptcp: functional tests for the userspace PM type") Cc: stable@vger.kernel.org Acked-by: Paolo Abeni pabeni@redhat.com Signed-off-by: Matthieu Baerts matthieu.baerts@tessares.net --- tools/testing/selftests/net/mptcp/userspace_pm.sh | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/net/mptcp/userspace_pm.sh b/tools/testing/selftests/net/mptcp/userspace_pm.sh index b1eb7bce599d..8092399d911f 100755 --- a/tools/testing/selftests/net/mptcp/userspace_pm.sh +++ b/tools/testing/selftests/net/mptcp/userspace_pm.sh @@ -1,6 +1,10 @@ #!/bin/bash # SPDX-License-Identifier: GPL-2.0
+. "$(dirname "${0}")/mptcp_lib.sh" + +mptcp_lib_check_mptcp + ip -Version > /dev/null 2>&1 if [ $? -ne 0 ];then echo "SKIP: Cannot not run test without ip tool"
Hello:
This series was applied to netdev/net.git (main) by Paolo Abeni pabeni@redhat.com:
On Sun, 28 May 2023 19:35:25 +0200 you wrote:
After a few years of increasing test coverage in the MPTCP selftests, we realised [1] the last version of the selftests is supposed to run on old kernels without issues.
Supporting older versions is not that easy for this MPTCP case: these selftests are often validating the internals by checking packets that are exchanged, when some MIB counters are incremented after some actions, how connections are getting opened and closed in some cases, etc. In other words, it is not limited to the socket interface between the userspace and the kernelspace. In addition, the current selftests run a lot of different sub-tests but the TAP13 protocol used in the selftests don't support sub-tests: in other words, one failure in sub-tests implies that the whole selftest is seen as failed at the end because sub-tests are not tracked. It is then important to skip sub-tests not supported by old kernels.
[...]
Here is the summary with links: - [net,1/8] selftests: mptcp: join: avoid using 'cmp --bytes' https://git.kernel.org/netdev/net/c/d328fe870674 - [net,2/8] selftests: mptcp: connect: skip if MPTCP is not supported https://git.kernel.org/netdev/net/c/d83013bdf90a - [net,3/8] selftests: mptcp: pm nl: skip if MPTCP is not supported https://git.kernel.org/netdev/net/c/0f4955a40daf - [net,4/8] selftests: mptcp: join: skip if MPTCP is not supported https://git.kernel.org/netdev/net/c/715c78a82e00 - [net,5/8] selftests: mptcp: diag: skip if MPTCP is not supported https://git.kernel.org/netdev/net/c/46565acdd29f - [net,6/8] selftests: mptcp: simult flows: skip if MPTCP is not supported https://git.kernel.org/netdev/net/c/9161f21c74a1 - [net,7/8] selftests: mptcp: sockopt: skip if MPTCP is not supported https://git.kernel.org/netdev/net/c/cf6f0fda7af7 - [net,8/8] selftests: mptcp: userspace pm: skip if MPTCP is not supported https://git.kernel.org/netdev/net/c/63212608a92a
You are awesome, thank you!
linux-kselftest-mirror@lists.linaro.org
-
Matthieu Baerts -
patchwork-bot+netdevbpf＠kernel.org