The goal of this series is to use helpers from net/lib.sh with MPTCP selftests.
- Patches 1 to 4 are some clean-ups and preparation in net/lib.sh:
- Patch 1 simplifies the code handling errexit by ignoring possible errors instead of disabling errexit temporary.
- Patch 2 removes the netns from the list after having cleaned it, not to try to clean it twice.
- Patch 3 removes the 'readonly' attribute for the netns variable, to allow using the same name in local variables.
- Patch 4 removes the local 'ns' var, not to conflict with the global one it needs to setup.
- Patch 5 uses helpers from net/lib.sh to create and delete netns in MPTCP selftests.
- Patch 6 uses wait_local_port_listen helper from net/net_helper.sh.
Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org --- Geliang Tang (3): selftests: net: lib: remove 'ns' var in setup_ns selftests: mptcp: lib: use setup/cleanup_ns helpers selftests: mptcp: lib: use wait_local_port_listen helper
Matthieu Baerts (NGI0) (3): selftests: net: lib: ignore possible errors selftests: net: lib: remove ns from list after clean-up selftests: net: lib: do not set ns var as readonly
tools/testing/selftests/net/lib.sh | 55 +++++++++++++++----------- tools/testing/selftests/net/mptcp/mptcp_lib.sh | 33 +++++----------- 2 files changed, 42 insertions(+), 46 deletions(-) --- base-commit: a999973236543f0b8f6daeaa7ecba7488c3a593b change-id: 20240607-upstream-net-next-20240607-selftests-mptcp-net-lib-365e43e2e1ca
Best regards,
No need to disable errexit temporary, simply ignore the only possible and not handled error.
Reviewed-by: Geliang Tang geliang@kernel.org Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org --- tools/testing/selftests/net/lib.sh | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/tools/testing/selftests/net/lib.sh b/tools/testing/selftests/net/lib.sh index 9155c914c064..b2572aff6286 100644 --- a/tools/testing/selftests/net/lib.sh +++ b/tools/testing/selftests/net/lib.sh @@ -128,25 +128,17 @@ slowwait_for_counter() cleanup_ns() { local ns="" - local errexit=0 local ret=0
- # disable errexit temporary - if [[ $- =~ "e" ]]; then - errexit=1 - set +e - fi - for ns in "$@"; do [ -z "${ns}" ] && continue - ip netns delete "${ns}" &> /dev/null + ip netns delete "${ns}" &> /dev/null || true if ! busywait $BUSYWAIT_TIMEOUT ip netns list | grep -vq "^$ns$" &> /dev/null; then echo "Warn: Failed to remove namespace $ns" ret=1 fi done
- [ $errexit -eq 1 ] && set -e return $ret }
Instead of only appending items to the list, removing them when the netns has been deleted.
By doing that, we can make sure 'cleanup_all_ns()' is not trying to remove already deleted netns.
Reviewed-by: Geliang Tang geliang@kernel.org Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org --- tools/testing/selftests/net/lib.sh | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/net/lib.sh b/tools/testing/selftests/net/lib.sh index b2572aff6286..c7a8cfb477cc 100644 --- a/tools/testing/selftests/net/lib.sh +++ b/tools/testing/selftests/net/lib.sh @@ -125,6 +125,20 @@ slowwait_for_counter() slowwait "$timeout" until_counter_is ">= $((base + delta))" "$@" }
+remove_ns_list() +{ + local item=$1 + local ns + local ns_list=("${NS_LIST[@]}") + NS_LIST=() + + for ns in "${ns_list[@]}"; do + if [ "${ns}" != "${item}" ]; then + NS_LIST+=("${ns}") + fi + done +} + cleanup_ns() { local ns="" @@ -136,6 +150,8 @@ cleanup_ns() if ! busywait $BUSYWAIT_TIMEOUT ip netns list | grep -vq "^$ns$" &> /dev/null; then echo "Warn: Failed to remove namespace $ns" ret=1 + else + remove_ns_list "${ns}" fi done
@@ -154,17 +170,14 @@ setup_ns() local ns="" local ns_name="" local ns_list=() - local ns_exist= for ns_name in "$@"; do # Some test may setup/remove same netns multi times if unset ${ns_name} 2> /dev/null; then ns="${ns_name,,}-$(mktemp -u XXXXXX)" eval readonly ${ns_name}="$ns" - ns_exist=false else eval ns='$'${ns_name} cleanup_ns "$ns" - ns_exist=true fi
if ! ip netns add "$ns"; then @@ -173,7 +186,7 @@ setup_ns() return $ksft_skip fi ip -n "$ns" link set lo up - ! $ns_exist && ns_list+=("$ns") + ns_list+=("$ns") done NS_LIST+=("${ns_list[@]}") }
On Fri, Jun 07, 2024 at 06:31:03PM +0200, Matthieu Baerts (NGI0) wrote:
Instead of only appending items to the list, removing them when the netns has been deleted.
By doing that, we can make sure 'cleanup_all_ns()' is not trying to remove already deleted netns.
Reviewed-by: Geliang Tang geliang@kernel.org Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org
Hi Matthieu,
I like this, and I am happy to see that it has been accepted.
I do wonder if we can go a step further and use an associative array for ns_list (maybe renamed). I think this would reduce remove_ns_list to something like:
unset ns_list["$item"]
OTOH, perhaps this breaks with older versions of bash that we still care about.
...
Hi Simon,
Thank you for your reply!
On 14/06/2024 12:40, Simon Horman wrote:
On Fri, Jun 07, 2024 at 06:31:03PM +0200, Matthieu Baerts (NGI0) wrote:
Instead of only appending items to the list, removing them when the netns has been deleted.
By doing that, we can make sure 'cleanup_all_ns()' is not trying to remove already deleted netns.
I do wonder if we can go a step further and use an associative array for ns_list (maybe renamed). I think this would reduce remove_ns_list to something like:
unset ns_list["$item"]
I agree that it would ease the removal of one item -- which is not complex to deal with the new helper :) -- but do you see any other benefits?
For the moment, there is no other value to associate with, so we would do something like NS_MAP["$ns"]=1. We could link the name of the global variable, but that's not needed for the tests for the moment.
Also, I don't know if it is important, but when we will iterate over the list of netns, it will not be done following the same order items have been added into the hashmap. So we will change the order in which items are deleted.
OTOH, perhaps this breaks with older versions of bash that we still care about.
Good point. I don't have the answer, but associative arrays are starting to be quite old now :)
Cheers, Matt
Hi Matthieu,
Likewise, thanks for your response.
On Fri, Jun 14, 2024 at 04:42:38PM +0200, Matthieu Baerts wrote:
Hi Simon,
Thank you for your reply!
On 14/06/2024 12:40, Simon Horman wrote:
On Fri, Jun 07, 2024 at 06:31:03PM +0200, Matthieu Baerts (NGI0) wrote:
Instead of only appending items to the list, removing them when the netns has been deleted.
By doing that, we can make sure 'cleanup_all_ns()' is not trying to remove already deleted netns.
I do wonder if we can go a step further and use an associative array for ns_list (maybe renamed). I think this would reduce remove_ns_list to something like:
unset ns_list["$item"]
I agree that it would ease the removal of one item -- which is not complex to deal with the new helper :) -- but do you see any other benefits?
For the moment, there is no other value to associate with, so we would do something like NS_MAP["$ns"]=1. We could link the name of the global variable, but that's not needed for the tests for the moment.
Also, I don't know if it is important, but when we will iterate over the list of netns, it will not be done following the same order items have been added into the hashmap. So we will change the order in which items are deleted.
I agree that it would probably end up being a NS_MAP["$ns"]=1, i.e. a dummy value as there is no natural one to use.
I had not considered the order issue.
And yes, the benefit I see would be limited to removal. Which as you point out is not a terrible burden with the helper you added. While, OTOH, my idea adds complexity and unknowns elsewhere.
So overall, perhaps it's best left as an idea for later. As the code changes for other reasons (who knows what?) an associative array may make more sense than it does now.
OTOH, perhaps this breaks with older versions of bash that we still care about.
Good point. I don't have the answer, but associative arrays are starting to be quite old now :)
Yes, I think so too. But I also thought it was worth mentioning.
It sounds good to mark the global netns variable as 'readonly', but Bash doesn't allow the creation of local variables with the same name.
Because it looks like 'readonly' is mainly used here to check if a netns with that name has already been set, it sounds fine to check if a variable with this name has already been set instead. By doing that, we avoid having to modify helpers from MPTCP selftests using the same variable name as the one used to store the created netns name.
While at it, also avoid an unnecessary call to 'eval' to set a local variable.
Reviewed-by: Geliang Tang geliang@kernel.org Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org --- tools/testing/selftests/net/lib.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/testing/selftests/net/lib.sh b/tools/testing/selftests/net/lib.sh index c7a8cfb477cc..114b927fee25 100644 --- a/tools/testing/selftests/net/lib.sh +++ b/tools/testing/selftests/net/lib.sh @@ -172,11 +172,11 @@ setup_ns() local ns_list=() for ns_name in "$@"; do # Some test may setup/remove same netns multi times - if unset ${ns_name} 2> /dev/null; then + if [ -z "${!ns_name}" ]; then ns="${ns_name,,}-$(mktemp -u XXXXXX)" - eval readonly ${ns_name}="$ns" + eval "${ns_name}=${ns}" else - eval ns='$'${ns_name} + ns="${!ns_name}" cleanup_ns "$ns" fi
From: Geliang Tang tanggeliang@kylinos.cn
The helper setup_ns() doesn't work when a net namespace named "ns" is passed to it.
For example, in net/mptcp/diag.sh, the name of the namespace is "ns". If "setup_ns ns" is used in it, diag.sh fails with errors:
Invalid netns name "./mptcp_connect" Cannot open network namespace "10000": No such file or directory Cannot open network namespace "10000": No such file or directory
That is because "ns" is also a local variable in setup_ns, and it will not set the value for the global variable that has been giving in argument. To solve this, we could rename the variable, but it sounds better to drop it, as we can resolve the name using the variable passed in argument instead.
The other local variables -- "ns_list" and "ns_name" -- are more unlikely to conflict with existing global variables. They don't seem to be currently used in any other net selftests.
Co-developed-by: Matthieu Baerts (NGI0) matttbe@kernel.org Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org Signed-off-by: Geliang Tang tanggeliang@kylinos.cn Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org --- tools/testing/selftests/net/lib.sh | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-)
diff --git a/tools/testing/selftests/net/lib.sh b/tools/testing/selftests/net/lib.sh index 114b927fee25..915f319bcc8b 100644 --- a/tools/testing/selftests/net/lib.sh +++ b/tools/testing/selftests/net/lib.sh @@ -167,26 +167,30 @@ cleanup_all_ns() # setup_ns local remote setup_ns() { - local ns="" local ns_name="" local ns_list=() for ns_name in "$@"; do - # Some test may setup/remove same netns multi times - if [ -z "${!ns_name}" ]; then - ns="${ns_name,,}-$(mktemp -u XXXXXX)" - eval "${ns_name}=${ns}" - else - ns="${!ns_name}" - cleanup_ns "$ns" + # avoid conflicts with local var: internal error + if [ "${ns_name}" = "ns_name" ]; then + echo "Failed to setup namespace '${ns_name}': invalid name" + cleanup_ns "${ns_list[@]}" + exit $ksft_fail fi
- if ! ip netns add "$ns"; then + # Some test may setup/remove same netns multi times + if [ -z "${!ns_name}" ]; then + eval "${ns_name}=${ns_name,,}-$(mktemp -u XXXXXX)" + else + cleanup_ns "${!ns_name}" + fi + + if ! ip netns add "${!ns_name}"; then echo "Failed to create namespace $ns_name" cleanup_ns "${ns_list[@]}" return $ksft_skip fi - ip -n "$ns" link set lo up - ns_list+=("$ns") + ip -n "${!ns_name}" link set lo up + ns_list+=("${!ns_name}") done NS_LIST+=("${ns_list[@]}") }
From: Geliang Tang tanggeliang@kylinos.cn
This patch includes lib.sh into mptcp_lib.sh, uses setup_ns helper defined in lib.sh to set up namespaces in mptcp_lib_ns_init(), and uses cleanup_ns to delete namespaces in mptcp_lib_ns_exit().
Signed-off-by: Geliang Tang tanggeliang@kylinos.cn Reviewed-by: Matthieu Baerts (NGI0) matttbe@kernel.org Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org --- tools/testing/selftests/net/mptcp/mptcp_lib.sh | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing/selftests/net/mptcp/mptcp_lib.sh index 6ffa9b7a3260..d9e30516dc72 100644 --- a/tools/testing/selftests/net/mptcp/mptcp_lib.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh @@ -1,6 +1,8 @@ #! /bin/bash # SPDX-License-Identifier: GPL-2.0
+. "$(dirname "${0}")/../lib.sh" + readonly KSFT_PASS=0 readonly KSFT_FAIL=1 readonly KSFT_SKIP=4 @@ -438,17 +440,13 @@ mptcp_lib_check_tools() { }
mptcp_lib_ns_init() { - local sec rndh - - sec=$(date +%s) - rndh=$(printf %x "${sec}")-$(mktemp -u XXXXXX) + if ! setup_ns ${@}; then + mptcp_lib_pr_fail "Failed to setup namespace ${@}" + exit ${KSFT_FAIL} + fi
local netns for netns in "${@}"; do - eval "${netns}=${netns}-${rndh}" - - ip netns add "${!netns}" || exit ${KSFT_SKIP} - ip -net "${!netns}" link set lo up ip netns exec "${!netns}" sysctl -q net.mptcp.enabled=1 ip netns exec "${!netns}" sysctl -q net.ipv4.conf.all.rp_filter=0 ip netns exec "${!netns}" sysctl -q net.ipv4.conf.default.rp_filter=0 @@ -456,9 +454,10 @@ mptcp_lib_ns_init() { }
mptcp_lib_ns_exit() { + cleanup_ns "${@}" + local netns for netns in "${@}"; do - ip netns del "${netns}" rm -f /tmp/"${netns}".{nstat,out} done }
From: Geliang Tang tanggeliang@kylinos.cn
This patch includes net_helper.sh into mptcp_lib.sh, uses the helper wait_local_port_listen() defined in it to implement the similar mptcp helper. This can drop some duplicate code.
It looks like this helper from net_helper.sh was originally coming from MPTCP, but MPTCP selftests have not been updated to use it from this shared place.
Signed-off-by: Geliang Tang tanggeliang@kylinos.cn Reviewed-by: Matthieu Baerts (NGI0) matttbe@kernel.org Signed-off-by: Matthieu Baerts (NGI0) matttbe@kernel.org --- tools/testing/selftests/net/mptcp/mptcp_lib.sh | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-)
diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing/selftests/net/mptcp/mptcp_lib.sh index d9e30516dc72..194c8fc2e55a 100644 --- a/tools/testing/selftests/net/mptcp/mptcp_lib.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh @@ -2,6 +2,7 @@ # SPDX-License-Identifier: GPL-2.0
. "$(dirname "${0}")/../lib.sh" +. "$(dirname "${0}")/../net_helper.sh"
readonly KSFT_PASS=0 readonly KSFT_FAIL=1 @@ -363,20 +364,7 @@ mptcp_lib_check_transfer() {
# $1: ns, $2: port mptcp_lib_wait_local_port_listen() { - local listener_ns="${1}" - local port="${2}" - - local port_hex - port_hex="$(printf "%04X" "${port}")" - - local _ - for _ in $(seq 10); do - ip netns exec "${listener_ns}" cat /proc/net/tcp* | \ - awk "BEGIN {rc=1} {if ($2 ~ /:${port_hex}$/ && $4 ~ /0A/) \ - {rc=0; exit}} END {exit rc}" && - break - sleep 0.1 - done + wait_local_port_listen "${@}" "tcp" }
mptcp_lib_check_output() {
Hello:
This series was applied to netdev/net-next.git (main) by Jakub Kicinski kuba@kernel.org:
On Fri, 07 Jun 2024 18:31:01 +0200 you wrote:
The goal of this series is to use helpers from net/lib.sh with MPTCP selftests.
Patches 1 to 4 are some clean-ups and preparation in net/lib.sh:
- Patch 1 simplifies the code handling errexit by ignoring possible errors instead of disabling errexit temporary.
[...]
Here is the summary with links: - [net-next,1/6] selftests: net: lib: ignore possible errors https://git.kernel.org/netdev/net-next/c/7e0620bc6a5e - [net-next,2/6] selftests: net: lib: remove ns from list after clean-up https://git.kernel.org/netdev/net-next/c/92fe5670271a - [net-next,3/6] selftests: net: lib: do not set ns var as readonly https://git.kernel.org/netdev/net-next/c/577db6bd5750 - [net-next,4/6] selftests: net: lib: remove 'ns' var in setup_ns https://git.kernel.org/netdev/net-next/c/f8a2d2f874b7 - [net-next,5/6] selftests: mptcp: lib: use setup/cleanup_ns helpers https://git.kernel.org/netdev/net-next/c/f265d3119a29 - [net-next,6/6] selftests: mptcp: lib: use wait_local_port_listen helper https://git.kernel.org/netdev/net-next/c/1af3bc912eac
You are awesome, thank you!
linux-kselftest-mirror@lists.linaro.org
-
Matthieu Baerts -
Matthieu Baerts (NGI0)
-
patchwork-bot+netdevbpf＠kernel.org -
Simon Horman