Currently the options for writing networking tests are C, bash or some mix of the two. YAML/Netlink gives us the ability to easily interface with Netlink in higher level laguages. In particular, there is a Python library already available in tree, under tools/net. Add the scaffolding which allows writing tests using this library.
The "scaffolding" is needed because the library lives under tools/net and uses YAML files from under Documentation/. So we need a small amount of glue code to find those things and add them to TEST_FILES.
This series adds both a basic SW sanity test and driver test which can be run against netdevsim or a real device. When I develop core code I usually test with netdevsim, then a real device, and then a backport to Meta's kernel. Because of the lack of integration, until now I had to throw away the (YNL-based) test script and netdevsim code.
Running tests in tree directly:
$ ./tools/testing/selftests/net/nl_netdev.py KTAP version 1 1..2 ok 1 nl_netdev.empty_check ok 2 nl_netdev.lo_check # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
in tree via make:
$ make -C tools/testing/selftests/ TARGETS=net \ TEST_PROGS=nl_netdev.py TEST_GEN_PROGS="" run_tests [ ... ]
and installed externally, all seem to work:
$ make -C tools/testing/selftests/ TARGETS=net \ install INSTALL_PATH=/tmp/ksft-net $ /tmp/ksft-net/run_kselftest.sh -t net:nl_netdev.py [ ... ]
For driver tests I followed the lead of net/forwarding and get the device name from env and/or a config file.
Jakub Kicinski (7): netlink: specs: define ethtool header flags tools: ynl: copy netlink error to NlError selftests: net: add scaffolding for Netlink tests in Python selftests: nl_netdev: add a trivial Netlink netdev test netdevsim: report stats by default, like a real device selftests: drivers: add scaffolding for Netlink tests in Python testing: net-drv: add a driver test for stats reporting
Documentation/netlink/specs/ethtool.yaml | 5 + drivers/net/netdevsim/ethtool.c | 11 ++ drivers/net/netdevsim/netdev.c | 45 +++++++ tools/net/ynl/lib/ynl.py | 3 +- tools/testing/selftests/Makefile | 8 ++ tools/testing/selftests/drivers/net/Makefile | 7 ++ .../testing/selftests/drivers/net/README.rst | 30 +++++ .../selftests/drivers/net/lib/py/__init__.py | 17 +++ .../selftests/drivers/net/lib/py/env.py | 41 ++++++ tools/testing/selftests/drivers/net/stats.py | 85 +++++++++++++ tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/lib/Makefile | 8 ++ .../testing/selftests/net/lib/py/__init__.py | 7 ++ tools/testing/selftests/net/lib/py/consts.py | 9 ++ tools/testing/selftests/net/lib/py/ksft.py | 96 ++++++++++++++ tools/testing/selftests/net/lib/py/nsim.py | 118 ++++++++++++++++++ tools/testing/selftests/net/lib/py/utils.py | 47 +++++++ tools/testing/selftests/net/lib/py/ynl.py | 49 ++++++++ tools/testing/selftests/net/nl_netdev.py | 24 ++++ 19 files changed, 610 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/drivers/net/Makefile create mode 100644 tools/testing/selftests/drivers/net/README.rst create mode 100644 tools/testing/selftests/drivers/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/drivers/net/lib/py/env.py create mode 100755 tools/testing/selftests/drivers/net/stats.py create mode 100644 tools/testing/selftests/net/lib/Makefile create mode 100644 tools/testing/selftests/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/net/lib/py/consts.py create mode 100644 tools/testing/selftests/net/lib/py/ksft.py create mode 100644 tools/testing/selftests/net/lib/py/nsim.py create mode 100644 tools/testing/selftests/net/lib/py/utils.py create mode 100644 tools/testing/selftests/net/lib/py/ynl.py create mode 100755 tools/testing/selftests/net/nl_netdev.py
When interfacing with the ethtool commands it's handy to be able to use the names of the flags. Example:
ethnl.pause_get({"header": {"dev-index": cfg.ifindex, "flags": {'stats'}}})
Note that not all commands accept all the flags, but the meaning of the bits does not change command to command.
Signed-off-by: Jakub Kicinski kuba@kernel.org --- Documentation/netlink/specs/ethtool.yaml | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/Documentation/netlink/specs/ethtool.yaml b/Documentation/netlink/specs/ethtool.yaml index 197208f419dc..616bb3eb545f 100644 --- a/Documentation/netlink/specs/ethtool.yaml +++ b/Documentation/netlink/specs/ethtool.yaml @@ -16,6 +16,10 @@ doc: Partial family for Ethtool Netlink. name: stringset type: enum entries: [] + - + name: header-flags + type: flags + entries: [ compact-bitsets, omit-reply, stats ]
attribute-sets: - @@ -30,6 +34,7 @@ doc: Partial family for Ethtool Netlink. - name: flags type: u32 + enum: header-flags
- name: bitset-bit
Typing e.nl_msg.error when processing exception is a bit tedious and counter-intuitive. Set a local .error member to the positive value of the netlink level error.
Signed-off-by: Jakub Kicinski kuba@kernel.org --- CC: donald.hunter@gmail.com CC: jiri@resnulli.us --- tools/net/ynl/lib/ynl.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/net/ynl/lib/ynl.py b/tools/net/ynl/lib/ynl.py index 82d3c98067aa..b30210f537f7 100644 --- a/tools/net/ynl/lib/ynl.py +++ b/tools/net/ynl/lib/ynl.py @@ -100,9 +100,10 @@ from .nlspec import SpecFamily class NlError(Exception): def __init__(self, nl_msg): self.nl_msg = nl_msg + self.error = -nl_msg.error
def __str__(self): - return f"Netlink error: {os.strerror(-self.nl_msg.error)}\n{self.nl_msg}" + return f"Netlink error: {os.strerror(self.error)}\n{self.nl_msg}"
class ConfigError(Exception):
Add glue code for accessing the YNL library which lives under tools/net and YAML spec files from under Documentation/. Automatically figure out if tests are run in tree or not. Since we'll want to use this library both from net and drivers/net test targets make the library a target as well, and automatically include it when net or drivers/net are included. Making net/lib a target ensures that we end up with only one copy of it, and saves us some path guessing.
Add a tiny bit of formatting support to be able to output KTAP from the start.
Signed-off-by: Jakub Kicinski kuba@kernel.org --- CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org --- tools/testing/selftests/Makefile | 7 ++ tools/testing/selftests/net/lib/Makefile | 8 ++ .../testing/selftests/net/lib/py/__init__.py | 6 ++ tools/testing/selftests/net/lib/py/consts.py | 9 ++ tools/testing/selftests/net/lib/py/ksft.py | 96 +++++++++++++++++++ tools/testing/selftests/net/lib/py/utils.py | 47 +++++++++ tools/testing/selftests/net/lib/py/ynl.py | 49 ++++++++++ 7 files changed, 222 insertions(+) create mode 100644 tools/testing/selftests/net/lib/Makefile create mode 100644 tools/testing/selftests/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/net/lib/py/consts.py create mode 100644 tools/testing/selftests/net/lib/py/ksft.py create mode 100644 tools/testing/selftests/net/lib/py/utils.py create mode 100644 tools/testing/selftests/net/lib/py/ynl.py
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index e1504833654d..0cffdfb4b116 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -116,6 +116,13 @@ TARGETS += zram TARGETS_HOTPLUG = cpu-hotplug TARGETS_HOTPLUG += memory-hotplug
+# Networking tests want the net/lib target, include it automatically +ifneq ($(filter net ,$(TARGETS)),) +ifeq ($(filter net/lib,$(TARGETS)),) + override TARGETS := $(TARGETS) net/lib +endif +endif + # User can optionally provide a TARGETS skiplist. By default we skip # BPF since it has cutting edge build time dependencies which require # more effort to install. diff --git a/tools/testing/selftests/net/lib/Makefile b/tools/testing/selftests/net/lib/Makefile new file mode 100644 index 000000000000..5730682aeffb --- /dev/null +++ b/tools/testing/selftests/net/lib/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_FILES := ../../../../../Documentation/netlink/s* +TEST_FILES += ../../../../net/* + +TEST_INCLUDES := $(wildcard py/*.py) + +include ../../lib.mk diff --git a/tools/testing/selftests/net/lib/py/__init__.py b/tools/testing/selftests/net/lib/py/__init__.py new file mode 100644 index 000000000000..81a8d14b68f0 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/__init__.py @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 + +from .ksft import * +from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily +from .consts import KSRC +from .utils import * diff --git a/tools/testing/selftests/net/lib/py/consts.py b/tools/testing/selftests/net/lib/py/consts.py new file mode 100644 index 000000000000..f518ce79d82c --- /dev/null +++ b/tools/testing/selftests/net/lib/py/consts.py @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0 + +import sys +from pathlib import Path + +KSFT_DIR = (Path(__file__).parent / "../../..").resolve() +KSRC = (Path(__file__).parent / "../../../../../..").resolve() + +KSFT_MAIN_NAME = Path(sys.argv[0]).with_suffix("").name diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py new file mode 100644 index 000000000000..7c296fe5e438 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/ksft.py @@ -0,0 +1,96 @@ +# SPDX-License-Identifier: GPL-2.0 + +import builtins +from .consts import KSFT_MAIN_NAME + +KSFT_RESULT = None + + +class KsftSkipEx(Exception): + pass + + +class KsftXfailEx(Exception): + pass + + +def ksft_pr(*objs, **kwargs): + print("#", *objs, **kwargs) + + +def ksft_eq(a, b, comment=""): + global KSFT_RESULT + if a != b: + KSFT_RESULT = False + ksft_pr("Check failed", a, "!=", b, comment) + + +def ksft_true(a, comment=""): + global KSFT_RESULT + if not a: + KSFT_RESULT = False + ksft_pr("Check failed", a, "does not eval to True", comment) + + +def ksft_in(a, b, comment=""): + global KSFT_RESULT + if a not in b: + KSFT_RESULT = False + ksft_pr("Check failed", a, "not in", b, comment) + + +def ksft_ge(a, b, comment=""): + global KSFT_RESULT + if a < b: + KSFT_RESULT = False + ksft_pr("Check failed", a, "<", b, comment) + + +def ktap_result(ok, cnt=1, case="", comment=""): + res = "" + if not ok: + res += "not " + res += "ok " + res += str(cnt) + " " + res += KSFT_MAIN_NAME + if case: + res += "." + str(case.__name__) + if comment: + res += " # " + comment + print(res) + + +def ksft_run(cases): + totals = {"pass": 0, "fail": 0, "skip": 0, "xfail": 0} + + print("KTAP version 1") + print("1.." + str(len(cases))) + + global KSFT_RESULT + cnt = 0 + for case in cases: + KSFT_RESULT = True + cnt += 1 + try: + case() + except KsftSkipEx as e: + ktap_result(True, cnt, case, comment="SKIP " + str(e)) + totals['skip'] += 1 + continue + except KsftXfailEx as e: + ktap_result(True, cnt, case, comment="XFAIL " + str(e)) + totals['xfail'] += 1 + continue + except Exception as e: + for line in str(e).split('\n'): + ksft_pr("Exception|", line) + ktap_result(False, cnt, case) + totals['fail'] += 1 + continue + + ktap_result(KSFT_RESULT, cnt, case) + totals['pass'] += 1 + + print( + f"# Totals: pass:{totals['pass']} fail:{totals['fail']} xfail:{totals['xfail']} xpass:0 skip:{totals['skip']} error:0" + ) diff --git a/tools/testing/selftests/net/lib/py/utils.py b/tools/testing/selftests/net/lib/py/utils.py new file mode 100644 index 000000000000..f0d425731fd4 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/utils.py @@ -0,0 +1,47 @@ +# SPDX-License-Identifier: GPL-2.0 + +import json as _json +import subprocess + +class cmd: + def __init__(self, comm, shell=True, fail=True, ns=None, background=False): + if ns: + if isinstance(ns, NetNS): + ns = ns.name + comm = f'ip netns exec {ns} ' + comm + + self.stdout = None + self.stderr = None + self.ret = None + + self.comm = comm + self.proc = subprocess.Popen(comm, shell=shell, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + if not background: + self.process(terminate=False, fail=fail) + + def process(self, terminate=True, fail=None): + if terminate: + self.proc.terminate() + stdout, stderr = self.proc.communicate() + self.stdout = stdout.decode("utf-8") + self.stderr = stderr.decode("utf-8") + self.proc.stdout.close() + self.proc.stderr.close() + self.ret = self.proc.returncode + + if self.proc.returncode != 0 and fail: + if len(stderr) > 0 and stderr[-1] == "\n": + stderr = stderr[:-1] + raise Exception("Command failed: %s\n%s" % (self.proc.args, stderr)) + + +def ip(args, json=None, ns=None): + cmd_str = "ip " + if json: + cmd_str += '-j ' + cmd_str += args + cmd_obj = cmd(cmd_str, ns=ns) + if json: + return _json.loads(cmd_obj.stdout) + return cmd_obj diff --git a/tools/testing/selftests/net/lib/py/ynl.py b/tools/testing/selftests/net/lib/py/ynl.py new file mode 100644 index 000000000000..298bbc6b93c5 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/ynl.py @@ -0,0 +1,49 @@ +# SPDX-License-Identifier: GPL-2.0 + +import sys +from pathlib import Path +from .consts import KSRC, KSFT_DIR +from .ksft import ksft_pr, ktap_result + +# Resolve paths +try: + if (KSFT_DIR / "kselftest-list.txt").exists(): + # Running in "installed" selftests + tools_full_path = KSFT_DIR + SPEC_PATH = KSFT_DIR / "net/lib/specs" + + sys.path.append(tools_full_path.as_posix()) + from net.lib.ynl.lib import YnlFamily, NlError + else: + # Running in tree + tools_full_path = Path(KSRC) / "tools" + SPEC_PATH = Path(KSRC) / "Documentation/netlink/specs" + + sys.path.append(tools_full_path.as_posix()) + from net.ynl.lib import YnlFamily, NlError +except ModuleNotFoundError as e: + ksft_pr("Failed importing `ynl` library from kernel sources") + ksft_pr(str(e)) + ktap_result(True, comment="SKIP") + sys.exit(4) + +# +# Wrapper classes, loading the right specs +# Set schema='' to avoid jsonschema validation, it's slow +# +class EthtoolFamily(YnlFamily): + def __init__(self): + super().__init__((SPEC_PATH / Path('ethtool.yaml')).as_posix(), + schema='') + + +class RtnlFamily(YnlFamily): + def __init__(self): + super().__init__((SPEC_PATH / Path('rt_link.yaml')).as_posix(), + schema='') + + +class NetdevFamily(YnlFamily): + def __init__(self): + super().__init__((SPEC_PATH / Path('netdev.yaml')).as_posix(), + schema='')
Jakub Kicinski kuba@kernel.org writes:
Add glue code for accessing the YNL library which lives under tools/net and YAML spec files from under Documentation/. Automatically figure out if tests are run in tree or not. Since we'll want to use this library both from net and drivers/net test targets make the library a target as well, and automatically include it when net or drivers/net are included. Making net/lib a target ensures that we end up with only one copy of it, and saves us some path guessing.
Add a tiny bit of formatting support to be able to output KTAP from the start.
Signed-off-by: Jakub Kicinski kuba@kernel.org
CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org
tools/testing/selftests/Makefile | 7 ++ tools/testing/selftests/net/lib/Makefile | 8 ++ .../testing/selftests/net/lib/py/__init__.py | 6 ++ tools/testing/selftests/net/lib/py/consts.py | 9 ++ tools/testing/selftests/net/lib/py/ksft.py | 96 +++++++++++++++++++ tools/testing/selftests/net/lib/py/utils.py | 47 +++++++++ tools/testing/selftests/net/lib/py/ynl.py | 49 ++++++++++ 7 files changed, 222 insertions(+) create mode 100644 tools/testing/selftests/net/lib/Makefile create mode 100644 tools/testing/selftests/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/net/lib/py/consts.py create mode 100644 tools/testing/selftests/net/lib/py/ksft.py create mode 100644 tools/testing/selftests/net/lib/py/utils.py create mode 100644 tools/testing/selftests/net/lib/py/ynl.py
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index e1504833654d..0cffdfb4b116 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -116,6 +116,13 @@ TARGETS += zram TARGETS_HOTPLUG = cpu-hotplug TARGETS_HOTPLUG += memory-hotplug +# Networking tests want the net/lib target, include it automatically +ifneq ($(filter net ,$(TARGETS)),) +ifeq ($(filter net/lib,$(TARGETS)),)
- override TARGETS := $(TARGETS) net/lib
+endif +endif
# User can optionally provide a TARGETS skiplist. By default we skip # BPF since it has cutting edge build time dependencies which require # more effort to install. diff --git a/tools/testing/selftests/net/lib/Makefile b/tools/testing/selftests/net/lib/Makefile new file mode 100644 index 000000000000..5730682aeffb --- /dev/null +++ b/tools/testing/selftests/net/lib/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0
+TEST_FILES := ../../../../../Documentation/netlink/s* +TEST_FILES += ../../../../net/*
+TEST_INCLUDES := $(wildcard py/*.py)
+include ../../lib.mk diff --git a/tools/testing/selftests/net/lib/py/__init__.py b/tools/testing/selftests/net/lib/py/__init__.py new file mode 100644 index 000000000000..81a8d14b68f0 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/__init__.py @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0
+from .ksft import * +from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily +from .consts import KSRC +from .utils import * diff --git a/tools/testing/selftests/net/lib/py/consts.py b/tools/testing/selftests/net/lib/py/consts.py new file mode 100644 index 000000000000..f518ce79d82c --- /dev/null +++ b/tools/testing/selftests/net/lib/py/consts.py @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0
+import sys +from pathlib import Path
+KSFT_DIR = (Path(__file__).parent / "../../..").resolve() +KSRC = (Path(__file__).parent / "../../../../../..").resolve()
+KSFT_MAIN_NAME = Path(sys.argv[0]).with_suffix("").name diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py new file mode 100644 index 000000000000..7c296fe5e438 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/ksft.py @@ -0,0 +1,96 @@ +# SPDX-License-Identifier: GPL-2.0
+import builtins +from .consts import KSFT_MAIN_NAME
+KSFT_RESULT = None
+class KsftSkipEx(Exception):
- pass
+class KsftXfailEx(Exception):
- pass
+def ksft_pr(*objs, **kwargs):
- print("#", *objs, **kwargs)
+def ksft_eq(a, b, comment=""):
- global KSFT_RESULT
- if a != b:
KSFT_RESULT = Falseksft_pr("Check failed", a, "!=", b, comment)+def ksft_true(a, comment=""):
- global KSFT_RESULT
- if not a:
KSFT_RESULT = Falseksft_pr("Check failed", a, "does not eval to True", comment)+def ksft_in(a, b, comment=""):
- global KSFT_RESULT
- if a not in b:
KSFT_RESULT = Falseksft_pr("Check failed", a, "not in", b, comment)+def ksft_ge(a, b, comment=""):
- global KSFT_RESULT
- if a < b:
KSFT_RESULT = False
Hmm, instead of this global KSFT_RESULT business, have you considered adding and raising an XsftFailEx, like for the other outcomes? We need to use KSFT_RESULT-like approach in bash tests, because, well, bash.
Doing it all through exceptions likely requires consistent use of context managers for resource clean-up. But if we do, we'll get guaranteed cleanups as well. I see that you use __del__ and explicit "finally: del cfg" later on, which is exactly the sort of lifetime management boilerplate that context managers encapsulate.
This stuff is going to get cut'n'pasted around, and I worry we'll end up with a mess of mutable globals and forgotten cleanups if the right patterns are not introduced early on.
ksft_pr("Check failed", a, "<", b, comment)+def ktap_result(ok, cnt=1, case="", comment=""):
- res = ""
- if not ok:
res += "not "- res += "ok "
- res += str(cnt) + " "
- res += KSFT_MAIN_NAME
- if case:
res += "." + str(case.__name__)- if comment:
res += " # " + comment- print(res)
+def ksft_run(cases):
- totals = {"pass": 0, "fail": 0, "skip": 0, "xfail": 0}
- print("KTAP version 1")
- print("1.." + str(len(cases)))
- global KSFT_RESULT
- cnt = 0
- for case in cases:
KSFT_RESULT = Truecnt += 1try:case()except KsftSkipEx as e:ktap_result(True, cnt, case, comment="SKIP " + str(e))totals['skip'] += 1continueexcept KsftXfailEx as e:ktap_result(True, cnt, case, comment="XFAIL " + str(e))totals['xfail'] += 1continueexcept Exception as e:for line in str(e).split('\n'):ksft_pr("Exception|", line)ktap_result(False, cnt, case)totals['fail'] += 1continuektap_result(KSFT_RESULT, cnt, case)totals['pass'] += 1- print(
f"# Totals: pass:{totals['pass']} fail:{totals['fail']} xfail:{totals['xfail']} xpass:0 skip:{totals['skip']} error:0"- )
diff --git a/tools/testing/selftests/net/lib/py/utils.py b/tools/testing/selftests/net/lib/py/utils.py new file mode 100644 index 000000000000..f0d425731fd4 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/utils.py @@ -0,0 +1,47 @@ +# SPDX-License-Identifier: GPL-2.0
+import json as _json +import subprocess
+class cmd:
- def __init__(self, comm, shell=True, fail=True, ns=None, background=False):
if ns:if isinstance(ns, NetNS):ns = ns.namecomm = f'ip netns exec {ns} ' + commself.stdout = Noneself.stderr = Noneself.ret = Noneself.comm = commself.proc = subprocess.Popen(comm, shell=shell, stdout=subprocess.PIPE,stderr=subprocess.PIPE)if not background:self.process(terminate=False, fail=fail)- def process(self, terminate=True, fail=None):
if terminate:self.proc.terminate()stdout, stderr = self.proc.communicate()self.stdout = stdout.decode("utf-8")self.stderr = stderr.decode("utf-8")self.proc.stdout.close()self.proc.stderr.close()self.ret = self.proc.returncodeif self.proc.returncode != 0 and fail:if len(stderr) > 0 and stderr[-1] == "\n":stderr = stderr[:-1]raise Exception("Command failed: %s\n%s" % (self.proc.args, stderr))+def ip(args, json=None, ns=None):
- cmd_str = "ip "
- if json:
cmd_str += '-j '- cmd_str += args
- cmd_obj = cmd(cmd_str, ns=ns)
- if json:
return _json.loads(cmd_obj.stdout)- return cmd_obj
diff --git a/tools/testing/selftests/net/lib/py/ynl.py b/tools/testing/selftests/net/lib/py/ynl.py new file mode 100644 index 000000000000..298bbc6b93c5 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/ynl.py @@ -0,0 +1,49 @@ +# SPDX-License-Identifier: GPL-2.0
+import sys +from pathlib import Path +from .consts import KSRC, KSFT_DIR +from .ksft import ksft_pr, ktap_result
+# Resolve paths +try:
- if (KSFT_DIR / "kselftest-list.txt").exists():
# Running in "installed" selfteststools_full_path = KSFT_DIRSPEC_PATH = KSFT_DIR / "net/lib/specs"sys.path.append(tools_full_path.as_posix())from net.lib.ynl.lib import YnlFamily, NlError- else:
# Running in treetools_full_path = Path(KSRC) / "tools"SPEC_PATH = Path(KSRC) / "Documentation/netlink/specs"
I think KSRC is already Path? So it should be enough to just say KSRC / "tools", like above with KSFT_DIR?
sys.path.append(tools_full_path.as_posix())from net.ynl.lib import YnlFamily, NlError+except ModuleNotFoundError as e:
- ksft_pr("Failed importing `ynl` library from kernel sources")
- ksft_pr(str(e))
- ktap_result(True, comment="SKIP")
- sys.exit(4)
+# +# Wrapper classes, loading the right specs +# Set schema='' to avoid jsonschema validation, it's slow +# +class EthtoolFamily(YnlFamily):
- def __init__(self):
super().__init__((SPEC_PATH / Path('ethtool.yaml')).as_posix(),schema='')+class RtnlFamily(YnlFamily):
- def __init__(self):
super().__init__((SPEC_PATH / Path('rt_link.yaml')).as_posix(),schema='')+class NetdevFamily(YnlFamily):
- def __init__(self):
super().__init__((SPEC_PATH / Path('netdev.yaml')).as_posix(),schema='')
On Tue, 2 Apr 2024 17:53:41 +0200 Petr Machata wrote:
+def ksft_ge(a, b, comment=""):
- global KSFT_RESULT
- if a < b:
KSFT_RESULT = FalseHmm, instead of this global KSFT_RESULT business, have you considered adding and raising an XsftFailEx, like for the other outcomes? We need to use KSFT_RESULT-like approach in bash tests, because, well, bash.
Doing it all through exceptions likely requires consistent use of context managers for resource clean-up. But if we do, we'll get guaranteed cleanups as well. I see that you use __del__ and explicit "finally: del cfg" later on, which is exactly the sort of lifetime management boilerplate that context managers encapsulate.
This stuff is going to get cut'n'pasted around, and I worry we'll end up with a mess of mutable globals and forgotten cleanups if the right patterns are not introduced early on.
I wanted to support the semantics which the C kselftest harness has, by which I mean EXPECT and ASSERT. The helpers don't raise, just record the failure and keep going. ASSERT semantics are provided by the exceptions.
I thought it may be easier to follow and write correct code if we raise ASSERTS explicitly in the test, rather than in the helpers. I mean - if the programmer has to type in "raise" they are more likely to realize they need to also add cleanup.
But TBH I'm happy to be persuaded otherwise, I couldn't find a strong reason to do it one way or the other. I have tried to integrate with unittest and that wasn't great (I have one huge test I need to port back now). I don't know if pytest is better, but I decided that we should probably roll our own. What "our own" exactly is I don't have strong opinion.
On 2024-04-01 18:05, Jakub Kicinski wrote:
Add glue code for accessing the YNL library which lives under tools/net and YAML spec files from under Documentation/. Automatically figure out if tests are run in tree or not. Since we'll want to use this library both from net and drivers/net test targets make the library a target as well, and automatically include it when net or drivers/net are included. Making net/lib a target ensures that we end up with only one copy of it, and saves us some path guessing.
Add a tiny bit of formatting support to be able to output KTAP from the start.
Signed-off-by: Jakub Kicinski kuba@kernel.org
CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org
tools/testing/selftests/Makefile | 7 ++ tools/testing/selftests/net/lib/Makefile | 8 ++ .../testing/selftests/net/lib/py/__init__.py | 6 ++ tools/testing/selftests/net/lib/py/consts.py | 9 ++ tools/testing/selftests/net/lib/py/ksft.py | 96 +++++++++++++++++++ tools/testing/selftests/net/lib/py/utils.py | 47 +++++++++ tools/testing/selftests/net/lib/py/ynl.py | 49 ++++++++++ 7 files changed, 222 insertions(+) create mode 100644 tools/testing/selftests/net/lib/Makefile create mode 100644 tools/testing/selftests/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/net/lib/py/consts.py create mode 100644 tools/testing/selftests/net/lib/py/ksft.py create mode 100644 tools/testing/selftests/net/lib/py/utils.py create mode 100644 tools/testing/selftests/net/lib/py/ynl.py
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index e1504833654d..0cffdfb4b116 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -116,6 +116,13 @@ TARGETS += zram TARGETS_HOTPLUG = cpu-hotplug TARGETS_HOTPLUG += memory-hotplug +# Networking tests want the net/lib target, include it automatically +ifneq ($(filter net ,$(TARGETS)),) +ifeq ($(filter net/lib,$(TARGETS)),)
style nit: consistency in spacing
- override TARGETS := $(TARGETS) net/lib
+endif +endif
# User can optionally provide a TARGETS skiplist. By default we skip # BPF since it has cutting edge build time dependencies which require # more effort to install. diff --git a/tools/testing/selftests/net/lib/Makefile b/tools/testing/selftests/net/lib/Makefile new file mode 100644 index 000000000000..5730682aeffb --- /dev/null +++ b/tools/testing/selftests/net/lib/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0
+TEST_FILES := ../../../../../Documentation/netlink/s* +TEST_FILES += ../../../../net/*
+TEST_INCLUDES := $(wildcard py/*.py)
+include ../../lib.mk diff --git a/tools/testing/selftests/net/lib/py/__init__.py b/tools/testing/selftests/net/lib/py/__init__.py new file mode 100644 index 000000000000..81a8d14b68f0 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/__init__.py @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0
+from .ksft import * +from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily +from .consts import KSRC +from .utils import *
style nit: sort alphabetically
diff --git a/tools/testing/selftests/net/lib/py/consts.py b/tools/testing/selftests/net/lib/py/consts.py new file mode 100644 index 000000000000..f518ce79d82c --- /dev/null +++ b/tools/testing/selftests/net/lib/py/consts.py @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0
+import sys +from pathlib import Path
+KSFT_DIR = (Path(__file__).parent / "../../..").resolve() +KSRC = (Path(__file__).parent / "../../../../../..").resolve()
+KSFT_MAIN_NAME = Path(sys.argv[0]).with_suffix("").name diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py new file mode 100644 index 000000000000..7c296fe5e438 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/ksft.py @@ -0,0 +1,96 @@ +# SPDX-License-Identifier: GPL-2.0
+import builtins +from .consts import KSFT_MAIN_NAME
+KSFT_RESULT = None
+class KsftSkipEx(Exception):
- pass
+class KsftXfailEx(Exception):
- pass
+def ksft_pr(*objs, **kwargs):
- print("#", *objs, **kwargs)
+def ksft_eq(a, b, comment=""):
- global KSFT_RESULT
- if a != b:
KSFT_RESULT = Falseksft_pr("Check failed", a, "!=", b, comment)+def ksft_true(a, comment=""):
- global KSFT_RESULT
- if not a:
KSFT_RESULT = Falseksft_pr("Check failed", a, "does not eval to True", comment)+def ksft_in(a, b, comment=""):
- global KSFT_RESULT
- if a not in b:
KSFT_RESULT = Falseksft_pr("Check failed", a, "not in", b, comment)+def ksft_ge(a, b, comment=""):
- global KSFT_RESULT
- if a < b:
KSFT_RESULT = Falseksft_pr("Check failed", a, "<", b, comment)+def ktap_result(ok, cnt=1, case="", comment=""):
- res = ""
- if not ok:
res += "not "- res += "ok "
- res += str(cnt) + " "
- res += KSFT_MAIN_NAME
- if case:
res += "." + str(case.__name__)- if comment:
res += " # " + comment- print(res)
+def ksft_run(cases):
- totals = {"pass": 0, "fail": 0, "skip": 0, "xfail": 0}
- print("KTAP version 1")
- print("1.." + str(len(cases)))
- global KSFT_RESULT
- cnt = 0
- for case in cases:
KSFT_RESULT = Truecnt += 1try:case()except KsftSkipEx as e:ktap_result(True, cnt, case, comment="SKIP " + str(e))totals['skip'] += 1continueexcept KsftXfailEx as e:ktap_result(True, cnt, case, comment="XFAIL " + str(e))totals['xfail'] += 1continueexcept Exception as e:for line in str(e).split('\n'):ksft_pr("Exception|", line)ktap_result(False, cnt, case)totals['fail'] += 1continuektap_result(KSFT_RESULT, cnt, case)totals['pass'] += 1- print(
f"# Totals: pass:{totals['pass']} fail:{totals['fail']} xfail:{totals['xfail']} xpass:0 skip:{totals['skip']} error:0"- )
diff --git a/tools/testing/selftests/net/lib/py/utils.py b/tools/testing/selftests/net/lib/py/utils.py new file mode 100644 index 000000000000..f0d425731fd4 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/utils.py @@ -0,0 +1,47 @@ +# SPDX-License-Identifier: GPL-2.0
+import json as _json +import subprocess
+class cmd:
- def __init__(self, comm, shell=True, fail=True, ns=None, background=False):
if ns:if isinstance(ns, NetNS):ns = ns.namecomm = f'ip netns exec {ns} ' + commself.stdout = Noneself.stderr = Noneself.ret = Noneself.comm = commself.proc = subprocess.Popen(comm, shell=shell, stdout=subprocess.PIPE,stderr=subprocess.PIPE)if not background:self.process(terminate=False, fail=fail)- def process(self, terminate=True, fail=None):
if terminate:self.proc.terminate()stdout, stderr = self.proc.communicate()self.stdout = stdout.decode("utf-8")self.stderr = stderr.decode("utf-8")self.proc.stdout.close()self.proc.stderr.close()self.ret = self.proc.returncodeif self.proc.returncode != 0 and fail:if len(stderr) > 0 and stderr[-1] == "\n":stderr = stderr[:-1]raise Exception("Command failed: %s\n%s" % (self.proc.args, stderr))+def ip(args, json=None, ns=None):
- cmd_str = "ip "
- if json:
cmd_str += '-j '- cmd_str += args
- cmd_obj = cmd(cmd_str, ns=ns)
- if json:
return _json.loads(cmd_obj.stdout)- return cmd_obj
diff --git a/tools/testing/selftests/net/lib/py/ynl.py b/tools/testing/selftests/net/lib/py/ynl.py new file mode 100644 index 000000000000..298bbc6b93c5 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/ynl.py @@ -0,0 +1,49 @@ +# SPDX-License-Identifier: GPL-2.0
+import sys +from pathlib import Path +from .consts import KSRC, KSFT_DIR +from .ksft import ksft_pr, ktap_result
+# Resolve paths +try:
- if (KSFT_DIR / "kselftest-list.txt").exists():
# Running in "installed" selfteststools_full_path = KSFT_DIRSPEC_PATH = KSFT_DIR / "net/lib/specs"sys.path.append(tools_full_path.as_posix())from net.lib.ynl.lib import YnlFamily, NlError- else:
# Running in treetools_full_path = Path(KSRC) / "tools"SPEC_PATH = Path(KSRC) / "Documentation/netlink/specs"sys.path.append(tools_full_path.as_posix())from net.ynl.lib import YnlFamily, NlError+except ModuleNotFoundError as e:
- ksft_pr("Failed importing `ynl` library from kernel sources")
- ksft_pr(str(e))
- ktap_result(True, comment="SKIP")
- sys.exit(4)
+# +# Wrapper classes, loading the right specs +# Set schema='' to avoid jsonschema validation, it's slow +# +class EthtoolFamily(YnlFamily):
- def __init__(self):
super().__init__((SPEC_PATH / Path('ethtool.yaml')).as_posix(),schema='')+class RtnlFamily(YnlFamily):
- def __init__(self):
super().__init__((SPEC_PATH / Path('rt_link.yaml')).as_posix(),schema='')+class NetdevFamily(YnlFamily):
- def __init__(self):
super().__init__((SPEC_PATH / Path('netdev.yaml')).as_posix(),schema='')
Add a trivial test using YNL.
$ ./tools/testing/selftests/net/nl_netdev.py KTAP version 1 1..2 ok 1 nl_netdev.empty_check ok 2 nl_netdev.lo_check
Instantiate the family once, it takes longer than the test itself.
Signed-off-by: Jakub Kicinski kuba@kernel.org --- CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org --- tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/nl_netdev.py | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+) create mode 100755 tools/testing/selftests/net/nl_netdev.py
diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile index cb418a2346bc..5e34c93aa51b 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -34,6 +34,7 @@ TEST_PROGS += gre_gso.sh TEST_PROGS += cmsg_so_mark.sh TEST_PROGS += cmsg_time.sh cmsg_ipv6.sh TEST_PROGS += netns-name.sh +TEST_PROGS += nl_netdev.py TEST_PROGS += srv6_end_dt46_l3vpn_test.sh TEST_PROGS += srv6_end_dt4_l3vpn_test.sh TEST_PROGS += srv6_end_dt6_l3vpn_test.sh diff --git a/tools/testing/selftests/net/nl_netdev.py b/tools/testing/selftests/net/nl_netdev.py new file mode 100755 index 000000000000..40a59857f984 --- /dev/null +++ b/tools/testing/selftests/net/nl_netdev.py @@ -0,0 +1,24 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 + +from lib.py import ksft_run, ksft_pr, ksft_eq, ksft_ge, NetdevFamily + + +nf = NetdevFamily() + + +def empty_check() -> None: + global nf + devs = nf.dev_get({}, dump=True) + ksft_ge(len(devs), 1) + + +def lo_check() -> None: + global nf + lo_info = nf.dev_get({"ifindex": 1}) + ksft_eq(len(lo_info['xdp-features']), 0) + ksft_eq(len(lo_info['xdp-rx-metadata-features']), 0) + + +if __name__ == "__main__": + ksft_run([empty_check, lo_check])
On 2024-04-01 18:05, Jakub Kicinski wrote:
Add a trivial test using YNL.
$ ./tools/testing/selftests/net/nl_netdev.py KTAP version 1 1..2 ok 1 nl_netdev.empty_check ok 2 nl_netdev.lo_check
Instantiate the family once, it takes longer than the test itself.
Signed-off-by: Jakub Kicinski kuba@kernel.org
CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org
tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/nl_netdev.py | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+) create mode 100755 tools/testing/selftests/net/nl_netdev.py
diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile index cb418a2346bc..5e34c93aa51b 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -34,6 +34,7 @@ TEST_PROGS += gre_gso.sh TEST_PROGS += cmsg_so_mark.sh TEST_PROGS += cmsg_time.sh cmsg_ipv6.sh TEST_PROGS += netns-name.sh +TEST_PROGS += nl_netdev.py TEST_PROGS += srv6_end_dt46_l3vpn_test.sh TEST_PROGS += srv6_end_dt4_l3vpn_test.sh TEST_PROGS += srv6_end_dt6_l3vpn_test.sh diff --git a/tools/testing/selftests/net/nl_netdev.py b/tools/testing/selftests/net/nl_netdev.py new file mode 100755 index 000000000000..40a59857f984 --- /dev/null +++ b/tools/testing/selftests/net/nl_netdev.py @@ -0,0 +1,24 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0
+from lib.py import ksft_run, ksft_pr, ksft_eq, ksft_ge, NetdevFamily
+nf = NetdevFamily()
+def empty_check() -> None:
- global nf
I know you're rolling your own instead of using unittest or pytest. How about adding a Test class of some sort and make each test case a method? Then you wouldn't need to do this for each test case.
Also it would allow you to share some base functionality across different test suites.
- devs = nf.dev_get({}, dump=True)
- ksft_ge(len(devs), 1)
+def lo_check() -> None:
- global nf
- lo_info = nf.dev_get({"ifindex": 1})
- ksft_eq(len(lo_info['xdp-features']), 0)
- ksft_eq(len(lo_info['xdp-rx-metadata-features']), 0)
+if __name__ == "__main__":
- ksft_run([empty_check, lo_check])
Real devices should implement qstats. Devices which support pause or FEC configuration should also report the relevant stats.
nsim was missing FEC stats completely, some of the qstats and pause stats required toggling a debugfs knob.
Note that the tests which used pause always initialize the setting so they shouldn't be affected by the different starting value.
Signed-off-by: Jakub Kicinski kuba@kernel.org --- drivers/net/netdevsim/ethtool.c | 11 ++++++++ drivers/net/netdevsim/netdev.c | 45 +++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+)
diff --git a/drivers/net/netdevsim/ethtool.c b/drivers/net/netdevsim/ethtool.c index bd546d4d26c6..3f9c9327f149 100644 --- a/drivers/net/netdevsim/ethtool.c +++ b/drivers/net/netdevsim/ethtool.c @@ -140,6 +140,13 @@ nsim_set_fecparam(struct net_device *dev, struct ethtool_fecparam *fecparam) return 0; }
+static void +nsim_get_fec_stats(struct net_device *dev, struct ethtool_fec_stats *fec_stats) +{ + fec_stats->corrected_blocks.total = 123; + fec_stats->uncorrectable_blocks.total = 4; +} + static int nsim_get_ts_info(struct net_device *dev, struct ethtool_ts_info *info) { @@ -163,6 +170,7 @@ static const struct ethtool_ops nsim_ethtool_ops = { .set_channels = nsim_set_channels, .get_fecparam = nsim_get_fecparam, .set_fecparam = nsim_set_fecparam, + .get_fec_stats = nsim_get_fec_stats, .get_ts_info = nsim_get_ts_info, };
@@ -182,6 +190,9 @@ void nsim_ethtool_init(struct netdevsim *ns)
nsim_ethtool_ring_init(ns);
+ ns->ethtool.pauseparam.report_stats_rx = true; + ns->ethtool.pauseparam.report_stats_tx = true; + ns->ethtool.fec.fec = ETHTOOL_FEC_NONE; ns->ethtool.fec.active_fec = ETHTOOL_FEC_NONE;
diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c index 8330bc0bcb7e..096ac0abbc02 100644 --- a/drivers/net/netdevsim/netdev.c +++ b/drivers/net/netdevsim/netdev.c @@ -19,6 +19,7 @@ #include <linux/module.h> #include <linux/netdevice.h> #include <linux/slab.h> +#include <net/netdev_queues.h> #include <net/netlink.h> #include <net/pkt_cls.h> #include <net/rtnetlink.h> @@ -330,6 +331,49 @@ static const struct net_device_ops nsim_vf_netdev_ops = { .ndo_set_features = nsim_set_features, };
+/* We don't have true par-queue stats, yet, so do some random fakery here. */ +static void nsim_get_queue_stats_rx(struct net_device *dev, int idx, + struct netdev_queue_stats_rx *stats) +{ + struct rtnl_link_stats64 rtstats = {}; + + nsim_get_stats64(dev, &rtstats); + + stats->packets = rtstats.rx_packets - !!rtstats.rx_packets; + stats->bytes = rtstats.rx_bytes; +} + +static void nsim_get_queue_stats_tx(struct net_device *dev, int idx, + struct netdev_queue_stats_tx *stats) +{ + struct rtnl_link_stats64 rtstats = {}; + + nsim_get_stats64(dev, &rtstats); + + stats->packets = rtstats.tx_packets - !!rtstats.tx_packets; + stats->bytes = rtstats.tx_bytes; +} + +static void nsim_get_base_stats(struct net_device *dev, + struct netdev_queue_stats_rx *rx, + struct netdev_queue_stats_tx *tx) +{ + struct rtnl_link_stats64 rtstats = {}; + + nsim_get_stats64(dev, &rtstats); + + rx->packets = !!rtstats.rx_packets; + rx->bytes = 0; + tx->packets = !!rtstats.tx_packets; + tx->bytes = 0; +} + +static const struct netdev_stat_ops nsim_stat_ops = { + .get_queue_stats_tx = nsim_get_queue_stats_tx, + .get_queue_stats_rx = nsim_get_queue_stats_rx, + .get_base_stats = nsim_get_base_stats, +}; + static void nsim_setup(struct net_device *dev) { ether_setup(dev); @@ -360,6 +404,7 @@ static int nsim_init_netdevsim(struct netdevsim *ns)
ns->phc = phc; ns->netdev->netdev_ops = &nsim_netdev_ops; + ns->netdev->stat_ops = &nsim_stat_ops;
err = nsim_udp_tunnels_info_create(ns->nsim_dev, ns->netdev); if (err)
On 2024-04-01 18:05, Jakub Kicinski wrote:
Real devices should implement qstats. Devices which support pause or FEC configuration should also report the relevant stats.
nsim was missing FEC stats completely, some of the qstats and pause stats required toggling a debugfs knob.
Note that the tests which used pause always initialize the setting so they shouldn't be affected by the different starting value.
Signed-off-by: Jakub Kicinski kuba@kernel.org
drivers/net/netdevsim/ethtool.c | 11 ++++++++ drivers/net/netdevsim/netdev.c | 45 +++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+)
diff --git a/drivers/net/netdevsim/ethtool.c b/drivers/net/netdevsim/ethtool.c index bd546d4d26c6..3f9c9327f149 100644 --- a/drivers/net/netdevsim/ethtool.c +++ b/drivers/net/netdevsim/ethtool.c @@ -140,6 +140,13 @@ nsim_set_fecparam(struct net_device *dev, struct ethtool_fecparam *fecparam) return 0; } +static void +nsim_get_fec_stats(struct net_device *dev, struct ethtool_fec_stats *fec_stats) +{
- fec_stats->corrected_blocks.total = 123;
- fec_stats->uncorrectable_blocks.total = 4;
+}
static int nsim_get_ts_info(struct net_device *dev, struct ethtool_ts_info *info) { @@ -163,6 +170,7 @@ static const struct ethtool_ops nsim_ethtool_ops = { .set_channels = nsim_set_channels, .get_fecparam = nsim_get_fecparam, .set_fecparam = nsim_set_fecparam,
- .get_fec_stats = nsim_get_fec_stats, .get_ts_info = nsim_get_ts_info,
}; @@ -182,6 +190,9 @@ void nsim_ethtool_init(struct netdevsim *ns) nsim_ethtool_ring_init(ns);
- ns->ethtool.pauseparam.report_stats_rx = true;
- ns->ethtool.pauseparam.report_stats_tx = true;
- ns->ethtool.fec.fec = ETHTOOL_FEC_NONE; ns->ethtool.fec.active_fec = ETHTOOL_FEC_NONE;
diff --git a/drivers/net/netdevsim/netdev.c b/drivers/net/netdevsim/netdev.c index 8330bc0bcb7e..096ac0abbc02 100644 --- a/drivers/net/netdevsim/netdev.c +++ b/drivers/net/netdevsim/netdev.c @@ -19,6 +19,7 @@ #include <linux/module.h> #include <linux/netdevice.h> #include <linux/slab.h> +#include <net/netdev_queues.h> #include <net/netlink.h> #include <net/pkt_cls.h> #include <net/rtnetlink.h> @@ -330,6 +331,49 @@ static const struct net_device_ops nsim_vf_netdev_ops = { .ndo_set_features = nsim_set_features, }; +/* We don't have true par-queue stats, yet, so do some random fakery here. */
nit: per-queue
+static void nsim_get_queue_stats_rx(struct net_device *dev, int idx,
struct netdev_queue_stats_rx *stats)+{
- struct rtnl_link_stats64 rtstats = {};
- nsim_get_stats64(dev, &rtstats);
- stats->packets = rtstats.rx_packets - !!rtstats.rx_packets;
Why subtract !!rtstats.rx_packets? This evaluates to 0 if rx_packets is 0 and 1 if rx_packets is non-zero.
- stats->bytes = rtstats.rx_bytes;
+}
+static void nsim_get_queue_stats_tx(struct net_device *dev, int idx,
struct netdev_queue_stats_tx *stats)+{
- struct rtnl_link_stats64 rtstats = {};
- nsim_get_stats64(dev, &rtstats);
- stats->packets = rtstats.tx_packets - !!rtstats.tx_packets;
- stats->bytes = rtstats.tx_bytes;
+}
+static void nsim_get_base_stats(struct net_device *dev,
struct netdev_queue_stats_rx *rx,struct netdev_queue_stats_tx *tx)+{
- struct rtnl_link_stats64 rtstats = {};
- nsim_get_stats64(dev, &rtstats);
- rx->packets = !!rtstats.rx_packets;
- rx->bytes = 0;
- tx->packets = !!rtstats.tx_packets;
- tx->bytes = 0;
+}
+static const struct netdev_stat_ops nsim_stat_ops = {
- .get_queue_stats_tx = nsim_get_queue_stats_tx,
- .get_queue_stats_rx = nsim_get_queue_stats_rx,
- .get_base_stats = nsim_get_base_stats,
+};
static void nsim_setup(struct net_device *dev) { ether_setup(dev); @@ -360,6 +404,7 @@ static int nsim_init_netdevsim(struct netdevsim *ns) ns->phc = phc; ns->netdev->netdev_ops = &nsim_netdev_ops;
- ns->netdev->stat_ops = &nsim_stat_ops;
err = nsim_udp_tunnels_info_create(ns->nsim_dev, ns->netdev); if (err)
Add drivers/net as a target for mixed-use tests. The setup is expected to work similarly to the forwarding tests. Since we only need one interface (unlike forwarding tests) read the target device name from NETIF. If not present we'll try to run the test against netdevsim.
Signed-off-by: Jakub Kicinski kuba@kernel.org --- tools/testing/selftests/Makefile | 3 +- tools/testing/selftests/drivers/net/Makefile | 7 ++ .../testing/selftests/drivers/net/README.rst | 30 +++++ .../selftests/drivers/net/lib/py/__init__.py | 17 +++ .../selftests/drivers/net/lib/py/env.py | 41 ++++++ .../testing/selftests/net/lib/py/__init__.py | 1 + tools/testing/selftests/net/lib/py/nsim.py | 118 ++++++++++++++++++ 7 files changed, 216 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/drivers/net/Makefile create mode 100644 tools/testing/selftests/drivers/net/README.rst create mode 100644 tools/testing/selftests/drivers/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/drivers/net/lib/py/env.py create mode 100644 tools/testing/selftests/net/lib/py/nsim.py
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 0cffdfb4b116..d015ec14a85e 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -17,6 +17,7 @@ TARGETS += devices TARGETS += dmabuf-heaps TARGETS += drivers/dma-buf TARGETS += drivers/s390x/uvdevice +TARGETS += drivers/net TARGETS += drivers/net/bonding TARGETS += drivers/net/team TARGETS += dt @@ -117,7 +118,7 @@ TARGETS_HOTPLUG = cpu-hotplug TARGETS_HOTPLUG += memory-hotplug
# Networking tests want the net/lib target, include it automatically -ifneq ($(filter net ,$(TARGETS)),) +ifneq ($(filter net drivers/net,$(TARGETS)),) ifeq ($(filter net/lib,$(TARGETS)),) override TARGETS := $(TARGETS) net/lib endif diff --git a/tools/testing/selftests/drivers/net/Makefile b/tools/testing/selftests/drivers/net/Makefile new file mode 100644 index 000000000000..379cdb1960a7 --- /dev/null +++ b/tools/testing/selftests/drivers/net/Makefile @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0 + +TEST_INCLUDES := $(wildcard lib/py/*.py) + +TEST_PROGS := stats.py + +include ../../lib.mk diff --git a/tools/testing/selftests/drivers/net/README.rst b/tools/testing/selftests/drivers/net/README.rst new file mode 100644 index 000000000000..5ef7c417d431 --- /dev/null +++ b/tools/testing/selftests/drivers/net/README.rst @@ -0,0 +1,30 @@ +Running tests +============= + +Tests are executed within kselftest framework like any other tests. +By default tests execute against software drivers such as netdevsim. +All tests must support running against a real device (SW-only tests +should instead be placed in net/ or drivers/net/netdevsim, HW-only +tests in drivers/net/hw). + +Set appropriate variables to point the tests at a real device. + +Variables +========= + +Variables can be set in the environment or by creating a net.config +file in the same directory as this README file. Example:: + + $ NETIF=eth0 ./some_test.sh + +or:: + + $ cat tools/testing/selftests/drivers/net/net.config + # Variable set in a file + NETIF=eth0 + +NETIF +~~~~~ + +Name of the netdevice against which the test should be executed. +When empty or not set software devices will be used. diff --git a/tools/testing/selftests/drivers/net/lib/py/__init__.py b/tools/testing/selftests/drivers/net/lib/py/__init__.py new file mode 100644 index 000000000000..4653dffcd962 --- /dev/null +++ b/tools/testing/selftests/drivers/net/lib/py/__init__.py @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0 + +import sys +from pathlib import Path + +KSFT_DIR = (Path(__file__).parent / "../../../..").resolve() + +try: + sys.path.append(KSFT_DIR.as_posix()) + from net.lib.py import * +except ModuleNotFoundError as e: + ksft_pr("Failed importing `net` library from kernel sources") + ksft_pr(str(e)) + ktap_result(True, comment="SKIP") + sys.exit(4) + +from .env import * diff --git a/tools/testing/selftests/drivers/net/lib/py/env.py b/tools/testing/selftests/drivers/net/lib/py/env.py new file mode 100644 index 000000000000..ee4a44555d83 --- /dev/null +++ b/tools/testing/selftests/drivers/net/lib/py/env.py @@ -0,0 +1,41 @@ +# SPDX-License-Identifier: GPL-2.0 + +import os +import shlex +from pathlib import Path +from lib.py import ip +from lib.py import NetdevSimDev + +class NetDrvEnv: + def __init__(self, src_path): + self.env = os.environ.copy() + self._load_env_file(src_path) + + if 'NETIF' in self.env: + self._ns = None + self.dev = ip("link show dev " + self.env['NETIF'], json=True)[0] + else: + self._ns = NetdevSimDev() + self.dev = self._ns.nsims[0].dev + self.ifindex = self.dev['ifindex'] + + def __del__(self): + if self._ns: + self._ns.remove() + + def _load_env_file(self, src_path): + src_dir = Path(src_path).parent.resolve() + if not (src_dir / "net.config").exists(): + return + + lexer = shlex.shlex(open((src_dir / "net.config").as_posix(), 'r').read()) + k = None + for token in lexer: + if k is None: + k = token + self.env[k] = "" + elif token == "=": + pass + else: + self.env[k] = token + k = None diff --git a/tools/testing/selftests/net/lib/py/__init__.py b/tools/testing/selftests/net/lib/py/__init__.py index 81a8d14b68f0..99cfc8dc4dca 100644 --- a/tools/testing/selftests/net/lib/py/__init__.py +++ b/tools/testing/selftests/net/lib/py/__init__.py @@ -3,4 +3,5 @@ from .ksft import * from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily from .consts import KSRC +from .nsim import * from .utils import * diff --git a/tools/testing/selftests/net/lib/py/nsim.py b/tools/testing/selftests/net/lib/py/nsim.py new file mode 100644 index 000000000000..13eb42c82829 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/nsim.py @@ -0,0 +1,118 @@ +# SPDX-License-Identifier: GPL-2.0 + +import json +import os +import random +import re +import time +from .utils import cmd, ip + + +class NetdevSim: + """ + Class for netdevsim netdevice and its attributes. + """ + + def __init__(self, nsimdev, port_index, ifname, ns=None): + # In case udev renamed the netdev to according to new schema, + # check if the name matches the port_index. + nsimnamere = re.compile("eni\d+np(\d+)") + match = nsimnamere.match(ifname) + if match and int(match.groups()[0]) != port_index + 1: + raise Exception("netdevice name mismatches the expected one") + + self.ifname = ifname + self.nsimdev = nsimdev + self.port_index = port_index + self.ns = ns + self.dfs_dir = "%s/ports/%u/" % (nsimdev.dfs_dir, port_index) + ret = ip("-j link show dev %s" % ifname, ns=ns) + self.dev = json.loads(ret.stdout)[0] + + def dfs_write(self, path, val): + self.nsimdev.dfs_write(f'ports/{self.port_index}/' + path, val) + + +class NetdevSimDev: + """ + Class for netdevsim bus device and its attributes. + """ + @staticmethod + def ctrl_write(path, val): + fullpath = os.path.join("/sys/bus/netdevsim/", path) + with open(fullpath, "w") as f: + f.write(val) + + def dfs_write(self, path, val): + fullpath = os.path.join(f"/sys/kernel/debug/netdevsim/netdevsim{self.addr}/", path) + with open(fullpath, "w") as f: + f.write(val) + + def __init__(self, port_count=1, ns=None): + # nsim will spawn in init_net, we'll set to actual ns once we switch it the.sre + self.ns = None + + if not os.path.exists("/sys/bus/netdevsim"): + cmd("modprobe netdevsim") + + addr = random.randrange(1 << 15) + while True: + try: + self.ctrl_write("new_device", "%u %u" % (addr, port_count)) + except OSError as e: + if e.errno == errno.ENOSPC: + addr = random.randrange(1 << 15) + continue + raise e + break + self.addr = addr + + # As probe of netdevsim device might happen from a workqueue, + # so wait here until all netdevs appear. + self.wait_for_netdevs(port_count) + + if ns: + cmd(f"devlink dev reload netdevsim/netdevsim{addr} netns {ns.name}") + self.ns = ns + + cmd("udevadm settle", ns=self.ns) + ifnames = self.get_ifnames() + + self.dfs_dir = "/sys/kernel/debug/netdevsim/netdevsim%u/" % addr + + self.nsims = [] + for port_index in range(port_count): + self.nsims.append(NetdevSim(self, port_index, ifnames[port_index], + ns=ns)) + + def get_ifnames(self): + ifnames = [] + listdir = cmd(f"ls /sys/bus/netdevsim/devices/netdevsim{self.addr}/net/", + ns=self.ns).stdout.split() + for ifname in listdir: + ifnames.append(ifname) + ifnames.sort() + return ifnames + + def wait_for_netdevs(self, port_count): + timeout = 5 + timeout_start = time.time() + + while True: + try: + ifnames = self.get_ifnames() + except FileNotFoundError as e: + ifnames = [] + if len(ifnames) == port_count: + break + if time.time() < timeout_start + timeout: + continue + raise Exception("netdevices did not appear within timeout") + + def remove(self): + self.ctrl_write("del_device", "%u" % (self.addr, )) + + def remove_nsim(self, nsim): + self.nsims.remove(nsim) + self.ctrl_write("devices/netdevsim%u/del_port" % (self.addr, ), + "%u" % (nsim.port_index, ))
On 2024-04-01 18:05, Jakub Kicinski wrote:
Add drivers/net as a target for mixed-use tests. The setup is expected to work similarly to the forwarding tests. Since we only need one interface (unlike forwarding tests) read the target device name from NETIF. If not present we'll try to run the test against netdevsim.
Signed-off-by: Jakub Kicinski kuba@kernel.org
tools/testing/selftests/Makefile | 3 +- tools/testing/selftests/drivers/net/Makefile | 7 ++ .../testing/selftests/drivers/net/README.rst | 30 +++++ .../selftests/drivers/net/lib/py/__init__.py | 17 +++ .../selftests/drivers/net/lib/py/env.py | 41 ++++++ .../testing/selftests/net/lib/py/__init__.py | 1 + tools/testing/selftests/net/lib/py/nsim.py | 118 ++++++++++++++++++ 7 files changed, 216 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/drivers/net/Makefile create mode 100644 tools/testing/selftests/drivers/net/README.rst create mode 100644 tools/testing/selftests/drivers/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/drivers/net/lib/py/env.py create mode 100644 tools/testing/selftests/net/lib/py/nsim.py
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 0cffdfb4b116..d015ec14a85e 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -17,6 +17,7 @@ TARGETS += devices TARGETS += dmabuf-heaps TARGETS += drivers/dma-buf TARGETS += drivers/s390x/uvdevice +TARGETS += drivers/net TARGETS += drivers/net/bonding TARGETS += drivers/net/team TARGETS += dt @@ -117,7 +118,7 @@ TARGETS_HOTPLUG = cpu-hotplug TARGETS_HOTPLUG += memory-hotplug # Networking tests want the net/lib target, include it automatically -ifneq ($(filter net ,$(TARGETS)),) +ifneq ($(filter net drivers/net,$(TARGETS)),) ifeq ($(filter net/lib,$(TARGETS)),) override TARGETS := $(TARGETS) net/lib endif diff --git a/tools/testing/selftests/drivers/net/Makefile b/tools/testing/selftests/drivers/net/Makefile new file mode 100644 index 000000000000..379cdb1960a7 --- /dev/null +++ b/tools/testing/selftests/drivers/net/Makefile @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0
+TEST_INCLUDES := $(wildcard lib/py/*.py)
+TEST_PROGS := stats.py
+include ../../lib.mk diff --git a/tools/testing/selftests/drivers/net/README.rst b/tools/testing/selftests/drivers/net/README.rst new file mode 100644 index 000000000000..5ef7c417d431 --- /dev/null +++ b/tools/testing/selftests/drivers/net/README.rst @@ -0,0 +1,30 @@ +Running tests +=============
+Tests are executed within kselftest framework like any other tests. +By default tests execute against software drivers such as netdevsim. +All tests must support running against a real device (SW-only tests +should instead be placed in net/ or drivers/net/netdevsim, HW-only +tests in drivers/net/hw).
+Set appropriate variables to point the tests at a real device.
+Variables +=========
+Variables can be set in the environment or by creating a net.config +file in the same directory as this README file. Example::
- $ NETIF=eth0 ./some_test.sh
+or::
- $ cat tools/testing/selftests/drivers/net/net.config
- # Variable set in a file
- NETIF=eth0
+NETIF +~~~~~
+Name of the netdevice against which the test should be executed. +When empty or not set software devices will be used. diff --git a/tools/testing/selftests/drivers/net/lib/py/__init__.py b/tools/testing/selftests/drivers/net/lib/py/__init__.py new file mode 100644 index 000000000000..4653dffcd962 --- /dev/null +++ b/tools/testing/selftests/drivers/net/lib/py/__init__.py @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0
+import sys +from pathlib import Path
+KSFT_DIR = (Path(__file__).parent / "../../../..").resolve()
+try:
- sys.path.append(KSFT_DIR.as_posix())
- from net.lib.py import *
+except ModuleNotFoundError as e:
- ksft_pr("Failed importing `net` library from kernel sources")
- ksft_pr(str(e))
- ktap_result(True, comment="SKIP")
- sys.exit(4)
+from .env import * diff --git a/tools/testing/selftests/drivers/net/lib/py/env.py b/tools/testing/selftests/drivers/net/lib/py/env.py new file mode 100644 index 000000000000..ee4a44555d83 --- /dev/null +++ b/tools/testing/selftests/drivers/net/lib/py/env.py @@ -0,0 +1,41 @@ +# SPDX-License-Identifier: GPL-2.0
+import os +import shlex +from pathlib import Path +from lib.py import ip +from lib.py import NetdevSimDev
nit: these could be on the same line.
+class NetDrvEnv:
- def __init__(self, src_path):
self.env = os.environ.copy()self._load_env_file(src_path)if 'NETIF' in self.env:self._ns = None
My brain interprets 'ns' as 'namespace'. How about something like nsimdev/nsdev/nsim?
self.dev = ip("link show dev " + self.env['NETIF'], json=True)[0]else:self._ns = NetdevSimDev()self.dev = self._ns.nsims[0].devself.ifindex = self.dev['ifindex']- def __del__(self):
if self._ns:self._ns.remove()- def _load_env_file(self, src_path):
src_dir = Path(src_path).parent.resolve()if not (src_dir / "net.config").exists():returnlexer = shlex.shlex(open((src_dir / "net.config").as_posix(), 'r').read())k = Nonefor token in lexer:if k is None:k = tokenself.env[k] = ""elif token == "=":passelse:self.env[k] = tokenk = Nonediff --git a/tools/testing/selftests/net/lib/py/__init__.py b/tools/testing/selftests/net/lib/py/__init__.py index 81a8d14b68f0..99cfc8dc4dca 100644 --- a/tools/testing/selftests/net/lib/py/__init__.py +++ b/tools/testing/selftests/net/lib/py/__init__.py @@ -3,4 +3,5 @@ from .ksft import * from .ynl import NlError, YnlFamily, EthtoolFamily, NetdevFamily, RtnlFamily from .consts import KSRC +from .nsim import * from .utils import * diff --git a/tools/testing/selftests/net/lib/py/nsim.py b/tools/testing/selftests/net/lib/py/nsim.py new file mode 100644 index 000000000000..13eb42c82829 --- /dev/null +++ b/tools/testing/selftests/net/lib/py/nsim.py @@ -0,0 +1,118 @@ +# SPDX-License-Identifier: GPL-2.0
+import json +import os +import random +import re +import time +from .utils import cmd, ip
+class NetdevSim:
- """
- Class for netdevsim netdevice and its attributes.
- """
- def __init__(self, nsimdev, port_index, ifname, ns=None):
# In case udev renamed the netdev to according to new schema,# check if the name matches the port_index.nsimnamere = re.compile("eni\d+np(\d+)")match = nsimnamere.match(ifname)if match and int(match.groups()[0]) != port_index + 1:raise Exception("netdevice name mismatches the expected one")self.ifname = ifnameself.nsimdev = nsimdevself.port_index = port_indexself.ns = nsself.dfs_dir = "%s/ports/%u/" % (nsimdev.dfs_dir, port_index)ret = ip("-j link show dev %s" % ifname, ns=ns)self.dev = json.loads(ret.stdout)[0]- def dfs_write(self, path, val):
self.nsimdev.dfs_write(f'ports/{self.port_index}/' + path, val)+class NetdevSimDev:
- """
- Class for netdevsim bus device and its attributes.
- """
- @staticmethod
- def ctrl_write(path, val):
fullpath = os.path.join("/sys/bus/netdevsim/", path)with open(fullpath, "w") as f:f.write(val)- def dfs_write(self, path, val):
fullpath = os.path.join(f"/sys/kernel/debug/netdevsim/netdevsim{self.addr}/", path)with open(fullpath, "w") as f:f.write(val)- def __init__(self, port_count=1, ns=None):
# nsim will spawn in init_net, we'll set to actual ns once we switch it the.sreself.ns = Noneif not os.path.exists("/sys/bus/netdevsim"):cmd("modprobe netdevsim")addr = random.randrange(1 << 15)while True:try:self.ctrl_write("new_device", "%u %u" % (addr, port_count))except OSError as e:if e.errno == errno.ENOSPC:addr = random.randrange(1 << 15)continueraise ebreakself.addr = addr# As probe of netdevsim device might happen from a workqueue,# so wait here until all netdevs appear.self.wait_for_netdevs(port_count)if ns:cmd(f"devlink dev reload netdevsim/netdevsim{addr} netns {ns.name}")self.ns = nscmd("udevadm settle", ns=self.ns)ifnames = self.get_ifnames()self.dfs_dir = "/sys/kernel/debug/netdevsim/netdevsim%u/" % addrself.nsims = []for port_index in range(port_count):self.nsims.append(NetdevSim(self, port_index, ifnames[port_index],ns=ns))- def get_ifnames(self):
ifnames = []listdir = cmd(f"ls /sys/bus/netdevsim/devices/netdevsim{self.addr}/net/",ns=self.ns).stdout.split()for ifname in listdir:ifnames.append(ifname)ifnames.sort()return ifnames- def wait_for_netdevs(self, port_count):
timeout = 5timeout_start = time.time()while True:try:ifnames = self.get_ifnames()except FileNotFoundError as e:ifnames = []if len(ifnames) == port_count:breakif time.time() < timeout_start + timeout:continueraise Exception("netdevices did not appear within timeout")- def remove(self):
self.ctrl_write("del_device", "%u" % (self.addr, ))
I really want this to be in the dtor, but couldn't get it to work because Python doesn't let you open files in __del__(). :(
- def remove_nsim(self, nsim):
self.nsims.remove(nsim)self.ctrl_write("devices/netdevsim%u/del_port" % (self.addr, ),"%u" % (nsim.port_index, ))
Add a very simple test to make sure drivers report expected stats. Drivers which implement FEC or pause configuration should report relevant stats. Qstats must be reported, at least packet and byte counts, and they must match total device stats.
Tested with netdevsim, bnxt, in-tree and installed.
Signed-off-by: Jakub Kicinski kuba@kernel.org --- tools/testing/selftests/drivers/net/stats.py | 85 ++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100755 tools/testing/selftests/drivers/net/stats.py
diff --git a/tools/testing/selftests/drivers/net/stats.py b/tools/testing/selftests/drivers/net/stats.py new file mode 100755 index 000000000000..751cca2869b8 --- /dev/null +++ b/tools/testing/selftests/drivers/net/stats.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 + +from lib.py import ksft_run, ksft_in, ksft_true, KsftSkipEx, KsftXfailEx +from lib.py import EthtoolFamily, NetdevFamily, RtnlFamily, NlError +from lib.py import NetDrvEnv + +cfg = None +ethnl = EthtoolFamily() +netfam = NetdevFamily() +rtnl = RtnlFamily() + + +def check_pause() -> None: + global cfg, ethnl + + try: + ethnl.pause_get({"header": {"dev-index": cfg.ifindex}}) + except NlError as e: + if e.error == 95: + raise KsftXfailEx("pause not supported by the device") + raise + + data = ethnl.pause_get({"header": {"dev-index": cfg.ifindex, + "flags": {'stats'}}}) + ksft_true(data['stats'], "driver does not report stats") + + +def check_fec() -> None: + global ethnl + + try: + ethnl.fec_get({"header": {"dev-index": cfg.ifindex}}) + except NlError as e: + if e.error == 95: + raise KsftXfailEx("FEC not supported by the device") + raise + + data = ethnl.fec_get({"header": {"dev-index": cfg.ifindex, + "flags": {'stats'}}}) + ksft_true(data['stats'], "driver does not report stats") + + +def pkt_byte_sum() -> None: + global cfg, netfam, rtnl + + def get_qstat(test): + global netfam + stats = netfam.qstats_get({}, dump=True) + if stats: + for qs in stats: + if qs["ifindex"]== test.ifindex: + return qs + + qstat = get_qstat(cfg) + if qstat is None: + raise KsftSkipEx("qstats not supported by the device") + + for key in ['tx-packets', 'tx-bytes', 'rx-packets', 'rx-bytes']: + ksft_in(key, qstat, "Drivers should always report basic keys") + + # Compare stats, rtnl stats and qstats must match, + # but the interface may be up, so do a series of dumps + # each time the more "recent" stats must be higher or same. + def stat_cmp(rstat, qstat): + for key in ['tx-packets', 'tx-bytes', 'rx-packets', 'rx-bytes']: + if rstat[key] != qstat[key]: + return rstat[key] - qstat[key] + return 0 + + for _ in range(10): + rtstat = rtnl.getlink({"ifi-index": cfg.ifindex})['stats'] + if stat_cmp(rtstat, qstat) < 0: + raise Exception("RTNL stats are lower, fetched later") + qstat = get_qstat(cfg) + if stat_cmp(rtstat, qstat) > 0: + raise Exception("Qstats are lower, fetched later") + + +if __name__ == "__main__": + cfg = NetDrvEnv(__file__) + try: + ksft_run([check_pause, check_fec, pkt_byte_sum]) + finally: + del cfg
Jakub Kicinski kuba@kernel.org writes:
Add a very simple test to make sure drivers report expected stats. Drivers which implement FEC or pause configuration should report relevant stats. Qstats must be reported, at least packet and byte counts, and they must match total device stats.
Tested with netdevsim, bnxt, in-tree and installed.
Signed-off-by: Jakub Kicinski kuba@kernel.org
tools/testing/selftests/drivers/net/stats.py | 85 ++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100755 tools/testing/selftests/drivers/net/stats.py
diff --git a/tools/testing/selftests/drivers/net/stats.py b/tools/testing/selftests/drivers/net/stats.py new file mode 100755 index 000000000000..751cca2869b8 --- /dev/null +++ b/tools/testing/selftests/drivers/net/stats.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0
+from lib.py import ksft_run, ksft_in, ksft_true, KsftSkipEx, KsftXfailEx +from lib.py import EthtoolFamily, NetdevFamily, RtnlFamily, NlError +from lib.py import NetDrvEnv
+cfg = None +ethnl = EthtoolFamily() +netfam = NetdevFamily() +rtnl = RtnlFamily()
+def check_pause() -> None:
- global cfg, ethnl
- try:
ethnl.pause_get({"header": {"dev-index": cfg.ifindex}})- except NlError as e:
if e.error == 95:raise KsftXfailEx("pause not supported by the device")raise- data = ethnl.pause_get({"header": {"dev-index": cfg.ifindex,
"flags": {'stats'}}})- ksft_true(data['stats'], "driver does not report stats")
+def check_fec() -> None:
- global ethnl
- try:
ethnl.fec_get({"header": {"dev-index": cfg.ifindex}})- except NlError as e:
if e.error == 95:raise KsftXfailEx("FEC not supported by the device")raise- data = ethnl.fec_get({"header": {"dev-index": cfg.ifindex,
"flags": {'stats'}}})- ksft_true(data['stats'], "driver does not report stats")
+def pkt_byte_sum() -> None:
- global cfg, netfam, rtnl
- def get_qstat(test):
global netfamstats = netfam.qstats_get({}, dump=True)if stats:for qs in stats:if qs["ifindex"]== test.ifindex:return qs- qstat = get_qstat(cfg)
- if qstat is None:
raise KsftSkipEx("qstats not supported by the device")- for key in ['tx-packets', 'tx-bytes', 'rx-packets', 'rx-bytes']:
ksft_in(key, qstat, "Drivers should always report basic keys")- # Compare stats, rtnl stats and qstats must match,
- # but the interface may be up, so do a series of dumps
- # each time the more "recent" stats must be higher or same.
- def stat_cmp(rstat, qstat):
for key in ['tx-packets', 'tx-bytes', 'rx-packets', 'rx-bytes']:if rstat[key] != qstat[key]:return rstat[key] - qstat[key]return 0- for _ in range(10):
rtstat = rtnl.getlink({"ifi-index": cfg.ifindex})['stats']if stat_cmp(rtstat, qstat) < 0:raise Exception("RTNL stats are lower, fetched later")qstat = get_qstat(cfg)if stat_cmp(rtstat, qstat) > 0:raise Exception("Qstats are lower, fetched later")+if __name__ == "__main__":
- cfg = NetDrvEnv(__file__)
- try:
ksft_run([check_pause, check_fec, pkt_byte_sum])- finally:
del cfg
Yeah, this would be usually done through context managers, as I mention in the other e-mail. But then cfg would be lexically scoped, which IMHO is a good thing, but then it needs to be passed around as an argument, and that makes the ksft_run() invocation a bit messy:
with NetDrvEnv(__file__) as cfg: ksft_run([lambda: check_pause(cfg), lambda: check_fec(cfg), lambda: pkt_byte_sum(cfg)])
Dunno, maybe it could forward *args **kwargs to the cases? But then it loses some of the readability again.
On Tue, 2 Apr 2024 18:37:44 +0200 Petr Machata wrote:
Yeah, this would be usually done through context managers, as I mention in the other e-mail. But then cfg would be lexically scoped, which IMHO is a good thing, but then it needs to be passed around as an argument, and that makes the ksft_run() invocation a bit messy:
with NetDrvEnv(__file__) as cfg: ksft_run([lambda: check_pause(cfg), lambda: check_fec(cfg), lambda: pkt_byte_sum(cfg)])Dunno, maybe it could forward *args **kwargs to the cases? But then it loses some of the readability again.
Yes, I was wondering about that. It must be doable, IIRC the multi-threading API "injects" args from a tuple. I was thinking something along the lines of:
with NetDrvEnv(__file__) as cfg: ksft_run([check_pause, check_fec, pkt_byte_sum], args=(cfg, ))
I got lazy, let me take a closer look. Another benefit will be that once we pass in "env" / cfg - we can "register" objects in there for auto-cleanup (in the future, current tests don't need cleanup)
On Tue, 2 Apr 2024 10:31:11 -0700 Jakub Kicinski wrote:
Yes, I was wondering about that. It must be doable, IIRC the multi-threading API "injects" args from a tuple. I was thinking something along the lines of:
with NetDrvEnv(__file__) as cfg: ksft_run([check_pause, check_fec, pkt_byte_sum], args=(cfg, ))
seems to work, is this good?
diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py index 7c296fe5e438..c7210525981c 100644 --- a/tools/testing/selftests/net/lib/py/ksft.py +++ b/tools/testing/selftests/net/lib/py/ksft.py @@ -60,7 +60,7 @@ KSFT_RESULT = None print(res)
-def ksft_run(cases): +def ksft_run(cases, args=()): totals = {"pass": 0, "fail": 0, "skip": 0, "xfail": 0}
print("KTAP version 1") @@ -72,7 +72,7 @@ KSFT_RESULT = None KSFT_RESULT = True cnt += 1 try: - case() + case(*args) except KsftSkipEx as e: ktap_result(True, cnt, case, comment="SKIP " + str(e)) totals['skip'] += 1
Jakub Kicinski kuba@kernel.org writes:
On Tue, 2 Apr 2024 10:31:11 -0700 Jakub Kicinski wrote:
Yes, I was wondering about that. It must be doable, IIRC the multi-threading API "injects" args from a tuple. I was thinking something along the lines of:
with NetDrvEnv(__file__) as cfg: ksft_run([check_pause, check_fec, pkt_byte_sum], args=(cfg, ))seems to work, is this good?
diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py index 7c296fe5e438..c7210525981c 100644 --- a/tools/testing/selftests/net/lib/py/ksft.py +++ b/tools/testing/selftests/net/lib/py/ksft.py @@ -60,7 +60,7 @@ KSFT_RESULT = None print(res) -def ksft_run(cases): +def ksft_run(cases, args=()): totals = {"pass": 0, "fail": 0, "skip": 0, "xfail": 0} print("KTAP version 1") @@ -72,7 +72,7 @@ KSFT_RESULT = None KSFT_RESULT = True cnt += 1 try:
case()
case(*args) except KsftSkipEx as e: ktap_result(True, cnt, case, comment="SKIP " + str(e)) totals['skip'] += 1
Yep, looks good.
Jakub Kicinski kuba@kernel.org writes:
On Tue, 2 Apr 2024 18:37:44 +0200 Petr Machata wrote:
Yeah, this would be usually done through context managers, as I mention in the other e-mail. But then cfg would be lexically scoped, which IMHO is a good thing, but then it needs to be passed around as an argument, and that makes the ksft_run() invocation a bit messy:
with NetDrvEnv(__file__) as cfg: ksft_run([lambda: check_pause(cfg), lambda: check_fec(cfg), lambda: pkt_byte_sum(cfg)])Dunno, maybe it could forward *args **kwargs to the cases? But then it loses some of the readability again.
Yes, I was wondering about that. It must be doable, IIRC the multi-threading API "injects" args from a tuple. I was thinking something along the lines of:
with NetDrvEnv(__file__) as cfg: ksft_run([check_pause, check_fec, pkt_byte_sum], args=(cfg, ))I got lazy, let me take a closer look. Another benefit will be that once we pass in "env" / cfg - we can "register" objects in there for auto-cleanup (in the future, current tests don't need cleanup)
Yeah, though some of those should probably just be their own context managers IMHO, not necessarily hooked to cfg. I'm thinking something fairly general, so that the support boilerplate doesn't end up costing an arm and leg:
with build("ip route add 192.0.2.1/28 nexthop via 192.0.2.17", "ip route del 192.0.2.1/28"), build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1): le_test()
Dunno. I guess it makes sense to have some of the common stuff predefined, e.g. "with vrf() as h1". And then the stuff that's typically in lib.sh's setup() and cleanup(), can be losslessly hooked up to cfg.
This is what I ended up gravitating towards after writing a handful of LNST tests anyway. The scoping makes it clear where the object exists, lifetime is taken care of, it's all ponies rainbows basically. At least as long as your object lifetimes can be cleanly nested, which admittedly is not always.
On Wed, 3 Apr 2024 00:04:14 +0200 Petr Machata wrote:
Yes, I was wondering about that. It must be doable, IIRC the multi-threading API "injects" args from a tuple. I was thinking something along the lines of:
with NetDrvEnv(__file__) as cfg: ksft_run([check_pause, check_fec, pkt_byte_sum], args=(cfg, ))I got lazy, let me take a closer look. Another benefit will be that once we pass in "env" / cfg - we can "register" objects in there for auto-cleanup (in the future, current tests don't need cleanup)
Yeah, though some of those should probably just be their own context managers IMHO, not necessarily hooked to cfg. I'm thinking something fairly general, so that the support boilerplate doesn't end up costing an arm and leg:
with build("ip route add 192.0.2.1/28 nexthop via 192.0.2.17", "ip route del 192.0.2.1/28"), build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1): le_test()Dunno. I guess it makes sense to have some of the common stuff predefined, e.g. "with vrf() as h1". And then the stuff that's typically in lib.sh's setup() and cleanup(), can be losslessly hooked up to cfg.
I was thinking of something along the lines of:
def test_abc(cfg): cfg.build("ip route add 192.0.2.1/28 nexthop via 192.0.2.17", "ip route del 192.0.2.1/28") cfg.build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1)
optionally we could then also:
thing = cfg.build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1)
# ... some code which may raise ...
# unlink to do something else with the device del thing # ... more code ...
cfg may not be best here, could be cleaner to create a "test" object, always pass it in as the first param, and destroy it after each test.
This is what I ended up gravitating towards after writing a handful of LNST tests anyway. The scoping makes it clear where the object exists, lifetime is taken care of, it's all ponies rainbows basically. At least as long as your object lifetimes can be cleanly nested, which admittedly is not always.
Should be fairly easy to support all cases - "with", "recording on cfg/test" and del. Unfortunately in the two tests I came up with quickly for this series cleanup is only needed for the env itself. It's a bit awkward to add the lifetime helpers without any users.
Jakub Kicinski kuba@kernel.org writes:
On Wed, 3 Apr 2024 00:04:14 +0200 Petr Machata wrote:
Yes, I was wondering about that. It must be doable, IIRC the multi-threading API "injects" args from a tuple. I was thinking something along the lines of:
with NetDrvEnv(__file__) as cfg: ksft_run([check_pause, check_fec, pkt_byte_sum], args=(cfg, ))I got lazy, let me take a closer look. Another benefit will be that once we pass in "env" / cfg - we can "register" objects in there for auto-cleanup (in the future, current tests don't need cleanup)
Yeah, though some of those should probably just be their own context managers IMHO, not necessarily hooked to cfg. I'm thinking something fairly general, so that the support boilerplate doesn't end up costing an arm and leg:
with build("ip route add 192.0.2.1/28 nexthop via 192.0.2.17", "ip route del 192.0.2.1/28"), build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1): le_test()Dunno. I guess it makes sense to have some of the common stuff predefined, e.g. "with vrf() as h1". And then the stuff that's typically in lib.sh's setup() and cleanup(), can be losslessly hooked up to cfg.
I was thinking of something along the lines of:
def test_abc(cfg): cfg.build("ip route add 192.0.2.1/28 nexthop via 192.0.2.17", "ip route del 192.0.2.1/28") cfg.build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1)
optionally we could then also:
thing = cfg.build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1) # ... some code which may raise ... # unlink to do something else with the device del thing # ... more code ...cfg may not be best here, could be cleaner to create a "test" object, always pass it in as the first param, and destroy it after each test.
I assume above you mean that cfg inherits the thing, but cfg lifetime currently looks like it spreads across several test cases. ksft_run() would need to know about it and call something to issue the postponed cleanups between cases.
Also, it's not clear what "del thing" should do in that context, because if cfg also keeps a reference, __del__ won't get called. There could be a direct method, like thing.exit() or whatever, but then you need bookkeeping so as not to clean up the second time through cfg. It's the less straightforward way of going about it IMHO.
I know that I must sound like a broken record at this point, but look:
with build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1) as thing: ... some code which may rise ... ... more code, interface detached, `thing' gone ...
It's just as concise, makes it very clear where the device is part of the bridge and where not anymore, and does away with the intricacies of lifetime management.
If lifetimes don't nest, I think it's just going to be ugly either way. But I don't think this comes up often.
I don't really see stuff that you could just throw at cfg to keep track of, apart from the suite configuration (i.e. topology set up). But I suppose if it comes up, we can do something like:
thing = cfg.retain(build(..., ...))
Or maybe have a dedicated retainer object, or whatever, it doesn't necessarily need to be cfg itself.
This is what I ended up gravitating towards after writing a handful of LNST tests anyway. The scoping makes it clear where the object exists, lifetime is taken care of, it's all ponies rainbows basically. At least as long as your object lifetimes can be cleanly nested, which admittedly is not always.
Should be fairly easy to support all cases - "with", "recording on cfg/test" and del. Unfortunately in the two tests I came up with
Yup.
quickly for this series cleanup is only needed for the env itself. It's a bit awkward to add the lifetime helpers without any users.
Yeah. I'm basically delving in this now to kinda try and steer future expectations.
On Wed, 3 Apr 2024 10:58:19 +0200 Petr Machata wrote:
Also, it's not clear what "del thing" should do in that context, because if cfg also keeps a reference, __del__ won't get called. There could be a direct method, like thing.exit() or whatever, but then you need bookkeeping so as not to clean up the second time through cfg. It's the less straightforward way of going about it IMHO.
I see, having read up on what del actually does - "del thing" would indeed not work here.
I know that I must sound like a broken record at this point, but look:
with build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1) as thing: ... some code which may rise ... ... more code, interface detached, `thing' gone ...It's just as concise, makes it very clear where the device is part of the bridge and where not anymore, and does away with the intricacies of lifetime management.
My experience [1] is that with "with" we often end up writing tests like this:
def test(): with a() as bunch, of() as things: ... entire body of the test indented ...
[1] https://github.com/kuba-moo/linux/blob/psp/tools/net/ynl/psp.py
Nothing wrong with that. I guess the question in my mind is whether we're aiming for making the tests "pythonic" (in which case "with" definitely wins), or more of a "bash with classes" style trying to avoid any constructs people may have to google. I'm on the fence on that one, as the del example proves my python expertise is not high. OTOH people who prefer bash will continue to write bash tests, so maybe we don't have to worry about non-experts too much. Dunno.
Jakub Kicinski kuba@kernel.org writes:
On Wed, 3 Apr 2024 10:58:19 +0200 Petr Machata wrote:
Also, it's not clear what "del thing" should do in that context, because if cfg also keeps a reference, __del__ won't get called. There could be a direct method, like thing.exit() or whatever, but then you need bookkeeping so as not to clean up the second time through cfg. It's the less straightforward way of going about it IMHO.
I see, having read up on what del actually does - "del thing" would indeed not work here.
I know that I must sound like a broken record at this point, but look:
with build("ip link set dev %s master %s" % (swp1, h1), "ip link set dev %s nomaster" % swp1) as thing: ... some code which may rise ... ... more code, interface detached, `thing' gone ...It's just as concise, makes it very clear where the device is part of the bridge and where not anymore, and does away with the intricacies of lifetime management.
My experience [1] is that with "with" we often end up writing tests like this:
def test(): with a() as bunch, of() as things: ... entire body of the test indented ...
[1] https://github.com/kuba-moo/linux/blob/psp/tools/net/ynl/psp.py
Yeah, that does end up happening. I think there are a couple places where you could have folded several withs in one, but it is going to be indented, yeah.
But you end up indenting for try: finally: to make the del work reliably anyway, so it's kinda lose/lose in that regard.
Nothing wrong with that. I guess the question in my mind is whether we're aiming for making the tests "pythonic" (in which case "with" definitely wins), or more of a "bash with classes" style trying to avoid any constructs people may have to google. I'm on the fence on that one, as the del example proves my python expertise is not high. OTOH people who prefer bash will continue to write bash tests, so maybe we don't have to worry about non-experts too much. Dunno.
What I'm saying is, bash is currently a bit of a mess when it comes to cleanups. It's hard to get right, annoying to review, and sometimes individual cases add state that they don't unwind in cleanup() but only later in the function, so when you C-c half-way through such case, stuff stays behind.
Python has tools to just magic all this away.
On Wed, 3 Apr 2024 18:52:50 +0200 Petr Machata wrote:
Nothing wrong with that. I guess the question in my mind is whether we're aiming for making the tests "pythonic" (in which case "with" definitely wins), or more of a "bash with classes" style trying to avoid any constructs people may have to google. I'm on the fence on that one, as the del example proves my python expertise is not high. OTOH people who prefer bash will continue to write bash tests, so maybe we don't have to worry about non-experts too much. Dunno.
What I'm saying is, bash is currently a bit of a mess when it comes to cleanups. It's hard to get right, annoying to review, and sometimes individual cases add state that they don't unwind in cleanup() but only later in the function, so when you C-c half-way through such case, stuff stays behind.
Python has tools to just magic all this away.
Understood, just to be clear what I was saying is that +/- bugs in my example it is possible to "attach" the lifetime of things to a test object or such. Maybe people would be less likely to remember to do that than use "with"? Dunno. In any case, IIUC we don't have to decide now, so I went ahead with the v2 last night.
On 2024-04-02 10:31, Jakub Kicinski wrote:
On Tue, 2 Apr 2024 18:37:44 +0200 Petr Machata wrote:
Yeah, this would be usually done through context managers, as I mention in the other e-mail. But then cfg would be lexically scoped, which IMHO is a good thing, but then it needs to be passed around as an argument, and that makes the ksft_run() invocation a bit messy:
with NetDrvEnv(__file__) as cfg: ksft_run([lambda: check_pause(cfg), lambda: check_fec(cfg), lambda: pkt_byte_sum(cfg)])Dunno, maybe it could forward *args **kwargs to the cases? But then it loses some of the readability again.
Yes, I was wondering about that. It must be doable, IIRC the multi-threading API "injects" args from a tuple. I was thinking something along the lines of:
with NetDrvEnv(__file__) as cfg: ksft_run([check_pause, check_fec, pkt_byte_sum], args=(cfg, ))I got lazy, let me take a closer look. Another benefit will be that once we pass in "env" / cfg - we can "register" objects in there for auto-cleanup (in the future, current tests don't need cleanup)
How about a TestSuite class as a context manager and individual tests being methods? Then running the test suite runs all test cases and you won't need to add each test case manually to ksft_run().
On 2024-04-01 18:05, Jakub Kicinski wrote:
Add a very simple test to make sure drivers report expected stats. Drivers which implement FEC or pause configuration should report relevant stats. Qstats must be reported, at least packet and byte counts, and they must match total device stats.
Tested with netdevsim, bnxt, in-tree and installed.
Signed-off-by: Jakub Kicinski kuba@kernel.org
tools/testing/selftests/drivers/net/stats.py | 85 ++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100755 tools/testing/selftests/drivers/net/stats.py
diff --git a/tools/testing/selftests/drivers/net/stats.py b/tools/testing/selftests/drivers/net/stats.py new file mode 100755 index 000000000000..751cca2869b8 --- /dev/null +++ b/tools/testing/selftests/drivers/net/stats.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0
+from lib.py import ksft_run, ksft_in, ksft_true, KsftSkipEx, KsftXfailEx +from lib.py import EthtoolFamily, NetdevFamily, RtnlFamily, NlError +from lib.py import NetDrvEnv
+cfg = None +ethnl = EthtoolFamily() +netfam = NetdevFamily() +rtnl = RtnlFamily()
+def check_pause() -> None:
- global cfg, ethnl
- try:
ethnl.pause_get({"header": {"dev-index": cfg.ifindex}})- except NlError as e:
if e.error == 95:raise KsftXfailEx("pause not supported by the device")raise- data = ethnl.pause_get({"header": {"dev-index": cfg.ifindex,
"flags": {'stats'}}})- ksft_true(data['stats'], "driver does not report stats")
+def check_fec() -> None:
- global ethnl
- try:
ethnl.fec_get({"header": {"dev-index": cfg.ifindex}})- except NlError as e:
if e.error == 95:raise KsftXfailEx("FEC not supported by the device")raise- data = ethnl.fec_get({"header": {"dev-index": cfg.ifindex,
"flags": {'stats'}}})- ksft_true(data['stats'], "driver does not report stats")
+def pkt_byte_sum() -> None:
- global cfg, netfam, rtnl
- def get_qstat(test):
global netfamstats = netfam.qstats_get({}, dump=True)if stats:for qs in stats:if qs["ifindex"]== test.ifindex:return qs- qstat = get_qstat(cfg)
- if qstat is None:
raise KsftSkipEx("qstats not supported by the device")- for key in ['tx-packets', 'tx-bytes', 'rx-packets', 'rx-bytes']:
ksft_in(key, qstat, "Drivers should always report basic keys")- # Compare stats, rtnl stats and qstats must match,
- # but the interface may be up, so do a series of dumps
- # each time the more "recent" stats must be higher or same.
- def stat_cmp(rstat, qstat):
for key in ['tx-packets', 'tx-bytes', 'rx-packets', 'rx-bytes']:if rstat[key] != qstat[key]:return rstat[key] - qstat[key]return 0- for _ in range(10):
rtstat = rtnl.getlink({"ifi-index": cfg.ifindex})['stats']if stat_cmp(rtstat, qstat) < 0:raise Exception("RTNL stats are lower, fetched later")qstat = get_qstat(cfg)if stat_cmp(rtstat, qstat) > 0:raise Exception("Qstats are lower, fetched later")+if __name__ == "__main__":
- cfg = NetDrvEnv(__file__)
- try:
ksft_run([check_pause, check_fec, pkt_byte_sum])
Would there ever be a case where you don't want to run every test case in a suite?
- finally:
del cfg
linux-kselftest-mirror@lists.linaro.org
-
David Wei -
Jakub Kicinski -
Petr Machata