January 2021 - Linux-kselftest-mirror

Re: [RFC PATCH bpf-next 1/2] bpf: share BTF "show" implementation between kernel and libbpf

by Andrii Nakryiko

On Mon, Jan 11, 2021 at 9:34 AM Alan Maguire <alan.maguire(a)oracle.com> wrote: > > libbpf already supports a "dumper" API for dumping type information, > but there is currently no support for dumping typed _data_ via libbpf. > However this functionality does exist in the kernel, in part to > facilitate the bpf_snprintf_btf() helper which dumps a string > representation of the pointer passed in utilizing the BTF type id > of the data pointed to. For example, the pair of a pointer to > a "struct sk_buff" and the BTF type id of "struct sk_buff" can be > used. > > Here the kernel code is generalized into btf_show_common.c. For the > most part, code is identical for userspace and kernel, beyond a few API > differences and missing functions. The only significant differences are > > - the "safe copy" logic used by the kernel to ensure we do not induce a > crash during BPF operation; and > - the BTF seq file support that is kernel-only. > > The mechanics are to maintain identical btf_show_common.c files in > kernel/bpf and tools/lib/bpf , and a common header btf_common.h in > include/linux/ and tools/lib/bpf/. This file duplication seems to > be the common practice with duplication between kernel and tools/ > so it's the approach taken here. > > The common code approach could likely be explored further, but here > the minimum common code required to support BTF show functionality is > used. > I don't think this approach will work. libbpf and kernel have considerably different restrictions and styles, I don't think it's appropriate to take kernel code and try to fit it into libbpf almost as is, with a bunch of #defines. It would be much cleaner, simpler, and more maintainable to just re-implement core logic for libbpf, IMO. > Currently the only "show" function for userspace is to write the > representation of the typed data to a string via > > LIBBPF_API int > btf__snprintf(struct btf *btf, char *buf, int len, __u32 id, void *obj, > __u64 flags); > > ...but other approaches could be pursued including printf()-based > show, or even a callback mechanism could be supported to allow > user-defined show functions. > It's strange that you saw btf_dump APIs, and yet decided to go with this API instead. snprintf() is not a natural "method" of struct btf. Using char buffer as an output is overly restrictive and inconvenient. It's appropriate for kernel and BPF program due to their restrictions, but there is no need to cripple libbpf APIs for that. I think it should follow btf_dump APIs with custom callback so that it's easy to just printf() everything, but also user can create whatever elaborate mechanism they need and that fits their use case. Code reuse is not the ultimate goal, it should facilitate maintainability, not harm it. There are times where sharing code introduces unnecessary coupling and maintainability issues. And I think this one is a very obvious case of that. See below a few comments as well. But overall it's really hard to review such a humongous patch, of course. So I so far just skimmed through it. > Here's an example usage, storing a string representation of > struct sk_buff *skb in buf: > > struct btf *btf = libbpf_find_kernel_btf(); > char buf[8192]; > __s32 skb_id; > > skb_id = btf__find_by_name_kind(btf, "sk_buff", BTF_KIND_STRUCT); > if (skb_id < 0) > fprintf(stderr, "no skbuff, err %d\n", skb_id); > else > btf__snprintf(btf, buf, sizeof(buf), skb_id, skb, 0); > > Suggested-by: Alexei Starovoitov <ast(a)kernel.org> > Signed-off-by: Alan Maguire <alan.maguire(a)oracle.com> > --- > include/linux/btf.h | 121 +--- > include/linux/btf_common.h | 286 +++++++++ > kernel/bpf/Makefile | 2 +- > kernel/bpf/arraymap.c | 1 + > kernel/bpf/bpf_struct_ops.c | 1 + > kernel/bpf/btf.c | 1215 +------------------------------------- > kernel/bpf/btf_show_common.c | 1218 +++++++++++++++++++++++++++++++++++++++ > kernel/bpf/core.c | 1 + > kernel/bpf/hashtab.c | 1 + > kernel/bpf/local_storage.c | 1 + > kernel/bpf/verifier.c | 1 + > kernel/trace/bpf_trace.c | 1 + > tools/lib/bpf/Build | 2 +- > tools/lib/bpf/btf.h | 7 + > tools/lib/bpf/btf_common.h | 286 +++++++++ > tools/lib/bpf/btf_show_common.c | 1218 +++++++++++++++++++++++++++++++++++++++ > tools/lib/bpf/libbpf.map | 1 + > 17 files changed, 3044 insertions(+), 1319 deletions(-) > create mode 100644 include/linux/btf_common.h > create mode 100644 kernel/bpf/btf_show_common.c > create mode 100644 tools/lib/bpf/btf_common.h > create mode 100644 tools/lib/bpf/btf_show_common.c > [...] > +/* For kernel u64 is long long unsigned int... */ > +#define FMT64 "ll" > + > +#else > +/* ...while for userspace it is long unsigned int. These definitions avoid > + * format specifier warnings. > + */ that's not true, it depends on the architecture > +#define FMT64 "l" > + > +/* libbpf names differ slightly to in-kernel function names. */ > +#define btf_type_by_id btf__type_by_id > +#define btf_name_by_offset btf__name_by_offset > +#define btf_str_by_offset btf__str_by_offset > +#define btf_resolve_size btf__resolve_size ugh... good luck navigating the code in libbpf.... > + > +#endif /* __KERNEL__ */ > +/* > + * Options to control show behaviour. > + * - BTF_SHOW_COMPACT: no formatting around type information > + * - BTF_SHOW_NONAME: no struct/union member names/types > + * - BTF_SHOW_PTR_RAW: show raw (unobfuscated) pointer values; > + * equivalent to %px. > + * - BTF_SHOW_ZERO: show zero-valued struct/union members; they > + * are not displayed by default > + * - BTF_SHOW_UNSAFE: skip use of bpf_probe_read() to safely read > + * data before displaying it. > + */ > +#define BTF_SHOW_COMPACT BTF_F_COMPACT > +#define BTF_SHOW_NONAME BTF_F_NONAME > +#define BTF_SHOW_PTR_RAW BTF_F_PTR_RAW > +#define BTF_SHOW_ZERO BTF_F_ZERO > +#define BTF_SHOW_UNSAFE (1ULL << 4) this (or some subset of them) should be done as opts struct's bool fields for libbpf > + > +/* > + * Copy len bytes of string representation of obj of BTF type_id into buf. > + * > + * @btf: struct btf object > + * @type_id: type id of type obj points to > + * @obj: pointer to typed data > + * @buf: buffer to write to > + * @len: maximum length to write to buf > + * @flags: show options (see above) > + * > + * Return: length that would have been/was copied as per snprintf, or > + * negative error. > + */ > +int btf_type_snprintf_show(const struct btf *btf, u32 type_id, void *obj, > + char *buf, int len, u64 flags); > + > +#define for_each_member(i, struct_type, member) \ > + for (i = 0, member = btf_type_member(struct_type); \ > + i < btf_type_vlen(struct_type); \ > + i++, member++) > + > +#define for_each_vsi(i, datasec_type, member) \ > + for (i = 0, member = btf_type_var_secinfo(datasec_type); \ > + i < btf_type_vlen(datasec_type); \ > + i++, member++) > + > +static inline bool btf_type_is_ptr(const struct btf_type *t) > +{ > + return BTF_INFO_KIND(t->info) == BTF_KIND_PTR; > +} > + > +static inline bool btf_type_is_int(const struct btf_type *t) > +{ > + return BTF_INFO_KIND(t->info) == BTF_KIND_INT; > +} > + > +static inline bool btf_type_is_small_int(const struct btf_type *t) > +{ > + return btf_type_is_int(t) && t->size <= sizeof(u64); > +} > + > +static inline bool btf_type_is_enum(const struct btf_type *t) > +{ > + return BTF_INFO_KIND(t->info) == BTF_KIND_ENUM; > +} > + > +static inline bool btf_type_is_typedef(const struct btf_type *t) > +{ > + return BTF_INFO_KIND(t->info) == BTF_KIND_TYPEDEF; > +} > + > +static inline bool btf_type_is_func(const struct btf_type *t) > +{ > + return BTF_INFO_KIND(t->info) == BTF_KIND_FUNC; > +} > + > +static inline bool btf_type_is_func_proto(const struct btf_type *t) > +{ > + return BTF_INFO_KIND(t->info) == BTF_KIND_FUNC_PROTO; > +} > + > +static inline bool btf_type_is_var(const struct btf_type *t) > +{ > + return BTF_INFO_KIND(t->info) == BTF_KIND_VAR; > +} > + > +/* union is only a special case of struct: > + * all its offsetof(member) == 0 > + */ > +static inline bool btf_type_is_struct(const struct btf_type *t) > +{ > + u8 kind = BTF_INFO_KIND(t->info); > + > + return kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION; > +} > + > +static inline bool btf_type_is_modifier(const struct btf_type *t) > +{ > + /* Some of them is not strictly a C modifier > + * but they are grouped into the same bucket > + * for BTF concern: > + * A type (t) that refers to another > + * type through t->type AND its size cannot > + * be determined without following the t->type. > + * > + * ptr does not fall into this bucket > + * because its size is always sizeof(void *). > + */ > + switch (BTF_INFO_KIND(t->info)) { > + case BTF_KIND_TYPEDEF: > + case BTF_KIND_VOLATILE: > + case BTF_KIND_CONST: > + case BTF_KIND_RESTRICT: > + return true; > + default: > + return false; > + } > +} > + > +static inline > +const struct btf_type *btf_type_skip_modifiers(const struct btf *btf, > + u32 id, u32 *res_id) > +{ > + const struct btf_type *t = btf_type_by_id(btf, id); > + > + while (btf_type_is_modifier(t)) { > + id = t->type; > + t = btf_type_by_id(btf, t->type); > + } > + > + if (res_id) > + *res_id = id; > + > + return t; > +} > + > +static inline u32 btf_type_int(const struct btf_type *t) > +{ > + return *(u32 *)(t + 1); > +} > + > +static inline const struct btf_array *btf_type_array(const struct btf_type *t) > +{ > + return (const struct btf_array *)(t + 1); > +} > + > +static inline const struct btf_enum *btf_type_enum(const struct btf_type *t) > +{ > + return (const struct btf_enum *)(t + 1); > +} > + > +static inline const struct btf_var *btf_type_var(const struct btf_type *t) > +{ > + return (const struct btf_var *)(t + 1); > +} > + > +static inline u16 btf_type_vlen(const struct btf_type *t) > +{ > + return BTF_INFO_VLEN(t->info); > +} > + > +static inline u16 btf_func_linkage(const struct btf_type *t) > +{ > + return BTF_INFO_VLEN(t->info); > +} > + > +/* size can be used */ > +static inline bool btf_type_has_size(const struct btf_type *t) > +{ > + switch (BTF_INFO_KIND(t->info)) { > + case BTF_KIND_INT: > + case BTF_KIND_STRUCT: > + case BTF_KIND_UNION: > + case BTF_KIND_ENUM: > + case BTF_KIND_DATASEC: > + return true; > + default: > + return false; > + } > +} > + > +static inline const struct btf_member *btf_type_member(const struct btf_type *t) > +{ > + return (const struct btf_member *)(t + 1); > +} > + > +static inline const struct btf_var_secinfo *btf_type_var_secinfo( > + const struct btf_type *t) > +{ > + return (const struct btf_var_secinfo *)(t + 1); > +} > + > +static inline const char *__btf_name_by_offset(const struct btf *btf, > + u32 offset) > +{ > + const char *name; > + > + if (!offset) > + return "(anon)"; > + > + name = btf_str_by_offset(btf, offset); > + return name ?: "(invalid-name-offset)"; > +} > + (almost?) all of the above helpers are already defined in libbpf's btf.h, no need to add all this duplication > +/* functions shared between btf.c and btf_show_common.c */ > +void btf_type_ops_show(const struct btf *btf, const struct btf_type *t, > + __u32 type_id, void *obj, u8 bits_offset, > + struct btf_show *show); [...] > diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map > index 1c0fd2d..35bd9dc 100644 > --- a/tools/lib/bpf/libbpf.map > +++ b/tools/lib/bpf/libbpf.map > @@ -346,6 +346,7 @@ LIBBPF_0.3.0 { > btf__parse_split; > btf__new_empty_split; > btf__new_split; > + btf__snprintf; It's LIBBPF_0.4.0 already, I or someone else should send a patch adding a new section in .map file. > ring_buffer__epoll_fd; > xsk_setup_xdp_prog; > xsk_socket__update_xskmap; > -- > 1.8.3.1 >

3 years, 11 months

2
2
0 0

[PATCH v4 1/3] kunit: tool: surface and address more typing issues

by Daniel Latypov

The authors of this tool were more familiar with a different type-checker, https://github.com/google/pytype. That's open source, but mypy seems more prevalent (and runs faster). And unlike pytype, mypy doesn't try to infer types so it doesn't check unanotated functions. So annotate ~all functions in kunit tool to increase type-checking coverage. Note: per https://www.python.org/dev/peps/pep-0484/, `__init__()` should be annotated as `-> None`. Doing so makes mypy discover a number of new violations. Exclude main() since we reuse `request` for the different types of requests, which mypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Reviewed-by: David Gow <davidgow(a)google.com> --- Changes since v1: none here, reworked last patch. Changes since v2: rebased onto torvalds/master. Changes since v3: rebased again torvalds/master. --- tools/testing/kunit/kunit.py | 14 ++++----- tools/testing/kunit/kunit_config.py | 7 +++-- tools/testing/kunit/kunit_json.py | 2 +- tools/testing/kunit/kunit_kernel.py | 37 ++++++++++++----------- tools/testing/kunit/kunit_parser.py | 46 ++++++++++++++--------------- 5 files changed, 54 insertions(+), 52 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 21516e293d17..5521e0a8201e 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -43,9 +43,9 @@ class KunitStatus(Enum): BUILD_FAILURE = auto() TEST_FAILURE = auto() -def get_kernel_root_path(): - parts = sys.argv[0] if not __file__ else __file__ - parts = os.path.realpath(parts).split('tools/testing/kunit') +def get_kernel_root_path() -> str: + path = sys.argv[0] if not __file__ else __file__ + parts = os.path.realpath(path).split('tools/testing/kunit') if len(parts) != 2: sys.exit(1) return parts[0] @@ -171,7 +171,7 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, exec_result.elapsed_time)) return parse_result -def add_common_opts(parser): +def add_common_opts(parser) -> None: parser.add_argument('--build_dir', help='As in the make command, it specifies the build ' 'directory.', @@ -183,13 +183,13 @@ def add_common_opts(parser): help='Run all KUnit tests through allyesconfig', action='store_true') -def add_build_opts(parser): +def add_build_opts(parser) -> None: parser.add_argument('--jobs', help='As in the make command, "Specifies the number of ' 'jobs (commands) to run simultaneously."', type=int, default=8, metavar='jobs') -def add_exec_opts(parser): +def add_exec_opts(parser) -> None: parser.add_argument('--timeout', help='maximum number of seconds to allow for all tests ' 'to run. This does not include time taken to build the ' @@ -198,7 +198,7 @@ def add_exec_opts(parser): default=300, metavar='timeout') -def add_parse_opts(parser): +def add_parse_opts(parser) -> None: parser.add_argument('--raw_output', help='don\'t format output from kernel', action='store_true') parser.add_argument('--json', diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py index 02ffc3a3e5dc..bdd60230764b 100644 --- a/tools/testing/kunit/kunit_config.py +++ b/tools/testing/kunit/kunit_config.py @@ -8,6 +8,7 @@ import collections import re +from typing import List, Set CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_(\w+) is not set$' CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+|".*")$' @@ -30,10 +31,10 @@ class KconfigParseError(Exception): class Kconfig(object): """Represents defconfig or .config specified using the Kconfig language.""" - def __init__(self): - self._entries = [] + def __init__(self) -> None: + self._entries = [] # type: List[KconfigEntry] - def entries(self): + def entries(self) -> Set[KconfigEntry]: return set(self._entries) def add_entry(self, entry: KconfigEntry) -> None: diff --git a/tools/testing/kunit/kunit_json.py b/tools/testing/kunit/kunit_json.py index 624b31b2dbd6..f5cca5c38cac 100644 --- a/tools/testing/kunit/kunit_json.py +++ b/tools/testing/kunit/kunit_json.py @@ -13,7 +13,7 @@ import kunit_parser from kunit_parser import TestStatus -def get_json_result(test_result, def_config, build_dir, json_path): +def get_json_result(test_result, def_config, build_dir, json_path) -> str: sub_groups = [] # Each test suite is mapped to a KernelCI sub_group diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 698358c9c0d6..e77ee06aa407 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -11,6 +11,7 @@ import subprocess import os import shutil import signal +from typing import Iterator from contextlib import ExitStack @@ -39,7 +40,7 @@ class BuildError(Exception): class LinuxSourceTreeOperations(object): """An abstraction over command line operations performed on a source tree.""" - def make_mrproper(self): + def make_mrproper(self) -> None: try: subprocess.check_output(['make', 'mrproper'], stderr=subprocess.STDOUT) except OSError as e: @@ -47,7 +48,7 @@ class LinuxSourceTreeOperations(object): except subprocess.CalledProcessError as e: raise ConfigError(e.output.decode()) - def make_olddefconfig(self, build_dir, make_options): + def make_olddefconfig(self, build_dir, make_options) -> None: command = ['make', 'ARCH=um', 'olddefconfig'] if make_options: command.extend(make_options) @@ -60,7 +61,7 @@ class LinuxSourceTreeOperations(object): except subprocess.CalledProcessError as e: raise ConfigError(e.output.decode()) - def make_allyesconfig(self, build_dir, make_options): + def make_allyesconfig(self, build_dir, make_options) -> None: kunit_parser.print_with_timestamp( 'Enabling all CONFIGs for UML...') command = ['make', 'ARCH=um', 'allyesconfig'] @@ -82,7 +83,7 @@ class LinuxSourceTreeOperations(object): kunit_parser.print_with_timestamp( 'Starting Kernel with all configs takes a few minutes...') - def make(self, jobs, build_dir, make_options): + def make(self, jobs, build_dir, make_options) -> None: command = ['make', 'ARCH=um', '--jobs=' + str(jobs)] if make_options: command.extend(make_options) @@ -100,7 +101,7 @@ class LinuxSourceTreeOperations(object): if stderr: # likely only due to build warnings print(stderr.decode()) - def linux_bin(self, params, timeout, build_dir): + def linux_bin(self, params, timeout, build_dir) -> None: """Runs the Linux UML binary. Must be named 'linux'.""" linux_bin = get_file_path(build_dir, 'linux') outfile = get_outfile_path(build_dir) @@ -110,23 +111,23 @@ class LinuxSourceTreeOperations(object): stderr=subprocess.STDOUT) process.wait(timeout) -def get_kconfig_path(build_dir): +def get_kconfig_path(build_dir) -> str: return get_file_path(build_dir, KCONFIG_PATH) -def get_kunitconfig_path(build_dir): +def get_kunitconfig_path(build_dir) -> str: return get_file_path(build_dir, KUNITCONFIG_PATH) -def get_outfile_path(build_dir): +def get_outfile_path(build_dir) -> str: return get_file_path(build_dir, OUTFILE_PATH) class LinuxSourceTree(object): """Represents a Linux kernel source tree with KUnit tests.""" - def __init__(self): + def __init__(self) -> None: self._ops = LinuxSourceTreeOperations() signal.signal(signal.SIGINT, self.signal_handler) - def clean(self): + def clean(self) -> bool: try: self._ops.make_mrproper() except ConfigError as e: @@ -134,17 +135,17 @@ class LinuxSourceTree(object): return False return True - def create_kunitconfig(self, build_dir, defconfig=DEFAULT_KUNITCONFIG_PATH): + def create_kunitconfig(self, build_dir, defconfig=DEFAULT_KUNITCONFIG_PATH) -> None: kunitconfig_path = get_kunitconfig_path(build_dir) if not os.path.exists(kunitconfig_path): shutil.copyfile(defconfig, kunitconfig_path) - def read_kunitconfig(self, build_dir): + def read_kunitconfig(self, build_dir) -> None: kunitconfig_path = get_kunitconfig_path(build_dir) self._kconfig = kunit_config.Kconfig() self._kconfig.read_from_file(kunitconfig_path) - def validate_config(self, build_dir): + def validate_config(self, build_dir) -> bool: kconfig_path = get_kconfig_path(build_dir) validated_kconfig = kunit_config.Kconfig() validated_kconfig.read_from_file(kconfig_path) @@ -158,7 +159,7 @@ class LinuxSourceTree(object): return False return True - def build_config(self, build_dir, make_options): + def build_config(self, build_dir, make_options) -> bool: kconfig_path = get_kconfig_path(build_dir) if build_dir and not os.path.exists(build_dir): os.mkdir(build_dir) @@ -170,7 +171,7 @@ class LinuxSourceTree(object): return False return self.validate_config(build_dir) - def build_reconfig(self, build_dir, make_options): + def build_reconfig(self, build_dir, make_options) -> bool: """Creates a new .config if it is not a subset of the .kunitconfig.""" kconfig_path = get_kconfig_path(build_dir) if os.path.exists(kconfig_path): @@ -186,7 +187,7 @@ class LinuxSourceTree(object): print('Generating .config ...') return self.build_config(build_dir, make_options) - def build_um_kernel(self, alltests, jobs, build_dir, make_options): + def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool: try: if alltests: self._ops.make_allyesconfig(build_dir, make_options) @@ -197,7 +198,7 @@ class LinuxSourceTree(object): return False return self.validate_config(build_dir) - def run_kernel(self, args=[], build_dir='', timeout=None): + def run_kernel(self, args=[], build_dir='', timeout=None) -> Iterator[str]: args.extend(['mem=1G', 'console=tty']) self._ops.linux_bin(args, timeout, build_dir) outfile = get_outfile_path(build_dir) @@ -206,6 +207,6 @@ class LinuxSourceTree(object): for line in file: yield line - def signal_handler(self, sig, frame): + def signal_handler(self, sig, frame) -> None: logging.error('Build interruption occurred. Cleaning console.') subprocess.call(['stty', 'sane']) diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 6614ec4d0898..8b5eb9507765 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -12,32 +12,32 @@ from collections import namedtuple from datetime import datetime from enum import Enum, auto from functools import reduce -from typing import List, Optional, Tuple +from typing import Iterator, List, Optional, Tuple TestResult = namedtuple('TestResult', ['status','suites','log']) class TestSuite(object): - def __init__(self): - self.status = None - self.name = None - self.cases = [] + def __init__(self) -> None: + self.status = None # type: Optional[TestStatus] + self.name = '' + self.cases = [] # type: List[TestCase] - def __str__(self): - return 'TestSuite(' + self.status + ',' + self.name + ',' + str(self.cases) + ')' + def __str__(self) -> str: + return 'TestSuite(' + str(self.status) + ',' + self.name + ',' + str(self.cases) + ')' - def __repr__(self): + def __repr__(self) -> str: return str(self) class TestCase(object): - def __init__(self): - self.status = None + def __init__(self) -> None: + self.status = None # type: Optional[TestStatus] self.name = '' - self.log = [] + self.log = [] # type: List[str] - def __str__(self): - return 'TestCase(' + self.status + ',' + self.name + ',' + str(self.log) + ')' + def __str__(self) -> str: + return 'TestCase(' + str(self.status) + ',' + self.name + ',' + str(self.log) + ')' - def __repr__(self): + def __repr__(self) -> str: return str(self) class TestStatus(Enum): @@ -51,7 +51,7 @@ kunit_start_re = re.compile(r'TAP version [0-9]+$') kunit_end_re = re.compile('(List of all partitions:|' 'Kernel panic - not syncing: VFS:)') -def isolate_kunit_output(kernel_output): +def isolate_kunit_output(kernel_output) -> Iterator[str]: started = False for line in kernel_output: line = line.rstrip() # line always has a trailing \n @@ -64,7 +64,7 @@ def isolate_kunit_output(kernel_output): elif started: yield line[prefix_len:] if prefix_len > 0 else line -def raw_output(kernel_output): +def raw_output(kernel_output) -> None: for line in kernel_output: print(line.rstrip()) @@ -72,26 +72,26 @@ DIVIDER = '=' * 60 RESET = '\033[0;0m' -def red(text): +def red(text) -> str: return '\033[1;31m' + text + RESET -def yellow(text): +def yellow(text) -> str: return '\033[1;33m' + text + RESET -def green(text): +def green(text) -> str: return '\033[1;32m' + text + RESET -def print_with_timestamp(message): +def print_with_timestamp(message) -> None: print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) -def format_suite_divider(message): +def format_suite_divider(message) -> str: return '======== ' + message + ' ========' -def print_suite_divider(message): +def print_suite_divider(message) -> None: print_with_timestamp(DIVIDER) print_with_timestamp(format_suite_divider(message)) -def print_log(log): +def print_log(log) -> None: for m in log: print_with_timestamp(m) base-commit: 146620506274bd24d52fb1c589110a30eed8240b -- 2.30.0.296.g2bfb1c46d8-goog

3 years, 11 months

2
5
0 0

[PATCH v3 1/3] kunit: tool: surface and address more typing issues

by Daniel Latypov

The authors of this tool were more familiar with a different type-checker, https://github.com/google/pytype. That's open source, but mypy seems more prevalent (and runs faster). And unlike pytype, mypy doesn't try to infer types so it doesn't check unanotated functions. So annotate ~all functions in kunit tool to increase type-checking coverage. Note: per https://www.python.org/dev/peps/pep-0484/, `__init__()` should be annotated as `-> None`. Doing so makes mypy discover a number of new violations. Exclude main() since we reuse `request` for the different types of requests, which mypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Reviewed-by: David Gow <davidgow(a)google.com> --- tools/testing/kunit/kunit.py | 14 ++++----- tools/testing/kunit/kunit_config.py | 7 +++-- tools/testing/kunit/kunit_json.py | 2 +- tools/testing/kunit/kunit_kernel.py | 37 ++++++++++++----------- tools/testing/kunit/kunit_parser.py | 46 ++++++++++++++--------------- 5 files changed, 54 insertions(+), 52 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 21516e293d17..5521e0a8201e 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -43,9 +43,9 @@ class KunitStatus(Enum): BUILD_FAILURE = auto() TEST_FAILURE = auto() -def get_kernel_root_path(): - parts = sys.argv[0] if not __file__ else __file__ - parts = os.path.realpath(parts).split('tools/testing/kunit') +def get_kernel_root_path() -> str: + path = sys.argv[0] if not __file__ else __file__ + parts = os.path.realpath(path).split('tools/testing/kunit') if len(parts) != 2: sys.exit(1) return parts[0] @@ -171,7 +171,7 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, exec_result.elapsed_time)) return parse_result -def add_common_opts(parser): +def add_common_opts(parser) -> None: parser.add_argument('--build_dir', help='As in the make command, it specifies the build ' 'directory.', @@ -183,13 +183,13 @@ def add_common_opts(parser): help='Run all KUnit tests through allyesconfig', action='store_true') -def add_build_opts(parser): +def add_build_opts(parser) -> None: parser.add_argument('--jobs', help='As in the make command, "Specifies the number of ' 'jobs (commands) to run simultaneously."', type=int, default=8, metavar='jobs') -def add_exec_opts(parser): +def add_exec_opts(parser) -> None: parser.add_argument('--timeout', help='maximum number of seconds to allow for all tests ' 'to run. This does not include time taken to build the ' @@ -198,7 +198,7 @@ def add_exec_opts(parser): default=300, metavar='timeout') -def add_parse_opts(parser): +def add_parse_opts(parser) -> None: parser.add_argument('--raw_output', help='don\'t format output from kernel', action='store_true') parser.add_argument('--json', diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py index 02ffc3a3e5dc..bdd60230764b 100644 --- a/tools/testing/kunit/kunit_config.py +++ b/tools/testing/kunit/kunit_config.py @@ -8,6 +8,7 @@ import collections import re +from typing import List, Set CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_(\w+) is not set$' CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+|".*")$' @@ -30,10 +31,10 @@ class KconfigParseError(Exception): class Kconfig(object): """Represents defconfig or .config specified using the Kconfig language.""" - def __init__(self): - self._entries = [] + def __init__(self) -> None: + self._entries = [] # type: List[KconfigEntry] - def entries(self): + def entries(self) -> Set[KconfigEntry]: return set(self._entries) def add_entry(self, entry: KconfigEntry) -> None: diff --git a/tools/testing/kunit/kunit_json.py b/tools/testing/kunit/kunit_json.py index 624b31b2dbd6..f5cca5c38cac 100644 --- a/tools/testing/kunit/kunit_json.py +++ b/tools/testing/kunit/kunit_json.py @@ -13,7 +13,7 @@ import kunit_parser from kunit_parser import TestStatus -def get_json_result(test_result, def_config, build_dir, json_path): +def get_json_result(test_result, def_config, build_dir, json_path) -> str: sub_groups = [] # Each test suite is mapped to a KernelCI sub_group diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 57c1724b7e5d..db7ed84ea410 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -11,6 +11,7 @@ import subprocess import os import shutil import signal +from typing import Iterator from contextlib import ExitStack @@ -39,7 +40,7 @@ class BuildError(Exception): class LinuxSourceTreeOperations(object): """An abstraction over command line operations performed on a source tree.""" - def make_mrproper(self): + def make_mrproper(self) -> None: try: subprocess.check_output(['make', 'mrproper'], stderr=subprocess.STDOUT) except OSError as e: @@ -47,7 +48,7 @@ class LinuxSourceTreeOperations(object): except subprocess.CalledProcessError as e: raise ConfigError(e.output.decode()) - def make_olddefconfig(self, build_dir, make_options): + def make_olddefconfig(self, build_dir, make_options) -> None: command = ['make', 'ARCH=um', 'olddefconfig'] if make_options: command.extend(make_options) @@ -60,7 +61,7 @@ class LinuxSourceTreeOperations(object): except subprocess.CalledProcessError as e: raise ConfigError(e.output.decode()) - def make_allyesconfig(self, build_dir, make_options): + def make_allyesconfig(self, build_dir, make_options) -> None: kunit_parser.print_with_timestamp( 'Enabling all CONFIGs for UML...') command = ['make', 'ARCH=um', 'allyesconfig'] @@ -82,7 +83,7 @@ class LinuxSourceTreeOperations(object): kunit_parser.print_with_timestamp( 'Starting Kernel with all configs takes a few minutes...') - def make(self, jobs, build_dir, make_options): + def make(self, jobs, build_dir, make_options) -> None: command = ['make', 'ARCH=um', '--jobs=' + str(jobs)] if make_options: command.extend(make_options) @@ -100,7 +101,7 @@ class LinuxSourceTreeOperations(object): if stderr: # likely only due to build warnings print(stderr.decode()) - def linux_bin(self, params, timeout, build_dir): + def linux_bin(self, params, timeout, build_dir) -> None: """Runs the Linux UML binary. Must be named 'linux'.""" linux_bin = get_file_path(build_dir, 'linux') outfile = get_outfile_path(build_dir) @@ -110,23 +111,23 @@ class LinuxSourceTreeOperations(object): stderr=subprocess.STDOUT) process.wait(timeout) -def get_kconfig_path(build_dir): +def get_kconfig_path(build_dir) -> str: return get_file_path(build_dir, KCONFIG_PATH) -def get_kunitconfig_path(build_dir): +def get_kunitconfig_path(build_dir) -> str: return get_file_path(build_dir, KUNITCONFIG_PATH) -def get_outfile_path(build_dir): +def get_outfile_path(build_dir) -> str: return get_file_path(build_dir, OUTFILE_PATH) class LinuxSourceTree(object): """Represents a Linux kernel source tree with KUnit tests.""" - def __init__(self): + def __init__(self) -> None: self._ops = LinuxSourceTreeOperations() signal.signal(signal.SIGINT, self.signal_handler) - def clean(self): + def clean(self) -> bool: try: self._ops.make_mrproper() except ConfigError as e: @@ -134,17 +135,17 @@ class LinuxSourceTree(object): return False return True - def create_kunitconfig(self, build_dir, defconfig=DEFAULT_KUNITCONFIG_PATH): + def create_kunitconfig(self, build_dir, defconfig=DEFAULT_KUNITCONFIG_PATH) -> None: kunitconfig_path = get_kunitconfig_path(build_dir) if not os.path.exists(kunitconfig_path): shutil.copyfile(defconfig, kunitconfig_path) - def read_kunitconfig(self, build_dir): + def read_kunitconfig(self, build_dir) -> None: kunitconfig_path = get_kunitconfig_path(build_dir) self._kconfig = kunit_config.Kconfig() self._kconfig.read_from_file(kunitconfig_path) - def validate_config(self, build_dir): + def validate_config(self, build_dir) -> bool: kconfig_path = get_kconfig_path(build_dir) validated_kconfig = kunit_config.Kconfig() validated_kconfig.read_from_file(kconfig_path) @@ -158,7 +159,7 @@ class LinuxSourceTree(object): return False return True - def build_config(self, build_dir, make_options): + def build_config(self, build_dir, make_options) -> bool: kconfig_path = get_kconfig_path(build_dir) if build_dir and not os.path.exists(build_dir): os.mkdir(build_dir) @@ -170,7 +171,7 @@ class LinuxSourceTree(object): return False return self.validate_config(build_dir) - def build_reconfig(self, build_dir, make_options): + def build_reconfig(self, build_dir, make_options) -> bool: """Creates a new .config if it is not a subset of the .kunitconfig.""" kconfig_path = get_kconfig_path(build_dir) if os.path.exists(kconfig_path): @@ -186,7 +187,7 @@ class LinuxSourceTree(object): print('Generating .config ...') return self.build_config(build_dir, make_options) - def build_um_kernel(self, alltests, jobs, build_dir, make_options): + def build_um_kernel(self, alltests, jobs, build_dir, make_options) -> bool: try: if alltests: self._ops.make_allyesconfig(build_dir, make_options) @@ -197,7 +198,7 @@ class LinuxSourceTree(object): return False return self.validate_config(build_dir) - def run_kernel(self, args=[], build_dir='', timeout=None): + def run_kernel(self, args=[], build_dir='', timeout=None) -> Iterator[str]: args.extend(['mem=1G']) self._ops.linux_bin(args, timeout, build_dir) outfile = get_outfile_path(build_dir) @@ -206,6 +207,6 @@ class LinuxSourceTree(object): for line in file: yield line - def signal_handler(self, sig, frame): + def signal_handler(self, sig, frame) -> None: logging.error('Build interruption occurred. Cleaning console.') subprocess.call(['stty', 'sane']) diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 6614ec4d0898..8b5eb9507765 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -12,32 +12,32 @@ from collections import namedtuple from datetime import datetime from enum import Enum, auto from functools import reduce -from typing import List, Optional, Tuple +from typing import Iterator, List, Optional, Tuple TestResult = namedtuple('TestResult', ['status','suites','log']) class TestSuite(object): - def __init__(self): - self.status = None - self.name = None - self.cases = [] + def __init__(self) -> None: + self.status = None # type: Optional[TestStatus] + self.name = '' + self.cases = [] # type: List[TestCase] - def __str__(self): - return 'TestSuite(' + self.status + ',' + self.name + ',' + str(self.cases) + ')' + def __str__(self) -> str: + return 'TestSuite(' + str(self.status) + ',' + self.name + ',' + str(self.cases) + ')' - def __repr__(self): + def __repr__(self) -> str: return str(self) class TestCase(object): - def __init__(self): - self.status = None + def __init__(self) -> None: + self.status = None # type: Optional[TestStatus] self.name = '' - self.log = [] + self.log = [] # type: List[str] - def __str__(self): - return 'TestCase(' + self.status + ',' + self.name + ',' + str(self.log) + ')' + def __str__(self) -> str: + return 'TestCase(' + str(self.status) + ',' + self.name + ',' + str(self.log) + ')' - def __repr__(self): + def __repr__(self) -> str: return str(self) class TestStatus(Enum): @@ -51,7 +51,7 @@ kunit_start_re = re.compile(r'TAP version [0-9]+$') kunit_end_re = re.compile('(List of all partitions:|' 'Kernel panic - not syncing: VFS:)') -def isolate_kunit_output(kernel_output): +def isolate_kunit_output(kernel_output) -> Iterator[str]: started = False for line in kernel_output: line = line.rstrip() # line always has a trailing \n @@ -64,7 +64,7 @@ def isolate_kunit_output(kernel_output): elif started: yield line[prefix_len:] if prefix_len > 0 else line -def raw_output(kernel_output): +def raw_output(kernel_output) -> None: for line in kernel_output: print(line.rstrip()) @@ -72,26 +72,26 @@ DIVIDER = '=' * 60 RESET = '\033[0;0m' -def red(text): +def red(text) -> str: return '\033[1;31m' + text + RESET -def yellow(text): +def yellow(text) -> str: return '\033[1;33m' + text + RESET -def green(text): +def green(text) -> str: return '\033[1;32m' + text + RESET -def print_with_timestamp(message): +def print_with_timestamp(message) -> None: print('[%s] %s' % (datetime.now().strftime('%H:%M:%S'), message)) -def format_suite_divider(message): +def format_suite_divider(message) -> str: return '======== ' + message + ' ========' -def print_suite_divider(message): +def print_suite_divider(message) -> None: print_with_timestamp(DIVIDER) print_with_timestamp(format_suite_divider(message)) -def print_log(log): +def print_log(log) -> None: for m in log: print_with_timestamp(m) base-commit: f5e6c330254ae691f6d7befe61c786eb5056007e -- 2.29.2.729.g45daf8777d-goog

3 years, 11 months

2
6
0 0

[PATCH] kunit: tool: Fix spelling of "diagnostic" in kunit_parser

by David Gow

Various helper functions were misspelling "diagnostic" in their names. It finally got annoying, so fix it. Signed-off-by: David Gow <davidgow(a)google.com> --- tools/testing/kunit/kunit_parser.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 6614ec4d0898..1a1e1d13f1d3 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -97,11 +97,11 @@ def print_log(log): TAP_ENTRIES = re.compile(r'^(TAP|[\s]*ok|[\s]*not ok|[\s]*[0-9]+\.\.[0-9]+|[\s]*#).*$') -def consume_non_diagnositic(lines: List[str]) -> None: +def consume_non_diagnostic(lines: List[str]) -> None: while lines and not TAP_ENTRIES.match(lines[0]): lines.pop(0) -def save_non_diagnositic(lines: List[str], test_case: TestCase) -> None: +def save_non_diagnostic(lines: List[str], test_case: TestCase) -> None: while lines and not TAP_ENTRIES.match(lines[0]): test_case.log.append(lines[0]) lines.pop(0) @@ -113,7 +113,7 @@ OK_NOT_OK_SUBTEST = re.compile(r'^[\s]+(ok|not ok) [0-9]+ - (.*)$') OK_NOT_OK_MODULE = re.compile(r'^(ok|not ok) ([0-9]+) - (.*)$') def parse_ok_not_ok_test_case(lines: List[str], test_case: TestCase) -> bool: - save_non_diagnositic(lines, test_case) + save_non_diagnostic(lines, test_case) if not lines: test_case.status = TestStatus.TEST_CRASHED return True @@ -139,7 +139,7 @@ SUBTEST_DIAGNOSTIC = re.compile(r'^[\s]+# (.*)$') DIAGNOSTIC_CRASH_MESSAGE = re.compile(r'^[\s]+# .*?: kunit test case crashed!$') def parse_diagnostic(lines: List[str], test_case: TestCase) -> bool: - save_non_diagnositic(lines, test_case) + save_non_diagnostic(lines, test_case) if not lines: return False line = lines[0] @@ -155,7 +155,7 @@ def parse_diagnostic(lines: List[str], test_case: TestCase) -> bool: def parse_test_case(lines: List[str]) -> Optional[TestCase]: test_case = TestCase() - save_non_diagnositic(lines, test_case) + save_non_diagnostic(lines, test_case) while parse_diagnostic(lines, test_case): pass if parse_ok_not_ok_test_case(lines, test_case): @@ -166,7 +166,7 @@ def parse_test_case(lines: List[str]) -> Optional[TestCase]: SUBTEST_HEADER = re.compile(r'^[\s]+# Subtest: (.*)$') def parse_subtest_header(lines: List[str]) -> Optional[str]: - consume_non_diagnositic(lines) + consume_non_diagnostic(lines) if not lines: return None match = SUBTEST_HEADER.match(lines[0]) @@ -179,7 +179,7 @@ def parse_subtest_header(lines: List[str]) -> Optional[str]: SUBTEST_PLAN = re.compile(r'[\s]+[0-9]+\.\.([0-9]+)') def parse_subtest_plan(lines: List[str]) -> Optional[int]: - consume_non_diagnositic(lines) + consume_non_diagnostic(lines) match = SUBTEST_PLAN.match(lines[0]) if match: lines.pop(0) @@ -202,7 +202,7 @@ def max_status(left: TestStatus, right: TestStatus) -> TestStatus: def parse_ok_not_ok_test_suite(lines: List[str], test_suite: TestSuite, expected_suite_index: int) -> bool: - consume_non_diagnositic(lines) + consume_non_diagnostic(lines) if not lines: test_suite.status = TestStatus.TEST_CRASHED return False @@ -235,7 +235,7 @@ def bubble_up_test_case_errors(test_suite: TestSuite) -> TestStatus: def parse_test_suite(lines: List[str], expected_suite_index: int) -> Optional[TestSuite]: if not lines: return None - consume_non_diagnositic(lines) + consume_non_diagnostic(lines) test_suite = TestSuite() test_suite.status = TestStatus.SUCCESS name = parse_subtest_header(lines) @@ -264,7 +264,7 @@ def parse_test_suite(lines: List[str], expected_suite_index: int) -> Optional[Te TAP_HEADER = re.compile(r'^TAP version 14$') def parse_tap_header(lines: List[str]) -> bool: - consume_non_diagnositic(lines) + consume_non_diagnostic(lines) if TAP_HEADER.match(lines[0]): lines.pop(0) return True @@ -274,7 +274,7 @@ def parse_tap_header(lines: List[str]) -> bool: TEST_PLAN = re.compile(r'[0-9]+\.\.([0-9]+)') def parse_test_plan(lines: List[str]) -> Optional[int]: - consume_non_diagnositic(lines) + consume_non_diagnostic(lines) match = TEST_PLAN.match(lines[0]) if match: lines.pop(0) @@ -286,7 +286,7 @@ def bubble_up_suite_errors(test_suite_list: List[TestSuite]) -> TestStatus: return bubble_up_errors(lambda x: x.status, test_suite_list) def parse_test_result(lines: List[str]) -> TestResult: - consume_non_diagnositic(lines) + consume_non_diagnostic(lines) if not lines or not parse_tap_header(lines): return TestResult(TestStatus.NO_TESTS, [], lines) expected_test_suite_num = parse_test_plan(lines) base-commit: 5f6b99d0287de2c2d0b5e7abcb0092d553ad804a -- 2.29.2.576.ga3fc446d84-goog

3 years, 11 months

2
2
0 0

[PATCH] Documentation: kunit: include example of a parameterized test

by Daniel Latypov

Commit fadb08e7c750 ("kunit: Support for Parameterized Testing") introduced support but lacks documentation for how to use it. This patch builds on commit 1f0e943df68a ("Documentation: kunit: provide guidance for testing many inputs") to show a minimal example of the new feature. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- Documentation/dev-tools/kunit/usage.rst | 57 +++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst index d9fdc14f0677..650f99590df5 100644 --- a/Documentation/dev-tools/kunit/usage.rst +++ b/Documentation/dev-tools/kunit/usage.rst @@ -522,6 +522,63 @@ There's more boilerplate involved, but it can: * E.g. if we wanted to also test ``sha256sum``, we could add a ``sha256`` field and reuse ``cases``. +* be converted to a "parameterized test", see below. + +Parameterized Testing +~~~~~~~~~~~~~~~~~~~~~ + +The table-driven testing pattern is common enough that KUnit has special +support for it. + +Reusing the same ``cases`` array from above, we can write the test as a +"parameterized test" with the following. + +.. code-block:: c + + // This is copy-pasted from above. + struct sha1_test_case { + const char *str; + const char *sha1; + }; + struct sha1_test_case cases[] = { + { + .str = "hello world", + .sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed", + }, + { + .str = "hello world!", + .sha1 = "430ce34d020724ed75a196dfc2ad67c77772d169", + }, + }; + + // Need a helper function to generate a name for each test case. + static void case_to_desc(const struct sha1_test_case *t, char *desc) + { + strcpy(desc, t->str); + } + // Creates `sha1_gen_params()` to iterate over `cases`. + KUNIT_ARRAY_PARAM(sha1, cases, case_to_desc); + + // Looks no different from a normal test. + static void sha1_test(struct kunit *test) + { + // This function can just contain the body of the for-loop. + // The former `cases[i]` is accessible under test->param_value. + char out[40]; + struct sha1_test_case *test_param = (struct sha1_test_case *)(test->param_value); + + sha1sum(test_param->str, out); + KUNIT_EXPECT_STREQ_MSG(test, (char *)out, test_param->sha1, + "sha1sum(%s)", test_param->str); + } + + // Instead of KUNIT_CASE, we use KUNIT_CASE_PARAM and pass in the + // function declared by KUNIT_ARRAY_PARAM. + static struct kunit_case sha1_test_cases[] = { + KUNIT_CASE_PARAM(sha1_test, sha1_gen_params), + {} + }; + .. _kunit-on-non-uml: KUnit on non-UML architectures base-commit: 5f6b99d0287de2c2d0b5e7abcb0092d553ad804a -- 2.29.2.684.gfbc64c5ab5-goog

3 years, 11 months

3
2
0 0

[PATCH v2 1/4] kunit: tool: fix unit test cleanup handling

by Daniel Latypov

* Stop leaking file objects. * Use self.addCleanup() to ensure we call cleanup functions even if setUp() fails. * use mock.patch.stopall instead of more error-prone manual approach Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- tools/testing/kunit/kunit_tool_test.py | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 497ab51bc170..3fbe1acd531a 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -288,19 +288,17 @@ class StrContains(str): class KUnitMainTest(unittest.TestCase): def setUp(self): path = get_absolute_path('test_data/test_is_test_passed-all_passed.log') - file = open(path) - all_passed_log = file.readlines() - self.print_patch = mock.patch('builtins.print') - self.print_mock = self.print_patch.start() + with open(path) as file: + all_passed_log = file.readlines() + + self.print_mock = mock.patch('builtins.print').start() + self.addCleanup(mock.patch.stopall) + self.linux_source_mock = mock.Mock() self.linux_source_mock.build_reconfig = mock.Mock(return_value=True) self.linux_source_mock.build_um_kernel = mock.Mock(return_value=True) self.linux_source_mock.run_kernel = mock.Mock(return_value=all_passed_log) - def tearDown(self): - self.print_patch.stop() - pass - def test_config_passes_args_pass(self): kunit.main(['config', '--build_dir=.kunit'], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 1 base-commit: 509a15421674b9e1a3e1916939d0d0efd3e578da -- 2.29.2.576.ga3fc446d84-goog

3 years, 11 months

3
12
0 0

[GIT PULL] Kselftest fixes for Linux 5.11-rc4

by Shuah Khan

Hi Linus, Please pull the following Kselftest fixes update for Linux 5.11-rc4 This Kselftest fixes update for Linux 5.11-rc4 consists of one single fix to skip BPF selftests by default. BPF selftests have a hard dependency on cutting edge versions of tools in the BPF ecosystem including LLVM. Skipping BPF allows by default will make it easier for users interested in running kselftest as a whole. Users can include BPF in Kselftest build by via SKIP_TARGETS variable. diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit e71ba9452f0b5b2e8dc8aa5445198cd9214a6a62: Linux 5.11-rc2 (2021-01-03 15:55:30 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-fixes-5.11-rc4 for you to fetch changes up to 7a6eb7c34a78498742b5f82543b7a68c1c443329: selftests: Skip BPF seftests by default (2021-01-04 09:27:20 -0700) ---------------------------------------------------------------- linux-kselftest-fixes-5.11-rc4 This Kselftest fixes update for Linux 5.11-rc4 consists of one single fix to skip BPF selftests by default. BPF selftests have a hard dependency on cutting edge versions of tools in the BPF ecosystem including LLVM. Skipping BPF allows by default will make it easier for users interested in running kselftest as a whole. Users can include BPF in Kselftest build by via SKIP_TARGETS variable. ---------------------------------------------------------------- Mark Brown (1): selftests: Skip BPF seftests by default tools/testing/selftests/Makefile | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) ----------------------------------------------------------------

3 years, 11 months

2
1
0 0

[PATCH] arm64: selftests: Fix spelling of 'Mismatch'

by Mark Brown

The SVE and FPSIMD stress tests have a spelling mistake in the output, fix it. Signed-off-by: Mark Brown <broonie(a)kernel.org> --- tools/testing/selftests/arm64/fp/fpsimd-test.S | 2 +- tools/testing/selftests/arm64/fp/sve-test.S | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/arm64/fp/fpsimd-test.S b/tools/testing/selftests/arm64/fp/fpsimd-test.S index 1c5556bdd11d..0dbd594c2747 100644 --- a/tools/testing/selftests/arm64/fp/fpsimd-test.S +++ b/tools/testing/selftests/arm64/fp/fpsimd-test.S @@ -457,7 +457,7 @@ function barf mov x11, x1 // actual data mov x12, x2 // data size - puts "Mistatch: PID=" + puts "Mismatch: PID=" mov x0, x20 bl putdec puts ", iteration=" diff --git a/tools/testing/selftests/arm64/fp/sve-test.S b/tools/testing/selftests/arm64/fp/sve-test.S index f95074c9b48b..9210691aa998 100644 --- a/tools/testing/selftests/arm64/fp/sve-test.S +++ b/tools/testing/selftests/arm64/fp/sve-test.S @@ -625,7 +625,7 @@ function barf mov x11, x1 // actual data mov x12, x2 // data size - puts "Mistatch: PID=" + puts "Mismatch: PID=" mov x0, x20 bl putdec puts ", iteration=" -- 2.20.1

3 years, 11 months

2
1
0 0

[PATCH bpf v2 1/2] bpf: support PTR_TO_MEM{, _OR_NULL} register spilling

by Gilad Reti

Add support for pointer to mem register spilling, to allow the verifier to track pointers to valid memory addresses. Such pointers are returned for example by a successful call of the bpf_ringbuf_reserve helper. The patch was partially contributed by CyberArk Software, Inc. Fixes: 457f44363a88 ("bpf: Implement BPF ring buffer and verifier support for it") Suggested-by: Yonghong Song <yhs(a)fb.com> Signed-off-by: Gilad Reti <gilad.reti(a)gmail.com> --- kernel/bpf/verifier.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 17270b8404f1..36af69fac591 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -2217,6 +2217,8 @@ static bool is_spillable_regtype(enum bpf_reg_type type) case PTR_TO_RDWR_BUF: case PTR_TO_RDWR_BUF_OR_NULL: case PTR_TO_PERCPU_BTF_ID: + case PTR_TO_MEM: + case PTR_TO_MEM_OR_NULL: return true; default: return false; -- 2.27.0

3 years, 11 months

4
5
0 0

[PATCH bpf-next v2 0/5] selftests/bpf: Some build fixes

by Jean-Philippe Brucker

A few fixes for cross-building the sefltests out of tree. This will enable wider automated testing on various Arm hardware. Changes since v1 [1]: * Use wildcard in patch 5 * Move the MAKE_DIRS declaration in patch 1 [1] https://lore.kernel.org/bpf/20210112135959.649075-1-jean-philippe@linaro.or… Jean-Philippe Brucker (5): selftests/bpf: Enable cross-building selftests/bpf: Fix out-of-tree build selftests/bpf: Move generated test files to $(TEST_GEN_FILES) selftests/bpf: Fix installation of urandom_read selftests/bpf: Install btf_dump test cases tools/testing/selftests/bpf/Makefile | 58 ++++++++++++++++++++-------- 1 file changed, 41 insertions(+), 17 deletions(-) -- 2.30.0

3 years, 11 months

2
6
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror January 2021