- Linux-kselftest-mirror - lists.linaro.org

[PATCH v4 0/9] drm: selftest: Convert to KUnit

by Maíra Canal

Hi everyone, Here is the v4 of the conversion of selftests to KUnit. Since the v3, there have been minor fixes, asked by Javier. Basically, fixes on the SoB chain and addition of Copywrites. Thanks for your attention and any feedback is welcomed! Best Regards, - Maíra Canal v1 -> v2: https://lore.kernel.org/dri-devel/20220615135824.15522-1-maira.canal@usp.br… - The suites not longer end in _tests (David Gow). - Remove the TODO entry involving the conversion of selftests to KUnit (Javier Martinez Canillas). - Change the filenames to match the documentation: use *_test.c (Javier Martinez Canillas). - Add MODULE_LICENSE to all tests (kernel test robot). - Make use of a generic symbol to group all tests - DRM_KUNIT_TEST (Javier Martinez Canillas). - Add .kunitconfig on the first patch (it was on the second patch of the series). - Straightforward conversion of the drm_cmdline_parser tests without functional changes (Shuah Khan) - Add David's Tested-by tags. v2 -> v3: https://lore.kernel.org/dri-devel/20220621200926.257002-1-maira.canal@usp.b… - Rebase it on top of the drm-misc-next with drm_format_helper KUnit tests. - Change KUNIT_EXPECT_FALSE to KUNIT_EXPECT_EQ on drm_format_test (Daniel Latypov). - Add Daniel's Acked-by tag. v3 -> v4: https://lore.kernel.org/dri-devel/9185aadb-e459-00fe-70be-3675f6f3ef4c@redh… - Add blank line after #include <kunit/test.h> (Javier Martinez Canillas). - Make the order of the tags chronological (Javier Martinez Canillas). - Add Copywrite to the tests (Javier Martinez Canillas). - Add Javier Martinez Canillas's Reviewed-by tag. Arthur Grillo (1): drm: selftest: convert drm_mm selftest to KUnit Maíra Canal (8): drm: selftest: convert drm_damage_helper selftest to KUnit drm: selftest: convert drm_cmdline_parser selftest to KUnit drm: selftest: convert drm_rect selftest to KUnit drm: selftest: convert drm_format selftest to KUnit drm: selftest: convert drm_plane_helper selftest to KUnit drm: selftest: convert drm_dp_mst_helper selftest to KUnit drm: selftest: convert drm_framebuffer selftest to KUnit drm: selftest: convert drm_buddy selftest to KUnit Documentation/gpu/todo.rst | 11 - drivers/gpu/drm/Kconfig | 20 +- drivers/gpu/drm/Makefile | 1 - drivers/gpu/drm/selftests/Makefile | 8 - .../gpu/drm/selftests/drm_buddy_selftests.h | 15 - .../gpu/drm/selftests/drm_cmdline_selftests.h | 68 - drivers/gpu/drm/selftests/drm_mm_selftests.h | 28 - .../gpu/drm/selftests/drm_modeset_selftests.h | 40 - drivers/gpu/drm/selftests/drm_selftest.c | 109 -- drivers/gpu/drm/selftests/drm_selftest.h | 41 - drivers/gpu/drm/selftests/test-drm_buddy.c | 994 -------------- .../drm/selftests/test-drm_cmdline_parser.c | 1141 ----------------- .../drm/selftests/test-drm_damage_helper.c | 668 ---------- drivers/gpu/drm/selftests/test-drm_format.c | 280 ---- .../drm/selftests/test-drm_modeset_common.c | 32 - .../drm/selftests/test-drm_modeset_common.h | 52 - drivers/gpu/drm/tests/Makefile | 4 +- drivers/gpu/drm/tests/drm_buddy_test.c | 750 +++++++++++ .../gpu/drm/tests/drm_cmdline_parser_test.c | 1080 ++++++++++++++++ .../gpu/drm/tests/drm_damage_helper_test.c | 637 +++++++++ .../drm_dp_mst_helper_test.c} | 87 +- drivers/gpu/drm/tests/drm_format_test.c | 287 +++++ .../drm_framebuffer_test.c} | 26 +- .../test-drm_mm.c => tests/drm_mm_test.c} | 1136 +++++++--------- .../drm_plane_helper_test.c} | 104 +- .../test-drm_rect.c => tests/drm_rect_test.c} | 125 +- 26 files changed, 3406 insertions(+), 4338 deletions(-) delete mode 100644 drivers/gpu/drm/selftests/Makefile delete mode 100644 drivers/gpu/drm/selftests/drm_buddy_selftests.h delete mode 100644 drivers/gpu/drm/selftests/drm_cmdline_selftests.h delete mode 100644 drivers/gpu/drm/selftests/drm_mm_selftests.h delete mode 100644 drivers/gpu/drm/selftests/drm_modeset_selftests.h delete mode 100644 drivers/gpu/drm/selftests/drm_selftest.c delete mode 100644 drivers/gpu/drm/selftests/drm_selftest.h delete mode 100644 drivers/gpu/drm/selftests/test-drm_buddy.c delete mode 100644 drivers/gpu/drm/selftests/test-drm_cmdline_parser.c delete mode 100644 drivers/gpu/drm/selftests/test-drm_damage_helper.c delete mode 100644 drivers/gpu/drm/selftests/test-drm_format.c delete mode 100644 drivers/gpu/drm/selftests/test-drm_modeset_common.c delete mode 100644 drivers/gpu/drm/selftests/test-drm_modeset_common.h create mode 100644 drivers/gpu/drm/tests/drm_buddy_test.c create mode 100644 drivers/gpu/drm/tests/drm_cmdline_parser_test.c create mode 100644 drivers/gpu/drm/tests/drm_damage_helper_test.c rename drivers/gpu/drm/{selftests/test-drm_dp_mst_helper.c => tests/drm_dp_mst_helper_test.c} (72%) create mode 100644 drivers/gpu/drm/tests/drm_format_test.c rename drivers/gpu/drm/{selftests/test-drm_framebuffer.c => tests/drm_framebuffer_test.c} (96%) rename drivers/gpu/drm/{selftests/test-drm_mm.c => tests/drm_mm_test.c} (58%) rename drivers/gpu/drm/{selftests/test-drm_plane_helper.c => tests/drm_plane_helper_test.c} (62%) rename drivers/gpu/drm/{selftests/test-drm_rect.c => tests/drm_rect_test.c} (53%) -- 2.36.1

2 years, 3 months

3
12
0 0

[PATCH v6 0/5] bpf: Add bpf_verify_pkcs7_signature() helper

by Roberto Sassu

One of the desirable features in security is the ability to restrict import of data to a given system based on data authenticity. If data import can be restricted, it would be possible to enforce a system-wide policy based on the signing keys the system owner trusts. This feature is widely used in the kernel. For example, if the restriction is enabled, kernel modules can be plugged in only if they are signed with a key whose public part is in the primary or secondary keyring. For eBPF, it can be useful as well. For example, it might be useful to authenticate data an eBPF program makes security decisions on. After a discussion in the eBPF mailing list, it was decided that the stated goal should be accomplished by introducing a new helper: bpf_verify_pkcs7_signature(), dedicated to verify PKCS#7 signatures. More helpers will be introduced later, as necessary. The job of bpf_verify_pkcs7_signature() is to retrieve the trusted keyring from function parameters, and to perform signature verification by calling verify_pkcs7_signature(). Data and signature can be provided to the new helper with two dynamic pointers, to reduce the number of parameters. The keyring can be provided by its serial, or by its special ID defined in verification.h, if the serial is zero (not a valid value). bpf_verify_pkcs7_signature() also accepts key lookup-specific flags, passed to lookup_user_key() when the helper searches the keyring by its serial. While passing the keyring serial to bpf_verify_pkcs7_signature() provides great flexibility, it seems suboptimal in terms of security guarantees, as even if the eBPF program is assumed to be trusted, that serial might come from untrusted user space not choosing one that the system administrator approves to enforce a mandatory policy. The same goal could be instead more easily achieved by setting a hardcoded keyring ID in the signed eBPF program, to be passed to bpf_verify_pkcs7_signature(). bpf_verify_pkcs7_signature() can be called only from sleepable programs, because of memory allocation (with lookup flag KEY_LOOKUP_CREATE) and crypto operations. For example, the lsm.s/bpf attach point is suitable, fexit/array_map_update_elem is not. A test was added to check the ability of bpf_verify_pkcs7_signature() to verify PKCS#7 signatures from the session keyring, a newly-created keyring, and from the primary and secondary keyring (taking the tcp_bic.ko kernel module for the verification). The test does not fail if that kernel module is not found (needs support from the CI). A consideration was made on whether bpf_verify_pkcs7_signature() should be a simple wrapper, doing as little as possible, or whether it could have more complex logic. Having a simple and flexible wrapper requires two additional helpers, bpf_lookup_user_key() and bpf_key_put(), to search and acquire a key reference, pass that key to the wrapper, and release the reference. More care is also required on the eBPF verifier side, to ensure that an eBPF program always releases an acquired reference. While that gives eBPF developers the greatest flexibility to use the helpers as necessary, it does not match the security of the solution of retrieving the key and using it within the same function, as for example in security/keys/keyctl.c. The risk is that an eBPF program requests a key for a purpose, and then uses the key in a different way with one of the available key-related helpers (to be added in the future). struct key is not like a file descriptor, carrying permissions requested during an open, that can be revalidated at the time a read or write is performed. It is more close to a struct inode, the function using the key cannot know reliably which permission was requested at lookup time. For that reason, the key lookup and usage cannot be separated, as the kernel will guarantee (also to other MAC mechanisms) that once a key has been requested with a specific purpose, it will be used accordingly, beyond the control of eBFP programs. The patch set is organized as follows. Patch 1 exports bpf_dynptr_get_size(), to obtain the real size of data carried by a dynamic pointer. Patch 2 makes available for new eBPF helpers some key-related definitions. Patch 3 fixes the helper prototype regular expression to accept unsigned as type prefix. Patch 4 introduces the bpf_verify_pkcs7_signature() helper and patch 5 adds the corresponding test. Changelog v5: - Move KEY_LOOKUP_ to include/linux/key.h for validation of bpf_verify_pkcs7_signature() parameter - Remove bpf_lookup_user_key() and bpf_key_put() helpers, and the corresponding tests - Replace struct key parameter of bpf_verify_pkcs7_signature() with the keyring serial and lookup flags - Call lookup_user_key() and key_put() in bpf_verify_pkcs7_signature() code, to ensure that the retrieved key is used according to the permission requested at lookup time - Clarified keyring precedence in the description of bpf_verify_pkcs7_signature() (suggested by John) - Remove newline in the second argument of ASSERT_ - Fix helper prototype regular expression in bpf_doc.py v4: - Remove bpf_request_key_by_id(), don't return an invalid pointer that other helpers can use - Pass the keyring ID (without ULONG_MAX, suggested by Alexei) to bpf_verify_pkcs7_signature() - Introduce bpf_lookup_user_key() and bpf_key_put() helpers (suggested by Alexei) - Add lookup_key_norelease test, to ensure that the verifier blocks eBPF programs which don't decrement the key reference count - Parse raw PKCS#7 signature instead of module-style signature in the verify_pkcs7_signature test (suggested by Alexei) - Parse kernel module in user space and pass raw PKCS#7 signature to the eBPF program for signature verification v3: - Rename bpf_verify_signature() back to bpf_verify_pkcs7_signature() to avoid managing different parameters for each signature verification function in one helper (suggested by Daniel) - Use dynamic pointers and export bpf_dynptr_get_size() (suggested by Alexei) - Introduce bpf_request_key_by_id() to give more flexibility to the caller of bpf_verify_pkcs7_signature() to retrieve the appropriate keyring (suggested by Alexei) - Fix test by reordering the gcc command line, always compile sign-file - Improve helper support check mechanism in the test v2: - Rename bpf_verify_pkcs7_signature() to a more generic bpf_verify_signature() and pass the signature type (suggested by KP) - Move the helper and prototype declaration under #ifdef so that user space can probe for support for the helper (suggested by Daniel) - Describe better the keyring types (suggested by Daniel) - Include linux/bpf.h instead of vmlinux.h to avoid implicit or redeclaration - Make the test selfcontained (suggested by Alexei) v1: - Don't define new map flag but introduce simple wrapper of verify_pkcs7_signature() (suggested by Alexei and KP) Roberto Sassu (5): bpf: Export bpf_dynptr_get_size() KEYS: Move KEY_LOOKUP_ to include/linux/key.h scripts: Handle unsigned type prefix in bpf_doc.py bpf: Add bpf_verify_pkcs7_signature() helper selftests/bpf: Add test for bpf_verify_pkcs7_signature() helper include/linux/bpf.h | 1 + include/linux/key.h | 3 + include/uapi/linux/bpf.h | 24 ++ kernel/bpf/bpf_lsm.c | 63 +++ kernel/bpf/helpers.c | 2 +- scripts/bpf_doc.py | 2 +- security/keys/internal.h | 2 - tools/include/uapi/linux/bpf.h | 24 ++ tools/testing/selftests/bpf/Makefile | 14 +- tools/testing/selftests/bpf/config | 2 + .../bpf/prog_tests/verify_pkcs7_sig.c | 359 ++++++++++++++++++ .../bpf/progs/test_verify_pkcs7_sig.c | 79 ++++ .../testing/selftests/bpf/verify_sig_setup.sh | 104 +++++ 13 files changed, 672 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/verify_pkcs7_sig.c create mode 100644 tools/testing/selftests/bpf/progs/test_verify_pkcs7_sig.c create mode 100755 tools/testing/selftests/bpf/verify_sig_setup.sh -- 2.25.1

2 years, 3 months

3
10
0 0

[PATCH net-next 1/1] net: dsa: mv88e6xxx: allow reading FID when handling ATU violations

by Hans Schultz

For convenience the function mv88e6xxx_g1_atu_op() has been used to read ATU violations, but the function has other purposes and does not enable the possibility to read the FID when reading ATU violations. The FID is needed to get hold of which VID was involved in the violation, thus the need for future purposes to be able to read the FID. Signed-off-by: Hans Schultz <netdev(a)kapio-technology.com> --- drivers/net/dsa/mv88e6xxx/global1_atu.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/net/dsa/mv88e6xxx/global1_atu.c b/drivers/net/dsa/mv88e6xxx/global1_atu.c index 40bd67a5c8e9..5d120d53823c 100644 --- a/drivers/net/dsa/mv88e6xxx/global1_atu.c +++ b/drivers/net/dsa/mv88e6xxx/global1_atu.c @@ -114,6 +114,19 @@ static int mv88e6xxx_g1_atu_op_wait(struct mv88e6xxx_chip *chip) return mv88e6xxx_g1_wait_bit(chip, MV88E6XXX_G1_ATU_OP, bit, 0); } +static int mv88e6xxx_g1_read_atu_violation(struct mv88e6xxx_chip *chip) +{ + int err; + + err = mv88e6xxx_g1_write(chip, MV88E6XXX_G1_ATU_OP, + MV88E6XXX_G1_ATU_OP_BUSY | + MV88E6XXX_G1_ATU_OP_GET_CLR_VIOLATION); + if (err) + return err; + + return mv88e6xxx_g1_atu_op_wait(chip); +} + static int mv88e6xxx_g1_atu_op(struct mv88e6xxx_chip *chip, u16 fid, u16 op) { u16 val; @@ -359,8 +372,7 @@ static irqreturn_t mv88e6xxx_g1_atu_prob_irq_thread_fn(int irq, void *dev_id) mv88e6xxx_reg_lock(chip); - err = mv88e6xxx_g1_atu_op(chip, 0, - MV88E6XXX_G1_ATU_OP_GET_CLR_VIOLATION); + err = mv88e6xxx_g1_read_atu_violation(chip); if (err) goto out; -- 2.30.2

2 years, 3 months

4
3
0 0

[PATCH V3 net-next 0/4] Extend locked port feature with FDB locked flag (MAC-Auth/MAB)

by Hans Schultz

This patch set extends the locked port feature for devices that are behind a locked port, but do not have the ability to authorize themselves as a supplicant using IEEE 802.1X. Such devices can be printers, meters or anything related to fixed installations. Instead of 802.1X authorization, devices can get access based on their MAC addresses being whitelisted. For an authorization daemon to detect that a device is trying to get access through a locked port, the bridge will add the MAC address of the device to the FDB with a locked flag to it. Thus the authorization daemon can catch the FDB add event and check if the MAC address is in the whitelist and if so replace the FDB entry without the locked flag enabled, and thus open the port for the device. This feature is known as MAC-Auth or MAC Authentication Bypass (MAB) in Cisco terminology, where the full MAB concept involves additional Cisco infrastructure for authorization. There is no real authentication process, as the MAC address of the device is the only input the authorization daemon, in the general case, has to base the decision if to unlock the port or not. With this patch set, an implementation of the offloaded case is supplied for the mv88e6xxx driver. When a packet ingresses on a locked port, an ATU miss violation event will occur. When handling such ATU miss violation interrupts, the MAC address of the device is added to the FDB with a zero destination port vector (DPV) and the MAC address is communicated through the switchdev layer to the bridge, so that a FDB entry with the locked flag enabled can be added. Hans Schultz (4): net: bridge: add fdb flag to extent locked port feature net: switchdev: add support for offloading of fdb locked flag net: dsa: mv88e6xxx: mac-auth/MAB implementation selftests: forwarding: add test of MAC-Auth Bypass to locked port tests drivers/net/dsa/mv88e6xxx/Makefile | 1 + drivers/net/dsa/mv88e6xxx/chip.c | 40 ++- drivers/net/dsa/mv88e6xxx/chip.h | 5 + drivers/net/dsa/mv88e6xxx/global1.h | 1 + drivers/net/dsa/mv88e6xxx/global1_atu.c | 35 ++- .../net/dsa/mv88e6xxx/mv88e6xxx_switchdev.c | 249 ++++++++++++++++++ .../net/dsa/mv88e6xxx/mv88e6xxx_switchdev.h | 40 +++ drivers/net/dsa/mv88e6xxx/port.c | 32 ++- drivers/net/dsa/mv88e6xxx/port.h | 2 + include/net/dsa.h | 6 + include/net/switchdev.h | 3 +- include/uapi/linux/neighbour.h | 1 + net/bridge/br.c | 3 +- net/bridge/br_fdb.c | 18 +- net/bridge/br_if.c | 1 + net/bridge/br_input.c | 11 +- net/bridge/br_private.h | 9 +- .../net/forwarding/bridge_locked_port.sh | 42 ++- 18 files changed, 470 insertions(+), 29 deletions(-) create mode 100644 drivers/net/dsa/mv88e6xxx/mv88e6xxx_switchdev.c create mode 100644 drivers/net/dsa/mv88e6xxx/mv88e6xxx_switchdev.h -- 2.30.2

2 years, 3 months

7
53
0 0

[PATCH v5 1/4] panic: Taint kernel if tests are run

by David Gow

Most in-kernel tests (such as KUnit tests) are not supposed to run on production systems: they may do deliberately illegal things to trigger errors, and have security implications (for example, KUnit assertions will often deliberately leak kernel addresses). Add a new taint type, TAINT_TEST to signal that a test has been run. This will be printed as 'N' (originally for kuNit, as every other sensible letter was taken.) This should discourage people from running these tests on production systems, and to make it easier to tell if tests have been run accidentally (by loading the wrong configuration, etc.) Acked-by: Luis Chamberlain <mcgrof(a)kernel.org> Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com> Signed-off-by: David Gow <davidgow(a)google.com> --- Documentation/admin-guide/tainted-kernels.rst | 1 + include/linux/panic.h | 3 ++- kernel/panic.c | 1 + 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/tainted-kernels.rst b/Documentation/admin-guide/tainted-kernels.rst index ceeed7b0798d..7d80e8c307d1 100644 --- a/Documentation/admin-guide/tainted-kernels.rst +++ b/Documentation/admin-guide/tainted-kernels.rst @@ -100,6 +100,7 @@ Bit Log Number Reason that got the kernel tainted 15 _/K 32768 kernel has been live patched 16 _/X 65536 auxiliary taint, defined for and used by distros 17 _/T 131072 kernel was built with the struct randomization plugin + 18 _/N 262144 an in-kernel test has been run === === ====== ======================================================== Note: The character ``_`` is representing a blank in this table to make reading diff --git a/include/linux/panic.h b/include/linux/panic.h index e71161da69c4..c7759b3f2045 100644 --- a/include/linux/panic.h +++ b/include/linux/panic.h @@ -68,7 +68,8 @@ static inline void set_arch_panic_timeout(int timeout, int arch_default_timeout) #define TAINT_LIVEPATCH 15 #define TAINT_AUX 16 #define TAINT_RANDSTRUCT 17 -#define TAINT_FLAGS_COUNT 18 +#define TAINT_TEST 18 +#define TAINT_FLAGS_COUNT 19 #define TAINT_FLAGS_MAX ((1UL << TAINT_FLAGS_COUNT) - 1) struct taint_flag { diff --git a/kernel/panic.c b/kernel/panic.c index a3c758dba15a..6b3369e21026 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -428,6 +428,7 @@ const struct taint_flag taint_flags[TAINT_FLAGS_COUNT] = { [ TAINT_LIVEPATCH ] = { 'K', ' ', true }, [ TAINT_AUX ] = { 'X', ' ', true }, [ TAINT_RANDSTRUCT ] = { 'T', ' ', true }, + [ TAINT_TEST ] = { 'N', ' ', true }, }; /** -- 2.37.0.rc0.161.g10f37bed90-goog

2 years, 3 months

4
10
0 0

[PATCH v4] kunit: tool: make --kunitconfig repeatable, blindly concat

by Daniel Latypov

It's come up a few times that it would be useful to have --kunitconfig be repeatable [1][2]. This could be done before with a bit of shell-fu, e.g. $ find fs/ -name '.kunitconfig' -exec cat {} + | \ ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin or equivalently: $ cat fs/ext4/.kunitconfig fs/fat/.kunitconfig | \ ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin But this can be fairly clunky to use in practice. And having explicit support in kunit.py opens the door to having more config fragments of interest, e.g. options for PCI on UML [1], UML coverage [2], variants of tests [3]. There's another argument to be made that users can just use multiple --kconfig_add's, but this gets very clunky very fast (e.g. [2]). Note: there's a big caveat here that some kconfig options might be incompatible. We try to give a clearish error message in the simple case where the same option appears multiple times with conflicting values, but more subtle ones (e.g. mutually exclusive options) will be potentially very confusing for the user. I don't know we can do better. Note 2: if you want to combine a --kunitconfig with the default, you either have to do to specify the current build_dir > --kunitconfig=.kunit --kunitconfig=additional.config or > --kunitconfig=tools/testing/kunit/configs/default.config --kunitconifg=additional.config each of which have their downsides (former depends on --build_dir, doesn't work if you don't have a .kunitconfig yet), etc. Example with conflicting values: > $ ./tools/testing/kunit/kunit.py config --kunitconfig=lib/kunit --kunitconfig=/dev/stdin <<EOF > CONFIG_KUNIT_TEST=n > CONFIG_KUNIT=m > EOF > ... > kunit_kernel.ConfigError: Multiple values specified for 2 options in kunitconfig: > CONFIG_KUNIT=y > vs from /dev/stdin > CONFIG_KUNIT=m > > CONFIG_KUNIT_TEST=y > vs from /dev/stdin > # CONFIG_KUNIT_TEST is not set [1] https://lists.freedesktop.org/archives/dri-devel/2022-June/357616.html [2] https://lore.kernel.org/linux-kselftest/CAFd5g45f3X3xF2vz2BkTHRqOC4uW6GZxtU… [3] https://lore.kernel.org/linux-kselftest/CANpmjNOdSy6DuO6CYZ4UxhGxqhjzx4tn0s… Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com> --- v1 -> v3: merge with kunitconfig refactor patch, rename differing_options() to conflicting_options() v3 -> v4: add Brendan's RB tag, rebase onto the -kselftest kunit branch. The 1/3 and 3/3 of the initial series applied cleanly, but this one didn't, so I'm sending just this one out by itself now. Specifically, there were significant merge conflicts with the --qemu_args patch. --- tools/testing/kunit/kunit.py | 7 ++-- tools/testing/kunit/kunit_config.py | 11 ++++- tools/testing/kunit/kunit_kernel.py | 38 +++++++++++------ tools/testing/kunit/kunit_tool_test.py | 56 ++++++++++++++++++++++---- 4 files changed, 89 insertions(+), 23 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index b686126afb40..e132b0654029 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -293,8 +293,9 @@ def add_common_opts(parser) -> None: parser.add_argument('--kunitconfig', help='Path to Kconfig fragment that enables KUnit tests.' ' If given a directory, (e.g. lib/kunit), "/.kunitconfig" ' - 'will get automatically appended.', - metavar='PATH') + 'will get automatically appended. If repeated, the files ' + 'blindly concatenated, which might not work in all cases.', + action='append', metavar='PATHS') parser.add_argument('--kconfig_add', help='Additional Kconfig options to append to the ' '.kunitconfig, e.g. CONFIG_KASAN=y. Can be repeated.', @@ -381,7 +382,7 @@ def tree_from_args(cli_args: argparse.Namespace) -> kunit_kernel.LinuxSourceTree qemu_args.extend(shlex.split(arg)) return kunit_kernel.LinuxSourceTree(cli_args.build_dir, - kunitconfig_path=cli_args.kunitconfig, + kunitconfig_paths=cli_args.kunitconfig, kconfig_add=cli_args.kconfig_add, arch=cli_args.arch, cross_compile=cli_args.cross_compile, diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py index 898b2a35eb29..48b5f34b2e5d 100644 --- a/tools/testing/kunit/kunit_config.py +++ b/tools/testing/kunit/kunit_config.py @@ -8,7 +8,7 @@ from dataclasses import dataclass import re -from typing import Dict, Iterable, Set +from typing import Dict, Iterable, List, Set, Tuple CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_(\w+) is not set$' CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+|".*")$' @@ -60,6 +60,15 @@ class Kconfig: return False return True + def conflicting_options(self, other: 'Kconfig') -> List[Tuple[KconfigEntry, KconfigEntry]]: + diff = [] # type: List[Tuple[KconfigEntry, KconfigEntry]] + for name, value in self._entries.items(): + b = other._entries.get(name) + if b and value != b: + pair = (KconfigEntry(name, value), KconfigEntry(name, b)) + diff.append(pair) + return diff + def merge_in_entries(self, other: 'Kconfig') -> None: for name, value in other._entries.items(): self._entries[name] = value diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 94ec9f65ef19..56492090e28e 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -177,6 +177,30 @@ def get_kunitconfig_path(build_dir: str) -> str: def get_old_kunitconfig_path(build_dir: str) -> str: return os.path.join(build_dir, OLD_KUNITCONFIG_PATH) +def get_parsed_kunitconfig(build_dir: str, + kunitconfig_paths: Optional[List[str]]=None) -> kunit_config.Kconfig: + if not kunitconfig_paths: + path = get_kunitconfig_path(build_dir) + if not os.path.exists(path): + shutil.copyfile(DEFAULT_KUNITCONFIG_PATH, path) + return kunit_config.parse_file(path) + + merged = kunit_config.Kconfig() + + for path in kunitconfig_paths: + if os.path.isdir(path): + path = os.path.join(path, KUNITCONFIG_PATH) + if not os.path.exists(path): + raise ConfigError(f'Specified kunitconfig ({path}) does not exist') + + partial = kunit_config.parse_file(path) + diff = merged.conflicting_options(partial) + if diff: + diff_str = '\n\n'.join(f'{a}\n vs from {path}\n{b}' for a, b in diff) + raise ConfigError(f'Multiple values specified for {len(diff)} options in kunitconfig:\n{diff_str}') + merged.merge_in_entries(partial) + return merged + def get_outfile_path(build_dir: str) -> str: return os.path.join(build_dir, OUTFILE_PATH) @@ -221,7 +245,7 @@ class LinuxSourceTree: def __init__( self, build_dir: str, - kunitconfig_path='', + kunitconfig_paths: Optional[List[str]]=None, kconfig_add: Optional[List[str]]=None, arch=None, cross_compile=None, @@ -238,17 +262,7 @@ class LinuxSourceTree: qemu_config_path = _default_qemu_config_path(self._arch) _, self._ops = _get_qemu_ops(qemu_config_path, extra_qemu_args, cross_compile) - if kunitconfig_path: - if os.path.isdir(kunitconfig_path): - kunitconfig_path = os.path.join(kunitconfig_path, KUNITCONFIG_PATH) - if not os.path.exists(kunitconfig_path): - raise ConfigError(f'Specified kunitconfig ({kunitconfig_path}) does not exist') - else: - kunitconfig_path = get_kunitconfig_path(build_dir) - if not os.path.exists(kunitconfig_path): - shutil.copyfile(DEFAULT_KUNITCONFIG_PATH, kunitconfig_path) - - self._kconfig = kunit_config.parse_file(kunitconfig_path) + self._kconfig = get_parsed_kunitconfig(build_dir, kunitconfig_paths) if kconfig_add: kconfig = kunit_config.parse_from_string('\n'.join(kconfig_add)) self._kconfig.merge_in_entries(kconfig) diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index e56544d58147..ad63d0d34f3f 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -356,17 +356,46 @@ class LinuxSourceTreeTest(unittest.TestCase): def test_invalid_kunitconfig(self): with self.assertRaisesRegex(kunit_kernel.ConfigError, 'nonexistent.* does not exist'): - kunit_kernel.LinuxSourceTree('', kunitconfig_path='/nonexistent_file') + kunit_kernel.LinuxSourceTree('', kunitconfig_paths=['/nonexistent_file']) def test_valid_kunitconfig(self): with tempfile.NamedTemporaryFile('wt') as kunitconfig: - kunit_kernel.LinuxSourceTree('', kunitconfig_path=kunitconfig.name) + kunit_kernel.LinuxSourceTree('', kunitconfig_paths=[kunitconfig.name]) def test_dir_kunitconfig(self): with tempfile.TemporaryDirectory('') as dir: with open(os.path.join(dir, '.kunitconfig'), 'w'): pass - kunit_kernel.LinuxSourceTree('', kunitconfig_path=dir) + kunit_kernel.LinuxSourceTree('', kunitconfig_paths=[dir]) + + def test_multiple_kunitconfig(self): + want_kconfig = kunit_config.Kconfig() + want_kconfig.add_entry('KUNIT', 'y') + want_kconfig.add_entry('KUNIT_TEST', 'm') + + with tempfile.TemporaryDirectory('') as dir: + other = os.path.join(dir, 'otherkunitconfig') + with open(os.path.join(dir, '.kunitconfig'), 'w') as f: + f.write('CONFIG_KUNIT=y') + with open(other, 'w') as f: + f.write('CONFIG_KUNIT_TEST=m') + pass + + tree = kunit_kernel.LinuxSourceTree('', kunitconfig_paths=[dir, other]) + self.assertTrue(want_kconfig.is_subset_of(tree._kconfig), msg=tree._kconfig) + + + def test_multiple_kunitconfig_invalid(self): + with tempfile.TemporaryDirectory('') as dir: + other = os.path.join(dir, 'otherkunitconfig') + with open(os.path.join(dir, '.kunitconfig'), 'w') as f: + f.write('CONFIG_KUNIT=y') + with open(other, 'w') as f: + f.write('CONFIG_KUNIT=m') + + with self.assertRaisesRegex(kunit_kernel.ConfigError, '(?s)Multiple values.*CONFIG_KUNIT'): + kunit_kernel.LinuxSourceTree('', kunitconfig_paths=[dir, other]) + def test_kconfig_add(self): want_kconfig = kunit_config.Kconfig() @@ -636,7 +665,7 @@ class KUnitMainTest(unittest.TestCase): kunit.main(['run', '--kunitconfig=mykunitconfig']) # Just verify that we parsed and initialized it correctly here. self.mock_linux_init.assert_called_once_with('.kunit', - kunitconfig_path='mykunitconfig', + kunitconfig_paths=['mykunitconfig'], kconfig_add=None, arch='um', cross_compile=None, @@ -647,18 +676,31 @@ class KUnitMainTest(unittest.TestCase): kunit.main(['config', '--kunitconfig=mykunitconfig']) # Just verify that we parsed and initialized it correctly here. self.mock_linux_init.assert_called_once_with('.kunit', - kunitconfig_path='mykunitconfig', + kunitconfig_paths=['mykunitconfig'], kconfig_add=None, arch='um', cross_compile=None, qemu_config_path=None, extra_qemu_args=[]) + @mock.patch.object(kunit_kernel, 'LinuxSourceTree') + def test_run_multiple_kunitconfig(self, mock_linux_init): + mock_linux_init.return_value = self.linux_source_mock + kunit.main(['run', '--kunitconfig=mykunitconfig', '--kunitconfig=other']) + # Just verify that we parsed and initialized it correctly here. + mock_linux_init.assert_called_once_with('.kunit', + kunitconfig_paths=['mykunitconfig', 'other'], + kconfig_add=None, + arch='um', + cross_compile=None, + qemu_config_path=None, + extra_qemu_args=[]) + def test_run_kconfig_add(self): kunit.main(['run', '--kconfig_add=CONFIG_KASAN=y', '--kconfig_add=CONFIG_KCSAN=y']) # Just verify that we parsed and initialized it correctly here. self.mock_linux_init.assert_called_once_with('.kunit', - kunitconfig_path=None, + kunitconfig_paths=None, kconfig_add=['CONFIG_KASAN=y', 'CONFIG_KCSAN=y'], arch='um', cross_compile=None, @@ -669,7 +711,7 @@ class KUnitMainTest(unittest.TestCase): kunit.main(['run', '--arch=x86_64', '--qemu_args', '-m 2048']) # Just verify that we parsed and initialized it correctly here. self.mock_linux_init.assert_called_once_with('.kunit', - kunitconfig_path=None, + kunitconfig_paths=None, kconfig_add=None, arch='x86_64', cross_compile=None, base-commit: 1d202d1496a0be94100d8cbc2b658dcd980a3edf -- 2.37.0.rc0.161.g10f37bed90-goog

2 years, 3 months

1
0
0 0

[PATCH] selftests/kvm: Add error messages before skipping tests in vm_xsave_req_perm()

by Gautam Menghani

Add messages in the checks that are performed before making a request with ARCH_REQ_XCOMP_GUEST_PERM. Signed-off-by: Gautam Menghani <gautammenghani201(a)gmail.com> --- tools/testing/selftests/kvm/lib/x86_64/processor.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c index ead7011ee8f6..c74e846661b7 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c @@ -643,11 +643,15 @@ void vm_xsave_req_perm(int bit) if (rc == -1 && (errno == ENXIO || errno == EINVAL)) exit(KSFT_SKIP); TEST_ASSERT(rc == 0, "KVM_GET_DEVICE_ATTR(0, KVM_X86_XCOMP_GUEST_SUPP) error: %ld", rc); - if (!(bitmask & (1ULL << bit))) + if (!(bitmask & (1ULL << bit))) { + print_skip("Userspace address of attr data is blank"); exit(KSFT_SKIP); + } - if (!is_xfd_supported()) + if (!is_xfd_supported()) { + print_skip("XFD is not supported"); exit(KSFT_SKIP); + } rc = syscall(SYS_arch_prctl, ARCH_REQ_XCOMP_GUEST_PERM, bit); -- 2.36.1

2 years, 3 months

2
1
0 0

[PATCH bpf-next v10 0/6] New BPF helpers to accelerate synproxy

by Maxim Mikityanskiy

The first patch of this series is a documentation fix. The second patch allows BPF helpers to accept memory regions of fixed size without doing runtime size checks. The two next patches add new functionality that allows XDP to accelerate iptables synproxy. v1 of this series [1] used to include a patch that exposed conntrack lookup to BPF using stable helpers. It was superseded by series [2] by Kumar Kartikeya Dwivedi, which implements this functionality using unstable helpers. The third patch adds new helpers to issue and check SYN cookies without binding to a socket, which is useful in the synproxy scenario. The fourth patch adds a selftest, which includes an XDP program and a userspace control application. The XDP program uses socketless SYN cookie helpers and queries conntrack status instead of socket status. The userspace control application allows to tune parameters of the XDP program. This program also serves as a minimal example of usage of the new functionality. The last two patches expose the new helpers to TC BPF and extend the selftest. The draft of the new functionality was presented on Netdev 0x15 [3]. v2 changes: Split into two series, submitted bugfixes to bpf, dropped the conntrack patches, implemented the timestamp cookie in BPF using bpf_loop, dropped the timestamp cookie patch. v3 changes: Moved some patches from bpf to bpf-next, dropped the patch that changed error codes, split the new helpers into IPv4/IPv6, added verifier functionality to accept memory regions of fixed size. v4 changes: Converted the selftest to the test_progs runner. Replaced some deprecated functions in xdp_synproxy userspace helper. v5 changes: Fixed a bug in the selftest. Added questionable functionality to support new helpers in TC BPF, added selftests for it. v6 changes: Wrap the new helpers themselves into #ifdef CONFIG_SYN_COOKIES, replaced fclose with pclose and fixed the MSS for IPv6 in the selftest. v7 changes: Fixed the off-by-one error in indices, changed the section name to "xdp", added missing kernel config options to vmtest in CI. v8 changes: Properly rebased, dropped the first patch (the same change was applied by someone else), updated the cover letter. v9 changes: Fixed selftests for no_alu32. v10 changes: Selftests for s390x were blacklisted due to lack of support of kfunc, rebased the series, split selftests to separate commits, created ARG_PTR_TO_FIXED_SIZE_MEM and packed arg_size, addressed the rest of comments. [1]: https://lore.kernel.org/bpf/20211020095815.GJ28644@breakpoint.cc/t/ [2]: https://lore.kernel.org/bpf/20220114163953.1455836-1-memxor@gmail.com/ [3]: https://netdevconf.info/0x15/session.html?Accelerating-synproxy-with-XDP Maxim Mikityanskiy (6): bpf: Fix documentation of th_len in bpf_tcp_{gen,check}_syncookie bpf: Allow helpers to accept pointers with a fixed size bpf: Add helpers to issue and check SYN cookies in XDP selftests/bpf: Add selftests for raw syncookie helpers bpf: Allow the new syncookie helpers to work with SKBs selftests/bpf: Add selftests for raw syncookie helpers in TC mode include/linux/bpf.h | 13 + include/net/tcp.h | 1 + include/uapi/linux/bpf.h | 88 +- kernel/bpf/verifier.c | 43 +- net/core/filter.c | 128 +++ net/ipv4/tcp_input.c | 3 +- scripts/bpf_doc.py | 4 + tools/include/uapi/linux/bpf.h | 88 +- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 3 +- .../selftests/bpf/prog_tests/xdp_synproxy.c | 183 ++++ .../selftests/bpf/progs/xdp_synproxy_kern.c | 833 ++++++++++++++++++ tools/testing/selftests/bpf/xdp_synproxy.c | 466 ++++++++++ 13 files changed, 1833 insertions(+), 21 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/xdp_synproxy.c create mode 100644 tools/testing/selftests/bpf/progs/xdp_synproxy_kern.c create mode 100644 tools/testing/selftests/bpf/xdp_synproxy.c -- 2.30.2

2 years, 3 months

4
9
0 0

[PATCH v2] kunit: tool: Enable virtio/PCI by default on UML

by David Gow

There are several tests which depend on PCI, and hence need a bunch of extra options to run under UML. This makes it awkward to give configuration instructions (whether in documentation, or as part of a .kunitconfig file), as two separate, incompatible sets of config options are required for UML and "most other architectures". For non-UML architectures, it's possible to add default kconfig options via the qemu_config python files, but there's no equivalent for UML. Add a new tools/testing/kunit/configs/arch_uml.config file containing extra kconfig options to use on UML. Tested-by: José Expósito <jose.exposito89(a)gmail.com> Reviewed-by: Daniel Latypov <dlatypov(a)google.com> Signed-off-by: David Gow <davidgow(a)google.com> --- NOTE: This depends on the refactor kconfig handling & repeatable --kunitconfig series here: https://lore.kernel.org/linux-kselftest/20220627221446.82157-1-dlatypov@goo… https://patchwork.kernel.org/project/linux-kselftest/list/?series=654332 Please apply those first. Changes since v1: https://lore.kernel.org/linux-kselftest/20220624084400.1454579-1-davidgow@g… - (Hopefully) fix a pytype warning re: architecture being None in the tests. (Thanks, Daniel) - Rebase on top of the new combined v3 of the kconfig/kunitconfig patchset. - Add José's Tested-by and Daniel's Reviewed-by. Changes since RFC: https://lore.kernel.org/linux-kselftest/20220622035326.759935-1-davidgow@go… - Rebase on top of the previous kconfig patches. - Fix a missing make_arch_qemuconfig->make_arch_config rename (Thanks Brendan) - Fix the tests to use the base LinuxSourceTreeOperations class, which has no default kconfig options (and so won't conflict with those set in the tests). Only test_build_reconfig_existing_config actually failed, but I updated a few more in case the defaults changed. --- tools/testing/kunit/configs/arch_uml.config | 5 +++++ tools/testing/kunit/kunit_kernel.py | 14 ++++++++++---- tools/testing/kunit/kunit_tool_test.py | 12 ++++++++++++ 3 files changed, 27 insertions(+), 4 deletions(-) create mode 100644 tools/testing/kunit/configs/arch_uml.config diff --git a/tools/testing/kunit/configs/arch_uml.config b/tools/testing/kunit/configs/arch_uml.config new file mode 100644 index 000000000000..e824ce43b05a --- /dev/null +++ b/tools/testing/kunit/configs/arch_uml.config @@ -0,0 +1,5 @@ +# Config options which are added to UML builds by default + +# Enable virtio/pci, as a lot of tests require it. +CONFIG_VIRTIO_UML=y +CONFIG_UML_PCI_OVER_VIRTIO=y diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index f65c996127c3..2698d4c51e6e 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -26,6 +26,7 @@ KUNITCONFIG_PATH = '.kunitconfig' OLD_KUNITCONFIG_PATH = 'last_used_kunitconfig' DEFAULT_KUNITCONFIG_PATH = 'tools/testing/kunit/configs/default.config' BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config' +UML_KCONFIG_PATH = 'tools/testing/kunit/configs/arch_uml.config' OUTFILE_PATH = 'test.log' ABS_TOOL_PATH = os.path.abspath(os.path.dirname(__file__)) QEMU_CONFIGS_DIR = os.path.join(ABS_TOOL_PATH, 'qemu_configs') @@ -53,7 +54,7 @@ class LinuxSourceTreeOperations: except subprocess.CalledProcessError as e: raise ConfigError(e.output.decode()) - def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: + def make_arch_config(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: return base_kunitconfig def make_allyesconfig(self, build_dir: str, make_options) -> None: @@ -109,7 +110,7 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations): self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot' self._extra_qemu_params = qemu_arch_params.extra_qemu_params - def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: + def make_arch_config(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: kconfig = kunit_config.parse_from_string(self._kconfig) kconfig.merge_in_entries(base_kunitconfig) return kconfig @@ -138,6 +139,11 @@ class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations): def __init__(self, cross_compile=None): super().__init__(linux_arch='um', cross_compile=cross_compile) + def make_arch_config(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: + kconfig = kunit_config.parse_file(UML_KCONFIG_PATH) + kconfig.merge_in_entries(base_kunitconfig) + return kconfig + def make_allyesconfig(self, build_dir: str, make_options) -> None: kunit_parser.print_with_timestamp( 'Enabling all CONFIGs for UML...') @@ -297,7 +303,7 @@ class LinuxSourceTree: if build_dir and not os.path.exists(build_dir): os.mkdir(build_dir) try: - self._kconfig = self._ops.make_arch_qemuconfig(self._kconfig) + self._kconfig = self._ops.make_arch_config(self._kconfig) self._kconfig.write_to_file(kconfig_path) self._ops.make_olddefconfig(build_dir, make_options) except ConfigError as e: @@ -328,7 +334,7 @@ class LinuxSourceTree: return self.build_config(build_dir, make_options) existing_kconfig = kunit_config.parse_file(kconfig_path) - self._kconfig = self._ops.make_arch_qemuconfig(self._kconfig) + self._kconfig = self._ops.make_arch_config(self._kconfig) if self._kconfig.is_subset_of(existing_kconfig) and not self._kunitconfig_changed(build_dir): return True diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 0c5ba3ed35e6..6b8887c79c50 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -430,6 +430,10 @@ class LinuxSourceTreeTest(unittest.TestCase): f.write('CONFIG_KUNIT=y') tree = kunit_kernel.LinuxSourceTree(build_dir) + # Stub out the source tree operations, so we don't have + # the defaults for any given architecture get in the + # way. + tree._ops = kunit_kernel.LinuxSourceTreeOperations('none', None) mock_build_config = mock.patch.object(tree, 'build_config').start() # Should generate the .config @@ -447,6 +451,10 @@ class LinuxSourceTreeTest(unittest.TestCase): f.write('CONFIG_KUNIT=y\nCONFIG_KUNIT_TEST=y') tree = kunit_kernel.LinuxSourceTree(build_dir) + # Stub out the source tree operations, so we don't have + # the defaults for any given architecture get in the + # way. + tree._ops = kunit_kernel.LinuxSourceTreeOperations('none', None) mock_build_config = mock.patch.object(tree, 'build_config').start() self.assertTrue(tree.build_reconfig(build_dir, make_options=[])) @@ -463,6 +471,10 @@ class LinuxSourceTreeTest(unittest.TestCase): f.write('CONFIG_KUNIT=y\nCONFIG_KUNIT_TEST=y') tree = kunit_kernel.LinuxSourceTree(build_dir) + # Stub out the source tree operations, so we don't have + # the defaults for any given architecture get in the + # way. + tree._ops = kunit_kernel.LinuxSourceTreeOperations('none', None) mock_build_config = mock.patch.object(tree, 'build_config').start() # ... so we should trigger a call to build_config() -- 2.37.0.rc0.161.g10f37bed90-goog

2 years, 3 months

2
1
0 0

[PATCH v3 1/3] kunit: tool: refactor internal kconfig handling, allow overriding

by Daniel Latypov

Currently, you cannot ovewrwrite what's in your kunitconfig via --kconfig_add. Nor can you override something in a qemu_config via either means. This patch makes it so we have this level of priority * --kconfig_add * kunitconfig file (the default or the one from --kunitconfig) * qemu_config The rationale for this order is that the more "dynamic" sources of kconfig options should take priority. --kconfig_add is obviously the most dynamic. And for kunitconfig, users probably tweak the file manually or specify --kunitconfig more often than they delve into qemu_config python files. And internally, we convert the kconfigs from a python list into a set or dict fairly often. We should just use a dict internally. We exposed the set transform in the past since we didn't define __eq__, so also take the chance to shore up the kunit_kconfig.Kconfig interface. Example ======= Let's consider the unrealistic example where someone would want to disable CONFIG_KUNIT. I.e. they run $ ./tools/testing/kunit/kunit.py config --kconfig_add=CONFIG_KUNIT=n Before ------ We'd write the following > # CONFIG_KUNIT is not set > CONFIG_KUNIT_ALL_TESTS=y > CONFIG_KUNIT_TEST=y > CONFIG_KUNIT=y > CONFIG_KUNIT_EXAMPLE_TEST=y And we'd error out with > ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config. > This is probably due to unsatisfied dependencies. > Missing: # CONFIG_KUNIT is not set After ----- We'd write the following > # CONFIG_KUNIT is not set > CONFIG_KUNIT_TEST=y > CONFIG_KUNIT_ALL_TESTS=y > CONFIG_KUNIT_EXAMPLE_TEST=y And we'd error out with > ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config. > This is probably due to unsatisfied dependencies. > Missing: CONFIG_KUNIT_EXAMPLE_TEST=y, CONFIG_KUNIT_TEST=y, CONFIG_KUNIT_ALL_TESTS=y Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Reviewed-by: David Gow <davidgow(a)google.com> --- v1 -> v2: fix validate_config() func. There was a bug found by David, see https://lore.kernel.org/linux-kselftest/CAGS_qxpF338dvbB+6QW1n8_agddeS10+nk… v2 -> v3: remove `set_diff()` helper, merge into other kunitconfig series --- tools/testing/kunit/kunit_config.py | 45 ++++++++++++++------------ tools/testing/kunit/kunit_kernel.py | 20 ++++++------ tools/testing/kunit/kunit_tool_test.py | 45 +++++++++++--------------- 3 files changed, 54 insertions(+), 56 deletions(-) diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py index 75a8dc1683d4..898b2a35eb29 100644 --- a/tools/testing/kunit/kunit_config.py +++ b/tools/testing/kunit/kunit_config.py @@ -8,7 +8,7 @@ from dataclasses import dataclass import re -from typing import List, Set +from typing import Dict, Iterable, Set CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_(\w+) is not set$' CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+|".*")$' @@ -32,35 +32,42 @@ class Kconfig: """Represents defconfig or .config specified using the Kconfig language.""" def __init__(self) -> None: - self._entries = [] # type: List[KconfigEntry] + self._entries = {} # type: Dict[str, str] - def entries(self) -> Set[KconfigEntry]: - return set(self._entries) + def __eq__(self, other) -> bool: + if not isinstance(other, self.__class__): + return False + return self._entries == other._entries - def add_entry(self, entry: KconfigEntry) -> None: - self._entries.append(entry) + def __repr__(self) -> str: + return ','.join(str(e) for e in self.as_entries()) + + def as_entries(self) -> Iterable[KconfigEntry]: + for name, value in self._entries.items(): + yield KconfigEntry(name, value) + + def add_entry(self, name: str, value: str) -> None: + self._entries[name] = value def is_subset_of(self, other: 'Kconfig') -> bool: - other_dict = {e.name: e.value for e in other.entries()} - for a in self.entries(): - b = other_dict.get(a.name) + for name, value in self._entries.items(): + b = other._entries.get(name) if b is None: - if a.value == 'n': + if value == 'n': continue return False - if a.value != b: + if value != b: return False return True def merge_in_entries(self, other: 'Kconfig') -> None: - if other.is_subset_of(self): - return - self._entries = list(self.entries().union(other.entries())) + for name, value in other._entries.items(): + self._entries[name] = value def write_to_file(self, path: str) -> None: with open(path, 'a+') as f: - for entry in self.entries(): - f.write(str(entry) + '\n') + for e in self.as_entries(): + f.write(str(e) + '\n') def parse_file(path: str) -> Kconfig: with open(path, 'r') as f: @@ -78,14 +85,12 @@ def parse_from_string(blob: str) -> Kconfig: match = config_matcher.match(line) if match: - entry = KconfigEntry(match.group(1), match.group(2)) - kconfig.add_entry(entry) + kconfig.add_entry(match.group(1), match.group(2)) continue empty_match = is_not_set_matcher.match(line) if empty_match: - entry = KconfigEntry(empty_match.group(1), 'n') - kconfig.add_entry(entry) + kconfig.add_entry(empty_match.group(1), 'n') continue if line[0] == '#': diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 3539efaf99ba..4115781185e1 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -53,8 +53,8 @@ class LinuxSourceTreeOperations: except subprocess.CalledProcessError as e: raise ConfigError(e.output.decode()) - def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None: - pass + def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: + return base_kunitconfig def make_allyesconfig(self, build_dir: str, make_options) -> None: raise ConfigError('Only the "um" arch is supported for alltests') @@ -109,9 +109,10 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations): self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot' self._extra_qemu_params = qemu_arch_params.extra_qemu_params - def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None: + def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: kconfig = kunit_config.parse_from_string(self._kconfig) - base_kunitconfig.merge_in_entries(kconfig) + kconfig.merge_in_entries(base_kunitconfig) + return kconfig def start(self, params: List[str], build_dir: str) -> subprocess.Popen: kernel_path = os.path.join(build_dir, self._kernel_path) @@ -267,10 +268,10 @@ class LinuxSourceTree: validated_kconfig = kunit_config.parse_file(kconfig_path) if self._kconfig.is_subset_of(validated_kconfig): return True - invalid = self._kconfig.entries() - validated_kconfig.entries() + missing = set(self._kconfig.as_entries()) - set(validated_kconfig.as_entries()) message = 'Not all Kconfig options selected in kunitconfig were in the generated .config.\n' \ 'This is probably due to unsatisfied dependencies.\n' \ - 'Missing: ' + ', '.join([str(e) for e in invalid]) + 'Missing: ' + ', '.join(str(e) for e in missing) if self._arch == 'um': message += '\nNote: many Kconfig options aren\'t available on UML. You can try running ' \ 'on a different architecture with something like "--arch=x86_64".' @@ -282,7 +283,7 @@ class LinuxSourceTree: if build_dir and not os.path.exists(build_dir): os.mkdir(build_dir) try: - self._ops.make_arch_qemuconfig(self._kconfig) + self._kconfig = self._ops.make_arch_qemuconfig(self._kconfig) self._kconfig.write_to_file(kconfig_path) self._ops.make_olddefconfig(build_dir, make_options) except ConfigError as e: @@ -303,7 +304,7 @@ class LinuxSourceTree: return True old_kconfig = kunit_config.parse_file(old_path) - return old_kconfig.entries() != self._kconfig.entries() + return old_kconfig != self._kconfig def build_reconfig(self, build_dir: str, make_options) -> bool: """Creates a new .config if it is not a subset of the .kunitconfig.""" @@ -313,7 +314,8 @@ class LinuxSourceTree: return self.build_config(build_dir, make_options) existing_kconfig = kunit_config.parse_file(kconfig_path) - self._ops.make_arch_qemuconfig(self._kconfig) + self._kconfig = self._ops.make_arch_qemuconfig(self._kconfig) + if self._kconfig.is_subset_of(existing_kconfig) and not self._kunitconfig_changed(build_dir): return True print('Regenerating .config ...') diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 25a2eb3bf114..0fbca18b6e65 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -45,7 +45,7 @@ class KconfigTest(unittest.TestCase): self.assertTrue(kconfig0.is_subset_of(kconfig0)) kconfig1 = kunit_config.Kconfig() - kconfig1.add_entry(kunit_config.KconfigEntry('TEST', 'y')) + kconfig1.add_entry('TEST', 'y') self.assertTrue(kconfig1.is_subset_of(kconfig1)) self.assertTrue(kconfig0.is_subset_of(kconfig1)) self.assertFalse(kconfig1.is_subset_of(kconfig0)) @@ -56,40 +56,28 @@ class KconfigTest(unittest.TestCase): kconfig = kunit_config.parse_file(kconfig_path) expected_kconfig = kunit_config.Kconfig() - expected_kconfig.add_entry( - kunit_config.KconfigEntry('UML', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('MMU', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('TEST', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('EXAMPLE_TEST', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('MK8', 'n')) - - self.assertEqual(kconfig.entries(), expected_kconfig.entries()) + expected_kconfig.add_entry('UML', 'y') + expected_kconfig.add_entry('MMU', 'y') + expected_kconfig.add_entry('TEST', 'y') + expected_kconfig.add_entry('EXAMPLE_TEST', 'y') + expected_kconfig.add_entry('MK8', 'n') + + self.assertEqual(kconfig, expected_kconfig) def test_write_to_file(self): kconfig_path = os.path.join(test_tmpdir, '.config') expected_kconfig = kunit_config.Kconfig() - expected_kconfig.add_entry( - kunit_config.KconfigEntry('UML', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('MMU', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('TEST', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('EXAMPLE_TEST', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('MK8', 'n')) + expected_kconfig.add_entry('UML', 'y') + expected_kconfig.add_entry('MMU', 'y') + expected_kconfig.add_entry('TEST', 'y') + expected_kconfig.add_entry('EXAMPLE_TEST', 'y') + expected_kconfig.add_entry('MK8', 'n') expected_kconfig.write_to_file(kconfig_path) actual_kconfig = kunit_config.parse_file(kconfig_path) - - self.assertEqual(actual_kconfig.entries(), - expected_kconfig.entries()) + self.assertEqual(actual_kconfig, expected_kconfig) class KUnitParserTest(unittest.TestCase): @@ -381,8 +369,11 @@ class LinuxSourceTreeTest(unittest.TestCase): kunit_kernel.LinuxSourceTree('', kunitconfig_path=dir) def test_kconfig_add(self): + want_kconfig = kunit_config.Kconfig() + want_kconfig.add_entry('NOT_REAL', 'y') + tree = kunit_kernel.LinuxSourceTree('', kconfig_add=['CONFIG_NOT_REAL=y']) - self.assertIn(kunit_config.KconfigEntry('NOT_REAL', 'y'), tree._kconfig.entries()) + self.assertTrue(want_kconfig.is_subset_of(tree._kconfig), msg=tree._kconfig) def test_invalid_arch(self): with self.assertRaisesRegex(kunit_kernel.ConfigError, 'not a valid arch, options are.*x86_64'): base-commit: 274295c6e53f8b8b8dfa8b24a3fcb8a9d670c22c -- 2.37.0.rc0.161.g10f37bed90-goog

2 years, 3 months

2
5
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror