Make it easier to enable all KUnit fragments. This is needed for kernel test-systems, so its easy to get all KUnit tests enabled and if new gets added they will be enabled as well. Fragments that has to be builtin will be missed if CONFIG_KUNIT_RUN_ALL is set as a module.
Adding 'if !KUNIT_RUN_ALL' so individual test can be turned of if someone wants that even though KUNIT_RUN_ALL is enabled.
Signed-off-by: Anders Roxell anders.roxell@linaro.org --- drivers/base/Kconfig | 3 ++- drivers/base/test/Kconfig | 3 ++- fs/ext4/Kconfig | 3 ++- lib/Kconfig.debug | 6 ++++-- lib/Kconfig.kcsan | 3 ++- lib/kunit/Kconfig | 15 ++++++++++++--- security/apparmor/Kconfig | 3 ++- 7 files changed, 26 insertions(+), 10 deletions(-)
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index 5f0bc74d2409..c48e6e4ef367 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -149,8 +149,9 @@ config DEBUG_TEST_DRIVER_REMOVE test this functionality.
config PM_QOS_KUNIT_TEST - bool "KUnit Test for PM QoS features" + bool "KUnit Test for PM QoS features" if !KUNIT_RUN_ALL depends on KUNIT=y + default KUNIT_RUN_ALL
config HMEM_REPORTING bool diff --git a/drivers/base/test/Kconfig b/drivers/base/test/Kconfig index 305c7751184a..0d662d689f6b 100644 --- a/drivers/base/test/Kconfig +++ b/drivers/base/test/Kconfig @@ -9,5 +9,6 @@ config TEST_ASYNC_DRIVER_PROBE
If unsure say N. config KUNIT_DRIVER_PE_TEST - bool "KUnit Tests for property entry API" + bool "KUnit Tests for property entry API" if !KUNIT_RUN_ALL depends on KUNIT=y + default KUNIT_RUN_ALL diff --git a/fs/ext4/Kconfig b/fs/ext4/Kconfig index 2a592e38cdfe..76785143259d 100644 --- a/fs/ext4/Kconfig +++ b/fs/ext4/Kconfig @@ -103,9 +103,10 @@ config EXT4_DEBUG echo 1 > /sys/module/ext4/parameters/mballoc_debug
config EXT4_KUNIT_TESTS - tristate "KUnit tests for ext4" + tristate "KUnit tests for ext4" if !KUNIT_RUN_ALL select EXT4_FS depends on KUNIT + default KUNIT_RUN_ALL help This builds the ext4 KUnit tests.
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 8e4aded46281..993e0c5549bc 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2123,8 +2123,9 @@ config TEST_SYSCTL If unsure, say N.
config SYSCTL_KUNIT_TEST - tristate "KUnit test for sysctl" + tristate "KUnit test for sysctl" if !KUNIT_RUN_ALL depends on KUNIT + default KUNIT_RUN_ALL help This builds the proc sysctl unit test, which runs on boot. Tests the API contract and implementation correctness of sysctl. @@ -2134,8 +2135,9 @@ config SYSCTL_KUNIT_TEST If unsure, say N.
config LIST_KUNIT_TEST - tristate "KUnit Test for Kernel Linked-list structures" + tristate "KUnit Test for Kernel Linked-list structures" if !KUNIT_RUN_ALL depends on KUNIT + default KUNIT_RUN_ALL help This builds the linked list KUnit test suite. It tests that the API and basic functionality of the list_head type diff --git a/lib/Kconfig.kcsan b/lib/Kconfig.kcsan index ea28245c6c1d..91398300a1bc 100644 --- a/lib/Kconfig.kcsan +++ b/lib/Kconfig.kcsan @@ -46,8 +46,9 @@ config KCSAN_SELFTEST works as intended.
config KCSAN_TEST - tristate "KCSAN test for integrated runtime behaviour" + tristate "KCSAN test for integrated runtime behaviour" if !KUNIT_RUN_ALL depends on TRACEPOINTS && KUNIT + default KUNIT_RUN_ALL select TORTURE_TEST help KCSAN test focusing on behaviour of the integrated runtime. Tests diff --git a/lib/kunit/Kconfig b/lib/kunit/Kconfig index 95d12e3d6d95..d6a912779816 100644 --- a/lib/kunit/Kconfig +++ b/lib/kunit/Kconfig @@ -15,7 +15,8 @@ menuconfig KUNIT if KUNIT
config KUNIT_DEBUGFS - bool "KUnit - Enable /sys/kernel/debug/kunit debugfs representation" + bool "KUnit - Enable /sys/kernel/debug/kunit debugfs representation" if !KUNIT_RUN_ALL + default KUNIT_RUN_ALL help Enable debugfs representation for kunit. Currently this consists of /sys/kernel/debug/kunit/<test_suite>/results files for each @@ -23,7 +24,8 @@ config KUNIT_DEBUGFS run that occurred.
config KUNIT_TEST - tristate "KUnit test for KUnit" + tristate "KUnit test for KUnit" if !KUNIT_RUN_ALL + default KUNIT_RUN_ALL help Enables the unit tests for the KUnit test framework. These tests test the KUnit test framework itself; the tests are both written using @@ -32,7 +34,8 @@ config KUNIT_TEST expected.
config KUNIT_EXAMPLE_TEST - tristate "Example test for KUnit" + tristate "Example test for KUnit" if !KUNIT_RUN_ALL + default KUNIT_RUN_ALL help Enables an example unit test that illustrates some of the basic features of KUnit. This test only exists to help new users understand @@ -41,4 +44,10 @@ config KUNIT_EXAMPLE_TEST is intended for curious hackers who would like to understand how to use KUnit for kernel development.
+config KUNIT_RUN_ALL + tristate "KUnit run all test" + help + Enables all KUnit tests. If they can be enabled. + That depends on if KUnit is enabled as a module or builtin. + endif # KUNIT diff --git a/security/apparmor/Kconfig b/security/apparmor/Kconfig index 0fe336860773..c4648426ea5d 100644 --- a/security/apparmor/Kconfig +++ b/security/apparmor/Kconfig @@ -70,8 +70,9 @@ config SECURITY_APPARMOR_DEBUG_MESSAGES the kernel message buffer.
config SECURITY_APPARMOR_KUNIT_TEST - bool "Build KUnit tests for policy_unpack.c" + bool "Build KUnit tests for policy_unpack.c" if !KUNIT_RUN_ALL depends on KUNIT=y && SECURITY_APPARMOR + default KUNIT_RUN_ALL help This builds the AppArmor KUnit tests.
On Fri, 1 May 2020 at 10:35, Anders Roxell anders.roxell@linaro.org wrote:
Make it easier to enable all KUnit fragments. This is needed for kernel test-systems, so its easy to get all KUnit tests enabled and if new gets added they will be enabled as well. Fragments that has to be builtin will be missed if CONFIG_KUNIT_RUN_ALL is set as a module.
Adding 'if !KUNIT_RUN_ALL' so individual test can be turned of if someone wants that even though KUNIT_RUN_ALL is enabled.
Signed-off-by: Anders Roxell anders.roxell@linaro.org
drivers/base/Kconfig | 3 ++- drivers/base/test/Kconfig | 3 ++- fs/ext4/Kconfig | 3 ++- lib/Kconfig.debug | 6 ++++-- lib/Kconfig.kcsan | 3 ++- lib/kunit/Kconfig | 15 ++++++++++++--- security/apparmor/Kconfig | 3 ++- 7 files changed, 26 insertions(+), 10 deletions(-)
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index 5f0bc74d2409..c48e6e4ef367 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -149,8 +149,9 @@ config DEBUG_TEST_DRIVER_REMOVE test this functionality.
config PM_QOS_KUNIT_TEST
bool "KUnit Test for PM QoS features"
bool "KUnit Test for PM QoS features" if !KUNIT_RUN_ALL depends on KUNIT=ydefault KUNIT_RUN_ALLconfig HMEM_REPORTING bool diff --git a/drivers/base/test/Kconfig b/drivers/base/test/Kconfig index 305c7751184a..0d662d689f6b 100644 --- a/drivers/base/test/Kconfig +++ b/drivers/base/test/Kconfig @@ -9,5 +9,6 @@ config TEST_ASYNC_DRIVER_PROBE
If unsure say N.config KUNIT_DRIVER_PE_TEST
bool "KUnit Tests for property entry API"
bool "KUnit Tests for property entry API" if !KUNIT_RUN_ALL depends on KUNIT=ydefault KUNIT_RUN_ALLdiff --git a/fs/ext4/Kconfig b/fs/ext4/Kconfig index 2a592e38cdfe..76785143259d 100644 --- a/fs/ext4/Kconfig +++ b/fs/ext4/Kconfig @@ -103,9 +103,10 @@ config EXT4_DEBUG echo 1 > /sys/module/ext4/parameters/mballoc_debug
config EXT4_KUNIT_TESTS
tristate "KUnit tests for ext4"
tristate "KUnit tests for ext4" if !KUNIT_RUN_ALL select EXT4_FS depends on KUNITdefault KUNIT_RUN_ALL help This builds the ext4 KUnit tests.diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 8e4aded46281..993e0c5549bc 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2123,8 +2123,9 @@ config TEST_SYSCTL If unsure, say N.
config SYSCTL_KUNIT_TEST
tristate "KUnit test for sysctl"
tristate "KUnit test for sysctl" if !KUNIT_RUN_ALL depends on KUNITdefault KUNIT_RUN_ALL help This builds the proc sysctl unit test, which runs on boot. Tests the API contract and implementation correctness of sysctl.@@ -2134,8 +2135,9 @@ config SYSCTL_KUNIT_TEST If unsure, say N.
config LIST_KUNIT_TEST
tristate "KUnit Test for Kernel Linked-list structures"
tristate "KUnit Test for Kernel Linked-list structures" if !KUNIT_RUN_ALL depends on KUNITdefault KUNIT_RUN_ALL help This builds the linked list KUnit test suite. It tests that the API and basic functionality of the list_head typediff --git a/lib/Kconfig.kcsan b/lib/Kconfig.kcsan index ea28245c6c1d..91398300a1bc 100644 --- a/lib/Kconfig.kcsan +++ b/lib/Kconfig.kcsan @@ -46,8 +46,9 @@ config KCSAN_SELFTEST works as intended.
config KCSAN_TEST
tristate "KCSAN test for integrated runtime behaviour"
tristate "KCSAN test for integrated runtime behaviour" if !KUNIT_RUN_ALL depends on TRACEPOINTS && KUNITdefault KUNIT_RUN_ALL select TORTURE_TEST help KCSAN test focusing on behaviour of the integrated runtime. Tests
I think if you want this patch to be picked up you need to split it, with one patch for each test that is not yet in mainline or the tree that should pick this patch up.
The KCSAN test is in the -rcu tree, but I don't expect it to be merged before 5.9. Most likely, we would only be able to pick up the patch that would make the chance to the KCSAN Kconfig entry once the rest here made it into mainline.
Thanks, -- Marco
diff --git a/lib/kunit/Kconfig b/lib/kunit/Kconfig index 95d12e3d6d95..d6a912779816 100644 --- a/lib/kunit/Kconfig +++ b/lib/kunit/Kconfig @@ -15,7 +15,8 @@ menuconfig KUNIT if KUNIT
config KUNIT_DEBUGFS
bool "KUnit - Enable /sys/kernel/debug/kunit debugfs representation"
bool "KUnit - Enable /sys/kernel/debug/kunit debugfs representation" if !KUNIT_RUN_ALLdefault KUNIT_RUN_ALL help Enable debugfs representation for kunit. Currently this consists of /sys/kernel/debug/kunit/<test_suite>/results files for each@@ -23,7 +24,8 @@ config KUNIT_DEBUGFS run that occurred.
config KUNIT_TEST
tristate "KUnit test for KUnit"
tristate "KUnit test for KUnit" if !KUNIT_RUN_ALLdefault KUNIT_RUN_ALL help Enables the unit tests for the KUnit test framework. These tests test the KUnit test framework itself; the tests are both written using@@ -32,7 +34,8 @@ config KUNIT_TEST expected.
config KUNIT_EXAMPLE_TEST
tristate "Example test for KUnit"
tristate "Example test for KUnit" if !KUNIT_RUN_ALLdefault KUNIT_RUN_ALL help Enables an example unit test that illustrates some of the basic features of KUnit. This test only exists to help new users understand@@ -41,4 +44,10 @@ config KUNIT_EXAMPLE_TEST is intended for curious hackers who would like to understand how to use KUnit for kernel development.
+config KUNIT_RUN_ALL
tristate "KUnit run all test"helpEnables all KUnit tests. If they can be enabled.That depends on if KUnit is enabled as a module or builtin.
s/tests. If/tests, if/ ?
endif # KUNIT diff --git a/security/apparmor/Kconfig b/security/apparmor/Kconfig index 0fe336860773..c4648426ea5d 100644 --- a/security/apparmor/Kconfig +++ b/security/apparmor/Kconfig @@ -70,8 +70,9 @@ config SECURITY_APPARMOR_DEBUG_MESSAGES the kernel message buffer.
config SECURITY_APPARMOR_KUNIT_TEST
bool "Build KUnit tests for policy_unpack.c"
bool "Build KUnit tests for policy_unpack.c" if !KUNIT_RUN_ALL depends on KUNIT=y && SECURITY_APPARMORdefault KUNIT_RUN_ALL help This builds the AppArmor KUnit tests.-- 2.20.1
On Fri, 1 May 2020 at 11:57, Marco Elver elver@google.com wrote:
On Fri, 1 May 2020 at 10:35, Anders Roxell anders.roxell@linaro.org wrote:
Make it easier to enable all KUnit fragments. This is needed for kernel test-systems, so its easy to get all KUnit tests enabled and if new gets added they will be enabled as well. Fragments that has to be builtin will be missed if CONFIG_KUNIT_RUN_ALL is set as a module.
Adding 'if !KUNIT_RUN_ALL' so individual test can be turned of if someone wants that even though KUNIT_RUN_ALL is enabled.
Signed-off-by: Anders Roxell anders.roxell@linaro.org
drivers/base/Kconfig | 3 ++- drivers/base/test/Kconfig | 3 ++- fs/ext4/Kconfig | 3 ++- lib/Kconfig.debug | 6 ++++-- lib/Kconfig.kcsan | 3 ++- lib/kunit/Kconfig | 15 ++++++++++++--- security/apparmor/Kconfig | 3 ++- 7 files changed, 26 insertions(+), 10 deletions(-)
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index 5f0bc74d2409..c48e6e4ef367 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -149,8 +149,9 @@ config DEBUG_TEST_DRIVER_REMOVE test this functionality.
config PM_QOS_KUNIT_TEST
bool "KUnit Test for PM QoS features"
bool "KUnit Test for PM QoS features" if !KUNIT_RUN_ALL depends on KUNIT=ydefault KUNIT_RUN_ALLconfig HMEM_REPORTING bool diff --git a/drivers/base/test/Kconfig b/drivers/base/test/Kconfig index 305c7751184a..0d662d689f6b 100644 --- a/drivers/base/test/Kconfig +++ b/drivers/base/test/Kconfig @@ -9,5 +9,6 @@ config TEST_ASYNC_DRIVER_PROBE
If unsure say N.config KUNIT_DRIVER_PE_TEST
bool "KUnit Tests for property entry API"
bool "KUnit Tests for property entry API" if !KUNIT_RUN_ALL depends on KUNIT=ydefault KUNIT_RUN_ALLdiff --git a/fs/ext4/Kconfig b/fs/ext4/Kconfig index 2a592e38cdfe..76785143259d 100644 --- a/fs/ext4/Kconfig +++ b/fs/ext4/Kconfig @@ -103,9 +103,10 @@ config EXT4_DEBUG echo 1 > /sys/module/ext4/parameters/mballoc_debug
config EXT4_KUNIT_TESTS
tristate "KUnit tests for ext4"
tristate "KUnit tests for ext4" if !KUNIT_RUN_ALL select EXT4_FS depends on KUNITdefault KUNIT_RUN_ALL help This builds the ext4 KUnit tests.diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 8e4aded46281..993e0c5549bc 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2123,8 +2123,9 @@ config TEST_SYSCTL If unsure, say N.
config SYSCTL_KUNIT_TEST
tristate "KUnit test for sysctl"
tristate "KUnit test for sysctl" if !KUNIT_RUN_ALL depends on KUNITdefault KUNIT_RUN_ALL help This builds the proc sysctl unit test, which runs on boot. Tests the API contract and implementation correctness of sysctl.@@ -2134,8 +2135,9 @@ config SYSCTL_KUNIT_TEST If unsure, say N.
config LIST_KUNIT_TEST
tristate "KUnit Test for Kernel Linked-list structures"
tristate "KUnit Test for Kernel Linked-list structures" if !KUNIT_RUN_ALL depends on KUNITdefault KUNIT_RUN_ALL help This builds the linked list KUnit test suite. It tests that the API and basic functionality of the list_head typediff --git a/lib/Kconfig.kcsan b/lib/Kconfig.kcsan index ea28245c6c1d..91398300a1bc 100644 --- a/lib/Kconfig.kcsan +++ b/lib/Kconfig.kcsan @@ -46,8 +46,9 @@ config KCSAN_SELFTEST works as intended.
config KCSAN_TEST
tristate "KCSAN test for integrated runtime behaviour"
tristate "KCSAN test for integrated runtime behaviour" if !KUNIT_RUN_ALL depends on TRACEPOINTS && KUNITdefault KUNIT_RUN_ALL select TORTURE_TEST help KCSAN test focusing on behaviour of the integrated runtime. TestsI think if you want this patch to be picked up you need to split it, with one patch for each test that is not yet in mainline or the tree that should pick this patch up.
OK, would it be ok to do one patch per subsystem if it's in the mainline tree, and another patch if it's only in the next tree for the same subsystem?
The KCSAN test is in the -rcu tree, but I don't expect it to be merged before 5.9. Most likely, we would only be able to pick up the patch that would make the chance to the KCSAN Kconfig entry once the rest here made it into mainline.
Thanks, -- Marco
diff --git a/lib/kunit/Kconfig b/lib/kunit/Kconfig index 95d12e3d6d95..d6a912779816 100644 --- a/lib/kunit/Kconfig +++ b/lib/kunit/Kconfig @@ -15,7 +15,8 @@ menuconfig KUNIT if KUNIT
config KUNIT_DEBUGFS
bool "KUnit - Enable /sys/kernel/debug/kunit debugfs representation"
bool "KUnit - Enable /sys/kernel/debug/kunit debugfs representation" if !KUNIT_RUN_ALLdefault KUNIT_RUN_ALL help Enable debugfs representation for kunit. Currently this consists of /sys/kernel/debug/kunit/<test_suite>/results files for each@@ -23,7 +24,8 @@ config KUNIT_DEBUGFS run that occurred.
config KUNIT_TEST
tristate "KUnit test for KUnit"
tristate "KUnit test for KUnit" if !KUNIT_RUN_ALLdefault KUNIT_RUN_ALL help Enables the unit tests for the KUnit test framework. These tests test the KUnit test framework itself; the tests are both written using@@ -32,7 +34,8 @@ config KUNIT_TEST expected.
config KUNIT_EXAMPLE_TEST
tristate "Example test for KUnit"
tristate "Example test for KUnit" if !KUNIT_RUN_ALLdefault KUNIT_RUN_ALL help Enables an example unit test that illustrates some of the basic features of KUnit. This test only exists to help new users understand@@ -41,4 +44,10 @@ config KUNIT_EXAMPLE_TEST is intended for curious hackers who would like to understand how to use KUnit for kernel development.
+config KUNIT_RUN_ALL
tristate "KUnit run all test"helpEnables all KUnit tests. If they can be enabled.That depends on if KUnit is enabled as a module or builtin.s/tests. If/tests, if/ ?
correct, I will fix that.
Cheers, Anders
endif # KUNIT diff --git a/security/apparmor/Kconfig b/security/apparmor/Kconfig index 0fe336860773..c4648426ea5d 100644 --- a/security/apparmor/Kconfig +++ b/security/apparmor/Kconfig @@ -70,8 +70,9 @@ config SECURITY_APPARMOR_DEBUG_MESSAGES the kernel message buffer.
config SECURITY_APPARMOR_KUNIT_TEST
bool "Build KUnit tests for policy_unpack.c"
bool "Build KUnit tests for policy_unpack.c" if !KUNIT_RUN_ALL depends on KUNIT=y && SECURITY_APPARMORdefault KUNIT_RUN_ALL help This builds the AppArmor KUnit tests.-- 2.20.1
On Fri, May 1, 2020 at 1:35 AM Anders Roxell anders.roxell@linaro.org wrote:
Make it easier to enable all KUnit fragments. This is needed for kernel test-systems, so its easy to get all KUnit tests enabled and if new gets added they will be enabled as well. Fragments that has to be builtin will be missed if CONFIG_KUNIT_RUN_ALL is set as a module.
Adding 'if !KUNIT_RUN_ALL' so individual test can be turned of if someone wants that even though KUNIT_RUN_ALL is enabled.
I would LOVE IT, if you could make this work! I have been trying to figure out the best way to run all KUnit tests for a long time now.
That being said, I am a bit skeptical that this approach will be much more successful than just using allyesconfig. Either way, there are tests coming down the pipeline that are incompatible with each other (the KASAN test and the KCSAN test will be incompatible). Even so, tests like the apparmor test require a lot of non-default configuration to compile. In the end, I am not sure how many tests we will really be able to turn on this way.
Thoughts?
On Sat, May 2, 2020 at 4:31 AM Brendan Higgins brendanhiggins@google.com wrote:
On Fri, May 1, 2020 at 1:35 AM Anders Roxell anders.roxell@linaro.org wrote:
Make it easier to enable all KUnit fragments. This is needed for kernel test-systems, so its easy to get all KUnit tests enabled and if new gets added they will be enabled as well. Fragments that has to be builtin will be missed if CONFIG_KUNIT_RUN_ALL is set as a module.
Adding 'if !KUNIT_RUN_ALL' so individual test can be turned of if someone wants that even though KUNIT_RUN_ALL is enabled.
I would LOVE IT, if you could make this work! I have been trying to figure out the best way to run all KUnit tests for a long time now.
That being said, I am a bit skeptical that this approach will be much more successful than just using allyesconfig. Either way, there are tests coming down the pipeline that are incompatible with each other (the KASAN test and the KCSAN test will be incompatible). Even so, tests like the apparmor test require a lot of non-default configuration to compile. In the end, I am not sure how many tests we will really be able to turn on this way.
Thoughts?
I think there's still some value in this which the allyesconfig option doesn't provide. As you point out, it's not possible to have a generic "run all tests" option due to potential conflicting dependencies, but this does provide a way to run all tests for things enabled in the current config. This could be really useful for downstream developers who want a way of running all tests relevant to their config without the overhead of running irrelevant tests (e.g., for drivers they don't build). Using allyesconfig doesn't make that distinction.
Ultimately, we'll probably still want something which enables a broader set of tests for upstream development: whether that's based on this, allyesconfig, or something else entirely remains to be seen, I think. I suspect we're going to end up with something subsystem-specific (having a kunitconfig per subsystem, or a testing line in MAINTAINERS or similar are ideas which have been brought up in the past).
This is a great looking tool to have in the toolbox, though.
-- David
On Sat, 2 May 2020 at 04:11, David Gow davidgow@google.com wrote:
On Sat, May 2, 2020 at 4:31 AM Brendan Higgins brendanhiggins@google.com wrote:
On Fri, May 1, 2020 at 1:35 AM Anders Roxell anders.roxell@linaro.org wrote:
Make it easier to enable all KUnit fragments. This is needed for kernel test-systems, so its easy to get all KUnit tests enabled and if new gets added they will be enabled as well. Fragments that has to be builtin will be missed if CONFIG_KUNIT_RUN_ALL is set as a module.
Adding 'if !KUNIT_RUN_ALL' so individual test can be turned of if someone wants that even though KUNIT_RUN_ALL is enabled.
I would LOVE IT, if you could make this work! I have been trying to figure out the best way to run all KUnit tests for a long time now.
That being said, I am a bit skeptical that this approach will be much more successful than just using allyesconfig. Either way, there are tests coming down the pipeline that are incompatible with each other (the KASAN test and the KCSAN test will be incompatible). Even so, tests like the apparmor test require a lot of non-default configuration to compile. In the end, I am not sure how many tests we will really be able to turn on this way.
Thoughts?
I think there's still some value in this which the allyesconfig option doesn't provide. As you point out, it's not possible to have a generic "run all tests" option due to potential conflicting dependencies, but this does provide a way to run all tests for things enabled in the current config. This could be really useful for downstream developers who want a way of running all tests relevant to their config without the overhead of running irrelevant tests (e.g., for drivers they don't build).
It will solve that as well as for a tester doesn't have to go through all KUnit tests fragments to turn them on.
Using allyesconfig doesn't make that distinction.
We could also create a config fragment file in kernel/configs/kunit.config where we set ------start CONFIG_KUNIT=y CONFIG_KUNIT_RUN_ALL=y CONFIG_SECURITY_APPARMOR=y ------end
So, these two can only be enabled if KUNIT=y CONFIG_KUNIT_DRIVER_PE_TEST=y CONFIG_PM_QOS_KUNIT_TEST=y
and for this one we have a pre-request of SECURITY_APPARMOR=y CONFIG_SECURITY_APPARMOR_KUNIT_TEST=y
Other tests solves the dependencies with 'select' like CONFIG_EXT4_KUNIT_TESTS, that adds this row in fs/ext4/Kconfig, 'select EXT4_FS'
But I think we should try to minimize the number of 'select' statements, in order to avoid circular dependencies and unexpected behaviours. Maybe we should add the CONFIG_EXT4_FS=y into the kunit.config file instead ?
Ultimately, we'll probably still want something which enables a broader set of tests for upstream development: whether that's based on this, allyesconfig, or something else entirely remains to be seen, I think. I suspect we're going to end up with something subsystem-specific (having a kunitconfig per subsystem, or a testing line in MAINTAINERS or similar are ideas which have been brought up in the past).
This is a great looking tool to have in the toolbox, though.
I agree!
I'll prepare a patchset with individual patches as was suggested by Marco shortly.
Cheers, Anders
linux-kselftest-mirror@lists.linaro.org
-
Anders Roxell -
Brendan Higgins -
David Gow -
Marco Elver