The testing effort is increasing throughout the community. The tests are generally merged into the subsystem trees, and are of relatively narrow interest. The patch volume on linux-kselftest@vger.kernel.org makes it hard to follow the changes to the framework, and discuss proposals.
Create a new ML for "all" of kselftests (tests and framework), replacing the old list. Use the old list for framework changes only. It would cause less churn to create a ML for just the framework, but I prefer to use the shorter name for the list which has much more practical use.
Signed-off-by: Jakub Kicinski kuba@kernel.org --- Posting as an RFC because we need to create the new ML.
CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org CC: workflows@vger.kernel.org --- MAINTAINERS | 12 ++++++++++++ 1 file changed, 12 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS index c27f3190737f..9a03dc1c8974 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12401,6 +12401,18 @@ S: Maintained Q: https://patchwork.kernel.org/project/linux-kselftest/list/ T: git git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git F: Documentation/dev-tools/kselftest* +F: tools/testing/selftests/kselftest/ +F: tools/testing/selftests/lib/ +F: tools/testing/selftests/lib.mk +F: tools/testing/selftests/Makefile +F: tools/testing/selftests/*.sh +F: tools/testing/selftests/*.h + +KERNEL SELFTEST TESTS +M: Shuah Khan shuah@kernel.org +M: Shuah Khan skhan@linuxfoundation.org +L: linux-kselftest-all@vger.kernel.org +S: Maintained F: tools/testing/selftests/
KERNEL SMB3 SERVER (KSMBD)
-----Original Message----- From: Jakub Kicinski kuba@kernel.org The testing effort is increasing throughout the community. The tests are generally merged into the subsystem trees, and are of relatively narrow interest. The patch volume on linux-kselftest@vger.kernel.org makes it hard to follow the changes to the framework, and discuss proposals.
Create a new ML for "all" of kselftests (tests and framework), replacing the old list. Use the old list for framework changes only. It would cause less churn to create a ML for just the framework, but I prefer to use the shorter name for the list which has much more practical use.
I'm not a fan of this split.
The approach taken seems like it would create a lot of churn. You have 2 kinds of developers - a) developers of the framework and b) developers of the tests themselves. Are you trying to shield framework developers from discussions about tests, or test developers from discussions about the framework? It seems like framework developers would be interested in seeing discussions about tests, and vice-versa.
I'm not sure a new list is needed, but if the discussion needs to be split, I think it would be better to create a new list for framework-only discussions, and leave the old list handling all (both framework and tests).
Like maybe have the new list be something like "linux-kselftest-core@vger.kernel.org"? -- Tim
Signed-off-by: Jakub Kicinski kuba@kernel.org
Posting as an RFC because we need to create the new ML.
CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org CC: workflows@vger.kernel.org
MAINTAINERS | 12 ++++++++++++ 1 file changed, 12 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS index c27f3190737f..9a03dc1c8974 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12401,6 +12401,18 @@ S: Maintained Q: https://patchwork.kernel.org/project/linux-kselftest/list/ T: git git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git F: Documentation/dev-tools/kselftest* +F: tools/testing/selftests/kselftest/ +F: tools/testing/selftests/lib/ +F: tools/testing/selftests/lib.mk +F: tools/testing/selftests/Makefile +F: tools/testing/selftests/*.sh +F: tools/testing/selftests/*.h
+KERNEL SELFTEST TESTS +M: Shuah Khan shuah@kernel.org +M: Shuah Khan skhan@linuxfoundation.org +L: linux-kselftest-all@vger.kernel.org +S: Maintained F: tools/testing/selftests/
KERNEL SMB3 SERVER (KSMBD)
2.46.2
On Thu, 3 Oct 2024 15:05:06 +0000 Bird, Tim wrote:
Create a new ML for "all" of kselftests (tests and framework), replacing the old list. Use the old list for framework changes only. It would cause less churn to create a ML for just the framework, but I prefer to use the shorter name for the list which has much more practical use.
I'm not a fan of this split.
Not a fan of the split or of reusing the existing list for core?
The approach taken seems like it would create a lot of churn. You have 2 kinds of developers - a) developers of the framework and b) developers of the tests themselves.
Are you viewing the two groups "kernel-wide"? My subjective experience is that tests for larger subsystems only get reviewed within their subsystem. My mental model is not people who write tests vs people who create framework. For networking a lot of people write tests when they develop features and fixes, a small subset of those people also care about the framework. But they/we do not care about non-networking tests.
So it's (a) people who write tests but also care about framework and need cross-subsystem forum vs (b) people who just write tests within their subsystem.
Slight sidebar but the concept of "framework developer" scares me. I've seen "framework developers" in my corporate life too often creating complex and unmaintainable software that has to be rewritten from scratch every 2 years.. Maybe just my experience, but normal SW engineers tend to produce better results.
Are you trying to shield framework developers from discussions about tests, or test developers from discussions about the framework? It seems like framework developers would be interested in seeing discussions about tests, and vice-versa.
Not exactly. I can't think of a single case where netdev tests got meaningful feedback from the kselftest ML. I'm probably forgetting something but it's very very rare.
Every now and then we'd appreciate comments or a discussion with the broader community. I'm guessing current list doesn't serve this purpose because the occasional framework change gets drowned out by a lot of pure tests getting posted.
I'd myself subscribe to a ML with framework changes and discussions.
I'm not sure a new list is needed, but if the discussion needs to be split, I think it would be better to create a new list for framework-only discussions, and leave the old list handling all (both framework and tests).
Like maybe have the new list be something like "linux-kselftest-core@vger.kernel.org"?
On 10/3/24 08:23, Jakub Kicinski wrote:
The testing effort is increasing throughout the community. The tests are generally merged into the subsystem trees, and are of relatively narrow interest. The patch volume on linux-kselftest@vger.kernel.org makes it hard to follow the changes to the framework, and discuss proposals.
I agree with you that the linux-kselftest mailing list is high volume list and it is hard to keep up with it.
Create a new ML for "all" of kselftests (tests and framework), replacing the old list. Use the old list for framework changes only. It would cause less churn to create a ML for just the framework, but I prefer to use the shorter name for the list which has much more practical use.
I am not sure if the split it helps with reducing the work for maintainers and reviewers. We might end up watching both lists. I know I have to.
It is hard enough with one list for people to cc the right people and the list itself when they send patches. I am concerned that with two lists it becomes harder.
selftests have grown year after year and we have about 109 subdirs under selftests as of 6.12.
Some of these are high volume, active, and longer term citizens of selftests: net, mm, and bpf. Developers and maintainers from these subsystems are very engaged in test development and using the tests for their development activities. This also includes they fit into the selftest common infrastructure or core.
These are the subsystems that don't require my attention as much to make sure builds and installs are working correctly. These tests have been around a long time and the problems if any were are ironed out. I sanity check the Makefile changes and if there is new test that gets added.
There are several other subsystems that aren't highly active. I take these patches through my tree. Whenever a new subsystem/directory gets added, I review to make sure Makefile that anchors the test to selftest core is correct.
Most subsystems' test patches get reviewed by their developers which is how it should be. The tests should be reviewed by developers and maintainers in those subsystems.
Can we solve the problem of hard to follow the changes to the framework, and discuss proposals another way?
- Naming convention selftests/core for core/common framework patches We have to get people to do this. New ML has the same issue of getting people to use the right ML.
Signed-off-by: Jakub Kicinski kuba@kernel.org
Posting as an RFC because we need to create the new ML.
CC: shuah@kernel.org CC: linux-kselftest@vger.kernel.org CC: workflows@vger.kernel.org
MAINTAINERS | 12 ++++++++++++ 1 file changed, 12 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS index c27f3190737f..9a03dc1c8974 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12401,6 +12401,18 @@ S: Maintained Q: https://patchwork.kernel.org/project/linux-kselftest/list/ T: git git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git F: Documentation/dev-tools/kselftest* +F: tools/testing/selftests/kselftest/ +F: tools/testing/selftests/lib/ +F: tools/testing/selftests/lib.mk +F: tools/testing/selftests/Makefile +F: tools/testing/selftests/*.sh +F: tools/testing/selftests/*.h
I would add these somehow to core/framework. Some of the issues CI runs into require fixes to individual test makefiles.
selftests/*testdirs*/Makefile
If we do decide on creating two lists, I would prefer creating an alias for exiting linux-kselftest to linux-kselftest-all and then create a new list kselftest-core
+KERNEL SELFTEST TESTS +M: Shuah Khan shuah@kernel.org +M: Shuah Khan skhan@linuxfoundation.org +L: linux-kselftest-all@vger.kernel.org +S: Maintained F: tools/testing/selftests/ KERNEL SMB3 SERVER (KSMBD)
thanks, -- Shuah
linux-kselftest-mirror@lists.linaro.org
-
Bird, Tim -
Jakub Kicinski -
Shuah