On Wed, 6 Dec 2023 at 02:45, Nikolai Kondrashov Nikolai.Kondrashov@redhat.com wrote:
Support referencing test suite documentation in the V: entries of MAINTAINERS file. Use the '*<name>' syntax (like C pointer dereference), where '<name>' is a second-level heading in the new Documentation/process/tests.rst file, with the suite's description. This syntax allows distinguishing the references from test commands.
Add a boiler-plate Documentation/process/tests.rst file, describing a way to add structured info to the test suites in the form of field lists. Apart from a "summary" and "command" fields, they can also contain a "superset" field specifying the superset of the test suite, helping reuse documentation and express both wider and narrower test sets.
Make scripts/checkpatch.pl load the tests from the file, along with the structured data, validate the references in MAINTAINERS, dereference them, and output the test suite information in the CHECK messages whenever the corresponding subsystems are changed. But only if there was no corresponding Tested-with: tag in the commit message, certifying it was executed successfully already.
This is supposed to help propose executing test suites which cannot be executed immediately, and need extra setup, as well as provide a place for extra documentation and information on directly-available suites.
Signed-off-by: Nikolai Kondrashov Nikolai.Kondrashov@redhat.com
I like the idea here, but wonder whether it makes sense to put all of these tests into a single 'tests.rst' file. There's already lots of existing documentation scattered around the tree, and while keeping all of the testing information in one place does have advantages, I think there's a lot to be said for keeping subsystem-specific test docs alongside the rest of the documentation for the subsystem itself. And it'd be less work, as the docs are already there.
So, could we just make this a path under Documentation/ (possibly with an #anchor if we need to reference just one part of a file)?
e.g., something like these, all of which are existing docs: V: *Documentation/dev-tools/kasan.rst#Tests or V: *Dcoumentation/RCU/torture.rst or V: *Documentation/gpu/automated_testing.rst or V: *Documentation/process/maintainer-kvm-x86.rst#Testing
(We could even get rid of the '*' and just use 'Documentation/' as a prefix, or the executable bit on the file, or similar to distinguish these from scripts.)
If we wanted to be very brave, we could extend this further to arbitrary webpages, like: V: https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/README
Thoughts?
-- David