Alright, here's a second version, attempting to address as many concerns as possible. It's likely I've missed something, though.
Changes from v1:
* Make scripts/get_maintainer.pl survive querying missing files, giving a warning instead. This is necessary to enable scripts/checkpatch.pl to query MAINTAINERS about files being deleted. * Start with the minimal change just documenting the V: entry, which accepts test commands directly, and tweaking the tools to deal with that. * However, require the commands accept the -h/--help option so that users have an easier time getting *some* help. The run_checks.py missing that is the reason why the patch proposing it for kunit subsystem is marked "DONOTMERGE" in this version. We can drop that requirement, or soften the language, if there's opposition. * Have a *separate* patch documenting 'Tested-with:' as the next (early) change. Mention that you can add a '#' followed by a results URL, on the end. Adjust the V: docs/checks to exclude '#'. * Have a *separate* patch making scripts/checkpatch.pl propose the execution of the test suite defined in MAINTAINERS whenever the corresponding subsystem is changed. * However, use 'CHECK', instead of 'WARNING', to allow submitters specify the exact (and potentially slightly different) command they used, and not have checkpatch.pl complain too loudly that they didn't run the (exact MAINTAINERS-specified) command. This unfortunately means that unless you use --strict, you won't see the message. We'll try to address that in a new change at the end. * Have a *separate* patch introducing the test catalog and accepting references to that everywhere, with a special syntax to distinguish them from verbatim/direct commands. The syntax is prepending the test name with a '*' (just like C pointer dereference). Make checkpatch.pl handle that. * Drop the recommendation to have the "Docs" and "Sources" fields in test descriptions, as the description text should focus on giving a good introduction and not prompt the user to go somewhere else immediately. They both can be referenced in the text where and how is appropriate. * Generally keep the previous changes adding V: entries and test suite docs, and try to accommodate all the requests, but refine the "Summary" fields to fit the checkpatch.pl messages better. * Have a separate patch cataloguing the complete kunit suite. * Finally, add a patch introducing the "proposal strength" keywords (SUGGESTED/RECOMMENDED/REQUIRED) to the syntax of V: entries, which directly affect which level of checkpatch.pl message missing 'Tested-with:' tags would generate: CHECK/WARNING/ERROR respectively. This allows subsystems to disable checkpatch.pl WARNINGS/ERRORS, and keep their test proposals inobtrusive, if they so wish. E.g. if they expect people to change their commands often. At the same time allow stricter workflows for subsystems with more uniform testing. Or e.g. for subsystems which expect the tests to explain their parameters in their output, and the submitters to upload and link their results in their 'Tested-with:' tags.
That seems to be all, but I'm sure I forgot something :D
Anyway, send me more corrections and I'll try to address them, but it's likely going to happen next year only.
Nick --- Nikolai Kondrashov (9): get_maintainer: Survive querying missing files MAINTAINERS: Introduce V: entry for tests MAINTAINERS: Propose kunit core tests for framework changes docs: submitting-patches: Introduce Tested-with: checkpatch: Propose tests to execute MAINTAINERS: Support referencing test docs in V: MAINTAINERS: Propose kvm-xfstests smoke for ext4 docs: tests: Document kunit in general MAINTAINERS: Add proposal strength to V: entries
Mark Brown (1): MAINTAINERS: Propose kunit tests for regmap
Documentation/process/index.rst | 1 + Documentation/process/submitting-patches.rst | 46 +++++++ Documentation/process/tests.rst | 96 +++++++++++++++ MAINTAINERS | 17 +++ scripts/checkpatch.pl | 174 ++++++++++++++++++++++++++- scripts/get_maintainer.pl | 23 +++- scripts/parse-maintainers.pl | 3 +- 7 files changed, 355 insertions(+), 5 deletions(-) ---