On Thu, 7 Nov 2024 at 01:01, Donald Zickus dzickus@redhat.com wrote:
Hi,
Thanks for the feedback. I created a more realistic test.yaml file to start (we can split it when more tests are added) and a parser. I was going to add patch support as input to mimic get_maintainers.pl output, but that might take some time. For now, you have to manually select a subsystem. I will try to find space on kernelci.org to grow this work but you can find a git tree here[0].
From the README.md """ An attempt to map kernel subsystems to kernel tests that should be run on patches or code by humans and CI systems.
Examples:
Find test info for a subsystem
./get_tests.py -s 'KUNIT TEST' --info
Subsystem: KUNIT TEST Maintainer: David Gow davidgow@google.com Mailing List: None Version: None Dependency: ['python3-mypy'] Test: smoke: Url: None Working Directory: None Cmd: ./tools/testing/kunit/kunit.py Env: None Param: run --kunitconfig lib/kunit Hardware: arm64, x86_64
Find copy-n-pastable tests for a subsystem
./get_tests.py -s 'KUNIT TEST'
./tools/testing/kunit/kunit.pyrun --kunitconfig lib/kunit """
Is this aligning with what people were expecting?
Awesome! I've been playing around a bit with this, and I think it's an excellent start.
There are definitely some more features I'd want in an ideal world (e.g., configuration matrices, etc), but this works well enough.
I've been playing around with a branch which adds the ability to actually run these tests, based on the 'run_checks.py' script we use for KUnit: https://github.com/sulix/test-catalog/tree/runtest-wip
In particular, this adds a '-r' option which runs the tests for the subsystem in parallel. This largely matches what I was doing manually — for instance, the KUnit section in test.yaml now has three different tests, and running it gives me this result: ../test-catalog/get_tests.py -r -s 'KUNIT TEST' Waiting on 3 checks (kunit-tool-test, uml, x86_64)... kunit-tool-test: PASSED x86_64: PASSED uml: PASSED
(Obviously, in the real world, I'd have more checks, including other architectures, checkpatch, etc, but this works as a proof-of-concept for me.)
I think the most interesting questions will be: - How do we make this work with more complicated dependencies (containers, special hardware, etc)? - How do we integrate it with CI systems — can we pull the subsystem name for a patch from MAINTAINERS and look it up here? - What about things like checkpatch, or general defconfig build tests which aren't subsystem-specific? - How can we support more complicated configurations or groups of configurations? - Do we add support for specific tools and/or parsing/combining output?
But I'm content to keep playing around with this a bit more for now.
Thanks, -- David