Hi,
On Tue, Jul 25, 2023 at 09:25:15PM +0000, Rae Moar wrote:
Add filtering of test attributes. Users can filter tests using the module_param called "filter".
Filters are imputed in the format: <attribute_name><operation><value>
Example: kunit.filter="speed>slow"
Operations include: >, <, >=, <=, !=, and =. These operations will act the same for attributes of the same type but may not between types.
Note multiple filters can be inputted by separating them with a comma. Example: kunit.filter="speed=slow, module!=example"
Since both suites and test cases can have attributes, there may be conflicts. The process of filtering follows these rules:
- Filtering always operates at a per-test level.
- If a test has an attribute set, then the test's value is filtered on.
- Otherwise, the value falls back to the suite's value.
- If neither are set, the attribute has a global "default" value, which is used.
Filtered tests will not be run or show in output. The tests can instead be skipped using the configurable option "kunit.filter_action=skip".
Note the default settings for running tests remains unfiltered.
Finally, add "filter" methods for the speed and module attributes to parse and compare attribute values.
Note this filtering functionality will be added to kunit.py in the next patch.
Reviewed-by: David Gow davidgow@google.com Signed-off-by: Rae Moar rmoar@google.com
This patch causes all my s390 boot tests to crash.
[ 17.009188] # module: kunit [ 17.009239] 1..8 [ 17.014967] ok 1 parse_filter_test [ 17.020755] ok 2 filter_suites_test [ 17.026350] ok 3 filter_suites_test_glob_test [ 17.033102] ok 4 filter_suites_to_empty_test [ 17.035422] Unable to handle kernel pointer dereference in virtual kernel address space [ 17.035546] Failing address: 0000000001501000 TEID: 0000000001501407 [ 17.035617] Fault in home space mode while using kernel ASCE. [ 17.035833] AS:0000000002164007 R3:000000001f71c007 S:000000001f711000 P:000000000150131f [ 17.036313] Oops: 0004 ilc:2 [#1] SMP [ 17.036851] Modules linked in: [ 17.037026] CPU: 0 PID: 193 Comm: kunit_try_catch Tainted: G N 6.5.0-01207-g1c59d383390f #1 [ 17.037106] Hardware name: QEMU 8561 QEMU (KVM/Linux) [ 17.037199] Krnl PSW : 0704e00180000000 0000000000950e18 (kunit_next_attr_filter+0xc8/0x1c8) [ 17.037451] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3 [ 17.037531] Krnl GPRS: 0000037f00000000 000000000150120a 000000000000003e 000000000000003e [ 17.037583] 0000038000197e04 0000038000197c04 0000038000197df0 000003800000000a [ 17.037629] 0000038000000005 0000000001501203 00000000015011fe 0000037f0000000c [ 17.037675] 0000000005404100 0000038000197e08 0000000000950dda 0000038000197cd8 [ 17.038176] Krnl Code: 0000000000950e0a: b9180098 agfr %r9,%r8 [ 17.038176] 0000000000950e0e: a7080000 lhi %r0,0 [ 17.038176] #0000000000950e12: e32090000090 llgc %r2,0(%r9) [ 17.038176] >0000000000950e18: 92009000 mvi 0(%r9),0 [ 17.038176] 0000000000950e1c: b904003a lgr %r3,%r10 [ 17.038176] 0000000000950e20: c418007bd73c lgrl %r1,00000000018cbc98 [ 17.038176] 0000000000950e26: b25d0013 clst %r1,%r3 [ 17.038176] 0000000000950e2a: a714fffe brc 1,0000000000950e26 [ 17.038625] Call Trace: [ 17.038679] [<0000000000950e18>] kunit_next_attr_filter+0xc8/0x1c8 [ 17.038749] [<000000000094e5f2>] parse_filter_attr_test+0x92/0x360 [ 17.038796] [<000000000094e39a>] kunit_generic_run_threadfn_adapter+0x32/0x48 [ 17.038840] [<0000000000186702>] kthread+0x142/0x148 [ 17.038884] [<0000000000103138>] __ret_from_fork+0x40/0x58 [ 17.038927] [<000000000102df9a>] ret_from_fork+0xa/0x30 [ 17.038984] INFO: lockdep is turned off. [ 17.039027] Last Breaking-Event-Address: [ 17.039062] [<0000000000950da4>] kunit_next_attr_filter+0x54/0x1c8 [ 17.039303] Kernel panic - not syncing: Fatal exception: panic_on_oops
Bisect log is attached.
Guenter
--- # bad: [1c59d383390f970b891b503b7f79b63a02db2ec5] Merge tag 'linux-kselftest-nolibc-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest # good: [2dde18cd1d8fac735875f2e4987f11817cc0bc2c] Linux 6.5 git bisect start '1c59d383390f' 'v6.5' # good: [42a7f6e3ffe06308c1ec43a7dac39a27de101574] Merge tag 'x86_microcode_for_v6.6_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip git bisect good 42a7f6e3ffe06308c1ec43a7dac39a27de101574 # good: [330235e87410349042468b52baff02af7cb7d331] Merge tag 'acpi-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm git bisect good 330235e87410349042468b52baff02af7cb7d331 # good: [5a31cc7297072a7266a910ca5266b640d27803b4] Merge tag 'linux-kselftest-next-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest git bisect good 5a31cc7297072a7266a910ca5266b640d27803b4 # good: [eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4] tools/nolibc: riscv: shrink _start with _start_c git bisect good eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4 # good: [faeb4e09fe77262f9a6b2f9f874eec0b6850721e] selftests/nolibc: add test support for ppc64 git bisect good faeb4e09fe77262f9a6b2f9f874eec0b6850721e # bad: [76066f93f1df27657eb937b7c9c091e3a6abf4db] kunit: add tests for filtering attributes git bisect bad 76066f93f1df27657eb937b7c9c091e3a6abf4db # good: [64bd4641310c41a1ecf07c13c67bc0ed61045dfd] MAINTAINERS: add Rust KUnit files to the KUnit entry git bisect good 64bd4641310c41a1ecf07c13c67bc0ed61045dfd # bad: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes git bisect bad 529534e8cba3e60f843a682e2a3149612b30d608 # good: [02c2d0c2a84172c3c7ec0229c61db55d23dd4730] kunit: Add speed attribute git bisect good 02c2d0c2a84172c3c7ec0229c61db55d23dd4730 # good: [a00a72709175a4d53096301a8792b8171d1223e5] kunit: Add module attribute git bisect good a00a72709175a4d53096301a8792b8171d1223e5 # first bad commit: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes