On Wed, 13 May 2020 at 06:30, David Gow davidgow@google.com wrote:
This is a proof-of-concept to support "skipping" tests.
The kunit_mark_skipped() macro marks the current test as "skipped", with the provided reason. The kunit_skip() macro will mark the test as skipped, and abort the test.
The TAP specification supports this "SKIP directive" as a comment after the "ok" / "not ok" for a test. See the "Directives" section of the TAP spec for details: https://testanything.org/tap-specification.html#directives
kunit_tool will parse this SKIP directive, and renders skipped tests in yellow and counts them. Skipped tests do not affect the result for a suite.
Signed-off-by: David Gow davidgow@google.com
Following on from discussions about the KCSAN test[1], which requires a multi-core/processor system to make sense, it would be useful for tests to be able to mark themselves as "skipped", where tests have runtime dependencies which aren't met.
As a proof-of-concept, this patch doesn't implement some things which we'd ideally like to have (e.g., non-static "reasons" for skipping the test, maybe some SKIP macros akin to the EXPECT and ASSERT ones), and the implementation is still pretty hacky, but I though I'd put this out there to see if there are any thoughts on the concept in general.
Cheers, -- David
include/kunit/test.h | 12 ++++++++++++ lib/kunit/kunit-example-test.c | 7 +++++++ lib/kunit/test.c | 23 ++++++++++++++++------- tools/testing/kunit/kunit_parser.py | 21 +++++++++++++++++---- 4 files changed, 52 insertions(+), 11 deletions(-)
diff --git a/include/kunit/test.h b/include/kunit/test.h index 9b0c46a6ca1f..7817c5580b2c 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -178,6 +178,7 @@ struct kunit_suite { /* private - internal use only */ struct dentry *debugfs; char *log;
const char *skip_directive;};
/** @@ -213,6 +214,8 @@ struct kunit { * protect it with some type of lock. */ struct list_head resources; /* Protected by lock. */
const char *skip_directive;};
void kunit_init_test(struct kunit *test, const char *name, char *log); @@ -391,6 +394,15 @@ void kunit_cleanup(struct kunit *test);
void kunit_log_append(char *log, const char *fmt, ...);
+#define kunit_mark_skipped(test_or_suite, reason) \
(test_or_suite)->skip_directive = "SKIP " reason
Would it be useful to make this accept any string possibly with a format? Otherwise, the reason will always need to be a constant string here.
I'm fine with printing more detailed info via pr_warn() or so, if that's an unreasonable request.
+#define kunit_skip(test_or_suite, reason) \
do { \kunit_mark_skipped(test_or_suite, reason); \kunit_try_catch_throw(&((test_or_suite)->try_catch)); \} while (0)
[...]
This looks good to me. One question I'd have is: will this work in test_init functions? Because in the KCSAN test, the test setup determines if we have enough CPUs, and then causes test_init to return a non-zero error code.
Thanks, -- Marco