When one parameter of a parameterised test failed, its failure would be propagated to the overall test, but not to the suite result (unless it was the last parameter).
This is because test_case->success was being reset to the test->success result after each parameter was used, so a failing test's result would be overwritten by a non-failing result. The overall test result was handled in a third variable, test_result, but this was disacarded after the status line was printed.
Instead, just propagate the result after each parameter run.
Signed-off-by: David Gow davidgow@google.com Fixes: fadb08e7c750 ("kunit: Support for Parameterized Testing") ---
This is fixing quite a serious bug where some test suites would appear to succeed even if some of their component tests failed. It'd be nice to get this into kunit-fixes ASAP.
(This will require a rework of some of the skip tests work, for which I'll send out a new version soon.)
Cheers, -- David
lib/kunit/test.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 2f6cc0123232..17973a4a44c2 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -376,7 +376,7 @@ static void kunit_run_case_catch_errors(struct kunit_suite *suite, context.test_case = test_case; kunit_try_catch_run(try_catch, &context);
- test_case->success = test->success; + test_case->success &= test->success; }
int kunit_run_tests(struct kunit_suite *suite) @@ -388,7 +388,7 @@ int kunit_run_tests(struct kunit_suite *suite)
kunit_suite_for_each_test_case(suite, test_case) { struct kunit test = { .param_value = NULL, .param_index = 0 }; - bool test_success = true; + test_case->success = true;
if (test_case->generate_params) { /* Get initial param. */ @@ -398,7 +398,6 @@ int kunit_run_tests(struct kunit_suite *suite)
do { kunit_run_case_catch_errors(suite, test_case, &test); - test_success &= test_case->success;
if (test_case->generate_params) { if (param_desc[0] == '\0') { @@ -420,7 +419,7 @@ int kunit_run_tests(struct kunit_suite *suite) } } while (test.param_value);
- kunit_print_ok_not_ok(&test, true, test_success, + kunit_print_ok_not_ok(&test, true, test_case->success, kunit_test_case_num(suite, test_case), test_case->name); }