On Fri, Jun 20, 2025 at 05:37:39PM +0800, David Gow wrote:
On Wed, 11 Jun 2025 at 15:38, Thomas Weißschuh thomas.weissschuh@linutronix.de wrote:
Currently there is no test validating the result reporting from nested tests. Add one, it will also be used to validate upcoming changes to the nested test parsing.
Signed-off-by: Thomas Weißschuh thomas.weissschuh@linutronix.de
This looks good, modulo a couple of minor suggestions below.
Regardless, Reviewed-by: David Gow davidgow@google.com
Cheers, -- David
tools/testing/kunit/kunit_tool_test.py | 9 +++++++++ .../kunit/test_data/test_is_test_passed-failure-nested.log | 7 +++++++ 2 files changed, 16 insertions(+)
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index bbba921e0eacb18663abfcabb2bccf330d8666f5..691cde9b030f7729128490c1bdb42ccee1967ad6 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -165,6 +165,15 @@ class KUnitParserTest(unittest.TestCase): self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status) self.assertEqual(result.counts.errors, 0)
def test_parse_failed_nested_tests_log(self):nested_log = test_data_path('test_is_test_passed-failure-nested.log')with open(nested_log) as file:result = kunit_parser.parse_run_tests(file.readlines(), stdout)self.assertEqual(kunit_parser.TestStatus.FAILURE, result.status)self.assertEqual(result.counts.failed, 2)self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[0].status)Is it worth also testing the value of the nested test's result here? i.e., self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[0].subtests[0].status)
This should be result.subtests[1].subtests[0].status. But Ack and done.
self.assertEqual(kunit_parser.TestStatus.FAILURE, result.subtests[1].status)def test_no_header(self): empty_log = test_data_path('test_is_test_passed-no_tests_run_no_header.log') with open(empty_log) as file:diff --git a/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log new file mode 100644 index 0000000000000000000000000000000000000000..835816e0a07715a514f5f5afab1b6250037feaf4 --- /dev/null +++ b/tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log @@ -0,0 +1,7 @@ +KTAP version 1 +1..2 +not ok 1 subtest 1
- KTAP version 1
- 1..1
not ok 1 test 1+not ok 2 subtest 2
Having these named 'subtest 1' and 'test 1' is a bit confusing to me (as it implies the outer tests are subtests of the inner ones, which isn't right).
Could we either swap 'subtest' and 'test' here, or -- if we want to preserve the match between 'subtest' here and the subtest in the python code -- label the inner one something like 'subsubtest'?
Ack.