On Fri, Aug 20, 2021 at 1:01 PM Rae Moar rmoar@google.com wrote:
Update to kunit_parser to improve compatibility with KTAP specification including arbitrarily nested tests. Patch accomplishes three major changes:
- Use a general Test object to represent all tests rather than TestCase
and TestSuite objects. This allows for easier implementation of arbitrary levels of nested test and promotes the idea that both test suites and test cases are tests.
- Print errors incrementally rather than all at once after the
parsing finishes to maximize information given to the user in the case of the parser given invalid input and to increase the helpfulness of the timestamps given during printing.
- Increase compatibility for different formats of input. Arbitrary levels
of nested tests supported. Also, test cases and test suites are now supported to be present on the same level of testing.
This patch now implements the KTAP specification as described here: https://lore.kernel.org/linux-kselftest/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqaJ....
This patch adjusts the kunit_tool_test.py file to check for the correct outputs from the new parser and adds a new test to check the parsing for a KTAP result log with correct format for multiple nested subtests (test_is_test_passed-all_passed_nested.log).
This patch also alters the kunit_json.py file to allow for arbitrarily nested tests.
Signed-off-by: Rae Moar rmoar@google.com
One minor question/potential issue below, otherwise:
Reviewed-by: Brendan Higgins brendanhiggins@google.com
[...]
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 75045aa0f8a1..ca760ee32096 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -106,10 +106,10 @@ class KUnitParserTest(unittest.TestCase): with open(log_path) as file: result = kunit_parser.extract_tap_lines(file.readlines()) self.assertContains('TAP version 14', result)
self.assertContains(' # Subtest: example', result)self.assertContains(' 1..2', result)self.assertContains(' ok 1 - example_simple_test', result)self.assertContains(' ok 2 - example_mock_test', result)
self.assertContains('# Subtest: example', result)self.assertContains('1..2', result)self.assertContains('ok 1 - example_simple_test', result)self.assertContains('ok 2 - example_mock_test', result) self.assertContains('ok 1 - example', result)
Do you have any test cases which accept "ok/not ok" lines without a "-"? (Or tests for any other KTAP compliant Kselftest output?)
def test_output_with_prefix_isolated_correctly(self):
Cheers!