On Tue, Jun 16, 2020 at 09:16:01PM +0000, Bird, Tim wrote:
So far, most of the CI systems don't parse out diagnostic data, so it doesn't really matter what the format is. If it's useful for humans, it's valuable as is. However, it would be nice if that could change. But without some formalization of the format of the diagnostic data, it's an intractable problem for CI systems to parse it. So it's really a chicken and egg problem. To solve it, we would have to determine what exactly needs to be provided on a consistent basis for diagnostic data across many tests. I think that it's too big a problem to handle right now. I'm not opposed to migrating to some structure with yaml in the future, but free form text output seems OK for now.
For a CI system, if I see a test has failed, I expect to be able to click a link to get the log of that test, which includes the diagnostic lines. The other reason to have them there is to show progress during a manual run.
Yeah, I think it would be nice if all test frameworks/libraries for the kernel output tests in the same language.
Agreed.
$ git grep "TAP version" exec/binfmt_script:print("TAP version 1.3") kselftest.h: printf("TAP version 13\n"); kselftest/runner.sh: echo "TAP version 13" resctrl/resctrl_tests.c: printf("TAP version 13\n"); size/get_size.c: print("TAP version 13\n");
Looks like there are 2 tests to convert to kselftest.h, and then we can just change the version to 14 in the header and the runner. ;)