Hi, Willy
On Thu, Jul 06, 2023 at 05:11:17PM +0800, Zhangjin Wu wrote:
two newlines are added around the test summary line to extrude the test status.
But then we're back to making it annoying to check, having to figure if we need to grep -A or grep -B etc. With grep 'status:' we would get a synthetic status and the counters together. Why do you think it's not convenient ? Or am I the only one considering it useful to just run grep "status:" on all output files and figure a global status at once ?
Sorry, Willy, my commit message may mislead you a little.
The newlines are added around the whole test summary line (with the status info), not only around the 'status info' ;-)
An example is added in our cover-letter (use '%3d' instead of '%03d' here):
... <-- newline here --> 138 test(s): 135 passed, 2 skipped, 1 failed => status: failure <-- newline here --> See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.out
Or:
... <-- newline here --> 137 test(s): 134 passed, 3 skipped, 0 failed => status: warning <-- newline here --> See all results in /labs/linux-lab/src/linux-stable/tools/testing/selftests/nolibc/run.out
It is not for status grep, it is for developers to easily see the whole summary line at a glance (I should add this in the commit message), especially when we run tests for lots of architectures one by one automatically, during the tests running, these newlines may help us to see the status at a glance.
And further, if not consider pure-text, the colors may be more helpful, for example, red for failed/failure, yellow for skipped/warning, green for passed/success, for example:
$ echo | awk 'END{printf("138 test(s): \033[32m135\033[0m passed, \033[33m 2\033[0m skipped, \033[31m 1\033[0m failed => status: \033[31mfailure\033[0m\n");}' 138 test(s): 135 passed, 2 skipped, 1 failed => status: failure
But as we can see, the color control code is not readable and it may break the simple "status: failure" grep, we should use something like "status: .*failure" ;-)
It is possible to filter out the color info in the last run.out and only reserve the colors info in the console.
$ cat run.tmp.out | sed 's/\x1b[[0-9;]*m//g' | col -bp > run.out
As a summary, the "status info grep" you proposed is very helpful to summarize all of the tests after the testing finish, I do like it:
$ grep "status: " /path/to/all/run.out 138 test(s): 135 passed, 2 skipped, 1 failed => status: failure 137 test(s): 134 passed, 3 skipped, 0 failed => status: warning
And these newlines (and even further with colors) are added to help developers to see what happens during the tests running at a glance.
Thanks, Zhangjin
Willy