-----Original Message----- From: Brendan Higgins
On Tue, Feb 18, 2020 at 12:49 PM Bird, Tim Tim.Bird@sony.com wrote:
-----Original Message----- From: Brendan Higgins
On Wed, Feb 12, 2020 at 7:25 PM Frank Rowand frowand.list@gmail.com wrote:
On 2/7/20 10:58 AM, Alan Maguire wrote:
...
diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 9242f93..aec607f 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -10,6 +10,7 @@ #include <linux/kernel.h> #include <linux/sched/debug.h>
+#include "debugfs.h" #include "string-stream.h" #include "try-catch-impl.h"
@@ -28,73 +29,91 @@ static void kunit_print_tap_version(void) } }
-static size_t kunit_test_cases_len(struct kunit_case *test_cases) +size_t kunit_suite_num_test_cases(struct kunit_suite *suite) { struct kunit_case *test_case; size_t len = 0;
for (test_case = test_cases; test_case->run_case; test_case++)
kunit_suite_for_each_test_case(suite, test_case) len++; return len;} +EXPORT_SYMBOL_GPL(kunit_suite_num_test_cases);
static void kunit_print_subtest_start(struct kunit_suite *suite) { kunit_print_tap_version();
pr_info("\t# Subtest: %s\n", suite->name);pr_info("\t1..%zd\n", kunit_test_cases_len(suite->test_cases));
kunit_log(KERN_INFO, suite, "# Subtest: %s", suite->name);kunit_log(KERN_INFO, suite, "1..%zd",kunit_suite_num_test_cases(suite));The subtest 'is a TAP stream indented 4 spaces'. (So the old code was also incorrect since it indented with a tab.)
Whoops.
I agree that fixing tabs to spaces is probably the easiest thing to do here; nevertheless, I think this might be a good time to talk about other deviations from the spec and what to do about it. This might also be a good time to bring up Tim's comment at LPC last year about forking TAP. Arguably I already have given that TAP14 is still under review and is consequently subject to change.
Additionally, the way I report expectation/assertion failures are my own extension to the TAP spec. I did this because at the time I wasn't ready to open the can of worms that was adding a YAML serializer to the Linux kernel; I mentioned adding a YAML serializer at LPC and people didn't seem super thrilled with the idea.
I'm not sure I follow. Are you talking about writing YAML or interpreting YAML. You don't need a serializer to write YAML. It can be done with straight text output. I guess it depends on the scope of what you envision. Even if you want to do more than trivial structured output, I don't think you'll need a full serializer. (IOW, I think you could sneak something in and just call it a test output formatter. Just don't call it YAML and most people won't notice. :-)
Yeah, for the first one or two things just printing things out directly is probably fine, and yes, I could have just snuck it in, but at the time it wasn't a hindrance for me to ask what people wanted: I had already worked around it.
In any case, I was just explaining part of why I did expectations and assertion failures the way that I did.
Further both the TAP implementation here as well as what is in kselftest have arbitrary kernel output mixed in with TAP output, which seems to be a further deviation from the spec.
Well that's a different kettle of worms, and really argues for staying with something that is strictly line-based.
In an effort to do this, and so that at the very least I could document what I have done here, I have been looking into getting a copy of TAP into the kernel. Unfortunately, TAP appears to have some licensing issues. TAP says that it can be used/modified "under the same terms as Perl itself" and then provides a dead link. I filed a pull request to update the licence to the Perl Artistic Licence 1.0 since I believe that is what they are referencing; however, I have not heard back from them yet.
When you say "getting a copy of TAP into the kernel", I presume you mean an existing implementation to produce TAP output? Or are you talking about a TAP interpreter? I'm not sure the former needs to use an existing implementation.
Sorry, that wasn't clear. I meant: get a copy of the TAP spec itself into the kernel documentation. KUnit already has an implementation.
Ah. OK. Thanks.
I previously volunteered (in Lisbon) to write up the TAP deviations, and never got around to it. Sorry about that. I can try to work on it now if people are still interested.
I think that would be useful. I would do it, but, as I mentioned, there are licensing issues with the TAP spec. I am trying to resolve those issues, and am currently waiting to hear back from somebody from TAP.
If that drags on at all, I'd be happy to write up some docs from scratch. Since we have deviations from TAP, and I was supposed to write up those deviations anyway, it might be worthwhile to combine those two actions. Also, a good chunk of the spec examples are spent on the YAML stuff, which we currently don't support.
We can reference the official spec of course. I think I suggested we probably need to fork it and define our own KTAP anyway (with an eye towards staying compatible with external parsers).
It *is* a bit weird that their licensing is unclear. I get the feeling, though, that the project is very much on people's backburner. (Mine included, unfortunately.) -- Tim