Hi David,
On Tue, May 10, 2022 at 6:45 AM David Gow davidgow@google.com wrote:
I've just sent out a pull request to get this working under UML as well, which would simplify running these further: https://github.com/Rust-for-Linux/linux/pull/766
Thanks a lot!
Yeah, these are all fair points: particularly for small doctests.
Maybe having an optional name, which more significant tests could use to override the file:line names? That could be useful for a few of the larger, more often referenced tests.
Sounds reasonable. I can add support for that.
Ugh: it's a bit ugly either way. I suspect that file:line is still probably better, if only because we need some way of looking up the test in the code if it fails. I'd hate for people to be randomly hashing bits of just to find out what test is failing.
One redeeming quality is that the assertion prints the line/file number in the generated file, so it would still be possible to check where it came from:
[13:13:43] # rust_kernel_doctest_str_rs_somehash: ASSERTION FAILED at rust/doctests_kernel_generated.rs:2209 [13:13:43] Expected 2 > 3 to be true, but is false [13:13:43] not ok 43 - rust_kernel_doctest_str_rs_somehash [13:13:43] [FAILED] rust_kernel_doctest_str_rs_somehash
Another alternative is to keep the file:line information around without embedding it into the test name, e.g. in a TAP comment or a mapping file (which `kunit.py` could read).
But, yeah, before doing hashes or things like that, I would just go for simplicity and keep things as they are unless some use case really needs doctests to be stable.
Oops: I missed that (one of the issues with testing this on a different machine which had a rust toolchain). Looks good to me.
Ah: I didn't realise the plan was always to have crate-specific suites, and possibly to split things up.
The KTAP output specification does actually support arbitrary nesting (though KUnit itself doesn't at the moment), which would potentially be an option if (e.g.) providing the complete module nesting made sense. I'm not convinced that'd make things easier to read, though.
That is useful to know in case we need it, thanks!
Cheers, Miguel