On Fri, Aug 28, 2020 at 12:17:05AM +0800, David Gow wrote:
On Thu, Aug 27, 2020 at 9:14 PM Marco Elver elver@google.com wrote:
Just an idea: Maybe the names are also an opportunity to distinguish real _unit_ style tests and then the rarer integration-style tests. I personally prefer using the more generic *-test.c, at least for the integration-style tests I've been working on (KUnit is still incredibly valuable for integration-style tests, because otherwise I'd have to roll my own poor-man's version of KUnit, so thank you!). Using *_kunit.c for such tests is unintuitive, because the word "unit" hints at "unit tests" -- and having descriptive (and not misleading) filenames is still important. So I hope you won't mind if *-test.c are still used where appropriate.
This is a good point, and I guess not one that has really been examined. I'm not sure what to think of some of the lib/ tests. test_user_copy seems to be a "unit" test -- it's validating the function family vs all kinds of arguments and conditions. But test_overflow is less unit and more integration, which includes "do all of these allocators end up acting the same way?" etc
I'm not really sure what to make of that -- I don't really have a formal testing background.
As Brendan alluded to in the talk, the popularity of KUnit for these integration-style tests came as something of a surprise (more due to our lack of imagination than anything else, I suspect). It's great that it's working, though: I don't think anyone wants the world filled with more single-use test "frameworks" than is necessary!
I guess the interesting thing to note is that we've to date not really made a distinction between KUnit the framework and the suite of all KUnit tests. Maybe having a separate file/module naming scheme could be a way of making that distinction, though it'd really only appear when loading tests as modules -- there'd be no indication in e.g., suite names or test results. The more obvious solution to me (at least, based on the current proposal) would be to have "integration" or similar be part of the suite name (and hence the filename, so _integration_kunit.c or similar), though even I admit that that's much uglier. Maybe the idea of having the subsystem/suite distinction be represented in the code could pave the way to having different suites support different suffixes like that.
Heh, yeah, let's not call them "_integration_kunit.c" ;) _behavior.c? _integration.c?
Do you know of any cases where something has/would have both unit-style tests and integration-style tests built with KUnit where the distinction needs to be clear?
This is probably the right question. :)