On 3/4/25 14:26, Mark Brown wrote:
On Thu, Apr 03, 2025 at 02:19:40PM +0800, David Gow wrote:
That being said, there are a _lot_ of Cirrus FW tests (over 5000 by my count when all of the parameterised ones are taken into account), so this can still trigger the default 5-minute kunit.py timeout. And because its split up over a lot of smaller suites, it doesn't really make sense to mark all of those tests/suites as slow (though some of them might merit it). If that's causing issues for people, maybe that could be a good reason to remove them from KUNIT_ALL_TESTS, though personally, I suspect the better answer might be to move it behind a config option which can then be excluded from the default .kunitconfig if necessary, as in [2]. (Or just increase the default timeut.)
I've not heard anyone mention hitting the timeouts, though now I run interactively I do see that the Cirrus stuff is a good proportion of the runtime for --alltests. We could potentially add a config option, though it'd need to end up in the config file so it gets turned on and we'd still run into any timeout issues. There's a tension with people expecting --alltests to actually run all the tests as well...
I don't want to get into the situation where nobody outside of Cirrus is running the tests. One of the main points of converting our tests to kunit was the hope that more people would run them on more configurations.
Ideally if they have any failures they would report those back with enough information for us to investigate. Disabling tests that fail and ignoring them isn't helpful.