This past RC cycle I think we exposed a weakness in what we have in LKFT where the ability to execute some key functional stacks in the system to drive the kernel would probably be useful for validation.
The networking bug involving dhclient for example.
So what if we used either Debian, Gentoo or akin that has a mechanism that has as part of it’s packaging system a test target for each package. Simplest build a package, runs ‘make test’ (or akin) for some key packages that exercises parts of the system that should help tickle the kernel in interesting ways to tease out regressions.
Thus wouldn’t work on modest boards but the socionext or Juno boards could probably work fine.
Thoughts?
Regards,
Tom
On Mon, Oct 09, 2017 at 09:44:36AM +0000, Linaro QA wrote:
> Summary
> ------------------------------------------------------------------------
>
> kernel: 4.9.54
> git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
> git branch: linux-4.9.y
> git commit: f37eb7b586f1dd24a86c50278c65322fc6787722
> git describe: v4.9.54
> Test details: https://qa-reports.linaro.org/lkft/linux-stable-rc-4.9-oe/build/v4.9.54
Hm, I don't seem to have a test result here for 4.9.55-rc1, so I'm
hijacking this thread to ask what happened to the tests there?
Turns out that 4.9.55-rc1 did not work at all for networking, yet no
tests seem to have caught it. Are we not testing something with a
network here? You said you were using NFS, how did that work?
Anything we can do to add to the tests to verify that basic dhcp works?
We should learn from our mistakes...
Heck, the simple build/boot tests that Google runs instantly found this
issue when I tried to merge 4.9.55 into the android-common trees, maybe
I should just rely on thost tests from now on?
Because of this, I had to release 4.9.56 a few hours later fixing the
issue. This doesn't make me feel like I can trust this testing effort
at all just yet, given the track record it has shown this past
week. We have had two known-bad kernel releases and none of this testing
caught it at all (both were caught by community members...) :(
Back to booting the kernels on my laptop and running 'make
allmodconfig', that's found more errors so far than anything else...
greg k-h