On Nov 21, 2017, at 10:55 AM, Fathi Boudra fathi.boudra@linaro.org wrote:
On 21 November 2017 at 18:39, Tom Gall tom.gall@linaro.org wrote:
Hi All,
So I’ve spent a little time checking into how one might take advantage of ‘useful’ test cases which are part of various distro packages.
Specifically this often runs the make check or akin target.
These are often wired into the various package build systems that distros have. The nice thing about using this mechanism is:
- the package already has been base lined.
- packages run a range of completely useless tests for testing the kernel (like checking internal apis to a package) to things which actually trigger kernel activity like running valgrind or driving some network activity.
So in my case I’ve been using gentoo to dork around since gentoo has a pretty nice process and happily is able to email testing results. More about why this is a good attribute in a moment.
The use scenario ends up looking like:
RC released -> build/deploy new kernel -> emerge package_foo; email package_foo result; emerge package_bar; email package_bar result; post process results and integrate into qa-reports.
Where this might be viewed as kind of weird is this class of testing really doesn’t belong in LAVA. One doesn’t need (or want) to do whole fresh image deployment. I know what you’re thinking, the system will be dirty after a run is made so one MUST do a redeploy or the results will be invalid.
That might be the case for some environments, but it’s not the case for gentoo. Package build and test is already ‘clean’ for the purposes of a system under test because in the land of gentoo we sandbox. If we didn’t distro package testing would be incredibly painful. So really the only ‘danger’ is that the local environment does need to build and boot a to the kernel under test but that’s not a big deal, since if something fails to build/boot, well kernelci will complain. This isn’t about testing that type of situation.
Anyway I should have some test test 4.14 results in a few hours as some examples. I’m thinking an upstream report is probably super super simple, unless a regression pops up.
Comments?
For OE RPB, we've recently enabled similar feature (ptest) on 410c builds.
ptest is dirty tho. It doesn’t sandbox.
We still use LAVA in our case and the rootfs grows significantly (which could be an issue for boards with 4G eMMC). It's been done only on 410c to see how useful it could be and gather some data for discussion.
That’s cool. To be useful on dev boards I think we’d really need sata based root filesystems or pretty close to it.
Where’s that socionext developer workstation :-)
Tom
Lts-dev mailing list Lts-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lts-dev