On Thu, Sep 23, 2010, Michael Hope wrote:
To overdo things, it would be nice to mix the compiler in as well. I've been planning to add a test case to the compiler build that builds the current Ubuntu kernel and boots it inside QEMU.
I think we should have batteries of test cases of each component before it sees a release, e.g. we should have a kernel build + run test before releasing a new toolchain, and a kernel run test before releasing a new QEMU, but this needs not be based on crack of the day. It's sufficient to build / run the latest released kernel for these tests rather than a git kernel. In the same way, I don't think we need to build kernels out of bzr toolchains; the latest released toolchain would be good.
We should get sufficient cross-testing via continuous integration of our releases in our development platforms (aka package repositories, like maverick or maverick+1).
It's entirely possible to do as you say, but I think it's basically testing too many things at once resulting in unstable x unstable x unstable combination, rather than stable x stable x unstable where we can blame changes quickly.