On 13/05/13 16:12, the mail apparently from Nicolas Dechesne included:
On Thu, May 9, 2013 at 7:56 PM, Fathi Boudra <fathi.boudra@linaro.org mailto:fathi.boudra@linaro.org> wrote:
> Hi all, > > I just wanted to forward this thread from LAKML to linaro-dev: > http://article.gmane.org/gmane.linux.ports.tegra/10683 > > Seems there is lots desire for an improvement to automated build > coverage and automated reporting along with it. I replied to it. We've got already such daily builds with boot testing: https://ci.linaro.org/jenkins/view/kernel-ci I'm surprised that some people involved in Linaro and this thread didn't mentioned it. Anyway, it's a good opportunity to remind people that we've got a Kernel CI and I'll be happy to get more feedback to improve it.Hi Fathi,
I have to admit that what we do in terms of Kernel CI is still a bit fuzzy to me, even now that I am an insider. When I was at TI and working closely with the TI Landing team, I don't believe we ever reached the point where Linaro kernel CI was useful for the 'products' we were jointly doing. Now that I am at Linaro, I am going to need LAVA and kernel CI for our project shortly. I have no doubt that what is being is worthwhile, but I believe a little bit of marketing and/or presentation would be very welcome. It might be nice to highlight the bugs that have been found (and fixed?) *thanks to* Linaro kernel CI too, for example. also in the link above all of the 7 'active' jobs are failing with 3 of them who always failed, and 2 of them failing for 2 weeks. so it's not clear what that means. i am sure it doesn't mean that none of our kernel ever boots ;-) if we want Kernel CI to be useful and kernel devs to rely on it, it should work all the time, so that failure are quickly identified and fixed. maybe this is why Linaro Kernel CI was not mentioned by Linaro people in that thread.
I think TI use of CI only evolved as far as compiling the thing, it's not hooked up to any actual testing.
The error mails we are still getting spammed with are partly my fault.
Previously, LAVA would remain silent if a build failed. That is quite a bad situation if you're committing to that tree, and the last thing you heard was everything is good then the build machine has a problem and stops testing. You can have committed something that broke build even, but continue thinking everything is good because nothing is telling you that it's not actually in test any more.
So after some prompting from me pointing out that false sense of security undermines the point of LAVA, now we get notifications that the build attempt failed. However since I didn't touch tilt-3.4 or tilt-tracking for months, it's surprising the number of failure reports we periodically get that are basically Lava infrastructure choking on trying to build it, not any actual problems.
-Andy