On 1 April 2012 22:07, Andy Doan andy.doan@linaro.org wrote:
On 04/01/2012 08:26 PM, Zach Pfeffer wrote:
In other words, are we really submitting LAVA jobs and not caring about the
results?
Since LAVA:
- Can't reliably boot all the builds in all configurations
- Doesn't use linaro-android-media-create (which we tell users to use)
- Doesn't use the right bootloaders
We've always hand tested our builds to ensure they work. Until LAVA:
- Can program a build in the same manner we tell users to
- Doesn't assume anything about the target, like it even booting
We have to keep hand testing.
I think even if LAVA were perfect, hand testing is still required. And I won't (in this thread) debate the limitations your bringing up.
In my case, LAVA has been working pretty reliably for Panda for about 4 months now (at least for my benchmark jobs). When I saw it broken, I pushed the issue and the team found a fix pretty quickly. So shouldn't we have someone paying attention to at least Panda builds and raise an issue when they trend from mostly working to completely broken?
Yeah, Panda's been pretty good. I think monitoring the builds fits pretty squarely in the new QA groups area. Paul, perhaps you can add, Android LAVA health to your daily checklist.