On Tue, 2012-04-24 at 01:39 -0500, Paul Larson wrote:
On Tue, Apr 24, 2012 at 1:30 AM, Jon Medhurst (Tixy) tixy@linaro.orgwrote:
[...]
I justify to myself that I didn't need to run them by using the fact that the tests weren't marked with and 'm' to say they should be run monthly. But to be honest, after 5 hour testing the release I wanted to be doing other things, not having my work laptop out of action for what would probably be the best part of another day.
If these tests are meant to run monthly, can't we automate this in 'the cloud' somewhere?
Thanks Tixy, that echoes *exactly* what I was trying to get at. These tests are unreasonably long, and unless they can be automated I think they are of limited value. The tests themselves are really not testing the images, but rather the scripts that are intended to make it easier for people to do these tasks.
Well, the scripts are something we're distributing so they should have some testing.
Assuming that the scripts for different boards are essentially the same but with different parameters couldn't we just test one board a month?
Alternatively, as they are meant to build the entire Android system, why don't we produce all the release images using these scripts? That way the final binaries we test and distribute should be pretty much identical to what a user would produce using the same scripts.