Paul,
What do you think about adding a condition to the "boots" test that stipulates that the board needs to boot 3 times?
I was looking at: https://docs.google.com/a/linaro.org/spreadsheet/ccc?key=0AkxwyUNxNaAadDE1Ml... and found that the "boots" column should have probably been yellow for a bit based on bug https://bugs.launchpad.net/linaro-android/+bug/900681 which I see again on iMX53.
That's one possibility, but is 3 enough? We could get into a game where 3, 10, 100 is never enough to catch the problem. The other obvious thing that jumps out at me here is that reboot and shutdown are tested on the ubuntu side, but not on android. If we covered both of those, we would already have multiple boots covered. I would be ok with 3 boots as a reasonable number, but I think it would be even better if we added it implicitly by adding reboot and halt as tests.
Thanks, Paul Larson
On Tue, May 8, 2012 at 8:32 AM, Zach Pfeffer zach.pfeffer@linaro.orgwrote:
Paul,
What do you think about adding a condition to the "boots" test that stipulates that the board needs to boot 3 times?
I was looking at:
https://docs.google.com/a/linaro.org/spreadsheet/ccc?key=0AkxwyUNxNaAadDE1Ml... and found that the "boots" column should have probably been yellow for a bit based on bug https://bugs.launchpad.net/linaro-android/+bug/900681 which I see again on iMX53.
-- Zach Pfeffer Android Platform Team Lead, Linaro Platform Teams Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
On 8 May 2012 18:57, Paul Larson paul.larson@linaro.org wrote:
That's one possibility, but is 3 enough? We could get into a game where 3, 10, 100 is never enough to catch the problem. The other obvious thing that jumps out at me here is that reboot and shutdown are tested on the ubuntu side, but not on android. If we covered both of those, we would already have multiple boots covered. I would be ok with 3 boots as a reasonable number, but I think it would be even better if we added it implicitly by adding reboot and halt as tests.
Its a start for a manual test. I think the 100 reboots that Alexander mentions in LAVA is also good. If you;re plus one would you add it to the Android test wiki?
Thanks, Paul Larson
On Tue, May 8, 2012 at 8:32 AM, Zach Pfeffer zach.pfeffer@linaro.org wrote:
Paul,
What do you think about adding a condition to the "boots" test that stipulates that the board needs to boot 3 times?
I was looking at:
https://docs.google.com/a/linaro.org/spreadsheet/ccc?key=0AkxwyUNxNaAadDE1Ml... and found that the "boots" column should have probably been yellow for a bit based on bug https://bugs.launchpad.net/linaro-android/+bug/900681 which I see again on iMX53.
-- Zach Pfeffer Android Platform Team Lead, Linaro Platform Teams Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
On Tue, May 8, 2012 at 6:48 PM, Zach Pfeffer zach.pfeffer@linaro.orgwrote:
On 8 May 2012 18:57, Paul Larson paul.larson@linaro.org wrote:
That's one possibility, but is 3 enough? We could get into a game where
3,
10, 100 is never enough to catch the problem. The other obvious thing
that
jumps out at me here is that reboot and shutdown are tested on the ubuntu side, but not on android. If we covered both of those, we would already have multiple boots covered. I would be ok with 3 boots as a reasonable number, but I think it would be even better if we added it implicitly by adding reboot and halt as tests.
Its a start for a manual test. I think the 100 reboots that Alexander mentions in LAVA is also good. If you;re plus one would you add it to the Android test wiki?
Adding what, the 3 boots instead of single boot, or the boot + reboot + halt and boot as I described (which would still be 3 boots, but would also cover additional cases that are currently not covered).
On 9 May 2012 00:15, Paul Larson paul.larson@linaro.org wrote:
On Tue, May 8, 2012 at 6:48 PM, Zach Pfeffer zach.pfeffer@linaro.org wrote:
On 8 May 2012 18:57, Paul Larson paul.larson@linaro.org wrote:
That's one possibility, but is 3 enough? We could get into a game where 3, 10, 100 is never enough to catch the problem. The other obvious thing that jumps out at me here is that reboot and shutdown are tested on the ubuntu side, but not on android. If we covered both of those, we would already have multiple boots covered. I would be ok with 3 boots as a reasonable number, but I think it would be even better if we added it implicitly by adding reboot and halt as tests.
Its a start for a manual test. I think the 100 reboots that Alexander mentions in LAVA is also good. If you;re plus one would you add it to the Android test wiki?
Adding what, the 3 boots instead of single boot, or the boot + reboot + halt and boot as I described (which would still be 3 boots, but would also cover additional cases that are currently not covered).
Yeah. So the boot test is:
Repeat 3 times: Boot to UI Shutdown
On Tue, 2012-05-08 at 20:48 -0500, Zach Pfeffer wrote:
I think the 100 reboots that Alexander mentions in LAVA is also good. If you;re plus one would you add it to the Android test wiki?
How long do we wait for a boot to complete before rebooting? On vexpress it takes about 5 minutes before logcat stops showing all the usual errors and warnings and the system seems to go idle. If we waited that log, this 100 reboot test would take 8 hours or so. It would seem a bit excessive to tie up limited testing resources for that long.
On Wed, May 9, 2012 at 10:36 AM, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Tue, 2012-05-08 at 20:48 -0500, Zach Pfeffer wrote:
I think the 100 reboots that Alexander mentions in LAVA is also good. If you;re plus one would you add it to the Android test wiki?
How long do we wait for a boot to complete before rebooting? On vexpress it takes about 5 minutes before logcat stops showing all the usual errors and warnings and the system seems to go idle. If we waited that log, this 100 reboot test would take 8 hours or so. It woul
Obviously, I don't know the right number, so we should make the amount of reboots to be easily tweakable for our needs.
linaro-android@lists.linaro.org
-
Alexander Sack -
Jon Medhurst (Tixy) -
Paul Larson -
Zach Pfeffer