On 8 March 2012 23:44, Paul Larson paul.larson@linaro.org wrote:
On Thu, Mar 8, 2012 at 4:07 PM, Zach Pfeffer zach.pfeffer@linaro.org wrote:
Right. If a command in the command log causes the unit-under-test to do any of those things, then the unit should be rebooted (in the case of a hang) or the reboot should be sensed (in case the command caused a reboot) and when the unit boots back up, LAVA would continue the test on the command after the one that caused it to hang, reboot, freeze, etc. LAVA should save the logs for the entire command file run, including all hangs and reboots.
As mentioned when we talked about this on the phone, I don't think this is the best way to approach the problem. Currently, if a test hangs, times out, reboots improperly (and thus times out because it won't get to the expected place), lava will reboot the system back into the test image. However, at this point, it will mark the previous testsuite it tried to run as a failed run. Then, if there are other tests queued up to run, it will continue running with the next one - NOT try to re-enter the existing test and continue where it left off. This is not a capability currently supported in lava-android-test.
The good news is, I think there's a much more straightforward way to do this. I haven't really seen an example of the kinds of things you want to run, but it seems that it's just a list of several tests, possibly with multiple steps in each test. And if it reboots, or has a problem, you want it to fail that one and continue to the next. This is very simple to do, if you simply define each of those tests as lava-android-tests. Then you can run them all in your test job, and I think it will do exactly what you are looking for here. Also, they can then be easily reused in other test jobs.
Hmm...
Here's what I'd like to do.
I'd like to pass a simple command file to LAVA. I'd like LAVA to execute each test and if that test hangs or causes a reboot I'd like LAVA to pick back up at the next test case. I'd like LAVA to collect all the logs and send them either to LAVA or back to us for post processing.
I'd like to do this, because I'd like to be able to include these command files in our builds so that people can run them themselves and include the post processing commands for people to see what passed and failed.
The text file also gives me a very easy way to add and remove tests on a per-build basis since it goes along with the build.
Thanks, Paul Larson