On 8 March 2012 23:44, Paul Larson <paul.larson@linaro.org> wrote:
>
>
> On Thu, Mar 8, 2012 at 4:07 PM, Zach Pfeffer <zach.pfeffer@linaro.org>
> wrote:
>>
>> Right. If a command in the command log causes the unit-under-test to
>> do any of those things, then the unit should be rebooted (in the case
>> of a hang) or the reboot should be sensed (in case the command caused
>> a reboot) and when the unit boots back up, LAVA would continue the
>> test on the command after the one that caused it to hang, reboot,
>> freeze, etc. LAVA should save the logs for the entire command file
>> run, including all hangs and reboots.
>
> As mentioned when we talked about this on the phone, I don't think this is
> the best way to approach the problem. Currently, if a test hangs, times out,
> reboots improperly (and thus times out because it won't get to the expected
> place), lava will reboot the system back into the test image.  However, at
> this point, it will mark the previous testsuite it tried to run as a failed
> run.  Then, if there are other tests queued up to run, it will continue
> running with the next one - NOT try to re-enter the existing test and
> continue where it left off.  This is not a capability currently supported in
> lava-android-test.
>
> The good news is, I think there's a much more straightforward way to do
> this.
> I haven't really seen an example of the kinds of things you want to run, but
> it seems that it's just a list of several tests, possibly with multiple
> steps in each test.  And if it reboots, or has a problem, you want it to
> fail that one and continue to the next.  This is very simple to do, if you
> simply define each of those tests as lava-android-tests.  Then you can run
> them all in your test job, and I think it will do exactly what you are
> looking for here.  Also, they can then be easily reused in other test jobs.

Hmm...

Here's what I'd like to do.

I'd like to pass a simple command file to LAVA. I'd like LAVA to
execute each test and if that test hangs or causes a reboot I'd like
LAVA to pick back up at the next test case. I'd like LAVA to collect
all the logs and send them either to LAVA or back to us for post
processing.

I'd like to do this, because I'd like to be able to include these
command files in our builds so that people can run them themselves and
include the post processing commands for people to see what passed and
failed.

The text file also gives me a very easy way to add and remove tests on
a per-build basis since it goes along with the build.