About the sync up on Monday

Alexander Sack asac at linaro.org
Fri Mar 9 13:20:13 UTC 2012


On Fri, Mar 9, 2012 at 8:05 AM, Paul Larson <paul.larson at linaro.org> wrote:

> On Fri, Mar 9, 2012 at 12:16 AM, Zach Pfeffer <zach.pfeffer at linaro.org>wrote:
>
>> On 8 March 2012 23:44, Paul Larson <paul.larson at linaro.org> wrote:
>> >
>> >
>> > On Thu, Mar 8, 2012 at 4:07 PM, Zach Pfeffer <zach.pfeffer at linaro.org>
>> > wrote:
>> >>
>> >> Right. If a command in the command log causes the unit-under-test to
>> >> do any of those things, then the unit should be rebooted (in the case
>> >> of a hang) or the reboot should be sensed (in case the command caused
>> >> a reboot) and when the unit boots back up, LAVA would continue the
>> >> test on the command after the one that caused it to hang, reboot,
>> >> freeze, etc. LAVA should save the logs for the entire command file
>> >> run, including all hangs and reboots.
>> >
>> > As mentioned when we talked about this on the phone, I don't think this
>> is
>> > the best way to approach the problem. Currently, if a test hangs, times
>> out,
>> > reboots improperly (and thus times out because it won't get to the
>> expected
>> > place), lava will reboot the system back into the test image.  However,
>> at
>> > this point, it will mark the previous testsuite it tried to run as a
>> failed
>> > run.  Then, if there are other tests queued up to run, it will continue
>> > running with the next one - NOT try to re-enter the existing test and
>> > continue where it left off.  This is not a capability currently
>> supported in
>> > lava-android-test.
>> >
>> > The good news is, I think there's a much more straightforward way to do
>> > this.
>> > I haven't really seen an example of the kinds of things you want to
>> run, but
>> > it seems that it's just a list of several tests, possibly with multiple
>> > steps in each test.  And if it reboots, or has a problem, you want it to
>> > fail that one and continue to the next.  This is very simple to do, if
>> you
>> > simply define each of those tests as lava-android-tests.  Then you can
>> run
>> > them all in your test job, and I think it will do exactly what you are
>> > looking for here.  Also, they can then be easily reused in other test
>> jobs.
>>
>> Hmm...
>>
>> Here's what I'd like to do.
>>
>> I'd like to pass a simple command file to LAVA. I'd like LAVA to
>> execute each test and if that test hangs or causes a reboot I'd like
>> LAVA to pick back up at the next test case. I'd like LAVA to collect
>> all the logs and send them either to LAVA or back to us for post
>> processing.
>>
>> I'd like to do this, because I'd like to be able to include these
>> command files in our builds so that people can run them themselves and
>> include the post processing commands for people to see what passed and
>> failed.
>>
>> The text file also gives me a very easy way to add and remove tests on
>> a per-build basis since it goes along with the build.
>>
> I get that, but I don't see why you can't have it both ways.  If you want
> a simple set of instructions for someone to manually type at a command line
> to run through the tests, there's nothing preventing you from including
> that in a readme, script, or however you want to do that.  But those tests
> described in there would be really nice to convert into reusable tests in
> lava.  Once you have those building blocks, you can arrange them in future
> builds however you like.
>
>
I think what Zach is saying here is that he wants to use a single file to
maintain the list of tests to run for a build. That file is then sourced
both by a tool used by developers for local testing as well as by LAVA.

He most likely also would want the local tool used by devs to be a standard
solution provided by the lava test framework...

-- 
Alexander Sack
Technical Director, Linaro Platform Teams
http://www.linaro.org | Open source software for ARM SoCs
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linaro.org/pipermail/linaro-android/attachments/20120309/88e147d4/attachment.html>


More information about the linaro-android mailing list