About the sync up on Monday

[Paul Larson]

I wouldn't think for every single command, no.  As mentioned earlier, I
suspect it could be broken up into tests comprised of one or more
commands.  The tests are what would go into lava-android-test.

But I'd like to get a better understanding of your perspective.  Would it
be possible to provide an example of one of these files so I can get a
better idea of what you're looking for here?

Thanks,
Paul Larson
On Mar 9, 2012 11:40 AM, "Zach Pfeffer" <zach.pfeffer at linaro.org> wrote:

> On 9 March 2012 07:33, Paul Larson <paul.larson at linaro.org> wrote:
> > On Fri, Mar 9, 2012 at 7:20 AM, Alexander Sack <asac at linaro.org> wrote:
> >>
> >> On Fri, Mar 9, 2012 at 8:05 AM, Paul Larson <paul.larson at linaro.org>
> >> wrote:
> >>>
> >>> On Fri, Mar 9, 2012 at 12:16 AM, Zach Pfeffer <zach.pfeffer at linaro.org
> >
> >>> wrote:
> >>>>
> >>>> On 8 March 2012 23:44, Paul Larson <paul.larson at linaro.org> wrote:
> >>>> >
> >>>> >
> >>>> > On Thu, Mar 8, 2012 at 4:07 PM, Zach Pfeffer <
> zach.pfeffer at linaro.org>
> >>>> > wrote:
> >>>> >>
> >>>> >> Right. If a command in the command log causes the unit-under-test
> to
> >>>> >> do any of those things, then the unit should be rebooted (in the
> case
> >>>> >> of a hang) or the reboot should be sensed (in case the command
> caused
> >>>> >> a reboot) and when the unit boots back up, LAVA would continue the
> >>>> >> test on the command after the one that caused it to hang, reboot,
> >>>> >> freeze, etc. LAVA should save the logs for the entire command file
> >>>> >> run, including all hangs and reboots.
> >>>> >
> >>>> > As mentioned when we talked about this on the phone, I don't think
> >>>> > this is
> >>>> > the best way to approach the problem. Currently, if a test hangs,
> >>>> > times out,
> >>>> > reboots improperly (and thus times out because it won't get to the
> >>>> > expected
> >>>> > place), lava will reboot the system back into the test image.
> >>>> > However, at
> >>>> > this point, it will mark the previous testsuite it tried to run as a
> >>>> > failed
> >>>> > run.  Then, if there are other tests queued up to run, it will
> >>>> > continue
> >>>> > running with the next one - NOT try to re-enter the existing test
> and
> >>>> > continue where it left off.  This is not a capability currently
> >>>> > supported in
> >>>> > lava-android-test.
> >>>> >
> >>>> > The good news is, I think there's a much more straightforward way to
> >>>> > do
> >>>> > this.
> >>>> > I haven't really seen an example of the kinds of things you want to
> >>>> > run, but
> >>>> > it seems that it's just a list of several tests, possibly with
> >>>> > multiple
> >>>> > steps in each test.  And if it reboots, or has a problem, you want
> it
> >>>> > to
> >>>> > fail that one and continue to the next.  This is very simple to do,
> if
> >>>> > you
> >>>> > simply define each of those tests as lava-android-tests.  Then you
> can
> >>>> > run
> >>>> > them all in your test job, and I think it will do exactly what you
> are
> >>>> > looking for here.  Also, they can then be easily reused in other
> test
> >>>> > jobs.
> >>>>
> >>>> Hmm...
> >>>>
> >>>> Here's what I'd like to do.
> >>>>
> >>>> I'd like to pass a simple command file to LAVA. I'd like LAVA to
> >>>> execute each test and if that test hangs or causes a reboot I'd like
> >>>> LAVA to pick back up at the next test case. I'd like LAVA to collect
> >>>> all the logs and send them either to LAVA or back to us for post
> >>>> processing.
> >>>>
> >>>> I'd like to do this, because I'd like to be able to include these
> >>>> command files in our builds so that people can run them themselves and
> >>>> include the post processing commands for people to see what passed and
> >>>> failed.
> >>>>
> >>>> The text file also gives me a very easy way to add and remove tests on
> >>>> a per-build basis since it goes along with the build.
> >>>
> >>> I get that, but I don't see why you can't have it both ways.  If you
> want
> >>> a simple set of instructions for someone to manually type at a command
> line
> >>> to run through the tests, there's nothing preventing you from
> including that
> >>> in a readme, script, or however you want to do that.  But those tests
> >>> described in there would be really nice to convert into reusable tests
> in
> >>> lava.  Once you have those building blocks, you can arrange them in
> future
> >>> builds however you like.
> >>>
> >>
> >> I think what Zach is saying here is that he wants to use a single file
> to
> >> maintain the list of tests to run for a build. That file is then sourced
> >> both by a tool used by developers for local testing as well as by LAVA.
> >>
> >> He most likely also would want the local tool used by devs to be a
> >> standard solution provided by the lava test framework...
> >>
> > I know, but what I'm seeing here doesn't look like something that's
> possible
> > right now, and I think might take some pretty serious effort and
> > re-engineering of how lava works around this particular use case to make
> it
> > work.  What I'm trying to show is that there's more than one way to think
> > about this, and that with a slight change it goes from something that we
> > need to discuss at the connect and get some effort behind making happen,
> to
> > something that he can have today.  And at the same time, doing it in this
> > alternative way builds up a reusable set of android tests that can be
> > applied to future builds.
>
> Paul,
>
> Your proposal is to create a linaro-android-test for each command I
> would put in a command file right?
>
> > Thanks,
> > Paul Larson
> >
>
>
>
> --
> Zach Pfeffer
> Android Platform Team Lead, Linaro Platform Teams
> Linaro.org | Open source software for ARM SoCs
> Follow Linaro: http://www.facebook.com/pages/Linaro
> http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linaro.org/pipermail/linaro-android/attachments/20120309/058f6e87/attachment-0001.html>