[Linaro-validation] LAVA QA Tracker dev status
paul.larson at linaro.org
Wed Mar 21 03:26:15 UTC 2012
Hi Jeff, a bit of background on lava-qatracker. A while back, we were
using another qatracker and having some issues. It allowed us to set up
milestones for testing, but just linked to a wiki page with steps to test,
which a tester would have to follow, then come back to the page and just
enter any bug numbers that were found. There was no more detail than that,
and just telling someone to follow a wiki made it too easy to miss steps
Just with some quick brainstorming, we thought it would be really useful to
have a system that would allow us to define the tests, take the user
through the tests and let them answer right on the page whether it passed,
failed, or was skipped. It wasn't meant to be a finished solution, just a
starting point for the conversation about what features should be included
in the first pass. I think already, it would be a huge improvement, but
there were some important things missing. Namely, we needed a good way to
link manual tests to automated tests on the same image, and visualize them
together for the milestone testing. This proved difficult because on
Ubuntu, we have separate build artifacts that are combined to make the
image. However, over the next month or so, we're looking at moving to a
different system for producing the builds, and creating an image at build
time that could be submitted. This should simplify the definition of "what
is an image" quite a bit.
There are still a number of issues that need to be worked out. For
instance, linking and visualization of the results, how to define the tests
(as a bzr branch with files, or in the db directly are two options that
have been brought up), granularity of products, milestones, etc. I'd love
to get input from other interested parties such as collabora so that it's
not only useful to us, but flexible enough to be useful to others as well.
In the interest of making it usable enough to demo what's there, and be
useful as a conversation starter again, I did a bit of testing on it today
and fixed a couple of bugs that might have been preventing you from seeing
how to use it. It looks like you've gotten far enough to install it and
make it run, so I'll skip over the boring details of that. Update your
branch and reinstall though, and I'll start from there:
1. Login as the admin user and open the admin interface
2. Click on the +Add button next to "milestones" on that page
3. Give it a name, bundle stream, etc. You'll also want to give it a
branch to use. For the branch just use "
https://code.launchpad.net/~mwhudson/+junk/test-cases" which has some
example tests in it
4. Go back out to the main lava interface and click on "QA Tracker at the
top, and select the milestone you just created", under there you'll see how
it asks you for test results, etc.
Let me know if you have any issues with getting to this point. I'd like to
talk further about your interest, and ideas for what kind of features
you're looking to have in a tool like this.
On Tue, Mar 20, 2012 at 5:51 AM, Alexander Sack <asac at linaro.org> wrote:
> On Tue, Mar 20, 2012 at 12:12:30AM -0400, Jean-François Fortin Tam wrote:
> > Hey all,
> > Lately I've been studying the LAVA QA Tracker by looking at its code,
> > setting it up on a LAVA dashboard instance in a VM and trying to use it,
> > and asking some questions on IRC.
> > I recall asking at the beginning of the month if there were plans on the
> > Linaro side to allocate developers to finish the QA Tracker in a
> > specific timeframe, and was told that some development for this
> > [monthly] cycle should start "next week".
> > However, except my own bazaar branch (crappy; it does not really
> > solve/fix anything so far), I haven't heard of recent development
> > activity. Or did anything happen that I missed?
> > Also, some people I spoke with were not aware at all of those plans for
> > this cycle. So I'd just like to check out a few things:
> > - Do you have a target deadline to meet for this?
> > - Who are the people who will specifically be working on this?
> LAVA QA tracker has been on/off the development agenda with recent
> activities indicating that we probably want a solution around those
> ATM, we are gathering requirements and decide what to do strategically
> on this topic. I suspect that the final input will come from the
> dedicated QA team that we are currently in the progress of
> I expect those to be the owners of requirements of such things for
> What is your interest in the QA tracker? What use-cases would you want
> to cover? If you want to work on this, I would suggest that we setup
> joint meeting with the right stakeholders to nail down what we want to
> do and how you can best help...
> Alexander Sack <asac at linaro.org>
> Technical Director, Linaro Platform Teams
> http://www.linaro.org | Open source software for ARM SoCs
> http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog
> linaro-validation mailing list
> linaro-validation at lists.linaro.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the linaro-validation