Hi Jeff, a bit of background on lava-qatracker. A while back, we were using another qatracker and having some issues. It allowed us to set up milestones for testing, but just linked to a wiki page with steps to test, which a tester would have to follow, then come back to the page and just enter any bug numbers that were found. There was no more detail than that, and just telling someone to follow a wiki made it too easy to miss steps completely.

Just with some quick brainstorming, we thought it would be really useful to have a system that would allow us to define the tests, take the user through the tests and let them answer right on the page whether it passed, failed, or was skipped. It wasn't meant to be a finished solution, just a starting point for the conversation about what features should be included in the first pass. I think already, it would be a huge improvement, but there were some important things missing. Namely, we needed a good way to link manual tests to automated tests on the same image, and visualize them together for the milestone testing. This proved difficult because on Ubuntu, we have separate build artifacts that are combined to make the image. However, over the next month or so, we're looking at moving to a different system for producing the builds, and creating an image at build time that could be submitted. This should simplify the definition of "what is an image" quite a bit.

There are still a number of issues that need to be worked out. For instance, linking and visualization of the results, how to define the tests (as a bzr branch with files, or in the db directly are two options that have been brought up), granularity of products, milestones, etc. I'd love to get input from other interested parties such as collabora so that it's not only useful to us, but flexible enough to be useful to others as well.

In the interest of making it usable enough to demo what's there, and be useful as a conversation starter again, I did a bit of testing on it today and fixed a couple of bugs that might have been preventing you from seeing how to use it. It looks like you've gotten far enough to install it and make it run, so I'll skip over the boring details of that. Update your branch and reinstall though, and I'll start from there:
1. Login as the admin user and open the admin interface
2. Click on the +Add button next to "milestones" on that page
3. Give it a name, bundle stream, etc. You'll also want to give it a branch to use. For the branch just use "https://code.launchpad.net/~mwhudson/+junk/test-cases" which has some example tests in it
4. Go back out to the main lava interface and click on "QA Tracker at the top, and select the milestone you just created", under there you'll see how it asks you for test results, etc.

Let me know if you have any issues with getting to this point. I'd like to talk further about your interest, and ideas for what kind of features you're looking to have in a tool like this.

Thanks,
Paul Larson

On Tue, Mar 20, 2012 at 5:51 AM, Alexander Sack <asac@linaro.org> wrote:

Hi,

On Tue, Mar 20, 2012 at 12:12:30AM -0400, Jean-François Fortin Tam wrote:
> Hey all,
> Lately I've been studying the LAVA QA Tracker by looking at its code,
> setting it up on a LAVA dashboard instance in a VM and trying to use it,
> and asking some questions on IRC.
>
> I recall asking at the beginning of the month if there were plans on the
> Linaro side to allocate developers to finish the QA Tracker in a
> specific timeframe, and was told that some development for this
> [monthly] cycle should start "next week".
>
> However, except my own bazaar branch (crappy; it does not really
> solve/fix anything so far), I haven't heard of recent development
> activity. Or did anything happen that I missed?
>
> Also, some people I spoke with were not aware at all of those plans for
> this cycle. So I'd just like to check out a few things:
> - Do you have a target deadline to meet for this?
> - Who are the people who will specifically be working on this?

LAVA QA tracker has been on/off the development agenda with recent
activities indicating that we probably want a solution around those
lines.

ATM, we are gathering requirements and decide what to do strategically
on this topic. I suspect that the final input will come from the
dedicated QA team that we are currently in the progress of
bootstrapping.

I expect those to be the owners of requirements of such things for
LAVA.

What is your interest in the QA tracker? What use-cases would you want
to cover? If you want to work on this, I would suggest that we setup
joint meeting with the right stakeholders to nail down what we want to
do and how you can best help...

--
Alexander Sack <asac@linaro.org>
Technical Director, Linaro Platform Teams
http://www.linaro.org | Open source software for ARM SoCs
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog

_______________________________________________
linaro-validation mailing list
linaro-validation@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-validation