Andy Doan andy.doan@linaro.org writes:
On 10/17/2012 02:55 PM, Michael Hudson-Doyle wrote:
Andy Doan andy.doan@linaro.org writes:
On 10/16/2012 07:47 PM, Michael Hudson-Doyle wrote:
We're going to be talking about test case management in LAVA at the Connect. I've brain-dumped some of my thoughts here:
https://wiki.linaro.org/Platform/LAVA/Specs/TestCaseManagementComments welcome. But if all you do is read it before coming to the session, that's enough for me :-)
It feels good just to see this all listed concisely.
I think we probably need one other section in the page like "testdef organization". This would describe any rules we have file filenames or directory hierarchies needed by the git/bzr testdef repo so that we'll know how to import things.
Yeah. I guess there is also an issue around namespacing if we allow pulling from multiple branches -- if you pull from two places and both define the test called "stream", what happens?
good point.
Also, suppose you use the web UI to update a test and then an update from a branch pulls in an update to the same test? (This is starting to feel a bit like the issues a dvcs-backed wiki like ikiwiki faces...)
Another thought -- do we want to have private tests?
I think we have to assume yes considering these test defs will/should contain descriptions of tests. However, a private repo doesn't feel like much of a difference other than the URL given to us to pull from. I'd think you'd just want to use a white-listing approach like we currently do for private builds.
I was thinking more like a test where only privileged people can submit a job that runs the test and where the results of the test can only be submitted to a stream that has the same visibility as the test, or things like that.
Cheers, mwh