On Mon, 20 Jun 2011 09:35:00 -0500, Zach Pfeffer zach.pfeffer@linaro.org wrote:
On 20 June 2011 08:29, James Westby james.westby@canonical.com wrote:
On Mon, 20 Jun 2011 12:10:08 +1200, Michael Hudson-Doyle michael.hudson@linaro.org wrote:
Something I don't completely have a plan for in my head at the moment is connecting the dots, or preserving identity between componenets (as I think Zygmunt put it). We have a build, which will trigger a test run, which will result in a test bundle being uploaded to the dashboard. I can see how to link the build to the test run and (mostly -- I'd like to talk about the details of this) see how to link the test run to the bundle, but I don't quite see how to have a link on the build page that goes straight to the results. I guess we can always have a $job_url/results link that does a redirect to the appropriate page in the dashboard...
I think it's vital for LAVA to connect a test run and the results from it, and I'm sure it will, but the details of that are something for you to decide.
As a consumer of LAVA though I think it's also vital that there is some way to link to a particular test run for things like android-build.linaro.org.
Clearly.
One way to do this would be to provide a test id back as the response to the API calls that schedule tests. That test id can then be used to contstruct URLs or make API calls.
Yes, that sounds fine.
If you don't want test ids to be the basis of APIs etc. then we could also make use of something that worked based on an externally defined identifier (e.g. the android-build.linaro.org URL.) If we could make API calls specifying our identifier, and maybe construct URLs too then we can also use that to display results or whatever on android-build.linaro.org.
Lets cut out one level of indirection and use the build URL as the id.
Is this expressing a preference or just trying to be helpful by getting some decisions made? Because in general I would rather those doing the implementation being the ones to make the implementation decisions. And requirements are best expressed using the "as a ... i want to ... so that i can ..." template.
(apologies if this sounds snarky)
Is being able to externally specify the job/test identifier a requirement?
In terms of implementation I think that the latter would be easier given where android-build.linaro.org is now, but I don't think the former would be that much harder for us to do.
As for android-build.linaro.org building other things I don't think that's really an issue for LAVA, as I think that we should push from android-build.linaro.org to LAVA, rather than having LAVA pull. android-build.linaro.org can then decide based on the build type what info to submit, including the type of artifacts, and LAVA can have a simpler job to map from say "android" to the set of tests to run for that.
Yeah, I agree that LAVA shouldn't pull anything. The build system should be requesting the tests at a high level, i.e. please run the "Android Benchmark and Test load on build x."
Cool.
The build should also probably wait until the test is done and get a pass fail, this should help set us up for Gerrit integration in the near future.
I guess the naive implementation of this would leave the EC2 instance running while the test runs which would be wasteful... but that's something for Paul to figure out.
I'll pull a BP together.
Cool. I don't *think* this is covered by an existing blueprint...
Cheers, mwh