On 10/08/2012 09:22 PM, Michael Hudson-Doyle wrote:
Michael Hudson-Doyle michael.hudson@linaro.org writes:
If you don't follow dispatcher commit or review mail, you might not be aware than Andy and I have been working an implementation of the "black box" tests we've been talking about for a while. This mail is just a summary of the things that have been mentioned in review that would be good to fix "in the next branch" so that we don't forget about them. The list probably doesn't make a great deal of sense if you haven't read the review mail, sorry about that.
Watching an execution of the dispatcher running a lava-test-shell carefully, I came up with another one:
- Failing to boot when we should be running lava-test-shell looks like it will just timeout looking for "<LAVA_TEST_RUNNER>: exiting". I think we should at least look for "<LAVA_TEST_RUNNER>: starting" and report a failure to see that differently. I'm not sure if we can reliably tell the difference between the device failing to boot and lava-test-runner failing to run because we goofed on installing the upstart job. Would be nice if we could.
I don't know if we can differentiate between boot failure and test-runner failure. But we could check for the "starting" message and then be able to log a more accurate error upon a timeout.
+1