On 28 January 2011 08:12, Paul Larson paul.larson@linaro.org wrote:
Other questions...
What does it do after running the test? How, and where does it leave the results? What get's added to the results (e.g. serial log) and how does that happen? What is the mechanism for then taking that bundle and pushing to the server? What happens if there were no results from the client, what do we capture in that case?
About the test result, I draft a test result bundle template, for a job test result bundle, it is stored in a text file.
Dispatcher server part will get the return value of the test case and the serial port log, then create test result together with some info from job message, client dispatcher test logs.
I think server dispatcher can fetch the client test logs initiatively after client dispatcher ends.
If there is no result from the client, after timeout, 1. server dispatcher send some "Ctrl+C" to client dispatcher to end it. 2. then restart the client dispatcher. 3. try to get the client dispatcher remained test logs. 4. mark the test case to TIMEOUT and create the test result. PS. If step 1 can not end the client dispatcher, it may need a reboot for server dispatcher have lost the control of serial line.
Test result bundle composed of every test case result, for each test case result, it includes:
-
LOG casename:Test case name testsuite:Test suite name testcmd:Test case Command timeout:Timeout retvalue:Return value version:kernel version(by "cat /proc/version") seriallog:
serial port log ENDLOG
e.g.
-
LOG casename:PERF001 testsuite:abrek testcmd:x11perf timeout:60000 retvalue:0 version:2.6.35-xxxx seriallog: xxx xxx xxx ENDLOG
And a test result bundle:
LOG casename:PERF001 testsuite:abrek testcmd:x11perf timeout:60000 retvalue:0 version:2.6.35-xxxx seriallog: xxx xxx xxx ENDLOG
LOG casename:USB002 testsuite:abrek testcmd:usb_app
timeout:60000 retvalue:0 version:2.6.35-xxxx seriallog: yyy yyy yyy ENDLOG
What do you think?
The flow to get test result: 1. Server dispatcher sends commands to client dispatcher via serial line to invoke client dispatcher, capture all the serial log and wait for the client dispatcher ends. 2. Client dispatcher executes the command, save logs to some place on the board. 3. If client dispatcher ends normally(return to the "root@testimage:~$" string), server dispatcher uses "echo $?" to get the return value. Or terminate the client dispatcher after timeout. 4. Server dispatcher gets client dispatcher test logs. 5. Server dispatcher uses the collected information to create test result.
On Tue, Feb 15, 2011 at 5:39 AM, Spring Zhang spring.zhang@linaro.orgwrote:
On 28 January 2011 08:12, Paul Larson paul.larson@linaro.org wrote:
Other questions...
What does it do after running the test? How, and where does it leave the results? What get's added to the results (e.g. serial log) and how does that happen? What is the mechanism for then taking that bundle and pushing to the server? What happens if there were no results from the client, what do we capture in that case?
About the test result, I draft a test result bundle template, for a job test result bundle, it is stored in a text file.
Dispatcher server part will get the return value of the test case and the serial port log, then create test result together with some info from job message, client dispatcher test logs.
I think server dispatcher can fetch the client test logs initiatively after client dispatcher ends.
If there is no result from the client, after timeout,
- server dispatcher send some "Ctrl+C" to client dispatcher to end it.
- then restart the client dispatcher.
- try to get the client dispatcher remained test logs.
- mark the test case to TIMEOUT and create the test result.
PS. If step 1 can not end the client dispatcher, it may need a reboot for server dispatcher have lost the control of serial line.
Test result bundle composed of every test case result, for each test case result, it includes:
He Spring, those questions were mainly to get you thinking about what
needed to happen. And I think you're seeing most of what I was trying to illustrate. If we use Abrek to run the test, we will probably want to add a flag to the run command to tell it to automatically generate the bundle after finishing, rather than have that as a separate step. We can point it to a location where we expect to always be able to retrieve the results later (for instance, /lava/results or something). Reboot the system back into the master image after the test regardless of whether the test passed, failed, completed, or not. If the tests are already in the abrek bundled format, we can append the necessary log information and send it on its way. If we use something other than abrek, then we will need to run the raw results through a filter specific to that test, generate a json bundle, and send it off.
The good news is that you don't need to worry about a results format. We already have that, and sorry I didn't specifically point that out sooner. If you pull lp:launch-control and look under the example_bundles directory, you can see some examples of what this looks like. This is the format the dashboard already understands.
Thanks, Paul Larson
-
Paul Larson -
Spring Zhang