In response to the connect session:
http://pad.linaro.org/lava-test-next-gen
I've put together a phased plan with Michael that will hopefully address the major points raised. Please let us know if you have comments/questions/concerns/etc.
The first phase of the approach will simply be getting this new "lava-test-shell" action working for FastModel on Android with the bigLITTLE test suite. As you'll see below, the idea's a little more baked for the stand-alone usage, but it should give us enough flexibility for host/test-buddy interactions in the future.
I'm also inlining some Michael question/answer notes that should help this make more sense.
LAVA Test Shell ===============
lava-test-shell is a lightweight application that can execute test suites in a way that works well with the LAVA architecture, specifically the lava-dispatcher.
//MWHUDSON: I presume lava-test-shell would already be installed in the images we test? //DOANAC: ultimately, however for now, I think including this code in the dispatcher and adding it to the FM image will be the best way to sort things out initially,
There are two modes this can be used under, stand-alone and host->target. Under most cases, the shell can run as a simple standalone application on the target device under test. However, for some tests may require host-side side interactions which can be run with a host->target approach.
Stand-alone Usage -----------------
Basic usage to execute a test suite:
lava-test-shell --output_dir <output directory> [test args to run]
Basic usage to send signal to host:
lava-test-shell --signal <SIGNAL_NAME>
Basic usage to create hardware and software context info files needed by LAVA for bundle results:
lava-test-shell --output_dir <output directory> --hwcontext --swcontext
Host->Target Usage ------------------
The host->target usage is done by launching a server on the target:
lava-test-shell --server
The server code on the target will then await the host's commands. The usage will be the same as the stand-alone usage but will use:
"--target <ip>:<port>"
bigLITTLE Example -----------------
The dispatcher JSON action will look like:
{ "command": "lava_test_shell", "parameters": { "cmd": "/usr/bin/run_stress_switcher_tests.sh -a", "pattern": "(?P<test_case_id>.*-*)\s+:\s+(?P<result>(PASS|FAIL))", "etc_part_no": 2, "timeout": OPTIONAL, "num_reboots": OPTIONAL(do we allow this to reboot and how many) } }
The customization logic in our deploy (android/ubuntu) actions will ensure we have the proper system modification to launch lava-test-shell with the proper arguments after the system has booted.
//MWHUDSON: So, to be clear, as far as a job submitter is concerned, lava-test-shell is an implementation detail? //DOANAC: Yes - just wanted to outline the mechanics of how this works.
Ubuntu might be an upstart job that reads the etc file created by the lava_test_shell action above, while Android could do something similar as well.
//MWHUDSON: How will we handle boot failure? //DOANAC: I think we'll have to make it a failed result in the "lava_results" bundle similar to how we currently do for a lava-test-run action.
After setting the etc file, the dispatcher will boot the device and wait for: * the lava-test-shell binary to send a signal over serial saying the command has completed. * the timeout has occurred * num_reboots has occurred
-andy