LAVA integration effort for Android validation
asac at linaro.org
Tue Mar 15 15:32:54 UTC 2011
On Tue, Mar 15, 2011 at 4:09 PM, Paul Larson <paul.larson at linaro.org> wrote:
> On Mon, Mar 14, 2011 at 11:22 AM, Jeremy Chang <jeremy.chang at linaro.org>
>> Hi, list:
>> I am working on Validation part for Android, in which mainly
>> inclusive of integrating existing benchmark and testing suites into
>> abrek, like 0xbench and android CTS. For now in my thought, if these
>> benchmark/testing can integrate with abrek, then it should be no much
>> problem or extra effort for integrating further into LAVA.
> As discussed before, I think abrek is probably not the right choice for
> running tests under android. Abrek was designed to work on linaro images
> running python in the client, specifically it was designed to run under
> Linaro or Ubuntu images, before we had even considered android at all. We
> probably need some kind of mechanism for running the tests from the server
> over adb instead, then outputting the results into a json bundle that the
> dashboard can understand.
we talked about this on IRC. I pointed Jeremy to our lp:lava branch
and explained to him the theory of how things work. We agreed to first
write a few example job descriptions and then decide what features a
generic testsuite framework would require (if we need this at all).
Also we might do a prototype to learn how things would work if we
implemented the actions for android deploy, boot and test run.
>> Though I have some questions regarding to validation.
>> How to detect the early fail through serial console? like
>> detecting the failure during kernel booting stage also before running
>> init or getting the shell. I think this will be a job issued by the
>> dispatcher though I don't know how this checking mechanism will be
>> done. Would there be anything here needed to do specifically for
> Right, the dispatcher currently handles this. We are using pexpect to drive
> things over the serial console. So once the system is booting, we expect to
> see a valid shell prompt within some reasonable timeout period. If we
> don't, then we know the boot failed. I expect the bit that detects the
> shell prompt may need to be changed somewhat for android. Perhaps for
> android we need to wait for a valid adb devices output? or a working adb
yes, its a timeout thing. we should give the adb connect a reasonable
timeout. In this way we can guess that the boot failed in a bad way as
no network came up.
>> How to connect the devices in the farm? I checked the wiki page,
>> Platform/Validation/Specs/HardwareSetup . The network will be used.
>> I am thinking what's needed for setting up a network environment for
>> this? The device is needed to fetch a fixed ip by dhcp just after
>> booting up? USB gadget is an alternative for Android and in most
>> situation could be more convenient for personal testing. adb (Android
>> Debug Bridge, running on host side) can connect to a device through
>> USB or TCP/IP.
> USB gadget is hard for us, if we have a lot of machines. Network would be
> better, but can we rely in it to be working? Maybe we just make that a
> requirement for running android tests. I don't currently have it set up,
> but my current thinking is that we want to define a new network for the
> validation machines, perhaps 10.1.x.x, and set up static assignments for
> them in a dhcp server running on the control node. This way, they could
> either be configured statically, or via dhcp.
yes, USB gadget is not that great for our current setup.
I like statically IP. How do we know which board would get which IP?
One idea would be to set up a DNS for internal board names. e.g.
panda01.internal.network would resolve to our preallocated static IP
for board "panda01".
This could then be used by the dispatcher to resolve the IP and pass
it to the board through kernel cmdline.
More information about the linaro-dev