On Mon, Mar 14, 2011 at 11:22 AM, Jeremy Chang jeremy.chang@linaro.orgwrote:
Hi, list: I am working on Validation part for Android, in which mainly inclusive of integrating existing benchmark and testing suites into abrek, like 0xbench and android CTS. For now in my thought, if these benchmark/testing can integrate with abrek, then it should be no much problem or extra effort for integrating further into LAVA.
As discussed before, I think abrek is probably not the right choice for running tests under android. Abrek was designed to work on linaro images running python in the client, specifically it was designed to run under Linaro or Ubuntu images, before we had even considered android at all. We probably need some kind of mechanism for running the tests from the server over adb instead, then outputting the results into a json bundle that the dashboard can understand.
Though I have some questions regarding to validation.
How to detect the early fail through serial console? like detecting the failure during kernel booting stage also before running init or getting the shell. I think this will be a job issued by the dispatcher though I don't know how this checking mechanism will be done. Would there be anything here needed to do specifically for Android?
Right, the dispatcher currently handles this. We are using pexpect to drive things over the serial console. So once the system is booting, we expect to see a valid shell prompt within some reasonable timeout period. If we don't, then we know the boot failed. I expect the bit that detects the shell prompt may need to be changed somewhat for android. Perhaps for android we need to wait for a valid adb devices output? or a working adb shell?
How to connect the devices in the farm? I checked the wiki page, Platform/Validation/Specs/HardwareSetup [1]. The network will be used. I am thinking what's needed for setting up a network environment for this? The device is needed to fetch a fixed ip by dhcp just after booting up? USB gadget is an alternative for Android and in most situation could be more convenient for personal testing. adb (Android Debug Bridge, running on host side) can connect to a device through USB or TCP/IP.
USB gadget is hard for us, if we have a lot of machines. Network would be better, but can we rely in it to be working? Maybe we just make that a requirement for running android tests. I don't currently have it set up, but my current thinking is that we want to define a new network for the validation machines, perhaps 10.1.x.x, and set up static assignments for them in a dhcp server running on the control node. This way, they could either be configured statically, or via dhcp.
Thanks, Paul Larson