2011/2/16 Zygmunt Krynicki zygmunt.krynicki@linaro.org:
W dniu 15.02.2011 21:01, Jim Huang pisze:
Hi Jim, great work!
hi Zygmunt,
Thanks. It is my pleasure to work with Linaro validation team.
** Why can't we execute LAVA/Abrek directly on Android devices?
I agree that direct abrek is not the right solution for Android. I think there is a class of use cases that also falls into this category: * testing early silicon * testing very primitive systems * testing other foreign systems They all have either no python or running python is not desirable.
Exactly. Thanks for your explanation.
[...]
Currently LAVA is just taking shape. Things like server and client are still not very well defined. You don't have to be constrained by this. The only thing that is somewhat well defined is abrek that was produced in the previous cycle. Abrek was designed to run _on_ the device. For use cases where Abrek is just interacting with the test device we may want to extend it sensibly to clearly separate those cases where necessary while retaining as much common code and user interface as possible.
There are a few things to consider here:
Device context: Currently abrek has some logic to probe the running system for context information. Context is loosely defined as the collection of relevant software and hardware information that might be interesting to analyze or that can be used to connect distinct test results in post-process analysis. If we are just interacting with a remote device we need to determine this information in some other way _and_ ensure we're not adding any dummy information from the "host" system.
Remote device registration: A single host may talk to possibly large number of such devices. At the very least we should plan how we intend to manage the process of defining/adding/removing a device and how to allow remote tests to be invoked on registered devices, if possible (if the device supports this test).
Test classes: So far abrek tests could run on any Linux box (where any was poorly defined as "any ubuntu-like system". This is no longer the case so we should be able to classify tests (and devices) somehow.
It sounds reasonable. Do you think whether the above changes are progressive to current LAVA implementation or not?
In Android, we can even re-use the infrastructure of ADB and DDMS [1] to provide remote device registration service.
IMHO we could define a new tool (or just a new sub-command class for abrek), say abrek-remote that will be used instead of the normal abrek call.
Yes, I like the idea.
Another crazy option would be to expose LAVA Job Dispatcher directly and allow people to run jobs. In this case one job would use abrek and some other tools to invoke tests, process results and send them to the dashboard while other job (one for android) would not run abrek at all, instead it would call some other helper, while still reusing identical components for "process results and send to result storage" phases. This is still in flux
[...][
Another upside is that such job definitions (and any required LAVA plugins) would just integrate with the rest of LAVA (farm backend, frontend, etc). You could work on a job description on your workstation and once it's finished you could add it to a job scheduler for automatic processing.
Agree.
** Show me the use case The attached patch is just trivial proof-of concept implementation done by Jeremy Chang that adds a 'monkey' test definition file for abrek. Once TCP/IP or USB is ready, for Android's monkey testing, the procedure is like as following: (1) abrek run monkey (2) abrek dashboard put /anonymous/ monkey1297752359.0
Could you please forgive my android ignorance and tell me how I can run this test? Please include any hardware/software I should have. Can I run this on a beagle board with some android installation? Do I need a real android phone? This is just so that I can participate in the discussion and not make clueless and pointless arguments later.
Jeremy, could you prepare the testing environment using Android emulator? We could write down the instructions in wiki for reference.
Any suggestion is appreciated. Thank you in advance.
I hope we can work on making this solid.
Sure!
Cheers, Jim Huang (jserv) http://0xlab.org/
[1] DDMS: http://developer.android.com/guide/developing/tools/ddms.html