W dniu 15.02.2011 21:01, Jim Huang pisze:
Hi Jim, great work!
** Why can't we execute LAVA/Abrek directly on Android devices?
I agree that direct abrek is not the right solution for Android. I think there is a class of use cases that also falls into this category: * testing early silicon * testing very primitive systems * testing other foreign systems
They all have either no python or running python is not desirable.
** So, what's agent-based remote validation invocation for LAVA?
There are three key items: (a) Agent (b) Remote validation invocation (c) LAVA
We keep in mind that we make no technical impact to LAVA architecture, and the "Agent" is just the "helper". Originally, the client-server communication looks like the following:
LAVA server<----> LAVA client --> Abrek test suite
Currently LAVA is just taking shape. Things like server and client are still not very well defined. You don't have to be constrained by this. The only thing that is somewhat well defined is abrek that was produced in the previous cycle. Abrek was designed to run _on_ the device. For use cases where Abrek is just interacting with the test device we may want to extend it sensibly to clearly separate those cases where necessary while retaining as much common code and user interface as possible.
There are a few things to consider here:
Device context: Currently abrek has some logic to probe the running system for context information. Context is loosely defined as the collection of relevant software and hardware information that might be interesting to analyze or that can be used to connect distinct test results in post-process analysis. If we are just interacting with a remote device we need to determine this information in some other way _and_ ensure we're not adding any dummy information from the "host" system.
Remote device registration: A single host may talk to possibly large number of such devices. At the very least we should plan how we intend to manage the process of defining/adding/removing a device and how to allow remote tests to be invoked on registered devices, if possible (if the device supports this test).
Test classes: So far abrek tests could run on any Linux box (where any was poorly defined as "any ubuntu-like system". This is no longer the case so we should be able to classify tests (and devices) somehow.
IMHO we could define a new tool (or just a new sub-command class for abrek), say abrek-remote that will be used instead of the normal abrek call.
Another crazy option would be to expose LAVA Job Dispatcher directly and allow people to run jobs. In this case one job would use abrek and some other tools to invoke tests, process results and send them to the dashboard while other job (one for android) would not run abrek at all, instead it would call some other helper, while still reusing identical components for "process results and send to result storage" phases. This is still in flux but has some advantages: 0) Jobs are simple text files that can be stored and shared with others 1) Jobs can encapsulate device information like which android device to connect to and how. 2) Jobs can still "call" to other parts of the LAVA stack such as result submission 3) Jobs can be extended locally (by LAVA plugins) so that anyone can develop specialized use cases for their very specific needs without altering the stack or having to write something completely custom.
Another upside is that such job definitions (and any required LAVA plugins) would just integrate with the rest of LAVA (farm backend, frontend, etc). You could work on a job description on your workstation and once it's finished you could add it to a job scheduler for automatic processing.
** Show me the use case
The attached patch is just trivial proof-of concept implementation done by Jeremy Chang that adds a 'monkey' test definition file for abrek. Once TCP/IP or USB is ready, for Android's monkey testing, the procedure is like as following: (1) abrek run monkey (2) abrek dashboard put /anonymous/ monkey1297752359.0
Could you please forgive my android ignorance and tell me how I can run this test? Please include any hardware/software I should have. Can I run this on a beagle board with some android installation? Do I need a real android phone? This is just so that I can participate in the discussion and not make clueless and pointless arguments later.
Any suggestion is appreciated. Thank you in advance.
I hope we can work on making this solid.