Hello list,
Regarding integration of LAVA [1] architecture with Android devices, we would like to reuse the existing infrastructure and framework design. Abrek [2] is a great testsuite framework for running test cases and benchmarks. However, due to the restrictions of unusual Android runtime, we consider to introduce the agent-based remote validation invocation mechanism for LAVA as the extension. Also, the proof-of-concept implementation is attached.
** Why can't we execute LAVA/Abrek directly on Android devices?
LAVA/Abrek is written in Python, which implies there must be a solid Python runtime for Android. CPython is verified and well-designed, but it is not well tested on Android. In fact, Android has its own libc implementation, bionic, which is the minimal and special libc originally taken from NetBSD libc. However, bionic libc only supports limited set of POSIX C APIs, and it is almost not feasible to maintain Linaro bionic modifications in early stage just to satisfy CPython. The bionic libc is always changed fast by Google engineers, and we have no idea about their plans.
Therefore, we prefer the way not to modify Android runtime. That is, don't execute LAVA/Abrek directly on Android environment.
** Yet another agent?
In fact, Android already provide an elegant approach for accessing target environment, adb (Android Debug Bridge)[3], which is a versatile tool allowing users to manage the state of Android-powered device. adb itself is a client-server program that includes three components: (1) A client, which runs on your development machine. You can invoke a client from a shell by issuing an adb command. (2) A server, which runs as a background process on your development machine. The server manages communication between the client and the adb daemon running on the device. (3) A daemon, which runs as a background process on each device instance.
The adb protocol can be established through USB (device needs to enable Android USB gadget driver) or TCP/IP. adb is very solid and powerful. For example, if you would like to test Android UI, you can use the command on Host side: $ adb shell monkey -v -p your.package.name 500
The above could be illustrated:
Host commands --> adb protocol (USB or TCP/IP) --> Target receives the command --> execute "monkey"
Monkey is a program on Android device that runs on your emulator or device and generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events.
In this example, your Android application is launched, and 500 pseudo-random events are sent to it. We call it as "agent-based remote invocation", and it is built-in.
** So, what's agent-based remote validation invocation for LAVA?
There are three key items: (a) Agent (b) Remote validation invocation (c) LAVA
We keep in mind that we make no technical impact to LAVA architecture, and the "Agent" is just the "helper". Originally, the client-server communication looks like the following:
LAVA server <----> LAVA client --> Abrek test suite
Since Python runtime is hard to support, the proposed model would be:
LAVA server <--> LAVA client --> Abrek test suite || adb extension (host) <--> adb (target) -> execute command
For integrating test items and benchmarks for Android, this proposal is running abrek on the host side. By using Android's standard tool, adb to communicate with the adbd (adb daemon) on target, the test case commands can be issued and the output can return back. Besides, the files can be pushed and pull to and from the target by adb as well. Sometimes the results may be stored in certain file on target, and we definitely could couple with the case. Thus, running abrek on host side along with "Agent" should be a workable approach.
** Show me the use case
The attached patch is just trivial proof-of concept implementation done by Jeremy Chang that adds a 'monkey' test definition file for abrek. Once TCP/IP or USB is ready, for Android's monkey testing, the procedure is like as following: (1) abrek run monkey (2) abrek dashboard put /anonymous/ monkey1297752359.0
In addition, if we wish to execute certain native application on Android device, abrek could ask adb to send the executable files and related data to target first. Then, execute it as expected. The command looks like: $ adb push <my-executable-file> /system/bin $ adb push <my-data-file> /data $ adb shell /system/bin/<my-executable-file>
The above instructions could be refined into "adb extension" as the part of LAVA client framework.
That's all. It could be straightforward and transparent.
Any suggestion is appreciated. Thank you in advance.
Sincerely, Jim Huang (jserv) http://0xlab.org/
[1] https://wiki.linaro.org/Platform/Validation/LAVA/Architecture [2] https://wiki.linaro.org/Platform/Validation/AbrekTestsuites [3] http://developer.android.com/guide/developing/tools/adb.html