About how to integrate android test into LAVA - linaro-dev

12 Apr 2012


      Hi, All
LAVA is an automated validation architecture, and it now has a test
framework for running android test tools and parsing the test output.
Please NOTE that lava-android-test is just used for running the test tools,
parsing the test output, and formatting the test result.
Here I will describe how to write and add a test wrapper script for
lava-android-test, and how to integrate the test into LAVA.
1. checkout the lava-android-test to your local machine
    bzr branch lp:lava-android-test
2. if the test tools are just command that can be run on android system,
and the output is well formatted,
    then congratulations, you can go directly to step 6.
    You don't need to wrapper script again.
3. If the test tools has already been build into the android image or in
the host image(normal Ubuntu image),
    then you won't need to define some scripts for organizing the test
tools, you can skip this step,
    Otherwise, put the actual test tools in some place, normally they are
in a sub directory of test_definitions, like the busybox test:
    the actual test tool is busybox_test.sh, and it is put in the *
lava_android_test/test_definitions/busybox *directory
4. add a test wrapper script for your test into the test_definitions
directory. like busybox.py:
    The content of the wrapper script should be something like below:
          Normally, you just need to redefine the red and bold part in the
above.
###############################################################################################
import os
import lava_android_test.testdef
test_name = '*test_sample'*
#linux commands that will be run on the host before INSTALL_STEPS_ADB_PRE"
*INSTALL_STEPS_HOST_PRE = []*
#adb commands that will be run before install apk file into android
*INSTALL_STEPS_ADB_PRE = []*
#APK file path list that will be intalled into android
*APKS= []*
#adb commands that will be run before install apk file into android
*INSTALL_STEPS_ADB_POST = []*
#linux commands that will be run on the host after INSTALL_STEPS_ADB_POST
*INSTALL_STEPS_HOST_POST = []*
#linux commands that will be run on the host before RUN_STEPS_ADB_PRE
*RUN_STEPS_HOST_PRE = []*
#adb commands that will be run before install apk file into android
*RUN_STEPS_ADB_PRE = []*
#commands that will be run on android
*ADB_SHELL_STEPS = []*
#adb commands that will be run before install apk file into android
*RUN_STEPS_ADB_POST = []*
#linux commands that will be run on the host after RUN_STEPS_ADB_POST
*RUN_STEPS_HOST_POST = []*
#pattern to parse the command output to generate the test result.
PATTERN =* **"^\s*(?P<test_case_id>\w+)=(?P<result>\w+)\s*$"*
inst = lava_android_test.testdef.AndroidTestInstaller(steps_host_pre=
INSTALL_STEPS_HOST_PRE,
steps_adb_pre=INSTALL_STEPS_ADB_PRE,
apks=APKS,
steps_adb_post=INSTALL_STEPS_ADB_POST,
steps_host_post=INSTALL_STEPS_HOST_POST)
run = lava_android_test.testdef.AndroidTestRunner(steps_host_pre=
RUN_STEPS_HOST_PRE,
steps_adb_pre=RUN_STEPS_ADB_PRE,
adbshell_steps=ADB_SHELL_STEPS,
steps_adb_post=RUN_STEPS_ADB_POST,
steps_host_post=RUN_STEPS_HOST_POST)
parser = lava_android_test.testdef.AndroidTestParser(PATTERN)
testobj = lava_android_test.testdef.AndroidTest(testname=test_name,
                                    installer=inst,
                                    runner=run,
                                    parser=parser)
###############################################################################################
And in the command part, you can use
"$(SERIAL)" to represent the device serial number, like:
RUN_STEPS_HOST_POST = [ 'python
%s/android-0xbenchmark/android_0xbenchmark_wait.py $(SERIAL)' % curdir]
and
"$(OPTIONS)" to represent the option string passed from command line. Like
INSTALL_STEPS_HOST_PRE = [ 'echo $(OPTION)'],
RUN_STEPS_HOST_PRE = [ 'echo $(OPTION)'],
then you can run *lava-android-test install -o "install options string" *or*
 **lava-android-test run -O "run options string"*
*Note:*
*          B**ecause lava-android-test will be run on lava-lab, and there
will be multiple devices connected simultaneously,*
*          So we should consider to pass the device serial number for each
test tools.*
*          If the test tools is defined
for steps_adb_pre/adbshell_steps/steps_adb_post,*
*          then there is no need to pass the device serial number,**
lava-android-test
will do this for you.*
5. you can
    use "lava-android-test list-tests" to check if the test wrapper created
can be recognized,
    use "lava-android-test install ${test_name}" to install the test,
    use "lava-android-test run ${test_name}" to execute the test,
    use "lava-android-test show ${result_id}" to show the output the test
 executed,
    use "lava-android-test parse ${result_id}" to to generate the result
bundle for the test executed.
And here is a blog about install/test lava-android-test that you can
reference:
http://www.linaro.org/linaro-blog/2011/12/01/local-lava-testing-of-android-i...
6. When you have done the above steps and verified your test that works
well, then you can integrate it in LAVA with the android-build.
    Here is a description about that.
    https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration
If you have any questions, please contact me.
Also I have copied it to google docs,
https://docs.google.com/a/linaro.org/document/d/1kVyuFZtnMsganZaszQZhgX3QPOF...
you can comment directly there.
Thanks,
Yongqin Liu
*---------------------------------
Mail:  yongqin.liu@linaro.org.
IRC Nickname: liuyq*