Hi
Marcus here has worked on CTS for 4.4 quite a bit so here is his reply.
We had to do some manual patching on 4.2 so that both the non-interactive mode and the disable-reboot options didn’t break the execution but it seems like those
problems are gone now. Just in case you have any such issues we can dig through our old stuff...
From: Marcus Oakland [mailto:marcus.oakland@arm.com]
Sent: 29 May 2014 12:13
To: Ramin Zaghi
Subject: Re: Any comments about CTS !?
Invoke CTS tests with the
--disable-reboot
option to stop cts-tradefed rebooting.
For example:
$ echo | cts-tradefed run cts --class libcore.java.lang.reflect.ProxyTest --method testMethodsImplementedByFarIndirectInterface --disable-reboot
That runs a single CTS test method (libcore.java.lang.reflect.ProxyTest#testMethodsImplementedByFarIndirectInterface in this case) in "Non-interactive mode" (so that cts-tradefed exits when the test has completed (caused by the use of "echo | " at the start
of the invocation line). Use of "Non-interactive mode" means that scripts can be used to invoke the tests.
You can run all methods in a CTS test by specifying the class (e.g. --class libcore.java.lang.reflect.ProxyTest) but omitting the method.
You can run all CTS test classes in a package by, for example:
$ echo | cts-tradefed run cts --package android.core.tests.libcore.package.libcore --disable-reboot
You can run all CTS test packages in a plan by, for example:
$ echo | cts-tradefed run cts --plan Java --disable-reboot
On the model running the CTS tests is sloooow. Figures I have from testing for 4.3 on the android64_2013Q4 branch, the Java plan for example took 9:21:34, the VM-TF plan took 16:00:10, and the Android plan took 26:32:31. However the model used to "die" fairly
frequently and tests had to be restarted.
There are more CTS tests in the complete suite for 4.4 than there were in 4.3 (43,021 compared with 24,491).
From: linaro-android-bounces@lists.linaro.org [mailto:linaro-android-bounces@lists.linaro.org]
On Behalf Of YongQin Liu
Sent: 29 May 2014 02:11
To: John Stultz
Cc: linaro-android@lists.linaro.org
Subject: Re: CTS help?
Hi, John
On 29 May 2014 08:10, John Stultz <john.stultz@linaro.org> wrote:
On 05/28/2014 05:04 PM, YongQin Liu wrote:
> Hi, John
>
> Since you are running in an emulated environment, you probably need
> some changing about timeout.
>
> Here are the 2 packages we are using for juice with some timeout changes:
>
> For 32 bit android userspace:
>
http://testdata.validation.linaro.org/cts/android-cts-32bit-master.zip
>
> For 64 bit android userspace:
>
http://testdata.validation.linaro.org/cts/android-cts-aarch64-master.zip
>
>
> After you got the cts-tf prompt via cts-tradefed, please run the "run
> cts" subcommands with "--disable-reboot" option specified.
> you can also have "--skip-device-info" specified since you probably
> already know the device information.
>
Thanks for the tips! Also any rough idea how long a basic CTS run should
take?
On emulator like fastmodels, it will take very longer time than with hardware.
Here are some information about the time it will run I recorded before.
Except the android.core.tests.libcore.package.org package, others seems still can be referenced.
The
android.core.tests.libcore.package.org will take 1 or 2 days now, so we split it into 7 smaller sub packages in the
above 2 mentioned cts packages.
I'm hoping to find something that stresses the hardware/kernel
side of things and finishes in a few short hours or less.
you can run "list packages" to see all the packages supported.
For hardware, you can select the packages from the name I think.(but for emulator...)
For kernel, IMO, android.core.tests.libcore.package.libcore and
android.core.tests.libcore.package.org are better to be used for kernel test,
since they can cover most basic tests(but this is just from my own perspective), but they will take long time:(
--
Best Regards,
Yongqin Liu
---------------------------------------------------------------
#mailing list