Hi Erin,
On Mon, Apr 16, 2018 at 06:50:12AM +0000, Erin Yang (楊雅婷) wrote:
Hi EAS-DEV,
We are fine tuning EAS and DVFS for our product, and use WA to do auto test. In the presentation, http://connect.linaro.org.s3.amazonaws.com/hkg18/presentations/hkg18-210.pdf, it lists Test cases at Page 9 & 10. Except Vellamo, can all the following test cases be verified via WA?
AFAIK, almost all testing cases can be verified with WA.
Except WA, I suggest you could take a look for LISA's wltest feature, which integrate LISA+WA, and it's friendly for results comparison within multiple runs [1].
In addition, there are some questions to consult.
Temple run and Beach buggy blitz are games, how to evaluate the performance?
AFAIK, We can use WA FPS as performance instrument for gaming [2].
Furthermore, sometimes we also can do statistics based on Janks for performance evaluation [3].
Is there any suitable benchmark for UIbench & Homescreen?
For UI benchmark, there have two programs:
- UIBench: which you could build by self in AOSP repo; - Jankbench: This is developed by Google team but I don't know if this APK has been released or not.
For Homescreen, we usually only measure the power data and for more detailed analysis, we can check the task placement on big/LITTLE CPUs.
Currently, audio and video are real-time playback, how to check the improvement?
Audio/Video are power sustainable scenarios, the system need to meet these two scenarios perofrmance requirement, e.g. audio should check overrun/underrun issue, and video can check FPS (be honest, I don't measure these metrics), we need to optimize the power data as possible, so WA can support energy instruments, in our case we are using ARM energy probe [4].
Thanks, Leo Yan
[1] https://github.com/ARM-software/lisa/blob/master/tools/wltests/README.md [2] https://pythonhosted.org/wlauto/extensions/instruments.html#fps [3] https://github.com/ARM-software/lisa/blob/master/tools/wa_user_directory/plu... [4] https://pythonhosted.org/wlauto/extensions/instruments.html#energy-probe