On Tue, Mar 1, 2011 at 4:32 AM, Jesse Barker jesse.barker@linaro.org wrote:
FWIW, skia certainly isn't android only and, at least for the purposes of getting the validation side of things up and running, could be run on a non-android build (Jammy is likely doing something like this for his work, though not oriented at abrek at the moment). Of course, I could be over-simplifying here ;-).
I'd like to use Skia as a toolchain benchmark but the upstream seems a bit messy. I'm using this export: http://people.linaro.org/~michaelh/skia-0~svn788.tar.xz
from http://code.google.com/p/skia/ which I then build and run using these rules: http://bazaar.launchpad.net/~linaro-toolchain-dev/cbuild/trunk/view/head:/li...
The code.google version seems a bit broken. autoconf scripts are included but they don't work. The non-autoconf Makefile assumes Intel with SSE2. The Android Skia seems to be a fork from some time ago and uses build scripts that are tightly tied into the Android build system.
What's the best version to use? I'd like to standardise across Linaro for benchmarking so that results can be compared across groups, or at least so we don't get confused.
-- Michael
On 1 March 2011 10:45, Michael Hope michael.hope@linaro.org wrote: [...]
I'd like to use Skia as a toolchain benchmark but the upstream seems a bit messy. I'm using this export: http://people.linaro.org/~michaelh/skia-0~svn788.tar.xz
hi Michael,
Thanks for your interest!
from http://code.google.com/p/skia/ which I then build and run using these rules: http://bazaar.launchpad.net/~linaro-toolchain-dev/cbuild/trunk/view/head:/li...
The code.google version seems a bit broken. autoconf scripts are included but they don't work. The non-autoconf Makefile assumes Intel with SSE2. The Android Skia seems to be a fork from some time ago and uses build scripts that are tightly tied into the Android build system.
Yes, don't use autotool inside skia package.
What's the best version to use? I'd like to standardise across Linaro for benchmarking so that results can be compared across groups, or at least so we don't get confused.
The skia maintainer, Mike Reed, made two branches: one is hosted in Google Code, and another is inside Android source tree: http://android.git.kernel.org/?p=platform/external/skia.git%3Ba=summary
I would suggest to use the later.
Also, here is my previous testing results: http://groups.google.com/group/0xlab-devel/browse_thread/thread/f76d8af5b01b...
Sincerely, -jserv
The skia code is also hold in the chromium project. I used chromium web browser to check the skia performance with some browser benchmarks.
I think pixman is also a good candidate for skia to benchmark toolchain.
Regards, Jammy
On Tue, Mar 1, 2011 at 10:53 AM, Jim Huang jim.huang@linaro.org wrote:
On 1 March 2011 10:45, Michael Hope michael.hope@linaro.org wrote: [...]
I'd like to use Skia as a toolchain benchmark but the upstream seems a bit messy. I'm using this export: http://people.linaro.org/~michaelh/skia-0~svn788.tar.xz
hi Michael,
Thanks for your interest!
from http://code.google.com/p/skia/ which I then build and run using these rules:
http://bazaar.launchpad.net/~linaro-toolchain-dev/cbuild/trunk/view/head:/li...
The code.google version seems a bit broken. autoconf scripts are included but they don't work. The non-autoconf Makefile assumes Intel with SSE2. The Android Skia seems to be a fork from some time ago and uses build scripts that are tightly tied into the Android build system.
Yes, don't use autotool inside skia package.
What's the best version to use? I'd like to standardise across Linaro for benchmarking so that results can be compared across groups, or at least so we don't get confused.
The skia maintainer, Mike Reed, made two branches: one is hosted in Google Code, and another is inside Android source tree: http://android.git.kernel.org/?p=platform/external/skia.git%3Ba=summary
I would suggest to use the later.
Also, here is my previous testing results:
http://groups.google.com/group/0xlab-devel/browse_thread/thread/f76d8af5b01b...
Sincerely, -jserv
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On Tue, Mar 1, 2011 at 3:53 PM, Jim Huang jim.huang@linaro.org wrote:
On 1 March 2011 10:45, Michael Hope michael.hope@linaro.org wrote: [...]
I'd like to use Skia as a toolchain benchmark but the upstream seems a bit messy. I'm using this export: http://people.linaro.org/~michaelh/skia-0~svn788.tar.xz
hi Michael,
Thanks for your interest!
from http://code.google.com/p/skia/ which I then build and run using these rules: http://bazaar.launchpad.net/~linaro-toolchain-dev/cbuild/trunk/view/head:/li...
The code.google version seems a bit broken. autoconf scripts are included but they don't work. The non-autoconf Makefile assumes Intel with SSE2. The Android Skia seems to be a fork from some time ago and uses build scripts that are tightly tied into the Android build system.
Yes, don't use autotool inside skia package.
What's the best version to use? I'd like to standardise across Linaro for benchmarking so that results can be compared across groups, or at least so we don't get confused.
The skia maintainer, Mike Reed, made two branches: one is hosted in Google Code, and another is inside Android source tree: http://android.git.kernel.org/?p=platform/external/skia.git%3Ba=summary
I would suggest to use the later.
Thanks Jim. The Android tree doesn't have any build scripts and I'm reluctant to stray too far away from upstream by adding my own. I'll stick with the code.google one for now.
I've updated: http://bazaar.launchpad.net/~linaro-toolchain-dev/cbuild/trunk/view/head:/li...
with interations so that each sub benchmark runs for approximately five seconds. This reduces the noise in the results, although I haven't checked to see if some of the very fast ones like fps_fill are dominated by the test harness overhead.
For example, running: bench -rotate -scale -clip -config 8888 -repeat 600 -match fps_blend
will take around 5 s to complete on a 1 GHz Cortex-A9.
-- Michael
On 2 March 2011 09:51, Michael Hope michael.hope@linaro.org wrote:
On Tue, Mar 1, 2011 at 3:53 PM, Jim Huang jim.huang@linaro.org wrote:
[...]
The skia maintainer, Mike Reed, made two branches: one is hosted in Google Code, and another is inside Android source tree: http://android.git.kernel.org/?p=platform/external/skia.git%3Ba=summary
Thanks Jim. The Android tree doesn't have any build scripts and I'm reluctant to stray too far away from upstream by adding my own. I'll stick with the code.google one for now.
I've updated: http://bazaar.launchpad.net/~linaro-toolchain-dev/cbuild/trunk/view/head:/li...
hi Michael,
Great to see more benchmark items.
with interations so that each sub benchmark runs for approximately five seconds. This reduces the noise in the results, although I haven't checked to see if some of the very fast ones like fps_fill are dominated by the test harness overhead.
For example, running: bench -rotate -scale -clip -config 8888 -repeat 600 -match fps_blend
will take around 5 s to complete on a 1 GHz Cortex-A9.
I just updated wiki page about Google's compiler benchmark suite: https://wiki.linaro.org/Platform/Android/UpstreamToolchain
Please check the section, "Evaluate Benchmark suite".
Through adb[1], it is highly convenient to bench generated binary files on Android device:
$ ../scripts/run_on_android.py ../scripts/bench.py --action=runcmd echo Android_TIME_STAMP_$(timestamp); /data/local/perflab/skia_bench -repeat 15; echo Android_TIME_STAMP_$(timestamp) ../scripts/bench.py --action=export ./out/skia_bench adb shell mkdir /data/local/perflab
['./out/skia_bench'] adb push ./out/skia_bench /data/local/perflab 823 KB/s (802944 bytes in 0.951s)
time adb shell 'cd /data/local/perflab; echo Android_TIME_STAMP_$(timestamp); /data/local/perflab/skia_bench -repeat 15; echo Android_TIME_STAMP_$(timestamp)'
0.00user 0.00system 4:33.25elapsed 0%CPU (0avgtext+0avgdata 4208maxresident)k 0inputs+0outputs (0major+325minor)pagefaults 0swaps timestamp: not found Android_TIME_STAMP_ running bench decode_4444_(null) 8888: 0.07 565: 0.00 4444: 0.00 A8: 0.07 running bench decode_565_(null) 8888: 0.00 565: 0.00 4444: 0.00 A8: 0.07 ...
From the above messages, Google introduced the automated approach to deploy
benchmark suite and evaluate it on the fly.
The function "timestamp" is broken at the moment, and I am still looking for the details.
Regards, -jserv
On Tue, Mar 8, 2011 at 7:37 AM, Jim Huang jim.huang@linaro.org wrote:
From the above messages, Google introduced the automated approach to deploy benchmark suite and evaluate it on the fly.
Hi jserv. That's quite cool. I'll have a look into their benchmark suite as there seems to be a lot of overlap with what I'm doing.
-- Michael
On 8 March 2011 02:37, Jim Huang jim.huang@linaro.org wrote:
I just updated wiki page about Google's compiler benchmark suite: https://wiki.linaro.org/Platform/Android/UpstreamToolchain Please check the section, "Evaluate Benchmark suite". Through adb[1], it is highly convenient to bench generated binary files on Android device:
[...]
time adb shell 'cd /data/local/perflab; echo Android_TIME_STAMP_$(timestamp); /data/local/perflab/skia_bench -repeat 15; echo Android_TIME_STAMP_$(timestamp)'
0.00user 0.00system 4:33.25elapsed 0%CPU (0avgtext+0avgdata 4208maxresident)k 0inputs+0outputs (0major+325minor)pagefaults 0swaps timestamp: not found
[...]
The function "timestamp" is broken at the moment, and I am still looking for the details.
Here "timestamp" is an Android command line tool which reports the current time in seconds.
You can get it from scripts/timestamp/src/timestamp.c http://android.git.kernel.org/?p=toolchain/benchmark.git%3Ba=blob%3Bf=script...
To build the tool, enter the timestamp directory and type
../bench.py --action=build --toolchain=<Android toolchain path>
adb push out/timestamp to /system/bin on Android device.
Then you are able to call timestamp to report time stamp in seconds.
Sincerely, -jserv
-
Jammy Zhou -
Jim Huang -
Michael Hope