Hi,
What was the timer used previously for benchmarks ?
Your page focuses on accuracy while I think an important point is already the availability/functionality of clock_gettime(CLOCK_PROCESS/THREAD_CPUTIME_ID, ...) on ARM, which measures process/thread execution time vs "wall time" for most other timer APIs. Good to know it looks accurate (and Internet seems to say that API is available on Android)
Generally your program is alone to be benchmarked so wall time converges to exec time but as you suggest, it would be more correct to use this clock.
On our side, team was using basic timer computing "wall time" + scheduler trace from perf or systemtap (to remove periods of non-execution in a pos-processing tool) in order for example to profile/debug a specific sequence in an application. We could get rid of post-processing tool now.
Regards Fred
Frederic Turgis OMAP Platform Business Unit - OMAP System Engineering - Platform Enablement
Texas Instruments France SA, 821 Avenue Jack Kilby, 06270 Villeneuve Loubet. 036 420 040 R.C.S Antibes. Capital de EUR 753.920
-----Original Message-----
From: linaro-dev-bounces@lists.linaro.org [mailto:linaro-dev-bounces@lists.linaro.org] On Behalf Of Michael Hope Sent: Tuesday, August 30, 2011 3:47 AM To: Linaro Dev Subject: Choosing the timer to use in benchmarking
Being able to accurately and consistently measure the elapsed CPU time is important for toolchain benchmarking. I ran a few experiments today and wrote up the results at:
https://wiki.linaro.org/WorkingGroups/ToolChain/Benchmarks/Tim erAccuracy
The original is available at:
http://bazaar.launchpad.net/~michaelh1/linaro-toolchain-benchm arks/trunk/view/head:/reports/timer-accuracy.rst
Short story:
- Use clock_gettime(CLOCK_PROCESS_CPUTIME_ID, ...)
- The mean is unaffected by CPU load
- I/O load has a significant effect
- Use nice -n -20 to reduce the variance
For a CPU bound, non-VFP, L1 bound, 5 s long program:
- The variation coefficient is < 0.01 % so we can reliably measure
0.1 % changes
- The accuracy is around 50 us
I've changed eembc-linaro and will change denbench-linaro next. I recommend anyone else measuring time on core based benchmarks to do the same.
-- Michael
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev