Choosing the timer to use in benchmarking
Michael Hope
michael.hope at linaro.org
Tue Aug 30 01:47:20 UTC 2011
Being able to accurately and consistently measure the elapsed CPU time
is important for toolchain benchmarking. I ran a few experiments today
and wrote up the results at:
https://wiki.linaro.org/WorkingGroups/ToolChain/Benchmarks/TimerAccuracy
The original is available at:
http://bazaar.launchpad.net/~michaelh1/linaro-toolchain-benchmarks/trunk/view/head:/reports/timer-accuracy.rst
Short story:
* Use clock_gettime(CLOCK_PROCESS_CPUTIME_ID, ...)
* The mean is unaffected by CPU load
* I/O load has a significant effect
* Use nice -n -20 to reduce the variance
For a CPU bound, non-VFP, L1 bound, 5 s long program:
* The variation coefficient is < 0.01 % so we can reliably measure
0.1 % changes
* The accuracy is around 50 us
I've changed eembc-linaro and will change denbench-linaro next. I
recommend anyone else measuring time on core based benchmarks to do
the same.
-- Michael
More information about the linaro-dev
mailing list