hard-float linker check benchmarks

[Steve McIntyre]

Hey folks,

My first prototype changes to ld.so (see the attached patch) seem to
work ok, and my code will now complain (and fail) appropriately if you
try to mix soft-float ABI and hard-float ABI binaries in the same
process. I think the working code for the patch itself is clean
enough, but at the moment it's hacked in and I'm looking for a cleaner
way to integrate it.

In the meantime, I've been benchmarking the cost of checking each of
the binaries at startup. I've got a script that does:

 * flush caches
 * time ld.so $binary --version

n times in a loop, and times each run so I can calculate the mean and
standard deviation. I've tested four different programs (ls, emacs,
iceweasel and kcalc) to give a spread of sizes. Full figures below.

Quick summary: the extra work of checking does not seem to have any
noticeable bad effect on program startup. In some cases it even seems
to make things faster! But in all cases the differences between
"check" and "nocheck" are well within the standard deviation from the
testing, so I'm happy.

I've measured the effects using Debian armel and armhf chroots on my
Panda board, but the specific versions of software and the target CPU
version vary armel/armhf so please don't compare the numbers in that
direction!

Debian armel (v4t, soft-float ABI), using --version:
====================================================

Test using /bin/ls --version, running 100 times
Results from /tmp/time.2592, 100 runs of "ls" using "check":
   Total times    real 35.20  user 0.00  sys 0.00
   Mean  times    real 0.352 user 0.000 sys 0.000
   stddev         real 0.068 user 0.000 sys 0.000
Test using /bin/ls --version, running 100 times
Results from /tmp/time.3305, 100 runs of "ls" using "nocheck":
   Total times    real 35.27  user 0.00  sys 0.00
   Mean  times    real 0.353 user 0.000 sys 0.000
   stddev         real 0.069 user 0.000 sys 0.000

Test using /usr/bin/emacs23-x --version, running 100 times
Results from /tmp/time.5448, 100 runs of "emacs23-x" using "check":
   Total times    real 519.23  user 1.32  sys 3.20
   Mean  times    real 5.192 user 0.013 sys 0.032
   stddev         real 0.430 user 0.015 sys 0.012
Test using /usr/bin/emacs23-x --version, running 100 times
Results from /tmp/time.6166, 100 runs of "emacs23-x" using "nocheck":
   Total times    real 507.24  user 1.31  sys 3.17
   Mean  times    real 5.072 user 0.013 sys 0.032
   stddev         real 0.302 user 0.012 sys 0.010

Test using /usr/lib/iceweasel/firefox-bin --version, running 100 times
Results from /tmp/time.6880, 100 runs of "/usr/lib/iceweasel/firefox-bin" using "check":
   Total times    real 668.45  user 1.69  sys 6.29
   Mean  times    real 6.684 user 0.017 sys 0.063
   stddev         real 0.692 user 0.018 sys 0.019
Test using /usr/lib/iceweasel/firefox-bin --version, running 100 times
Results from /tmp/time.7696, 100 runs of "/usr/lib/iceweasel/firefox-bin" using "nocheck":
   Total times    real 651.27  user 1.73  sys 6.27
   Mean  times    real 6.513 user 0.017 sys 0.063
   stddev         real 0.557 user 0.020 sys 0.019

Test using /usr/bin/kcalc --version, running 100 times
Results from /tmp/time.4018, 100 runs of "kcalc" using "check":
   Total times    real 1249.40  user 5.77  sys 7.41
   Mean  times    real 12.494 user 0.058 sys 0.074
   stddev         real 0.887 user 0.022 sys 0.021
Test using /usr/bin/kcalc --version, running 100 times
Results from /tmp/time.4733, 100 runs of "kcalc" using "nocheck":
   Total times    real 1240.01  user 5.29  sys 7.89
   Mean  times    real 12.400 user 0.053 sys 0.079
   stddev         real 0.786 user 0.020 sys 0.019

Debian armhf (v7, hard-float ABI), using --version:
===================================================

Results from /tmp/time.7128, 100 runs of "ls" using "check":
   Total times    real 26.52  user 0.00  sys 0.00
   Mean  times    real 0.265 user 0.000 sys 0.000
   stddev         real 0.058 user 0.000 sys 0.000
Test using /bin/ls --version, running 100 times
Results from /tmp/time.7843, 100 runs of "ls" using "nocheck":
   Total times    real 24.74  user 0.00  sys 0.00
   Mean  times    real 0.247 user 0.000 sys 0.000
   stddev         real 0.038 user 0.000 sys 0.000

Test using /usr/bin/emacs23-x --version, running 100 times
Results from /tmp/time.9991, 100 runs of "emacs23-x" using "check":
   Total times    real 548.08  user 2.26  sys 2.40
   Mean  times    real 5.481 user 0.023 sys 0.024
   stddev         real 0.630 user 0.017 sys 0.015
Test using /usr/bin/emacs23-x --version, running 100 times
Results from /tmp/time.10705, 100 runs of "emacs23-x" using "nocheck":
   Total times    real 549.29  user 2.15  sys 2.43
   Mean  times    real 5.493 user 0.022 sys 0.024
   stddev         real 0.600 user 0.019 sys 0.016
Test using /usr/lib/iceweasel/firefox-bin --version, running 100 times

Results from /tmp/time.11423, 100 runs of "/usr/lib/iceweasel/firefox-bin" using "check":
   Total times    real 620.18  user 1.75  sys 5.98
   Mean  times    real 6.202 user 0.018 sys 0.060
   stddev         real 0.682 user 0.019 sys 0.019
Test using /usr/lib/iceweasel/firefox-bin --version, running 100 times
Results from /tmp/time.12237, 100 runs of "/usr/lib/iceweasel/firefox-bin" using "nocheck":
   Total times    real 636.31  user 1.77  sys 5.95
   Mean  times    real 6.363 user 0.018 sys 0.060
   stddev         real 0.766 user 0.018 sys 0.020

Test using /usr/bin/kcalc --version, running 100 times
Results from /tmp/time.8556, 100 runs of "kcalc" using "check":
   Total times    real 1338.36  user 7.00  sys 5.41
   Mean  times    real 13.384 user 0.070 sys 0.054
   stddev         real 1.223 user 0.031 sys 0.029
Test using /usr/bin/kcalc --version, running 100 times
Results from /tmp/time.9275, 100 runs of "kcalc" using "nocheck":
   Total times    real 1345.21  user 7.46  sys 5.02
   Mean  times    real 13.452 user 0.075 sys 0.050
   stddev         real 1.197 user 0.031 sys 0.030

Cheers,
-- 
Steve McIntyre                                steve.mcintyre at linaro.org
<http://www.linaro.org/> Linaro.org | Open source software for ARM SoCs
-------------- next part --------------
A non-text attachment was scrubbed...
Name: check.patch
Type: text/x-diff
Size: 7415 bytes
Desc: not available
URL: <http://lists.linaro.org/pipermail/cross-distro/attachments/20111017/7bf99f9a/attachment.patch>