I'd like to record the running of a typical program such as Firefox, GCC, or ffmpeg and capture the calls and arguments to functions like strcpy() and memcpy(). The idea is to generate a usage profile so we can tell what standard library functions and what variants (i.e. aligned/unaligned, small copy/large copy) to attack first.
Does anyone know of existing research with this information, or existing tools that could capture it?
My fallback is to use a LD_PRELOAD trick to replace the interesting functions with tracing versions.
-- Michael
Will Oprofile help you on that?
BR, Jason
-----Original Message----- From: linaro-dev-bounces@lists.linaro.org [mailto:linaro-dev-bounces@lists.linaro.org] On Behalf Of Michael Hope Sent: Monday, August 09, 2010 10:47 PM To: linaro-dev@lists.linaro.org Subject: Capturing usage information
I'd like to record the running of a typical program such as Firefox, GCC, or ffmpeg and capture the calls and arguments to functions like strcpy() and memcpy(). The idea is to generate a usage profile so we can tell what standard library functions and what variants (i.e. aligned/unaligned, small copy/large copy) to attack first.
Does anyone know of existing research with this information, or existing tools that could capture it?
My fallback is to use a LD_PRELOAD trick to replace the interesting functions with tracing versions.
-- Michael
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Michael Hope wrote:
I'd like to record the running of a typical program such as Firefox, GCC, or ffmpeg and capture the calls and arguments to functions like strcpy() and memcpy(). The idea is to generate a usage profile so we can tell what standard library functions and what variants (i.e. aligned/unaligned, small copy/large copy) to attack first.
Micheal, There is a 2006 GCC Summit paper "Low-Level Performance Analysis", written by Steven Munroe and Peter Steinmetz. They introduced how do they improve the performance of some methods, like memcmp() on POWER processor. This paper is similar to what we are doing, and hope it is useful.
Does anyone know of existing research with this information, or existing tools that could capture it?
Looks like you need a Binary Instrumentation tool for method tracing. Pin[1]/Valgrind[2] are good DBI platforms, and not hard to write a plugin/extension to them for our needs. However, I am not sure Pin is well-supported on ARM, and I also concern about overhead of valgrind for performance profiling.
My fallback is to use a LD_PRELOAD trick to replace the interesting functions with tracing versions.
Yeah, LD_PRELOAD is not bad if we can't find any other tools to do method tracing.
[1] Pin Tool. http://www.pintool.org/ [2] Valgrind. valgrind.org
Does ltrace do what you want?
On Mon, Aug 9, 2010 at 8:47 PM, Michael Hope michael.hope@linaro.org wrote:
I'd like to record the running of a typical program such as Firefox, GCC, or ffmpeg and capture the calls and arguments to functions like strcpy() and memcpy(). The idea is to generate a usage profile so we can tell what standard library functions and what variants (i.e. aligned/unaligned, small copy/large copy) to attack first.
Does anyone know of existing research with this information, or existing tools that could capture it?
My fallback is to use a LD_PRELOAD trick to replace the interesting functions with tracing versions.
-- Michael
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Hi
On Mon, Aug 9, 2010 at 8:47 PM, Michael Hope michael.hope@linaro.org wrote:
I'd like to record the running of a typical program such as Firefox, GCC, or ffmpeg and capture the calls and arguments to functions like strcpy() and memcpy(). The idea is to generate a usage profile so we can tell what standard library functions and what variants (i.e. aligned/unaligned, small copy/large copy) to attack first.
The linux perf tool may also help you: https://perf.wiki.kernel.org/index.php/Main_Page
Think of it as "Oprofile 2" (though no doubt someone would flame me for that). It's a sampling profiler, so it won't count function calls AFAIK. But you can identify hotsports, both in libraries and the kernel as well as the target application.
Note - some patches to the perf tools is needed if you want to get correct profiling information from prebuilt Ubuntu packages; there's also one minor ARM-specific fix - see https://bugs.launchpad.net/ubuntu/+source/linux/+bug/608775
You will also need the debug images for any libraries you want profiling information for. Debug package names not completely predictable, but are usually either: * <binary-package-name>-dbg (from the standard archives), or * <binary-package-name>-dbgsym (from http://ddebs.ubuntu.com/) - see https://wiki.edubuntu.org/DebuggingProgramCrash
Common packages are more likely to have an explicitly-generated debug package (*-dbg); *-dbgsym are generated automatically for most other packages.
Cheers ---Dave
On Mon, Aug 09, 2010, John Rigby wrote:
Does ltrace do what you want?
This is what comes to my mind as well.
There is a catch: I think eglibc (in fact almost all Ubuntu packages) is built with -Bsymbolic-functions, which means that you can't intercept internal calls to memcpy() from other eglibc functions using LD_PRELOAD.
It might be more effective to instrument the eglibc source code to record what you want.
On Tue, Aug 10, 2010 at 10:16 AM, Loïc Minier loic.minier@linaro.org wrote:
On Mon, Aug 09, 2010, John Rigby wrote:
Does ltrace do what you want?
This is what comes to my mind as well.
There is a catch: I think eglibc (in fact almost all Ubuntu packages) is built with -Bsymbolic-functions, which means that you can't intercept internal calls to memcpy() from other eglibc functions using LD_PRELOAD.
memcpy is a particular scary example because the compiler may treat it as an intrinsic and/or inline it (as for mem*() and str*() in general).
I noticed when experimenting with memory allocation tracking that glibc seems to avoid applying -Bsymbolic-functions to some things ... so to my surprise you can intercept the malloc() family of calls and it seems to work properly, even for "implicit" allocations (e.g., calling fopen or asprintf etc.) This is probably deliberate -- I guess this will only work for specific groups of functions though.
Didn't know about ltrace before; it does look useful :)
Cheers ---Dave
On Tue, 2010-08-10 at 10:26 +0100, Dave Martin wrote:
On Tue, Aug 10, 2010 at 10:16 AM, Loïc Minier loic.minier@linaro.org wrote:
On Mon, Aug 09, 2010, John Rigby wrote:
Does ltrace do what you want?
This is what comes to my mind as well.
There is a catch: I think eglibc (in fact almost all Ubuntu packages) is built with -Bsymbolic-functions, which means that you can't intercept internal calls to memcpy() from other eglibc functions using LD_PRELOAD.
memcpy is a particular scary example because the compiler may treat it as an intrinsic and/or inline it (as for mem*() and str*() in general).
I think that's ok. We should probably look to see if there are some cases we don't currently inline, but probably should. Trivial examples would include things like moving 4 bytes when the alignment is unknown. (actually, that's two instructions on v7 if alignment checks in h/w are turned off).
R.
On 10/08/10 03:47, Michael Hope wrote:
I'd like to record the running of a typical program such as Firefox, GCC, or ffmpeg and capture the calls and arguments to functions like strcpy() and memcpy(). The idea is to generate a usage profile so we can tell what standard library functions and what variants (i.e. aligned/unaligned, small copy/large copy) to attack first.
Does anyone know of existing research with this information, or existing tools that could capture it?
The obvious answer is to rebuild both firefox and libc with -pg.
Maybe Ubuntu already has a glibc package build for profiling?
Running the program then produces a gmon.out file that can be read with gprof.
Andrew
On Tue, Aug 10, 2010 at 9:21 AM, Andrew Stubbs ams@codesourcery.com wrote:
[...]
The obvious answer is to rebuild both firefox and libc with -pg.
Maybe Ubuntu already has a glibc package build for profiling?
I believe there may be a profiling C library already in the archive, but most other libraries won't have a pre-existing profiling build ... but my memory is a bit hazy on that. Try apt-cache search :)
I seem to remeber that -pg does not work usefully with shared libraries... is this (still) the case?
Cheers ---Dave
Does anyone know of existing research with this information, or existing tools that could capture it?
The obvious answer is to rebuild both firefox and libc with -pg.
Maybe Ubuntu already has a glibc package build for profiling?
Yip, but I want the arguments as well so we can tell what size copies are being done and what alignment the arguments are.
ltrace looks perfect so far.
-- Michael
On Tue, 2010-08-10 at 14:47 +1200, Michael Hope wrote:
I'd like to record the running of a typical program such as Firefox, GCC, or ffmpeg and capture the calls and arguments to functions like strcpy() and memcpy(). The idea is to generate a usage profile so we can tell what standard library functions and what variants (i.e. aligned/unaligned, small copy/large copy) to attack first.
Does anyone know of existing research with this information, or existing tools that could capture it?
In a previous life I've played with sprof which gave a developer such information about a single shared library. That might be useful to play with if you are interested in profiling a single shared library. I don't know what the current status is but it's certainly worth a try and prevents you from having to write tracing versions of the functions that you might be interested in.
Still seems to come by default with eglibc and might be worth exploring though your mileage may vary.
HTH Ramana
-
Andrew Stubbs -
Dave Martin -
John Rigby -
Liu Hui-R64343 -
Loïc Minier -
Michael Hope -
Ramana Radhakrishnan -
Richard Earnshaw -
Yao Qi