Hi Andrea,
I've incorporated the patch into the latest release, while making a
few of the adjustments that Suzuki suggested.
Thanks
Mike
On Thu, 26 Mar 2020 at 15:09, Mike Leach <mike.leach@linaro.org> wrote:
>
> Hi Andrea, Suzuki
>
> On Thu, 26 Mar 2020 at 11:16, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:
> >
> > On 03/10/2020 02:02 PM, Andrea Brunato wrote:
> > > This patch updates the documentation to better capture
> > > the current status with the latest kernel releases and
> > > defines new scripts for controlling strobing accordingly.
> >
> > Personally I would prefer this patch to be split into two
> > logical components.
> >
> > 1) Add new script for setting the strobe period
> > 2) Update the documentation.
> >
> > It is upto Mike to see what fits best.
> >
>
> I'm OK either way, but makes sense if altering the scripts after
> fixing comments below.
>
> > > Signed-off-by: Andrea Brunato <andrea.brunato@arm.com>
> > > ---
> > > HOWTO.md | 54 +++++++++---------
> > > decoder/tests/auto-fdo/autofdo.md | 73 +++++++++++++++++++------
> > > decoder/tests/auto-fdo/record.sh | 68 -----------------------
> > > decoder/tests/auto-fdo/set_strobing.sh | 29 ++++++++++
> > > decoder/tests/auto-fdo/show_strobing.sh | 6 ++
> > > 5 files changed, 115 insertions(+), 115 deletions(-)
> > > delete mode 100644 decoder/tests/auto-fdo/record.sh
> > > create mode 100755 decoder/tests/auto-fdo/set_strobing.sh
> > > create mode 100755 decoder/tests/auto-fdo/show_strobing.sh
> > >
> > > diff --git a/HOWTO.md b/HOWTO.md
> > > index b16294a..ebf44eb 100644
> > > --- a/HOWTO.md
> > > +++ b/HOWTO.md
> > > @@ -6,35 +6,31 @@ HOWTO - using the library with perf {#howto_perf}
> > > This HOWTO explains how to use the perf cmd line tools and the openCSD
> > > library to collect and extract program flow traces generated by the
> > > CoreSight IP blocks on a Linux system. The examples have been generated using
> > > -an aarch64 Juno-r0 platform. All information is considered accurate and tested
> > > -using the latest version of the library and the `master` branch on the
> > > -[perf-opencsd github repository][1].
> > > +an aarch64 Juno-r0 platform.
> > >
> > >
> > > On Target Trace Acquisition - Perf Record
> > > -----------------------------------------
> > > -All the enhancement to the Perf tools that support the new `cs_etm` pmu have
> > > -not been upstreamed yet. To get the required functionality branch
> > > -`perf-opencsd-master` needs to be downloaded to the target system where
> > > -traces are to be collected. This branch is a vanilla upstream kernel
> > > -supplemented with modifications to the CoreSight framework and drivers to be
> > > -usable by the Perf core. The remaining out of tree patches are being
> > > -upstreamed incrementally.
> > > -
> > > -From there compiling the perf tools with `make -C tools/perf CORESIGHT=1` will
> > > -yield a `perf` executable that will support CoreSight trace collection. Note
> > > -that if traces are to be decompressed *off* target, there is no need to download
> > > +
> > > +Compile the perf tool from the same kernel source code version you are using,
> > > +with `make -C tools/perf`.
> >
> > Please make the commands explicitly visible by keeping them in a new
> > line, preferrable guarded by one blank line above and below, so that it
> > is easily visible for anyone looking back again. i.e
> >
> > Compile the perf tool from the same kernel source code version you are
> > using:
> >
> > make -C tools/perf
> >
> > > +This will yield a `perf` executable that will support CoreSight trace collection.
> > > +Note that if traces are to be decompressed *off* target, there is no need to download
> > > and compile the openCSD library (on the target).
> > >
> > > +If you are instead planning to use perf to record and decode the trace on the target,
> > > +compile the perf tool linking against the openCSD library, in the following way:
> >
> > +
> > > +`make -C tools/perf VF=1 CORESIGHT=1`
> > > +
> > > +
> >
> > Same here ^^
>
> Perhaps also at this point refer to the more detailed build
> information towards the end of the file.
>
> >
> > [...]
> >
> > > diff --git a/decoder/tests/auto-fdo/autofdo.md b/decoder/tests/auto-fdo/autofdo.md
> > > index b1f2241..d7d37b1 100644
> > > --- a/decoder/tests/auto-fdo/autofdo.md
> > > +++ b/decoder/tests/auto-fdo/autofdo.md
> > > @@ -99,6 +99,8 @@ You can include these backports in your kernel by either merging the
> > > appropriate branch using git or generating patches (using `git
> > > format-patch`).
> > >
> > > +For 5.5 based kernel, the only patch which needs to be applied is the one enabling strobing - etm4x: `Enable strobing of ETM`.
> >
> > Rather, could you be more specific:
> >
> > From linux kernel version v5.x onwards (of course fin x ), the only
> > patch...
> >
> >
> > > +
> > > For 4.9 based kernels, use the `coresight-4.9-etr-etm_strobe` branch:
> > +
> > > +## Capturing modes
> > > +
> > > +You can trace a single-threaded program in two different ways:
> > > +
> > > +1. By specifying `--per-thread`, and in this case the CoreSight subsystem will
> > > +record only a trace relative to the given program.
> > > +
> > > +2. By NOT specifying `--per-thread`, and in this case CPU-wide tracing will
> > > +be enabled. In this scenario the trace will contain both the target program trace
> > > +and other workloads that were executing on the same CPU
> > > +
> >
> > This is *not a recommended* way for capturing trace for a
> > single-threaded process. So you may as well skip it. On the other hand
> > you may list it as a separate option where people may want to
> > trace system-wide.
> >
> >
> > >
> > >
> > > ## Processing trace and profiles
> > > @@ -241,26 +259,42 @@ For example, a typical configuration is to use a window size of 5000 cycles
> > > and a period of 10000 - this will collect 5000 cycles of trace every 50M
> > > cycles. With these proof-of-concept patches, the strobe parameters are
> > > configured via sysfs - each ETM will have `strobe_window` and
> > > -`strobe_period` parameters in `/sys/bus/coresight/devices/NNNNNNNN.etm` and
> > > +`strobe_period` parameters in `/sys/bus/coresight/devices/<sink>` and
> > > these values will have to be written to each (In a future version, this
> > > -will be integrated into the drivers and perf tool). The `record.sh`
> > > -script in this directory [`<opencsd>/decoder/tests/auto-fdo`] automates this process.
> > > +will be integrated into the drivers and perf tool).
> > > +The `set_strobing.sh` script in this directory [`<opencsd>/decoder/tests/auto-fdo`] automates this process.
> > >
> > > To collect trace from an application using ETM strobing, run:
> > >
> > > ```
> > > -taskset -c 0 ./record.sh --strobe 5000 10000 28c06000.etr ./my_application arg1 arg2
> > > +sudo ./set_strobing.sh 5000 10000
> > > +perf record -e cs_etm/@tmc_etr0/u --per-thread -- <your app>"
> > > ```
> > >
> > > -The taskset command is used to ensure the process stays on the same CPU
> > > -during execution.
> > > -
> > > The raw trace can be examined using the `perf report` command:
> > >
> > > ```
> > > perf report -D -i perf.data --stdio
> > > ```
> > >
> > > +Perf needs to be built from your linux kernel version souce code repository against the OpenCSD library in order to be able to properly read ETM-gathered samples and post-process them.
> > > +If running `perf report` produces an error like:
> > > +
> > > +```
> > > +0x1f8 [0x268]: failed to process type: 70 [Operation not permitted]
> > > +Error:
> > > +failed to process sample
> > > +```
> > > +or
> > > +
> > > +```
> > > +"file uses a more recent and unsupported ABI (8 bytes extra). incompatible file format".
> > > +```
> > > +
> > > +You are probably using a perf version which is not using this library: please make sure to install this project in your system by either compiling it from [Source Code]( <
https://github.com/Linaro/OpenCSD>)
from v0.9.1 or later and compile perf using this library.
> > > +Otherwise, this project is packaged for debian (install the libopencsd0, libopencsd-dev packages).
> > > +
> > > +
> > > For example:
> > >
> > > ```
> > > @@ -295,6 +329,8 @@ an embedded target). The `perf inject` command
> > > decodes the execution trace and generates periodic instruction samples,
> > > with branch histories:
> > >
> > > +!! Careful: if you are using a device different than the one used to collect the profiling data,
> > > +you'll need to run `perf buildid-cache` as described below.
> > > ```
> > > perf inject -i perf.data -o inj.data --itrace=i100000il
> > > ```
> > > @@ -393,7 +429,8 @@ clang -O2 -fprofile-sample-use=program.llvmprof -o program program.c
> > > The basic commands to run an application and create a compiler profile are:
> > >
> > > ```
> > > -taskset -c 0 ./record.sh --strobe 5000 10000 28c06000.etr ./my_application arg1 arg2
> > > +sudo ./set_strobing.sh 5000 10000
> > > +perf record -e cs_etm/@tmc_etr0/u --per-thread -- <your app>"
> > > perf inject -i perf.data -o inj.data --itrace=i100000il
> > > create_llvm_prof -binary=/path/to/binary -profile=inj.data -out=program.llvmprof
> > > ```
> > > diff --git a/decoder/tests/auto-fdo/record.sh b/decoder/tests/auto-fdo/record.sh
> > > deleted file mode 100644
> > > index 16d4ba2..0000000
> > > --- a/decoder/tests/auto-fdo/record.sh
> > > +++ /dev/null
> > > @@ -1,68 +0,0 @@
> > > -#!/bin/sh
> > > -
> > > -BUFFER_ETF_A53=ec802000.etf
> > > -BUFFER_ETF_A73=ed002000.etf
> > > -BUFFER_ETF_SYS=ec036000.etf
> > > -BUFFER_ETR=ec033000.etr
> > > -
> > > -OUT_FILE=perf.data
> > > -
> > > -STROBE=
> > > -
> > > -while :; do
> > > - case $1 in
> > > - --strobe)
> > > - STROBE=y
> > > - WINDOW=$2
> > > - PERIOD=$3
> > > - shift 3
> > > - ;;
> > > -
> > > - *)
> > > - break ;;
> > > - esac
> > > -done
> > > -
> > > -case $1 in
> > > - etr)
> > > - BUFFER=$BUFFER_ETR
> > > - ;;
> > > -
> > > - etf-sys)
> > > - BUFFER=$BUFFER_ETF_SYS
> > > - ;;
> > > -
> > > - "")
> > > - BUFFER=$BUFFER_ETR
> > > - ;;
> > > -
> > > - *)
> > > - BUFFER=$1
> > > - ;;
> > > -esac
> > > -
> > > -shift 1
> > > -
> > > -case $0 in
> > > - /*) F=$0 ;;
> > > - *) F=$(pwd)/$0 ;;
> > > -esac
> > > -
> > > -SCRIPT_DIR=$(dirname $F)
> > > -
> > > -if [ "$STROBE" ]; then
> > > - for e in /sys/bus/coresight/devices/*.etm/; do
> > > - printf "%x" $WINDOW | sudo tee $e/strobe_window > /dev/null
> > > - printf "%x" $PERIOD | sudo tee $e/strobe_period > /dev/null
> > > - done
> > > -fi
> > > -
> > > -PERF=$SCRIPT_DIR/perf
> > > -
> > > -export LD_LIBRARY_PATH=$SCRIPT_DIR:$LD_LIBRARY_PATH
> > > -
> > > -sudo LD_LIBRARY_PATH=$SCRIPT_DIR:$LD_LIBRARY_PATH $PERF record $PERF_ARGS -e cs_etm/@$BUFFER/u --per-thread "$@"
> > > -
> > > -sudo chown $(id -u):$(id -g) $OUT_FILE
> > > -
> > > -
> > > diff --git a/decoder/tests/auto-fdo/set_strobing.sh b/decoder/tests/auto-fdo/set_strobing.sh
> > > new file mode 100755
> > > index 0000000..e11e62d
> > > --- /dev/null
> > > +++ b/decoder/tests/auto-fdo/set_strobing.sh
> > > @@ -0,0 +1,29 @@
> > > +#!/bin/bash
> > > +
> > > +WINDOW=$1
> > > +PERIOD=$2
> > > +
> > > +if [[ -z $WINDOW ]] || [[ -z $PERIOD ]]; then
> > > + echo "Window or Period not specified!"
> > > + echo "Example usage: ./set_strobing.sh <WINDOW VALUE> <PERIOD VALUE>"
> > > + echo "Example usage: ./set_strobing.sh 5000 10000"
> > > + exit
> >
> > Please set a non-zero exit code, for scripts to detect that this has failed.
> >
> >
> > > +fi
> > > +
> > > +
> > > +if [[ $EUID != 0 ]]; then
> > > + echo "Please run as root"
> > > + exit
> >
> > Same here.
> >
> > > +fi
> > > +
> > > +for e in /sys/bus/coresight/devices/etm*/; do
> > > + printf "%x" $WINDOW | tee $e/strobe_window > /dev/null
> > > + printf "%x" $PERIOD | tee $e/strobe_period > /dev/null
> > > + echo "Strobing period for $e set to $((`cat $e/strobe_period`))"
> > > + echo "Strobing window for $e set to $((`cat $e/strobe_window`))"
> > > +done
> > > +
> > > +## Shows the user a simple usage example
> > > +echo ">> Done! <<"
> > > +echo "You can now run perf to trace your application, for example:"
> > > +echo "perf record -e cs_etm/@tmc_etr0/u -- <your app>"
> >
> > This would mean that you are probably running the perf from the standard
> > installation path. Not the one you built and the user will need to spend
> > time debugging what is wrong.
> >
>
> The HOWTO.md goes into some depth about setting up build and execution
> environments - I think we have to assume at this point any user has
> this in hand.
>
>
> >
> > > diff --git a/decoder/tests/auto-fdo/show_strobing.sh b/decoder/tests/auto-fdo/show_strobing.sh
> > > new file mode 100755
> > > index 0000000..d80d84c
> > > --- /dev/null
> > > +++ b/decoder/tests/auto-fdo/show_strobing.sh
> > > @@ -0,0 +1,6 @@
> > > +#!/bin/bash
> > > +
> > > +for e in /sys/bus/coresight/devices/etm*/; do
> > > + echo "Strobing period for $e is $((`cat $e/strobe_period`))"
> > > + echo "Strobing window for $e is $((`cat $e/strobe_window`))"
> > > +done
> > > --
> > > 2.17.1
> > >
> > > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for
any purpose, or store or copy the information in any medium. Thank you.
> >
> > Please no disclaimers to the mailing lists.
> >
> >
> > Suzuki
>
>
> Regards
>
> Mike
>
>
> --
> Mike Leach
> Principal Engineer, ARM Ltd.
> Manchester Design Centre. UK
--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK