On 05/07/2023 12:48, Pandey, Radhey Shyam wrote:
-----Original Message----- From: James Clark james.clark@arm.com Sent: Wednesday, July 5, 2023 4:29 PM To: Pandey, Radhey Shyam radhey.shyam.pandey@amd.com Cc: coresight@lists.linaro.org; linux-arm-kernel@lists.infradead.org; Suzuki K Poulose suzuki.poulose@arm.com; mathieu.poirier@linaro.org; mike.leach@linaro.org; leo.yan@linaro.org; alexander.shishkin@linux.intel.com Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
On 05/07/2023 10:56, Pandey, Radhey Shyam wrote:
-----Original Message----- From: Suzuki K Poulose suzuki.poulose@arm.com Sent: Wednesday, July 5, 2023 1:57 PM To: Pandey, Radhey Shyam radhey.shyam.pandey@amd.com; mathieu.poirier@linaro.org; mike.leach@linaro.org; leo.yan@linaro.org; alexander.shishkin@linux.intel.com Cc: coresight@lists.linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
Hi Radhe Shyam
On 05/07/2023 05:38, Pandey, Radhey Shyam wrote:
Hi,
I am using linux 6.1 kernel coresight framework to capture ETM trace. Enabled coresight driver and added coresight component device node in DTS. With that, we could probe all coresight components.
But when we do capture using sysfs and then read the trace.bin using ptm2human we see invalid trace data. Trace content changes on each capture. Any pointers to help narrow down the issue?
Tried: sysfs capture and perf (with OpenCSD enabled)
Development board: VCK190 : https://docs.xilinx.com/r/en-US/ug1366-vck190-eval-bd https://www.xilinx.com/support/documents/architecture-
manuals/am011-ve
rsal-acap-trm.pdf
xilinx-vck190-20231:/home/petalinux# dmesg | grep -i amba [ 0.301054] Serial: AMBA PL011 UART driver [ 0.306226] amba f0c20000.funnel: Fixing up cyclic dependency with
f0d70000.etm
[ 0.313624] amba f0c20000.funnel: Fixing up cyclic dependency with
f0d30000.etm
[ 0.321110] amba f0c30000.etf: Fixing up cyclic dependency with
f0c20000.funnel
[ 1.470247] Serial: AMBA driver xilinx-vck190-20231:/home/petalinux# dmesg | grep -i coresight [ 1.865006] cs_system_cfg: CoreSight Configuration manager initialised [ 1.876858] coresight etm0: CPU0: etm v4.0 initialized [ 1.885666] coresight etm1: CPU1: etm v4.0 initialized [ 1.894435] coresight-cpu-debug f0d00000.debug1: Coresight debug-
CPU0
initialized
[ 1.902072] coresight-cpu-debug f0d40000.debug1: Coresight debug-
CPU1
initialized
xilinx-vck190-20231:/home/petalinux# ls /sys/bus/coresight/devices/ etm0 etm1 funnel0 tmc_etf0
cd /sys/bus/coresight/devices/ echo 1 > tmc_etf0/enable_sink echo 1 > etm0/enable_source echo 0 > etm0/enable_source echo 0 > tmc_etf0/enable_sink cd /root/ dd if=/dev/tmc_etf0 of=cstrace_28Jun.bin
./ptm2human/ptm2human -e -i cstrace.bin -d
Please note that ptm2human is for PTM trace decoding and ETMv4 uses a different format and thus is not compatible.
I also tried OpenCSD integration with PERF.
Have you made sure the perf is "linked" to the opencsd ?
Earlier I statically linked but after adding CORESIGHT=1.
linux-xlnx$ make ARCH=arm64 NO_LIBELF=1 NO_JVMTI=1 VF=1
CORESIGHT=1 -C
tools/perf/ make: Entering directory 'linux-xlnx/tools/perf' BUILD: Doing 'make -j24' parallel build
Makefile.config:520: *** Error: No libopencsd library found or the version
is not up-to-date.
Please install recent libopencsd to build with CORESIGHT=1. Stop.
echo $CSINCLUDES /scratch/development/coresight/my-opencsd/decoder/include radheys@xhdradheys41:/scratch/development/linux-xlnx$ ls /scratch/development/coresight/my-opencsd/decoder/include common i_dec interfaces mem_acc opencsd opencsd.h pkt_printers
echo $CSLIBS /scratch/development/coresight/my-opencsd/decoder/lib/builddir radheys@xhdradheys41:/scratch/development/linux-xlnx$ ls
/scratch/development/coresight/my-opencsd/decoder/lib/builddir
libopencsd.a libopencsd_c_api.so libopencsd_c_api.so.1.4.0
libopencsd.so.1
libopencsd_c_api.a libopencsd_c_api.so.1 libopencsd.so
libopencsd.so.1.4.0
Anything I am missing to fix this opencsd lib not found?
There is a minimum version of 1.1.1 but it looks like you are using higher than that so that part should be fine.
Personally I "make install" OpenCSD to the system path. I had a play around with $CSLIBS it and maybe there is also some stickyness to the feature detection so a make clean might help to make sure the errors you are seeing are real.
Lastly you can print the output of why the feature test compilation failed which might help. I see some linking issues in there if I delete my system installed version of OpenCSD, so there might be a bug with just using CSINCLUDES and CSLIBS:
$ cat tools/build/feature/test-libopencsd.make.output
/usr/bin/ld: warning: libopencsd.so.1, needed by opencsd- local/decoder/lib/builddir//libopencsd_c_api.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so: undefined reference to `vtable for StmTrcPacket' /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so: undefined reference to `DecodeTree::getDecoderStats(unsigned char, _ocsd_decode_stats**)' /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so: undefined reference to `DecodeTree::addRawFramePrinter(RawFramePrinter**, unsigned int)'
Can you try the make install and see if that works? If that works but the other way doesn't I can try looking into why its not and make a fix.
I tried make install but still it errors out. Looking at make.output I am seeing multiple undefined references.
NOTE I have to still set CSINCLUDES and CSLIBS as we are cross-compiling Perf on a x86 host machine.
This should still work if you make install them to the arch specific folder for example /usr/lib/aarch64-linux-gnu/. If do 'apt install libopencsd-dev:arm64' I would get the arm ones in there.
Then if I cross build like this I get openCSD linked:
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/perf CORESIGHT=1
So for now you should be able to work around the issue in this way.
cat tools/build/feature/test-libopencsd.make.output <snip library paths>
aarch64-xilinx-linux-ld.real: warning: libopencsd.so.1, needed by libopencsd_c_api.so, not found (try using -rpath or -rpath-link) aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `vtable for StmTrcPacket' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::getDecoderStats(unsigned char, _ocsd_decode_stats**)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::addRawFramePrinter(RawFramePrinter**, unsigned int)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::logMappedRanges()' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `OcsdLibDcdRegister::deregisterAllDecoders()' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `PtmTrcPacket::toString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) const' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::s_error_logger' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::addBinFileRegionMemAcc(_ocsd_file_mem_region const*, int, _ocsd_mem_space_acc_t, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `EtmV4ITrcPacket::~EtmV4ITrcPacket()' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `OcsdLibDcdRegister::getDecoderRegister()' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `typeinfo for TraceComponent' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `EtmV3TrcPacket::~EtmV3TrcPacket()' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `OcsdTraceElement::toString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) const' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::addGenElemPrinter(TrcGenericElementPrinter**)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::DestroyDecodeTree(DecodeTree*)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `ocsdError::getErrorString[abi:cxx11](ocsdError const&)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::setGenTraceElemOutI(ITrcGenElemIn*)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `ocsdError::ocsdError(ocsdError const*)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `ocsdError::ocsdError(_ocsd_err_severity_t, _ocsd_err_t, unsigned int, unsigned char, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `EtmV4ITrcPacket::toString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) const' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `DecodeTree::createDecoder(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, CSConfig const*)' aarch64-xilinx-linux-ld.real: libopencsd_c_api.so: undefined reference to `StmTrcPacket::StmTrcPacket()'
Yes this looks like what I see as well. There is probably some issue with CSLIBS not being propagated to all parts of the build. I would recommend using the system library path for now and I will look into a fix next week. Unless you are happy to look into it and send a fix that would be great too.
James
Thanks, Radhey
Thanks James