On 07/07/2023 09:49, James Clark wrote:
On 06/07/2023 13:47, Pandey, Radhey Shyam wrote:
-----Original Message----- From: Pandey, Radhey Shyam Sent: Thursday, July 6, 2023 5:40 PM To: James Clark james.clark@arm.com Cc: coresight@lists.linaro.org; linux-arm-kernel@lists.infradead.org; Suzuki K Poulose suzuki.poulose@arm.com; mathieu.poirier@linaro.org; mike.leach@linaro.org; leo.yan@linaro.org; alexander.shishkin@linux.intel.com; Sarangi, Anirudha anirudha.sarangi@amd.com Subject: RE: Linux coresight arm64 : Incorrect data in cstrace.bin
-----Original Message----- From: James Clark james.clark@arm.com Sent: Wednesday, July 5, 2023 5:44 PM To: Pandey, Radhey Shyam radhey.shyam.pandey@amd.com Cc: coresight@lists.linaro.org; linux-arm-kernel@lists.infradead.org; Suzuki K Poulose suzuki.poulose@arm.com; mathieu.poirier@linaro.org; mike.leach@linaro.org; leo.yan@linaro.org; alexander.shishkin@linux.intel.com; Sarangi, Anirudha anirudha.sarangi@amd.com Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
On 05/07/2023 12:48, Pandey, Radhey Shyam wrote:
-----Original Message----- From: James Clark james.clark@arm.com Sent: Wednesday, July 5, 2023 4:29 PM To: Pandey, Radhey Shyam radhey.shyam.pandey@amd.com Cc: coresight@lists.linaro.org; linux-arm-kernel@lists.infradead.org; Suzuki K Poulose suzuki.poulose@arm.com; mathieu.poirier@linaro.org; mike.leach@linaro.org; leo.yan@linaro.org; alexander.shishkin@linux.intel.com Subject: Re: Linux coresight arm64 : Incorrect data in cstrace.bin
On 05/07/2023 10:56, Pandey, Radhey Shyam wrote: >> -----Original Message----- >> From: Suzuki K Poulose suzuki.poulose@arm.com >> Sent: Wednesday, July 5, 2023 1:57 PM >> To: Pandey, Radhey Shyam radhey.shyam.pandey@amd.com; >> mathieu.poirier@linaro.org; mike.leach@linaro.org; >> leo.yan@linaro.org; alexander.shishkin@linux.intel.com >> Cc: coresight@lists.linaro.org; >> linux-arm-kernel@lists.infradead.org >> Subject: Re: Linux coresight arm64 : Incorrect data in >> cstrace.bin >> >> Hi Radhe Shyam >> >> On 05/07/2023 05:38, Pandey, Radhey Shyam wrote: >>> Hi, >>> >>> I am using linux 6.1 kernel coresight framework to capture ETM
trace.
>>> Enabled coresight driver and added coresight component device >>> node in DTS. With that, we could probe all coresight components. >>> >>> But when we do capture using sysfs and then read the trace.bin >>> using ptm2human we see invalid trace data. Trace content >>> changes on each capture. Any pointers to help narrow down the
issue?
>>> >>> Tried: sysfs capture and perf (with OpenCSD enabled) >>> >>> Development board: VCK190 : >>> https://docs.xilinx.com/r/en-US/ug1366-vck190-eval-bd >>> https://www.xilinx.com/support/documents/architecture- >> manuals/am011-ve >>> rsal-acap-trm.pdf >>> >>> xilinx-vck190-20231:/home/petalinux# dmesg | grep -i amba >>> [ 0.301054] Serial: AMBA PL011 UART driver >>> [ 0.306226] amba f0c20000.funnel: Fixing up cyclic dependency
with
>> f0d70000.etm >>> [ 0.313624] amba f0c20000.funnel: Fixing up cyclic dependency
with
>> f0d30000.etm >>> [ 0.321110] amba f0c30000.etf: Fixing up cyclic dependency with >> f0c20000.funnel >>> [ 1.470247] Serial: AMBA driver >>> xilinx-vck190-20231:/home/petalinux# dmesg | grep -i coresight >>> [ 1.865006] cs_system_cfg: CoreSight Configuration manager
initialised
>>> [ 1.876858] coresight etm0: CPU0: etm v4.0 initialized >>> [ 1.885666] coresight etm1: CPU1: etm v4.0 initialized >>> [ 1.894435] coresight-cpu-debug f0d00000.debug1: Coresight
debug-
CPU0 >> initialized >>> [ 1.902072] coresight-cpu-debug f0d40000.debug1: Coresight
debug-
CPU1 >> initialized >>> >>> xilinx-vck190-20231:/home/petalinux# ls
/sys/bus/coresight/devices/
>>> etm0 etm1 funnel0 tmc_etf0 >>> >>> cd /sys/bus/coresight/devices/ echo 1 > tmc_etf0/enable_sink >>> echo 1 > etm0/enable_source echo 0 > etm0/enable_source echo 0 > >>> tmc_etf0/enable_sink cd /root/ dd if=/dev/tmc_etf0 >>> of=cstrace_28Jun.bin >>> >>> ./ptm2human/ptm2human -e -i cstrace.bin -d >> >> Please note that ptm2human is for PTM trace decoding and >> ETMv4 uses a different format and thus is not compatible. >> >>> >>> I also tried OpenCSD integration with PERF. >> >> Have you made sure the perf is "linked" to the opencsd ? > > Earlier I statically linked but after adding CORESIGHT=1. > > > linux-xlnx$ make ARCH=arm64 NO_LIBELF=1 NO_JVMTI=1 VF=1 CORESIGHT=1 -C > tools/perf/ > make: Entering directory 'linux-xlnx/tools/perf' > BUILD: Doing 'make -j24' parallel build > > Makefile.config:520: *** Error: No libopencsd library found or the > version is not up-to-date. > Please install recent libopencsd to build with CORESIGHT=1. Stop. > > > echo $CSINCLUDES > /scratch/development/coresight/my-opencsd/decoder/include > radheys@xhdradheys41:/scratch/development/linux-xlnx$ ls > /scratch/development/coresight/my-opencsd/decoder/include > common i_dec interfaces mem_acc opencsd opencsd.h > pkt_printers > > echo $CSLIBS > /scratch/development/coresight/my-opencsd/decoder/lib/builddir > radheys@xhdradheys41:/scratch/development/linux-xlnx$ ls /scratch/development/coresight/my-opencsd/decoder/lib/builddir > libopencsd.a libopencsd_c_api.so libopencsd_c_api.so.1.4.0 libopencsd.so.1 > libopencsd_c_api.a libopencsd_c_api.so.1 libopencsd.so libopencsd.so.1.4.0 > > Anything I am missing to fix this opencsd lib not found?
There is a minimum version of 1.1.1 but it looks like you are using higher than that so that part should be fine.
Personally I "make install" OpenCSD to the system path. I had a play around with $CSLIBS it and maybe there is also some stickyness to the feature detection so a make clean might help to make sure the errors you are seeing are real.
Lastly you can print the output of why the feature test compilation failed which might help. I see some linking issues in there if I delete my system installed version of OpenCSD, so there might be a bug with just using CSINCLUDES and CSLIBS:
$ cat tools/build/feature/test-libopencsd.make.output
/usr/bin/ld: warning: libopencsd.so.1, needed by opencsd- local/decoder/lib/builddir//libopencsd_c_api.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so: undefined reference to `vtable for StmTrcPacket' /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so: undefined reference to `DecodeTree::getDecoderStats(unsigned char, _ocsd_decode_stats**)' /usr/bin/ld: opencsd-local/decoder/lib/builddir//libopencsd_c_api.so: undefined reference to `DecodeTree::addRawFramePrinter(RawFramePrinter**, unsigned int)'
Can you try the make install and see if that works? If that works but the other way doesn't I can try looking into why its not and make a
fix.
I tried make install but still it errors out. Looking at make.output I am seeing multiple undefined references.
NOTE I have to still set CSINCLUDES and CSLIBS as we are cross-compiling Perf on a x86 host machine.
This should still work if you make install them to the arch specific folder for example /usr/lib/aarch64-linux-gnu/. If do 'apt install
libopencsd-dev:arm64'
I would get the arm ones in there.
Then if I cross build like this I get openCSD linked:
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -C tools/perf CORESIGHT=1
So for now you should be able to work around the issue in this way.
I tried copying opencsd libraries to /usr/lib/aarch64-linux-gnu but still I see undefined reference. Is it because I am using cross-compiler from server (toolchain path set to $PATH variable)?
/proj/<snip>/Vitis/2023.2/gnu/aarch64/lin/aarch64-linux/bin/aarch64-linux- gnu-gcc I assume CSLIBS and CSINCLUDES should be sufficient to cross compile perf with coresight support?
FYI, I tried with static linking and could proceed to perf compilation.
Static compilation works because the link line is slightly different (it includes -lopencsd rather than just -lopencsd_c_api:
ifeq ($(findstring -static,${LDFLAGS}),-static) OPENCSDLIBS += -lopencsd -lstdc++
I think that this is related to the issue and there have been some recent changes around here. So yes I'll look into the fix for the non static and $CSLIBS mode.
There was an issue, thanks for the report. The fix should be here: https://lore.kernel.org/all/20230707154546.456720-1-james.clark@arm.com/T/#u
Please give it a go and reply with the outcome on that thread.