Hi Bharat,
Looking at the stack trace - I see that an illegal trace header has caused the library to error out.
On Mon, 21 Oct 2019 at 17:53, Mathieu Poirier mathieu.poirier@linaro.org wrote:
The openCSD library has been stable for a very long while now. The only time I've seen it misbehave is when it was fed wrong information about the traces being decoded. With the traces you provided Mike may be in a position to figure out what is happening but in the meantime, I'd like to make sure we aren't doing something that is not supported. As such can you give us more information on the platform you are tracing on and the CS topology found on the HW?
Thanks, Mathieu
On Mon, 21 Oct 2019 at 10:35, Bharat Bhushan bbhushan2@marvell.com wrote:
Hi All,
Tracing the coredump show ocsd_dt_process_data() called. Below is the coredump, in case it can give some quick suggestion.
================= #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x0000ffff93ca3e68 in __GI_abort () at abort.c:79 #2 0x0000ffff933e01e0 in __gnu_cxx::__verbose_terminate_handler() () from /lib/aarch64-linux-gnu/libstdc++.so.6 #3 0x0000ffff933dddb4 in ?? () from /lib/aarch64-linux-gnu/libstdc++.so.6 #4 0x0000ffff933dde00 in std::terminate() () from /lib/aarch64-linux-gnu/libstdc++.so.6 #5 0x0000ffff933de128 in __cxa_rethrow () from /lib/aarch64-linux-gnu/libstdc++.so.6 #6 0x0000ffff933e017c in __gnu_cxx::__verbose_terminate_handler() () from /lib/aarch64-linux-gnu/libstdc++.so.6 #7 0x0000ffff933dddb4 in ?? () from /lib/aarch64-linux-gnu/libstdc++.so.6 #8 0x0000ffff933dde00 in std::terminate() () from /lib/aarch64-linux-gnu/libstdc++.so.6 #9 0x0000ffff933de0c4 in __cxa_throw () from /lib/aarch64-linux-gnu/libstdc++.so.6 #10 0x0000ffff93577e38 in EtmV4IPktProcImpl::iPktReserved (this=0xaaaad843b8c0, lastByte=<optimized out>)
Reserved packet here ^^^ Unfortunately there is no indication of what that packet data is as it appears to be optimised out.
Which core / trace architecture are you trying to decode? This error can be caused by: 1) Architecture not supported by the current version of the library - there are some variants that we do not support as we have not had test cases to check them out. 2) Error in the trace generation / collection.
at /root/OpenCSD/decoder/include/opencsd/etmv4/trc_cmp_cfg_etmv4.h:378#11 0x0000ffff93579b34 in EtmV4IPktProcImpl::processData (this=0xaaaad843b8c0, index=<optimized out>, dataBlockSize=<optimized out>, pDataBlock=<optimized out>, numBytesProcessed=0xffffe4255184) at /root/OpenCSD/decoder/source/etmv4/trc_pkt_proc_etmv4i_impl.cpp:158 #12 0x0000ffff935779f8 in TrcPktProcBase<EtmV4ITrcPacket, _ocsd_etmv4_i_pkt_type, EtmV4Config>::TraceDataIn (this=0xaaaad843b750, op=OCSD_OP_DATA, index=0, dataBlockSize=<optimized out>, pDataBlock=<optimized out>, numBytesProcessed=<optimized out>) at /root/OpenCSD/decoder/include/common/comp_attach_pt_t.h:198 #13 0x0000aaaad6fde9ac in cs_etm_decoder__process_data_block (decoder=0xaaaad843aec0, indx=0, buf=0xffff93041998 <error: Cannot access memory at address 0xffff93041998>, len=24, consumed=consumed@entry=0xffffe4255210) at util/cs-etm-decoder/cs-etm-decoder.c:736
Which brings me to the error here ^^^ - cannot access memory
Could this be the point where bad data is passed into the decoder ?
Regards
Mike
#14 0x0000aaaad6fdba90 in cs_etm__decode_data_block (etmq=0xaaaad843ae30) at util/cs-etm.c:1916 #15 cs_etm__setup_queue (queue_nr=0, queue=<optimized out>, etm=0xaaaad83e4a80) at util/cs-etm.c:789 #16 cs_etm__setup_queues (etm=etm@entry=0xaaaad83e4a80) at util/cs-etm.c:838 #17 0x0000aaaad6fdd100 in cs_etm__update_queues (etm=0xaaaad83e4a80) at util/cs-etm.c:850 #18 cs_etm__update_queues (etm=0xaaaad83e4a80) at util/cs-etm.c:846 #19 cs_etm__process_event (session=<optimized out>, event=0xffff93041808, sample=<optimized out>, tool=<optimized out>) at util/cs-etm.c:2274 #20 0x0000aaaad6f8bfe4 in perf_session__deliver_event (session=0xaaaad83c7670, event=0xffff93041808, tool=0xffffe4255dc8, file_offset=137224) at util/session.c:1480 #21 0x0000aaaad6f900dc in do_flush (show_progress=false, oe=0xaaaad83ce190) at util/ordered-events.c:244 #22 __ordered_events__flush (oe=0xaaaad83ce190, how=OE_FLUSH__ROUND, timestamp=<optimized out>) at util/ordered-events.c:323 #23 0x0000aaaad6f8c95c in perf_session__process_user_event (session=session@entry=0xaaaad83c7670, event=event@entry=0xffff93041a40, file_offset=file_offset@entry=137792) at util/session.c:1528 #24 0x0000aaaad6f8ce58 in perf_session__process_event (file_offset=137792, event=0xffff93041a40, session=0xaaaad83c7670) at util/session.c:1660 #25 process_simple (session=session@entry=0xaaaad83c7670, event=event@entry=0xffff93041a40, file_offset=file_offset@entry=137792) at util/session.c:2146 #26 0x0000aaaad6f8e2e0 in reader__process_events (prog=0xffffe4255b78, session=0xaaaad83c7670, rd=<synthetic pointer>) at util/session.c:2111 #27 __perf_session__process_events (session=0xaaaad83c7670) at util/session.c:2169 #28 perf_session__process_events (session=session@entry=0xaaaad83c7670) at util/session.c:2202 #29 0x0000aaaad6ece000 in __cmd_report (rep=0xffffe4255dc8) at builtin-report.c:868 #30 cmd_report (argc=<optimized out>, argv=<optimized out>) at builtin-report.c:1468 #31 0x0000aaaad6f341b4 in run_builtin (p=0xaaaad7128288 <commands+240>, argc=1, argv=0xffffe42583a0) at perf.c:310
Thanks -Bharat
-----Original Message----- From: Mike Leach mike.leach@linaro.org Sent: Monday, October 21, 2019 9:01 PM To: Leo Yan leo.yan@linaro.org; Coresight ML coresight@lists.linaro.org Cc: Bharat Bhushan bbhushan2@marvell.com; Mathieu Poirier mathieu.poirier@linaro.org Subject: [EXT] Re: Coresight: ocsdError error
External Email
Hi,
From the little information in this mail, this looks like an issue with how the version of perf you have is interacting with the OpenCSD library. An ocsdError will usually be generated if the library detects input errors in configuration or data being passed to it.
ocsdErrors in general should provide a code + string - though output could be being suppressed by perf. I would follow Leo's suggestion to see if you can spot the library call from the backtrace that causes the issue.
Regards
Mike
On Mon, 21 Oct 2019 at 06:39, Leo Yan leo.yan@linaro.org wrote:
Hi Bharat,
[ + Mike ]
On Mon, Oct 21, 2019 at 05:28:54AM +0000, Bharat Bhushan wrote:
Hi Leo, Mathieu,
I found your email-id from linux git-log, Please ignore if you are not right
person.
I am using perf for hwtracing (coresight).
$perf record -C 0 -e cs_etm/@tmc_etr0/u ./test Couldn't synthesize bpf events. [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.124 MB perf.data ] $ perf report terminate called after throwing an instance of 'ocsdError' terminate called recursively Aborted (core dumped) $ perf report -v build id event received for [kernel.kallsyms]:473d5d96eceb209a58f19ec8f3e4f74891d49c9f
build id event received for /usr/lib/systemd/systemd:16460ffcbc368221e1e71c587c059a44497eb3de
build id event received for /usr/lib/aarch64-linux-gnu/libudev.so.1.6.12: bf0152a1c65039afe462800eb834190830a5c8d2
. <snip> . build id event received for /usr/lib/aarch64-linux-gnu/libdw-0.176.so:ad9dc3a40b2f39629fcc70ad2bb931d75f89b748
build id event received for /usr/lib/aarch64-linux-gnu/libelf-0.176.so:0b36ddcd158758dcb8356713eff8fe12c1326d21
build id event received for /usr/lib/libopencsd_c_api.so.0.12.0:84a1cf9a8b6efb9670b902d87118f05c81856b53
Aborted (core dumped) terminate called after throwing an instance of 'ocsdError' terminate called recursively root@ubuntu:~# Also I added prints in "perf" source code before calling and opencsdlibrary function (please find attached file) and no prints observed.
I am not sure why we are getting "ocsdError". Can you please point what's wrong I am doing.I think the error is reported internally in OpenCSD rather than the failure in perf tool.
And you could see there have core dump file, suggest you could to use the core dump file with gdb to dump the backtrace. Then this can help us to find the flow for the failure.
I loop Mike since this issue is more likely related with decoder.
Thanks, Leo Yan
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK