On Tue, 5 Nov 2019 at 23:10, Bharat Bhushan bbhushan2@marvell.com wrote:
Hello,
First of all I am sorry, I was supposed to run "./perf" while I was running "perf" when taking output "perf --debug verbose=10 report --stdio > perf_report_dbg.log"
I am attaching the output of " perf --debug verbose=10 report --stdio > perf_report_dbg.log"
A thought just struck me this morning... Aren't you dealing with a 1:1 source/sink topology in your design?
Thanks -Bharat
-----Original Message----- From: Mathieu Poirier mathieu.poirier@linaro.org Sent: Tuesday, November 5, 2019 10:21 PM To: Bharat Bhushan bbhushan2@marvell.com Cc: Leo Yan leo.yan@linaro.org; Coresight ML coresight@lists.linaro.org Subject: Re: [EXT] Re: perf: user-space trace decoding
On Tue, 5 Nov 2019 at 07:07, Bharat Bhushan bbhushan2@marvell.com wrote:
Hello,
I will look in more details in code you mentioned.
Yes, I think this is the right thing to do. I'm pretty sure function cs_etm__process_auxtrace_info() is finding something it doesn't like. Why it works for kernel traces but not user space trace is puzzling though. Usually it is the other way around.
I've also seen problems such as this one when the decoder can't lookup the address of the instructions being decoded in cs_etm__mem_access[1].
There may be several problems here and it is very likely you will have to trace a fair amount of code to find the root cause of the issue(s).
[1]. https://urldefense.proofpoint.com/v2/url?u=https- 3A__elixir.bootlin.com_linux_latest_source_tools_perf_util_cs-2Detm.c- 23L634&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=PAAlWswPe7d8gHlGb CLmy2YezyK7O3Hv_t2heGnouBw&m=xzqQHPtXHXIolLyGp6WrHnfjOr99WId oBnEkbzYFcwc&s=RZ-L4pyrmAq8F60e8nxlhIGBYchDeSEPVWKMv1_Ortc&e=
While below is the Auxtrace info:
. 0000: 46 00 00 00 00 00 70 00 03 00 00 00 00 00 00 00 F.....p......... . 0010: 00 00 00 00 00 00 00 00 01 00 00 00 07 00 00 00 ................ . 0020: 00 00 00 00 00 00 00 00 40 40 40 40 40 40 40 40 ........@@@@@@@@ . 0030: 01 00 00 00 00 00 00 00 40 08 00 00 00 00 00 00
........@.......
. 0040: 12 00 00 00 00 00 00 00 01 02 00 26 00 00 00 00 ...........&.... . 0050: 30 f4 00 43 00 00 00 00 88 10 00 20 00 00 00 00 0ô.C....... .... . 0060: 00 00 00 00 00 00 00 00 cc 00 00 00 00 00 00 00 ........Ì.......
0 0 0x178 [0x70]: PERF_RECORD_AUXTRACE_INFO type: 3 Header version 0 PMU type/num cpus 700000001 Snapshot 0 Magic number 4040404040404040 CPU 1 TRCCONFIGR 840 TRCTRACEIDR 12 TRCIDR0 26000201 TRCIDR1 4300f430 TRCIDR2 20001088 TRCIDR8 0 TRCAUTHSTATUS cc
Thanks -Bharat
-----Original Message----- From: Leo Yan leo.yan@linaro.org Sent: Tuesday, November 5, 2019 7:03 PM To: Bharat Bhushan bbhushan2@marvell.com Cc: Coresight ML coresight@lists.linaro.org Subject: Re: [EXT] Re: perf: user-space trace decoding
On Tue, Nov 05, 2019 at 12:51:24PM +0000, Bharat Bhushan wrote:
Hello,
Below is output of "perf --debug verbose=10 report --stdio >
perf_report_dbg.log"
It fails with type 0x70. Is this "ignore packet header "? It should workn because
Earlier I have seen this failing with "Reserved Packet" as h/w was ETM4.2
but it adds "Ignore Packet - 0x70".
Now I changed the TRCIDR1 to read out as 0x4.3 (git diff in the end). Maybe
I missed something.
[...]
0x178 [0x70]: failed to process type: 70 Error: failed to process sample
It should be related with auxtrace info handling.
We can use 'perf script -D' to dump related info, e.g. I can see auxtrace info as below for CoreSight:
# perf script -D
[...]
0 0 0x178 [0x70]: PERF_RECORD_AUXTRACE_INFO type: 3 Header version 0 PMU type/num cpus 900000001 Snapshot 0 Magic number 4040404040404040 CPU 1 TRCCONFIGR 840 TRCTRACEIDR 12 TRCIDR0 28000ea1 TRCIDR1 4100f400 TRCIDR2 488 TRCIDR8 0 TRCAUTHSTATUS cc
I looked the code, but cannot narrow down the issue. But I think the error might be caused by wrongly handle meta data (in the function cs_etm_info_fill(), file tools/perf/arch/arm/util/cs-etm.c) or the error in auxtrace info handling (in function cs_etm__process_auxtrace_info(), file tools/perf/util/cs-etm.c).
Thanks, Leo Yan
bharat@bharat-OptiPlex-3070:~/trace$
=======
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c index 219c10eb752c..afbdba22eb2a 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c @@ -2076,6 +2076,17 @@ static u32 etmv4_cross_read(const struct device
*dev, u32 offset)
* accessing the ETMv4 trace core registers */ smp_call_function_single(drvdata->cpu, do_smp_cross_read,®,
1);
- /*
- OcteonTx2 h/w reports ETMv4.2 but it supports Ignore Packet
- feature of ETMv4.3, Treat this h/w as ETMv4.3 compatible.
- */
- if ((offset == TRCIDR1) &&
(drvdata->etm_options & CORESIGHT_OPTS_TREAT_ETMv43)) {reg.data &= ~0xF0;reg.data |= 0x30;- }
- return reg.data;
}
diff --git a/drivers/hwtracing/coresight/coresight-etm4x.c b/drivers/hwtracing/coresight/coresight-etm4x.c index 6296837ca00d..ada87018317b 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.c +++ b/drivers/hwtracing/coresight/coresight-etm4x.c @@ -625,6 +625,16 @@ static void etm4_init_arch_data(void *info)
/* base architecture of trace unit */ etmidr1 = readl_relaxed(drvdata->base + TRCIDR1);
- /*
- OcteonTx2 h/w reports ETMv4.2 but it supports Ignore Packet
- feature of ETMv4.3, Treat this h/w as ETMv4.3 compatible.
- */
- if (drvdata->etm_options & CORESIGHT_OPTS_TREAT_ETMv43) {
etmidr1 &= ~0xF0;etmidr1 |= 0x30;- }
- /*
- TRCARCHMIN, bits[7:4] architecture the minor version number
- TRCARCHMAJ, bits[11:8] architecture major versin number @@
-1115,6 +1125,10 @@ static int etm4_probe(struct amba_device *adev,
const struct amba_id *id)
if (drvdata->cpu < 0) return drvdata->cpu;
- /* Enable options for Silicon issues */
- if (id->id == OCTEONTX_CN9XXX_ETM)
drvdata->etm_options = CORESIGHT_OPTS_TREAT_ETMv43;- desc.name = devm_kasprintf(dev, GFP_KERNEL, "etm%d", drvdata-
cpu); if (!desc.name) return -ENOMEM; diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h index 4523f10ddd0f..b16f66be8606 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.h +++ b/drivers/hwtracing/coresight/coresight-etm4x.h @@ -191,6 +191,12 @@ #define ETM_EXLEVEL_NS_HYP BIT(14) #define ETM_EXLEVEL_NS_NA BIT(15)
+/* Marvell OcteonTx CN9xxx ETM HW issues */ +#define CORESIGHT_OPTS_TREAT_ETMv43 (0x1U << 0) /* Treat
ETMv4.2
as ETMv4.3 */
+/* Marvell OcteonTx CN9xxx ETM device */ +#define OCTEONTX_CN9XXX_ETM 0x000cc210
/**
- struct etmv4_config - configuration information related to an ETMv4
- @mode: Controls various modes supported by this ETM.
@@ -287,6 +293,7 @@ struct etmv4_config {
- @csdev: Component vitals needed by the framework.
- @spinlock: Only one at a time pls.
- @mode: This tracer's mode, i.e sysFS, Perf or disabled.
- @etm_options: Bitmask of options to manage ETMv4 Silicon
- issues
- @cpu: The cpu this component is affined to.
- @arch: ETM version number.
- @nr_pe: The number of processing entity available for tracing.
@@ -342,6 +349,7 @@ struct etmv4_drvdata { struct coresight_device *csdev; spinlock_t spinlock; local_t mode;
- u32 etm_options; int cpu; u8 arch; u8 nr_pe;
==============
Thanks -Bharat
-----Original Message----- From: Leo Yan leo.yan@linaro.org Sent: Tuesday, November 5, 2019 4:38 PM To: Bharat Bhushan bbhushan2@marvell.com Cc: Coresight ML coresight@lists.linaro.org Subject: [EXT] Re: perf: user-space trace decoding
External Email
-- Hi Bharat,
On Tue, Nov 05, 2019 at 08:30:25AM +0000, Bharat Bhushan wrote:
> Hi All,
>
> Linux h/w trace decoding is working as expected but user-space > trace decoding not showing sybmols (dump raw trace shows h/w trace info) of user-space application/libraries.
>
> Here is how I am using:
>
> 1) Kernel trace
> => perf record -C 1 -e cs_etm/@tmc_etr1/k taskset 0x2 > uname
>
> => tar czf cs_example.tgz perf.data .debug
>
> Transfer cs_example.tgz on x86 machine, untar, copy .debug > in ~/ (home)
> => perf report --vmlinux=./vmlinux
>
> # Samples: 937 of event 'branches:k'
> # Event count (approx.): 937
> #
> # Children Self Command Shared Object Symbol
> # ........ ........ ....... ................. ..............................................
> #
> 15.58% 15.58% swapper [kernel.kallsyms] [k] __delay
> 8.54% 8.54% swapper [kernel.kallsyms] [k] arch_counter_get_cntpct
> 3.31% 3.31% swapper [kernel.kallsyms] [k] event_sched_in.isra.37
> 2.67% 2.67% swapper [kernel.kallsyms] [k]
group_sched_in
> 2.56% 2.56% swapper [kernel.kallsyms] [k] do_idle
> 2.35% 2.35% swapper [kernel.kallsyms] [k] perf_event_update_userpage
> 2.24% 2.24% swapper [kernel.kallsyms] [k] clocks_calc_mult_shift
>
> 2) User-space trace
>
> => perf record -C 1 -e cs_etm/@tmc_etr1/u taskset 0x2 > uname
I verified this command on DB410c, it can record and report well.
> => tar czf cs_example.tgz perf.data .debug
>
> Transfer cs_example.tgz on x86 machine, untar, copy > .debug in ~/ (home)
> => perf report
> Error:
> The perf.data data has no samples!
> # To display the perf.data header info, please use > --header/--header- only options.
> #
> But if dump raw trace then I can see ISYNC/ATOM packets
One potential reason is perf cannot find the correct DSO file. Could
you use below command to output more verbose log?
# perf --debug verbose=10 report --stdio > perf_report_dbg.log
Maybe can find more hints for this failure.
Thanks,
Leo Yan
> => perf report --dump
> . ... CoreSight ETM Trace data: size 25960 bytes
> Idx:24; ID:12; I_ASYNC : Alignment Synchronisation.
> Idx:36; ID:12; I_TRACE_INFO : Trace Info.; INFO=0x0
> Idx:39; ID:12; I_TRACE_ON : Trace On.
> Idx:40; ID:12; I_CTXT : Context Packet.; Ctxt: > AArch64,EL2, NS; CID=0x00000000;
> Idx:46; ID:12; I_TRACE_ON : Trace On.
> Idx:47; ID:12; I_CTXT : Context Packet.; Ctxt: > AArch64,EL0, NS; CID=0x00000000;
> Idx:53; ID:12; I_ADDR_L_64IS0 : Address, Long, 64 > bit, IS0.; Addr=0x0000FFFF801CCF7C;
> Idx:62; ID:12; I_ATOM_F1 : Atom format 1.; N
> Idx:63; ID:12; I_TIMESTAMP : Timestamp.; Updated val > = 0xede4824ce5
> . . .
>
> Any idea what I am missing.
>
> Thanks
> -Bharat
> _______________________________________________
> CoreSight mailing list
> CoreSight@lists.linaro.org
3A__lists.linaro.org_mailman_listinfo_coresight&d=DwIBAg&c=nKjWec2b6
R0
mOyPaz7xtfQ&r=PAAlWswPe7d8gHlGbCLmy2YezyK7O3Hv_t2heGnouBw&m =
GZeDPhs62scs3v9Csh9IZRB6pw8UMJYewwhrv_U5xpc&s=ezH1P6fe2-
wWc8g-
N02AMoUehx2BriCCEbr9VPmuaDw&e=
CoreSight mailing list CoreSight@lists.linaro.org https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.linaro.org_
mailman_listinfo_coresight&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=P AAlWsw
Pe7d8gHlGbCLmy2YezyK7O3Hv_t2heGnouBw&m=xzqQHPtXHXIolLyGp6WrH nfjOr99WId
oBnEkbzYFcwc&s=jHuu9gSD-
LeOv0dKLThWRZwwcY1_1CgcU5FBtLFxZoc&e=