Hi Mike,
Mike Leach wrote:
> Hi,
> On Tue, 21 Jan 2025 at 21:29, V E vincent.ernst@web.de wrote:
> > Hi Mike,
> > Mike Leach wrote:
> > HI Vincent
> > On Mon, 20 Jan 2025 at 09:33, V E vincent.ernst@web.de wrote:
> > Hi Mike,
> > Mike Leach wrote:
> > Hi VIncent
> > On Wed, 15 Jan 2025 at 09:11, V E vincent.ernst@web.de wrote:
> > Hi Mike,
> > I was able to initialise the ETMs by replacing the hex phandle references with labels.
> > For the funnels, I used "arm,coresight-funnel" like specified in the bindings of kernel 4.9, which seems to fix it.
> > When enabling the sinks and sources via sysfs, I get the expected messages in dmesg. There are no connections folders, though.
> > OK thats good. The connections information was added sometime in the
> > kernel 5.x series.
> > So registering the CoreSight devices seems to works, but I guess that there is only limited tracing functionality because the drivers are so old?
> > sysfs tracing should be OK - this is no more than switching on a
> > source to trace into a sink and collect data.
> > I tried collecting the trace data via "dd if=/dev/72030000.etf of=~/trace.bin" after turning the devices on and off again, but all I get is an empty file:
> > 0+1 records in
> > 0+1 records out
> > 64 bytes copied, 0.000981324 s, 65.2 kB/s
> > Do you have an idea what I can do to fix this?
> > In general that would suggest a break in the path between the ETM
> > source and ETF sink - hence no trace reaching the sink, or the CPU
> > source is idle so nothing being generated.
> > Ensure that there is a clear path in your DTS from the chosen source
> > to the sink. When you enable the source, the coresight code will
> > follow the connections looking for an active sink.
> > I am pretty sure that the connections are correct. When I echo 1 into enable_sink of the ETF and enable_source of the ETM, dmesg shows that the path was activated:
> > [ 3171.085440] coresight-tmc 72030000.etf: TMC-ETB/ETF enabled
> > [ 3171.085455] coresight-funnel 72010000.funnel_major: FUNNEL inport 2 enabled
> > [ 3171.085468] coresight-funnel 73010000.funnel_ccplex: FUNNEL inport 1 enabled
> > [ 3171.085662] coresight-etm4x 73440000.etm0: ETM tracing enabled
> > It is ETM - Funnel_CCPLEX - Funnel_Major - ETF as expected. And it also shows that the ETF is read:
> > [ 3716.865028] coresight-tmc 72030000.etf: TMC read start
> > [ 3716.867437] coresight-tmc 72030000.etf: TMC read end
> > Is there probably something else that I have to do before activating the devices or between activation and reading of the ETF?
> > Is it possible that there are target specific things not related to
> default coresight that might need to be done - clocks / power /
> permissions / etc?
I am not sure about that, I couldn't find anything regarding that. The documentation on the Nvidia side is very sparse, basically just the instructions that I linked earlier and some general information on the implemented CoreSight devices in the SoC manual.
I stumbled across two files in the Nvidia kernel sources which seem to be Nvidias own approach to implementing a CoreSight driver (see attachments).
tegra210_ptm.c compiles for kernel 4.9 after a few small adjustments, but I am still looking if there is a way to use it.
Nvidia mentions some of the registers defined in the two files in the SoC manual (I can provide it if you want to have a look at it), so it might be possible that they stuck to their own implementation.
Could it be that the standard CoreSight drivers do not work because Nvidia implements their own one?
> > Where you may suffer is if you try to use perf. We have made a lot of
> > enhancements and bugfixes on how perf processes trace data since 4.9
> > Do you know if OpenCSD is already supported for my perf version? I was able to get some perf.data, but ptm2human does not seem to work. I am not that comfortable with these tools yet, though.
> > As far as I know, ptm2human will only work on raw trace buffers - i.e.
> > those from the sysfs dumps above. perf.data contains a whole lot of
> > records, only a few of which are related to trace.
> > You only need OpenCSD for decode. Generally I ensure that my target
> > version of perf supports the cs_etm device for collection, and build a
> > host version of perf with OpenCSD for off target decode.
> > I would like to do on-target tracing (preferable in real-time), so this is not a solution for me, unfortunately. Do you think this is possible?
> > The solution above is on target capture, with off target decode. The
> reason I generally do this is that trace files tend to be very large
> and decode is processor intensive.
> If you build a version of on target perf that contains OpenCSD, then
> on target decode is possible.
> Not quite sure by what you mean by "real-time" here, but the rate at
> which trace is generated will always far exceed the rate at which it
> can be decoded.
With real-time I mean (permanent) collecting, decoding and analyzing of the trace on the target while the CoreSight devices keep running.
> > There are
> > options in the perf makefiles to pull in the OpenCSD library - this is
> > not done by default so you will probably have to build your own
> > version of perf - at least for decode.
> > Is this another approach to using OpenCSD than the one explained in the GitHub HOWTO?
> No, this is the same approach. The HowTo describes compiling perf with
> OpenCSD included as a library - though I probably need to check that
> the method is up to date.
It would be really nice to know if it is. The sample trace bundle from section "Trace Decoding with Perf Report", however, is not (I get a 404 when trying to download the .tgz).
> Using perf to decode the trace is generally the only way to decode
> trace captured using perf, as the perf.data file is a perf specific
> format, which importantly contains additional records that associate
> running program images with captured trace - the only way to get full
> decode.
I followed the build steps in the howto, but it seems like kernel v4.9 perf is too old to support OpenCSD.
The build completes without errors, but OpenCSD is nowhere to be found.
> Tools like ptm2human will only tell you the trace packets generated.
> See the docs in OpenCSD and the ETM trm for more detail
You mean the E/N-Atoms etc., right?
Best regards,
Vincent
> Regards
> Mike
> > Thanks for the help
> > Vincent
> > Regards
> > Mike
> > Unfortunately, Nvidia's Linux for my Jetson Nano ("Linux for Tegra") only supports kernel 4.9.
> > I guessed this might be the case
> > There exist community solutions for getting a version 5 kernel running (on Ubuntu), but I would prefer keeping the standard Linux for Tegra if possible.
> > What surprises me is that ...0000.etf/status (shown here [1]) does not exist on my device and I also couldn't find it in the ABI documentation. Do you have an idea what that is?
> > This looks like a printout of many of the ETF registers in a single
> > sysfs file. e.g. RRP, RWP, STATUS etc.
> > Having multiple output lines for a single sysfs file is something that
> > the upstream maintainers frown upon so has likely been dropped. You
> > should still be able to see the same output by reading each individual
> > register
> > Yes, this is the case, thanks for the info.
> > Kind regards,
> > Vincent
> > Regards
> > Mike
> > Best regards,
> > Vincent
> > [1] https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3275/index.html#pag...
> > _______________________________________________
> > CoreSight mailing list -- coresight@lists.linaro.org
> > To unsubscribe send an email to coresight-leave@lists.linaro.org
> > --
> > Mike Leach
> > Principal Engineer, ARM Ltd.
> > Manchester Design Centre. UK
> > CoreSight mailing list -- coresight@lists.linaro.org
> > To unsubscribe send an email to coresight-leave@lists.linaro.org
> > --
> > Mike Leach
> > Principal Engineer, ARM Ltd.
> > Manchester Design Centre. UK
> >
> > CoreSight mailing list -- coresight@lists.linaro.org
> > To unsubscribe send an email to coresight-leave@lists.linaro.org
> > --
> Mike Leach
> Principal Engineer, ARM Ltd.
> Manchester Design Centre. UK
From: Mike Leach <mike.leach@linaro.org>
Sent: Wednesday, January 22, 2025 15:52
To: V E <vincent.ernst@web.de>
Cc: coresight@lists.linaro.org <coresight@lists.linaro.org>
Subject: Re: CoreSight on Nvidia Jetson
Hi,
On Tue, 21 Jan 2025 at 21:29, V E <vincent.ernst@web.de> wrote:
>
> Hi Mike,
>
> Mike Leach wrote:
> > HI Vincent
> > On Mon, 20 Jan 2025 at 09:33, V E vincent.ernst@web.de wrote:
> > > Hi Mike,
> > > Mike Leach wrote:
> > > Hi VIncent
> > > On Wed, 15 Jan 2025 at 09:11, V E vincent.ernst@web.de wrote:
> > > Hi Mike,
> > > I was able to initialise the ETMs by replacing the hex phandle references with labels.
> > > For the funnels, I used "arm,coresight-funnel" like specified in the bindings of kernel 4.9, which seems to fix it.
> > > When enabling the sinks and sources via sysfs, I get the expected messages in dmesg. There are no connections folders, though.
> > > OK thats good. The connections information was added sometime in the
> > > kernel 5.x series.
> > > So registering the CoreSight devices seems to works, but I guess that there is only limited tracing functionality because the drivers are so old?
> > > sysfs tracing should be OK - this is no more than switching on a
> > > source to trace into a sink and collect data.
> > > I tried collecting the trace data via "dd if=/dev/72030000.etf of=~/trace.bin" after turning the devices on and off again, but all I get is an empty file:
> > > 0+1 records in
> > > 0+1 records out
> > > 64 bytes copied, 0.000981324 s, 65.2 kB/s
> > > Do you have an idea what I can do to fix this?
> > > In general that would suggest a break in the path between the ETM
> > source and ETF sink - hence no trace reaching the sink, or the CPU
> > source is idle so nothing being generated.
> > Ensure that there is a clear path in your DTS from the chosen source
> > to the sink. When you enable the source, the coresight code will
> > follow the connections looking for an active sink.
>
> I am pretty sure that the connections are correct. When I echo 1 into enable_sink of the ETF and enable_source of the ETM, dmesg shows that the path was activated:
>
> [ 3171.085440] coresight-tmc 72030000.etf: TMC-ETB/ETF enabled
> [ 3171.085455] coresight-funnel 72010000.funnel_major: FUNNEL inport 2 enabled
> [ 3171.085468] coresight-funnel 73010000.funnel_ccplex: FUNNEL inport 1 enabled
> [ 3171.085662] coresight-etm4x 73440000.etm0: ETM tracing enabled
>
> It is ETM - Funnel_CCPLEX - Funnel_Major - ETF as expected. And it also shows that the ETF is read:
>
> [ 3716.865028] coresight-tmc 72030000.etf: TMC read start
> [ 3716.867437] coresight-tmc 72030000.etf: TMC read end
>
> Is there probably something else that I have to do before activating the devices or between activation and reading of the ETF?
>
Is it possible that there are target specific things not related to
default coresight that might need to be done - clocks / power /
permissions / etc?
> > > Where you may suffer is if you try to use perf. We have made a lot of
> > > enhancements and bugfixes on how perf processes trace data since 4.9
> > > Do you know if OpenCSD is already supported for my perf version? I was able to get some perf.data, but ptm2human does not seem to work. I am not that comfortable with these tools yet, though.
> > > As far as I know, ptm2human will only work on raw trace buffers - i.e.
> > those from the sysfs dumps above. perf.data contains a whole lot of
> > records, only a few of which are related to trace.
> > You only need OpenCSD for decode. Generally I ensure that my target
> > version of perf supports the cs_etm device for collection, and build a
> > host version of perf with OpenCSD for off target decode.
>
> I would like to do on-target tracing (preferable in real-time), so this is not a solution for me, unfortunately. Do you think this is possible?
The solution above is on target capture, with off target decode. The
reason I generally do this is that trace files tend to be very large
and decode is processor intensive.
If you build a version of on target perf that contains OpenCSD, then
on target decode is possible.
Not quite sure by what you mean by "real-time" here, but the rate at
which trace is generated will always far exceed the rate at which it
can be decoded.
>
> > There are
> > options in the perf makefiles to pull in the OpenCSD library - this is
> > not done by default so you will probably have to build your own
> > version of perf - at least for decode.
>
> Is this another approach to using OpenCSD than the one explained in the GitHub HOWTO?
>
No, this is the same approach. The HowTo describes compiling perf with
OpenCSD included as a library - though I probably need to check that
the method is up to date.
Using perf to decode the trace is generally the only way to decode
trace captured using perf, as the perf.data file is a perf specific
format, which importantly contains additional records that associate
running program images with captured trace - the only way to get full
decode.
Tools like ptm2human will only tell you the trace packets generated.
See the docs in OpenCSD and the ETM trm for more detail
Regards
Mike
> Thanks for the help
>
> Vincent
>
> > Regards
> > Mike
> > > Unfortunately, Nvidia's Linux for my Jetson Nano ("Linux for Tegra") only supports kernel 4.9.
> > > I guessed this might be the case
> > > There exist community solutions for getting a version 5 kernel running (on Ubuntu), but I would prefer keeping the standard Linux for Tegra if possible.
> > > What surprises me is that ...0000.etf/status (shown here [1]) does not exist on my device and I also couldn't find it in the ABI documentation. Do you have an idea what that is?
> > > This looks like a printout of many of the ETF registers in a single
> > > sysfs file. e.g. RRP, RWP, STATUS etc.
> > > Having multiple output lines for a single sysfs file is something that
> > > the upstream maintainers frown upon so has likely been dropped. You
> > > should still be able to see the same output by reading each individual
> > > register
> > > Yes, this is the case, thanks for the info.
> > > Kind regards,
> > > Vincent
> > > Regards
> > > Mike
> > > Best regards,
> > > Vincent
> > > [1]
https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3275/index.html#pag...
> > > _______________________________________________
> > > CoreSight mailing list -- coresight@lists.linaro.org
> > > To unsubscribe send an email to coresight-leave@lists.linaro.org
> > > --
> > > Mike Leach
> > > Principal Engineer, ARM Ltd.
> > > Manchester Design Centre. UK
> > >
> > > CoreSight mailing list -- coresight@lists.linaro.org
> > > To unsubscribe send an email to coresight-leave@lists.linaro.org
> > > --
> > Mike Leach
> > Principal Engineer, ARM Ltd.
> > Manchester Design Centre. UK
> _______________________________________________
> CoreSight mailing list -- coresight@lists.linaro.org
> To unsubscribe send an email to coresight-leave@lists.linaro.org
--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK