On Feb 19, 2021, at 17:11, Mike Leach <mike.leach@linaro.org> wrote:

Hi Daniel,

The CoreSight mailing list is the best place to ask these questions -
I've added it to the to: section of this mail.

On Fri, 19 Feb 2021 at 13:43, Daniel Su <daniel.sun.su@gmail.com> wrote:

Sorry I made a mistake, I need to switch those bits but those aren't in the ETM mode sysFS parameter.

On Feb 19, 2021, at 14:21, Daniel Su <daniel.sun.su@gmail.com> wrote:

Thanks,

To just give a bit of background I am doing an internship to improve tracing and debugging for secure world applications in TrustZone. Originally the idea was to come up with a software solution but I thought using the hardware tracing features might be very powerful as the secure world trusted execution environments typically have less debugging and tracing functionalities built into the trusted operating system.

I have been doing further reading and I believe I need to configure the ETM mode sysFS parameter.

To switch these bits:

/* secure state access levels - TRCACATRn */
#define ETM_EXLEVEL_S_APP             BIT(8)
#define ETM_EXLEVEL_S_OS              BIT(9)
#define ETM_EXLEVEL_S_HYP             BIT(10)
#define ETM_EXLEVEL_S_MON             BIT(11)
/* non-secure state access levels - TRCACATRn */
#define ETM_EXLEVEL_NS_APP            BIT(12)
#define ETM_EXLEVEL_NS_OS             BIT(13)
#define ETM_EXLEVEL_NS_HYP            BIT(14)
#define ETM_EXLEVEL_NS_NA             BIT(15)

ns_exlevel_vinst and s_exlevel_vinst control these bits in the TRCVICTLR.

addr_exlevel_s_ns  sets both S and NS bits in the TRCACATRn register
currently addressed by addr_idx.

In the kernel tree -
Documentation/trace/coresight/coresight-etm4x-reference.rst - has lots
of information on programming these sysfs files.

I would advise setting mode first - which will set up some default
values for TRCVITLR, then make any further adjustments you want.

Right now I am able to capture the traces and read them out thanks to Leo Yan's command examples and presentation.

I still need to figure out how to only trace branching instructions, and also how to best decode the traces. Right now even though I can read out /dev/ec036000.etf , I am unsure how to best go about interpreting the raw trace data.

Hardware trace will trace everything - subject to certain filters -
such as address filtering, exception filtering etc.
It is not possible to trace only branch instructions. You will need to
capture trace, decode then run an analysis on that decode to determine
what branches have been taken.
ETMv4 is program flow trace - so all branches are marked as taken or not taken.
Consider the program:-

===============
0x1000   start: < some code >
...
0x1100            B func1

0x1200 func1: <some code>
....
0x1240           mov r0, &func2
                      B r0

0x2000 func2:  <some code>
================

This will result in trace as follows:-
TRACE_ON
ADDR(0x1000)
ATOM(E)

I believe ptm2human and OpenCSD are two open source libraries that allow for decoding of the ETMv4 trace data. I understand there are paid solutions as well such as the DS-5 and Trace32 from Lauterbach to decode.

Do you have any recommendations on other decoders or any tips in general as to how to decode the raw trace data into human readable format?

Best,
Daniel

On Feb 18, 2021, at 18:38, Suzuki K Poulose <suzuki.poulose@arm.com> wrote:

Hi Daniel

On 2/18/21 4:12 PM, Daniel Su wrote:

Hello,
First of all I want to say thanks for your work on the Coresight support as well as the presentation slides. Currently I am working with the Hikey960 which I see is supported and has been used as a reference platform in your slides.
I believe it should be possible to setup self-hosted Coresight to trace only for branch instructions in the secure world. Currently I am investigating how to set this up on the Hikey960 without access to a hardware debugger.
I am wondering if any of you have done this before, if there is additional documentation, or if there are any good support groups/chats/channels to ask questions. I have some experience with Coresight mainly from reading the ARM reference manuals but it is still a lot to parse.

I haven't done anything similar. You should be able to imitate how the
CoreSight drivers program the components ( ETM, Funnels, Replicator and ETR).

For ETM, you just need to make sure you don't filter out the Secure EL1/EL2/EL0.
You would need to read the manuals for the components or user the kernel driver
as the reference.

Cheers
Suzuki

I believe in Linux I should be using `ns_exlevel_vinst` to filter out Non-Secure instructions. Also since the Hikey960 is multicore, I guess I would need to configure the top level Coresight in order to trace all cores? Would this be the ETR then?
Best,
Daniel

--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK