Dear Sir,

Please look at inline comments for my reply.

My apologies for such a late reply.


From: Leo Yan <leo.yan@linaro.org>
Sent: 10 June 2019 17:32
To: Student - Ng Yi Zher Jeremy
Cc: Mathieu Poirier; Suzuki K Poulose; Coresight ML
Subject: Re: Request for Guidance in Using sysFS for Coresight in HiKey960 (Cortex A53/A73)
 
Hi Jeremy,

Please see several quick inline comments.

On Mon, Jun 10, 2019 at 08:49:48AM +0000, Student - Ng Yi Zher Jeremy wrote:

[...]

> ```bash
> -rw-r--r-- 1 root root 4096 1970-01-01 00:11 ecc40000.etm/addr_start
> hikey960:/sys/bus/coresight/devices # echo d10583ff > ecc40000.etm/addr_start
> 1|hikey960:/sys/bus/coresight/devices # cat ecc40000.etm/addr_start
> cat: ecc40000.etm/addr_start: Operation not permitted
> ```

After read the code, I think you need firstly to specify addr_idx:

  echo 3 > ecc40000.etm/addr_idx
  echo d10583ff > ecc40000.etm/addr_start
  cat ecc40000.etm/addr_start
I did as per your instruction. I was wondering if it is normal that once I
specified a value for addr_start, I am unable to insert a value into addr_stop.

An example of the snippet is found here:

hikey960: # echo 3 > addr_idx
hikey960: # cat addr*
addr_context:      0x0
addr_ctxtype:      none
addr_idx:          0x3
addr_instdatatype: instr
cat: addr_range:   Operation not permitted
addr_single:       0x0
addr_start:        0x0
addr_stop:         0x0

hikey960: # echo 0x55555557b0 > addr_start
hikey960: # cat addr*
addr_context:      0x0
addr_ctxtype:      none
addr_idx:          0x3
addr_instdatatype:   instr
cat: addr_range:   Operation not permitted
addr_single:       Operation not permitted
addr_start:        0x55555557b0
addr_stop:         Operation not permitted

> I thought it was not important that the Operation was 'not permitted'. Nonetheless, I continued with the tutorial as depicted [here](https://github.com/torvalds/linux/blob/master/Documentation/trace/coresight.txt). One thing I will like to note is that my `rwp` did move when i initially activated the sink and source. However, the 2nd time i checked the `rwp`, it went back `0x0`, the same position as that of my `rrp`.
>
> Regardless, I run the program in another kernel, performed CPU affinity to the program to point at the CPU in which the ETM is activated (in this case, CPU0). While the program is running, i looked at the `rwp/rrp` and see that neither of the pointers are moving. I thought perhaps if i perform `dd` operation, I might be able to get something.
>
> I performed `dd if=/dev/ec036000.etf of=/data/cs.bin bs=1M` but my device crashed and reboot itself without saving any files.

Could you try below commands before dd command?

  echo 1 > /sys/bus/coresight/devices/ec036000.etf
  # Enable ETM for CPU0
  echo 1 > /sys/bus/coresight/devices/ecc40000.etm/enable_source
  dd if=/dev/ec036000.etf of=/data/cs.bin bs=1M

Please note, you need firstly configure the address range, and then
enable the ETM and sink.  If afterwards change the address range
again, you need disable etf/etm and re-enable etf/etm so can set
address range configuration into registers properly.
I have also tried your above suggestion. Currently, I find
addr_start and addr_stop traces to better fit what I want rather
than addr_range. addr_start and addr_stop will be able to
capture traces of what the system perform in terms of context
switches, what syscalls it makes, etc.

Regardless, I have tried addr_range and I was able to
acquire some traces from my program. I will like to share
with you some of my findings because I find the result of
the traces to be peculiar in what it has chosen to trace and
ignore.

I have switched off ASLR, so I am under the assumption
that the instruction address from trace dumps should
be similar to that of GDB


=== simple_math_function1.c ===
void math(int *i){
    (*i)++;
    (*i)++;
    (*i)++;
    (*i)++;
}

main(int i){
    // many function calls, loop once
    while (i <= 20){
            math(&i);
            math(&i);
            math(&i);
            math(&i);
            math(&i);

            printf("%d\n",i);
    }
}
=======================

this is a snippet of my decoded trace log:
    Address - Instruction address 0x0000005555555cf4, Instruction set Aarch32 (Thumb)
    Address - Instruction address 0x0000005555555cfc, Instruction set Aarch32 (Thumb)
    Address - Instruction address 0x0000005555555d04, Instruction set Aarch32 (Thumb)
    Address - Instruction address 0x0000005555555d0c, Instruction set Aarch32 (Thumb)
    Address - Instruction address 0x0000005555555d14, Instruction set Aarch32 (Thumb)

compared against GDB:
    |0x5555555cf4 <main+972>         ldr    x0, [sp,#64]
    |0x5555555cf8 <main+976>         bl     0x55555558d8 <math>
    |0x5555555cfc <main+980>         ldr    x0, [sp,#64]
    |0x5555555d00 <main+984>         bl     0x55555558d8 <math>
    |0x5555555d04 <main+988>         ldr    x0, [sp,#64]
    |0x5555555d08 <main+992>         bl     0x55555558d8 <math>
    |0x5555555d0c <main+996>         ldr    x0, [sp,#64]
    |0x5555555d10 <main+1000>        bl     0x55555558d8 <math>
    |0x5555555d14 <main+1004>        ldr    x0, [sp,#64]

comparing the gdb and trace dumps, it seems to only trace ldr and ignore bl instructions.

this is the 2nd program, just a slight tweak from the above
program:

=== simple_math_function2.c ===
void math(int *i){
    (*i)++;
    (*i)++;
    (*i)++;
    (*i)++;
}

main(int i){
    // 1 function call, looped many times
    while (i <= 20){
            math(&i);
            printf("%d\n",i);
    }
}
=======================
trace dump:

Address - Instruction address 0x0000005555555b9c, Instruction set Aarch32 (Thumb)
ATOM - E
Commit - 1
Address - Instruction address 0x0000005555555b9c, Instruction set Aarch32 (Thumb)
ATOM - E
Commit - 1
Address - Instruction address 0x0000005555555b9c, Instruction set Aarch32 (Thumb)
ATOM - E
Commit - 1
Address - Instruction address 0x0000005555555b9c, Instruction set Aarch32 (Thumb)
ATOM - E
Commit - 1
Address - Instruction address 0x0000005555555b9c, Instruction set Aarch32 (Thumb)
ATOM - E
Commit - 1

gdb log:

|0x5555555b94 <main+620> ldr    x0, [sp,#64]
|0x5555555b98 <main+624> bl     0x55555558d8 <math>
|0x5555555b9c <main+628> b      0x5555555b78 <main+592>
|0x5555555ba0 <main+632> ldr    x8, [sp,#64]

This time, i notice that the etm only trace the `b`
instructions but not the `ldr` instructions.

I am rather puzzled by the outcome of the traces as I could not understand why the result might be different for a somewhat similar code...

I don't try at my side, if have any question or issue, please report
back so I can test on Hikey960.

Thanks,
Leo Yan
Thank you very much for your kindness and assistance the
last time!

Also, one last question: I will like to ask how I might be
able to activate global timestamp for instruction trace? i
noticed trcidr0 supports timestamp (tssize of 64 bits).
However, trcconfigr have disabled all my options. Is there
a workaround this?

Since perf is still not supported in android, i have been
using ptm2human to decode etm dumps. I am also
wondering if there are other decoders that I have not
explored? From what I have seen, DS-5 seems to request
DSTREAM to activate ETM trace and decode the trace
dumps. I tried to do it without the DSTREAM and DS-5 was
unable to recognise the trace dumps.

Any help will be greatly appreciated!

Sincerely,
Jeremy

The above message may contain confidential and/or proprietary information that is exempt from disclosure under applicable law and is intended for receipt and use solely by the addressee(s) named above. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this message, or any attachment, is strictly prohibited. If you have received this email in error, please inform the sender immediately by reply e-mail or telephone, reversing the charge if necessary. Please delete the message thereafter. Thank you.