Hi Al,
On 2020-09-26 01:47, Al Grant wrote:
Hi Al,
Somehow I couldn't find your message in my codeaurora inbox, so copied your message from coresight list.
On 2020-09-25 21:37, Sai Prakash Ranjan wrote: ... <snip>...
In system wide mode we do automatically enable timestamps to allow perf to correlate the trace streams from different cores.
Yes it is enabled by default as you mentioned even without specifying timestamp config for system wide mode.
In the past, 0 or fixed timestamps have been caused by the clock used to drive the timestamp counters not being enabled.
Yes this was the first suspect and so I tried setting CLK_IS_CRITICAL flag so that the clock is not gated ever. So after this, we don't see 0 timestamps but the timestamp freeze (fixed) was still observed. Only thing making a difference was the CPU idle.
There is nothing in the ETM configuration that could cause zero timestamps or frozen timestamps in trace - if CPU idle manages to restore an ETM state that disables timestamps, you won't see timestamp packets at all. We think what's causing zero or frozen timestamps is a zero or frozen timestamp input to the ETM, from the global timestamp distribution. Somehow, CPU idle has either disabled the global timestamp generator, or disabled something in the timestamp distribution network.
Are you able to see the global timestamp generator in the memory map?
I couldn't find it, I'll have to ask around.
If you use csscan.py from https://github.com/ARM-software/CSAL/coresight-tools to scan the ROM table, the timestamp generator might show up as a memory mapped device (or it might not...), if it does, and you can scan it in a zero timestamp situation, you can check if the timestamp generator is enabled.
The ROM table base address printed 0 with the csinfo module in CSAL. If the csscan.py takes rom table base address? then I guess I won't be able to get timestamp generator info. But nice to know such tools exist.
There's almost certainly a ROM table somewhere - if csinfo is showing the base address as 0, it just means the CPU isn't reporting it. You may be able to find it some other way. Generally, if you can find the overall address range allocated to CoreSight blocks, the system ROM table is at the lowest address, and then the table can be used to locate all the other blocks. But every system is different!
Thanks for all this information, very much appreciated, I'll try to collect more details regarding these. BTW looking into etm4 save and restore functions in etm4x driver, I just found that there seems to be a small bug where we save TRCVMIDCCTLR1 also into trcvmidcctlr0 var in etm4_cpu_save() and then write trcvmidcctlr0 into both TRCVMIDCCTLR0 and TRCVMIDCCTLR1 in etm4_cpu_restore(), patch incoming soon. But more surprisingly, after correcting this I am not able to reproduce the timestamp issue which was already very hard to reproduce. I don't understand the relation between these, but I will have to keep an eye on this just in case I am able to reproduce the timestamp issue again.
Thanks, Sai