On Thu, 19 Sep 2019 at 17:45, Yabin Cui yabinc@google.com wrote:
Thanks for both of your suggestions! I will try return stack and stall control later.
Another problem I found recently is about performance degradation when using ETM:
When running a program with ETM enabled on it using perf interface, ETM records data and sends to ETR, and ETR moves the data to memory. And kernel copies the data from ETR memory to a per-cpu buffer. So I think the performance cost is: ETR uses memory bus to move data, and the kernel uses both cpu and memory bus to copy data. In my experiments, the kernel moving data can take a lot of cpu-cycles, which slows down the running program.
You are correct - double buffering mode is definitely not an optimal way to work but at this time the HW doesn't give us another choice.
When testing a single thread busy loop program, the overhead in kernel is about 11% cpu-cycles (comparing to all cpu-cycles running the program). When testing a two threads busy looping program, the overhead in kernel is about 74% cpu-cycles. In both cases, the overhead in user space is very small, about 0.2% cpu-cycles.
I am surprised at the performance difference between single thread and multi thread programs. I haven't spent much time on this.
I suppose that depends on the mode operation, i.e per-thread of CPU-wide. It is hard for me to comment without more information about how you came up with those metrics.
I will check the accuracy and reason for the difference. Any suggestions or comments are welcomed.
On Thu, Sep 19, 2019 at 5:04 AM Mike Leach mike.leach@linaro.org wrote:
HI Yabin,
A couple of additional suggestions:-
- If available use the return stack - this can remove a number of
address packets from the trace stream. 2) Check that both SYSSTALL and STALLCTL are 1 on IDR3 on your device
- otherwise using stall will not work.
- Ensure that the coresight system is correctly described in the
device tree in respect of possible replicators and TPIU devices on the system. TPIU will ordinarily be disabled, but the bus to the TPIU may go through a downsizer - which can produce backpressure beyond any replicator. The coresight infrastucture should disable the TPIU branch on a correctly described system
Regards
Mike
On Wed, 18 Sep 2019 at 16:35, Mathieu Poirier mathieu.poirier@linaro.org wrote:
Hi Yabin,
On Tue, 17 Sep 2019 at 15:53, Yabin Cui yabinc@google.com wrote:
Hi guys,
I am trying to reduce ETM data lose. There seems to have two types of data lose:
- caused by instruction trace buffer overflow, which generates an Overflow packet after recovering.
- caused by ETR buffer overflow, which generates an PERF_RECORD_AUX with PERF_AUX_FLAG_TRUNCATED.
In practice, the second one is unlikely to happen when I set ETR buffer size to 4M, and can be improved by setting higher buffer size. The first one happens much more frequently, can happen about 21K times when generating 5.3M etm data. So I want to know if there is any way to reduce that.
I found in the ETM arch manual that the overflow seems controlled by TRCSTALLCTLR, a stall control register. But TRCIDR3.NOOVERFLOW is not supported on my experiment device, which seems using qualcomm sdm845. TRCSTALLCTLR.ISTALL seems can be set by writing to mode file in /sys. But I didn't find any way to set TRCSTALLCTLR.LEVEL in linux kernel. So I wonder if it is ever used.
Do you guys have any suggestions?
In order to get a timely response to your questions I advise to CC the coresight mailing list (which I have included). There is a lot of knowledgeable people there that can also help, especially with architecture specific configuration.
TRCSTALLCTLR.LEVEL is currently not accessible to users because we simply never needed to use the feature. If using the ISTALL and LEVEL parameters help with your use case send a patch that exposes the entire TRCSTALLCTLR register (rather than just the LEVEL field) and I'll be happy to fold it in.
Thanks, Mathieu
Best, Yabin
CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK