Hi Leo,
Sorry for the delayed reply.
On Tue, May 18, 2021 at 7:00 AM Leo Yan leo.yan@linaro.org wrote:
Hi Denis,
Performance overhead is indeed higher than with ETR polling patch. Here are some numbers for comparison (measured on browser Speedometer2 benchmark): Runtime overhead of ETM tracing with ETR poll period 100ms is less than 0.5%. Snapshot mode gives 2.1%. With 10ms period I see 4.6% with ETR polling and 22% in snapshot mode.
It's not expected that the snapshot mode causes so big overload. In my head, two factors might cause the overload:
- The perf interaction between the user space and kernel space;
- The data copying from the ETR's buffer to the AUX ring buffer.
Check one thing: what's the buffer size for ETR polling mode and for snapshot mode in your experiments?
AUX buffer size was 64KB in both modes. I used small buffers to keep the overall perf.data size under 200-500MB with a long-running perf record (3-5 min).
If I remember correctly, by default the snapshot mode uses 4MB for ETR buffer, if copying 4MB per 10ms, then it's likely to cause big overload. So at the first glance, the overhead difference might be caused by the by the different buffer size between ETR poll mode and snapshot mode.
As I said, the buffer was small. But I can go ahead and check the difference with a bigger buffer. I will also double check how strobing affects runtime overhead. It should be lower.
We could probably utilize the ETM strobing feature and reduce frequency of data collection but I see a problem when I'm using both. Within a minute of profiling the ETM generates a reasonable profile size (with strobing autofdo,preset=9 with period 0x1000 it is up to 20MB). But then the size grows unproportionally. With a 4 minute run I got a 6.3GB profile.
Just check, as Mathieu has suggested, have you applied the patch [1] on your local code base for fixing the data copying for snapshot mode?
That was my mistake. When I switched to the Strobing patch series I forgot to apply [1]. When applied I don't see this issue any more. It's not obvious from the description that the patch would fix my issue. So it sounds like your patch fixes multiple problems :)
Thanks, Denis
[...]
Thanks, Leo
[1]. https://lists.linaro.org/pipermail/coresight/2021-April/006254.html