Hi Al, Suzuki,
On Wed, May 06, 2020 at 10:17:15AM +0100, Suzuki Kuruppassery Poulose wrote:
On 05/05/2020 05:00 PM, Al Grant wrote:
The patch below does the kernel and userspace side but is not complete. The problem is that userspace perf creates the metadata copy of TRCCONFIGR based on its request (and fills in the other id registers by reading sysfs), but the detection of EL2/E2H happens in the kernel which adjusts TRCCONFIGR, and it's this config which is needed for decode. I
see three ways round this:
- have userspace test to see if the kernel is EL2 (somehow) and adjust
the metadata to mirror what the kernel is doing
- have the kernel pass the adjusted TRCCONFIGR back so perf can put it
in the metadata
- have the perf decoder get the thread id from whichever of VMID and
CONTEXTID is available in a PE_CONTEXT element
Obviously, the last is simplest, but it's a bodge, and means that OpenCSD will see VMIDs when its TRCCONFIGR says it won't. It's kind of cleanest to get the real TRCCONFIGR somehow, but how do we do that?
We do get TRCCONFIGR in the perf records. We should simply make sure we get the uptodate value (wherever we are getting it from).
The copy in PERF_RECORD_AUXINFO (which is a synthetic record created by userspace perf) is, I believe, as I said:
"userspace perf creates the metadata copy of TRCCONFIGR based on its request".
So if the kernel modifies it based on information only the kernel knows, there's no current way to get the actual value. That was what I was trying to address with my suggestions.
Have I missed some place the actual TRCCONFIGR is already being returned in other perf records?
The sysfs is one place where we could expose this and this could be then consumed by the perf tool while creating the TRCCONFIGR. Since the EL can't change, this could be a onetime probe activity.
I spent some time to look into this issue, I think we need to note one thing is: since the metadata is captured at the beginning when execute Perf tool, it should be prior to setting TRCCONFIGR in the CoreSight driver when enable perf event. Especially, if we consider for the per-thread case, if the traced task is not migrated on a specific CPU (let's say CPU_a), then CPU_a's TRCCONFIGR will not be set properly until the trace task is migrated to CPU_a.
So we can see in the perf tool it has the function cs_etmv4_get_config(), it doesn't read out any value from sysfs for the register TRCCONFIGR, alternatively it just generates a value for TRCCONFIGR based on perf's 'attr.config'. On the other hand, in the kernel side, it needs to maintain the same logic in the function etm4_parse_event_config(), which also parses 'attr.config' and set into TRCCONFIGR.
For fixing this issue, I think one potential direction is to change the function etm_event_init() in the kernel, we can use it to invoke a function like etm4_parse_event_config(), so this can allow the register TRCCONFIGR to be ready in the initialisation phase. Then, as Suzuki suggested, in the perf tool we can use sysfs node to read register TRCCONFIGR. To be honest, I don't verify this is feasible or not, but from reading the code, looks like this is feasible with below flow:
record__open() `-> evsel__open() `-> evsel__open_cpu() `-> perf_event_open() `-> perf_init_event() `-> perf_try_init_event() `-> etm_event_init() `-> Set TRCCONFIGR
record__synthesize() `-> perf_event__synthesize_auxtrace_info() `-> auxtrace_record__info_fill() `-> cs_etm_info_fill() `-> Use sysfs to read out TRCCONFIGR
Thanks, Leo