From: CoreSight coresight-bounces@lists.linaro.org On Behalf Of Mathieu Poirier Sent: 08 May 2020 16:44 To: leo.yan@linaro.org Cc: Coresight ML coresight@lists.linaro.org Subject: Re: Context id tracing with kernel at EL2
On Fri, 8 May 2020 at 00:11, Leo Yan <leo.yan@linaro.orgmailto:leo.yan@linaro.org> wrote: Hi Al, Suzuki,
On Wed, May 06, 2020 at 10:17:15AM +0100, Suzuki Kuruppassery Poulose wrote:
On 05/05/2020 05:00 PM, Al Grant wrote:
The patch below does the kernel and userspace side but is not complete. The problem is that userspace perf creates the metadata copy of TRCCONFIGR based on its request (and fills in the other id registers by reading sysfs), but the detection of EL2/E2H happens in the kernel which adjusts TRCCONFIGR, and it's this config which is needed for decode. I
see three ways round this:
- have userspace test to see if the kernel is EL2 (somehow) and adjust
the metadata to mirror what the kernel is doing
- have the kernel pass the adjusted TRCCONFIGR back so perf can put it
in the metadata
- have the perf decoder get the thread id from whichever of VMID and
CONTEXTID is available in a PE_CONTEXT element
Obviously, the last is simplest, but it's a bodge, and means that OpenCSD will see VMIDs when its TRCCONFIGR says it won't. It's kind of cleanest to get the real TRCCONFIGR somehow, but how do we do that?
We do get TRCCONFIGR in the perf records. We should simply make sure we get the uptodate value (wherever we are getting it from).
The copy in PERF_RECORD_AUXINFO (which is a synthetic record created by userspace perf) is, I believe, as I said:
"userspace perf creates the metadata copy of TRCCONFIGR based on its request".
So if the kernel modifies it based on information only the kernel knows, there's no current way to get the actual value. That was what I was trying to address with my suggestions.
Have I missed some place the actual TRCCONFIGR is already being returned in other perf records?
The sysfs is one place where we could expose this and this could be then consumed by the perf tool while creating the TRCCONFIGR. Since the EL can't change, this could be a onetime probe activity.
I'm joining this thread on the late and as such don't have all the context. But sysfs is never a good place to export run-time configuration registers as there could be multiple perf sessions active at the same time. This isn't a big issue for N:1 topologies but very real for 1:1.
I agree. Event-specific data (e.g. actual TRCCONFIGR) should be returned through some mechanism specific to the event. For example, we could have a new ioctl() specific to CoreSight events (“return trace config”). Or we could use one of the flag bits in the PERF_RECORD_AUX record. There are 64 bits, of which 4 are currently used as generic flags (PERF_AUX_FLAG_TRUNCATED etc.). If we could reserve some of those as event-specific flags (say PERF_AUX_FLAG_EVENT0..7), then for ETM AUX buffers we could use a flag to indicate if the thread id is in CID or VMID in the trace. That would be my preference.
The suggestion might have been to use sysfs to indicate if the kernel is at EL1 or EL2. That’s not runtime varying, so sysfs should be fine for that. However it then forces perf userspace to make the inference that at EL2 the thread id is always in VMID. That’s currently the case, but it’s the sort of heuristic rule we really don’t want to be pushing into perf.
And if we do that it will need to be captured at “perf record” time and stored in a new field in PERF_RECORD_AUXTRACE_INFO. The advantage of using an AUX flag is that we don’t change any formats – it’s just using a flag in an existing flags word to help us decode traces that are currently undecodeable.
Al
I spent some time to look into this issue, I think we need to note one thing is: since the metadata is captured at the beginning when execute Perf tool, it should be prior to setting TRCCONFIGR in the CoreSight driver when enable perf event. Especially, if we consider for the per-thread case, if the traced task is not migrated on a specific CPU (let's say CPU_a), then CPU_a's TRCCONFIGR will not be set properly until the trace task is migrated to CPU_a.
So we can see in the perf tool it has the function cs_etmv4_get_config(), it doesn't read out any value from sysfs for the register TRCCONFIGR, alternatively it just generates a value for TRCCONFIGR based on perf's 'attr.config'. On the other hand, in the kernel side, it needs to maintain the same logic in the function etm4_parse_event_config(), which also parses 'attr.config' and set into TRCCONFIGR.
For fixing this issue, I think one potential direction is to change the function etm_event_init() in the kernel, we can use it to invoke a function like etm4_parse_event_config(), so this can allow the register TRCCONFIGR to be ready in the initialisation phase. Then, as Suzuki suggested, in the perf tool we can use sysfs node to read register TRCCONFIGR. To be honest, I don't verify this is feasible or not, but from reading the code, looks like this is feasible with below flow:
record__open() `-> evsel__open() `-> evsel__open_cpu() `-> perf_event_open() `-> perf_init_event() `-> perf_try_init_event() `-> etm_event_init() `-> Set TRCCONFIGR
record__synthesize() `-> perf_event__synthesize_auxtrace_info() `-> auxtrace_record__info_fill() `-> cs_etm_info_fill() `-> Use sysfs to read out TRCCONFIGR
Thanks, Leo _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.orgmailto:CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.