From: CoreSight <coresight-bounces@lists.linaro.org>
On Behalf Of Mathieu Poirier
Sent: 08 May 2020 16:44
To: leo.yan@linaro.org
Cc: Coresight ML <coresight@lists.linaro.org>
Subject: Re: Context id tracing with kernel at EL2
On Fri, 8 May 2020 at 00:11, Leo Yan <leo.yan@linaro.org> wrote:
Hi Al, Suzuki,
On Wed, May 06, 2020 at 10:17:15AM +0100, Suzuki Kuruppassery Poulose wrote:
> On 05/05/2020 05:00 PM, Al Grant wrote:
> > > > The patch below does the kernel and userspace side but is not complete.
> > > > The problem is that userspace perf creates the metadata copy of
> > > > TRCCONFIGR based on its request (and fills in the other id registers
> > > > by reading sysfs), but the detection of EL2/E2H happens in the kernel
> > > > which adjusts TRCCONFIGR, and it's this config which is needed for decode. I
> > > see three ways round this:
> > > >
> > > > - have userspace test to see if the kernel is EL2 (somehow) and adjust
> > > > the metadata to mirror what the kernel is doing
> > > >
> > > > - have the kernel pass the adjusted TRCCONFIGR back so perf can put it
> > > > in the metadata
> > > >
> > > > - have the perf decoder get the thread id from whichever of VMID and
> > > > CONTEXTID is available in a PE_CONTEXT element
> > > >
> > > > Obviously, the last is simplest, but it's a bodge, and means that
> > > > OpenCSD will see VMIDs when its TRCCONFIGR says it won't. It's kind of
> > > > cleanest to get the real TRCCONFIGR somehow, but how do we do that?
> > >
> > > We do get TRCCONFIGR in the perf records. We should simply make sure we get
> > > the uptodate value (wherever we are getting it from).
> >
> > The copy in PERF_RECORD_AUXINFO (which is a synthetic record created
> > by userspace perf) is, I believe, as I said:
> >
> > "userspace perf creates the metadata copy of TRCCONFIGR based on
> > its request".
> >
> > So if the kernel modifies it based on information only the kernel knows,
> > there's no current way to get the actual value. That was what I was trying
> > to address with my suggestions.
> >
> > Have I missed some place the actual TRCCONFIGR is already being returned
> > in other perf records?
>
> The sysfs is one place where we could expose this and this could be then
> consumed by the perf tool while creating the TRCCONFIGR. Since the EL
> can't change, this could be a onetime probe activity.
I'm joining this thread on the late and as such don't have all the context. But sysfs is never a good place to export run-time configuration registers as there could be multiple perf sessions active at the same
time. This isn't a big issue for N:1 topologies but very real for 1:1.
I agree. Event-specific data (e.g. actual TRCCONFIGR) should be returned
through some mechanism specific to the event. For example, we could
have a new ioctl() specific to CoreSight events (“return trace config”).
Or we could use one of the flag bits in the PERF_RECORD_AUX record.
There are 64 bits, of which 4 are currently used as generic flags
(PERF_AUX_FLAG_TRUNCATED etc.). If we could reserve some of those as
event-specific flags (say PERF_AUX_FLAG_EVENT0..7), then for ETM AUX
buffers we could use a flag to indicate if the thread id is in CID or VMID
in the trace. That would be my preference.
The suggestion might have been to use sysfs to indicate if the kernel is
at EL1 or EL2. That’s not runtime varying, so sysfs should be fine for that.
However it then forces perf userspace to make the inference that at EL2
the thread id is always in VMID. That’s currently the case, but it’s the
sort of heuristic rule we really don’t want to be pushing into perf.
And if we do that it will need to be captured at “perf record” time and
stored in a new field in PERF_RECORD_AUXTRACE_INFO. The advantage
of using an AUX flag is that we don’t change any formats – it’s just using
a flag in an existing flags word to help us decode traces that are currently
undecodeable.
Al
I spent some time to look into this issue, I think we need to note one
thing is: since the metadata is captured at the beginning when execute
Perf tool, it should be prior to setting TRCCONFIGR in the CoreSight
driver when enable perf event. Especially, if we consider for the
per-thread case, if the traced task is not migrated on a specific CPU
(let's say CPU_a), then CPU_a's TRCCONFIGR will not be set properly
until the trace task is migrated to CPU_a.
So we can see in the perf tool it has the function
cs_etmv4_get_config(), it doesn't read out any value from sysfs
for the register TRCCONFIGR, alternatively it just generates a value
for TRCCONFIGR based on perf's 'attr.config'. On the other hand, in
the kernel side, it needs to maintain the same logic in the function
etm4_parse_event_config(), which also parses 'attr.config' and set
into TRCCONFIGR.
For fixing this issue, I think one potential direction is to change
the function etm_event_init() in the kernel, we can use it to invoke
a function like etm4_parse_event_config(), so this can allow the
register TRCCONFIGR to be ready in the initialisation phase. Then,
as Suzuki suggested, in the perf tool we can use sysfs node to read
register TRCCONFIGR. To be honest, I don't verify this is feasible
or not, but from reading the code, looks like this is feasible with
below flow:
record__open()
`-> evsel__open()
`-> evsel__open_cpu()
`-> perf_event_open()
`-> perf_init_event()
`-> perf_try_init_event()
`-> etm_event_init()
`-> Set TRCCONFIGR
record__synthesize()
`-> perf_event__synthesize_auxtrace_info()
`-> auxtrace_record__info_fill()
`-> cs_etm_info_fill()
`-> Use sysfs to read out TRCCONFIGR
Thanks,
Leo
_______________________________________________
CoreSight mailing list
CoreSight@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/coresight