Cc: Mathieu, Al, Leo Anshuman
Please let me know what you think about the proposal below.
On 09/16/2020 10:19 AM, Suzuki K Poulose wrote:
Hi all,
Resurrecting an old thread. See more below. Short summary:
With the kernel running at EL2, the PID is in CONTEXTIDR_EL2 and the normal CONTEXTID tracing (which uses CONTEXTIDR_EL1) is not useful. The solution is to use VMID (with CONTEXTIDR_EL2 emitted as VMID) for the PID on an EL2 kernel.
This needs to be communicated to the perf tool somehow, so that it can decode the trace appropriately. One of the proposal was to expose the TRCCONFIGR bits for each of the individual event->attr.config options. I have hacked up a patch [0] to do this and here are two ways to expose this :
Option 1: Trace Config exposed matching the format name:
$ pwd /sys/bus/event_source/devices/cs_etm
$ ls trace_config/ contextid cycacc retstack timestamp cs_etm# for f in $(find trace_config -type f) do echo $(basename $f) : $(cat $f) done
timestamp : 0x800 contextid : 0x40 cycacc : 0x10 retstack : 0x1000
Disadvantages : It is really tricky for the perf tool figure out the name of the "config" option from the event->attr.config bits. As the parsing of the format is not exported elsewhere (or I didn't look deep enough)
Adding a bit more clarity here : The perf tool needs to know the name of the "config" option for a given bit position to decode the traceconfig. i.e, if event->attr.config == BIT(12) | BIT(14), The perf tool now needs to go back and look at
/sys/bus/event_sources/../cs_etm/format/
and figure out the name corresponding to BIT(12) and BIT(14). Since the initial conversion of the name to config bit is performed by the perf tool parsing logic, a reverse mapping is not available to cs_etm code. Unless we hard code BIT(12) => cycacc in the perf tool, which is something we are trying to avoid to support the future changes seemlessly.
Option 2: Trace Config exposed as the "bit" positions.
$ ls trace_config/ 12 14 28 29
$ for f in $(find . -type f) do echo $(basename $f) : $(cat $f) done
28 : 0x800 14 : 0x40 12 : 0x10
29 : 0x1000
Advantage: The perf tool can iterate over the event->attr.config bits and do something like:
trconfigr = 0
for_each_set_bit(i, event->attr.config) trconfigr |= sysfs_read_trace_config(i)
where:
sysfs_read_trace_config(int i) { char path[MAX_LEN]; FILE *fp; unsigned long trcfg = 0;
sprintf(path, "%s/%d", $TRACE_CONFIG_PATH, i); fp = fopen(path, "r"); if (fp) fscanf(fp, "%lx", &trcfg); return trcfg; }
Disadvantage: It is not human reader friendly. But it doesn't have to be, as it is for the tools to construct the trcconfigr.
Option 3:
Another completely different option is to fixup the TRCIDR2.CID = 0, while running in VHE, via sysfs. This will indicate that the CONTEXTID tracing is not supported by the ETM, but the VMID support should be visible. This can be used by the perf-tool to detect that the VMID is used as the PID.
Thoughts ?
Suzuki
[0] ---- Expose trace config via sysfs
HACK: coresight: etm4x: Expose trace config via sysfs
Not-yet-signed-off-by: Suzuki K Poulose suzuki.poulose@arm.com
diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c index 9d61a71da96f..58e86267fa37 100644 --- a/drivers/hwtracing/coresight/coresight-etm-perf.c +++ b/drivers/hwtracing/coresight/coresight-etm-perf.c @@ -35,6 +35,55 @@ PMU_FORMAT_ATTR(retstack, "config:" __stringify(ETM_OPT_RETSTK)); /* Sink ID - same for all ETMs */ PMU_FORMAT_ATTR(sinkid, "config2:0-31");
+static struct etm_trace_config_map *etm_cfg_map;
+#define ETM_TRCONFIGR_ATTR(_config) \ + &((struct dev_ext_attribute []) { \ + { \ + __ATTR(_config, S_IRUGO, etm_trace_config_show, NULL), \ + (void *)_config \ + } \ + })[0].attr.attr
+static unsigned long etm_trace_config_map_cfg(unsigned long cfg) +{ + struct etm_trace_config_map *map = etm_cfg_map;
+ if (!map) + return 0;
+ for (; map->cfg; map++) + if (map->cfg == cfg) + return map->trcfg; + return 0; +}
+static ssize_t etm_trace_config_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct dev_ext_attribute *eattr; + unsigned long cfg;
+ eattr = container_of(attr, struct dev_ext_attribute, attr); + cfg = (unsigned long)eattr->var;
+ return snprintf(buf, PAGE_SIZE, "0x%lx\n", etm_trace_config_map_cfg(cfg)); +}
+static struct attribute *etm_pmu_trace_config_attr[] = { + ETM_TRCONFIGR_ATTR(ETM_OPT_CYCACC), + ETM_TRCONFIGR_ATTR(ETM_OPT_CTXTID), + ETM_TRCONFIGR_ATTR(ETM_OPT_TS), + ETM_TRCONFIGR_ATTR(ETM_OPT_RETSTK), + NULL, +};
+static const struct attribute_group etm_pmu_trace_config_group = { + .name = "trace_config", + .attrs = etm_pmu_trace_config_attr, +};
static struct attribute *etm_config_formats_attr[] = { &format_attr_cycacc.attr, &format_attr_contextid.attr, @@ -61,6 +110,7 @@ static const struct attribute_group etm_pmu_sinks_group = { static const struct attribute_group *etm_pmu_attr_groups[] = { &etm_pmu_format_group, &etm_pmu_sinks_group, + &etm_pmu_trace_config_group, NULL, };
@@ -600,6 +650,12 @@ void etm_perf_del_symlink_sink(struct coresight_device *csdev) csdev->ea = NULL; }
+void etm_install_trace_config_map(struct etm_trace_config_map *cfg_map) +{ + if (!etm_cfg_map && cfg_map) + etm_cfg_map = cfg_map; +}
static int __init etm_perf_init(void) { int ret; diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.h b/drivers/hwtracing/coresight/coresight-etm-perf.h index 015213abe00a..01679988debb 100644 --- a/drivers/hwtracing/coresight/coresight-etm-perf.h +++ b/drivers/hwtracing/coresight/coresight-etm-perf.h @@ -57,6 +57,16 @@ struct etm_event_data { struct list_head * __percpu *path; };
+/**
- struct etm_trace_config_map - Map perf config to TRCONFIGR bits for userspace
- @cfg: event->attr.config value
- @trcfg: TRCONFIGR setting for @cfg
- */
+struct etm_trace_config_map { + unsigned long cfg; + unsigned long trcfg; +};
#ifdef CONFIG_CORESIGHT int etm_perf_symlink(struct coresight_device *csdev, bool link); int etm_perf_add_symlink_sink(struct coresight_device *csdev); @@ -69,6 +79,8 @@ static inline void *etm_perf_sink_config(struct perf_output_handle *handle) return data->snk_config; return NULL; }
+void etm_install_trace_config_map(struct etm_trace_config_map *cfg_map); #else static inline int etm_perf_symlink(struct coresight_device *csdev, bool link) { return -EINVAL; } @@ -80,6 +92,7 @@ static inline void *etm_perf_sink_config(struct perf_output_handle *handle) return NULL; }
+static void etm_install_trace_config_map(struct etm_trace_config_map *cfg_map) {} #endif /* CONFIG_CORESIGHT */
#endif diff --git a/drivers/hwtracing/coresight/coresight-etm4x.c b/drivers/hwtracing/coresight/coresight-etm4x.c index 00c9f0bb8b1a..e528afc9a56e 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.c +++ b/drivers/hwtracing/coresight/coresight-etm4x.c @@ -1591,6 +1591,15 @@ static struct amba_driver etm4x_driver = { .id_table = etm4_ids, };
+#define ETM4x_TRACE_CONFIG_MAP_SIZE 5 +static struct etm_trace_config_map etm4x_map[ETM4x_TRACE_CONFIG_MAP_SIZE] = { + { ETM_OPT_CTXTID, 0 }, /* filled in dynamically for ETM_OPT_CTXTID */ + { ETM_OPT_CYCACC, BIT(4) }, + { ETM_OPT_TS, BIT(11) }, + { ETM_OPT_RETSTK, BIT(12) }, + { 0, 0 }, +};
static int __init etm4x_init(void) { int ret; @@ -1605,6 +1614,10 @@ static int __init etm4x_init(void) if (ret) { pr_err("Error registering etm4x driver\n"); etm4_pm_clear(); + } else { + etm4x_map[0].trcfg = is_kernel_in_hyp_mode() ? + (BIT(7) | BIT(15)) : BIT(6); + etm_install_trace_config_map(&etm4x_map[0]); }
return ret;
On 05/27/2020 10:20 AM, Suzuki K Poulose wrote:
On 05/26/2020 05:33 PM, Mike Leach wrote:
Hi,
On Tue, 26 May 2020 at 17:21, Suzuki K Poulose suzuki.poulose@arm.com wrote:
On 05/25/2020 04:56 PM, Mathieu Poirier wrote:
On Fri, 22 May 2020 at 09:30, Suzuki K Poulose suzuki.poulose@arm.com wrote:
On 05/08/2020 04:43 PM, Mathieu Poirier wrote: > > > On Fri, 8 May 2020 at 00:11, Leo Yan <leo.yan@linaro.org
> I'm joining this thread on the late and as such don't have all the context. > But sysfs is never a good place to export run-time configuration registers as > there could be multiple perf sessions active at the same time. This isn't a > big issue for N:1 topologies but very real for 1:1.
I don't see how this affected by the sink topology, as this is an ETM configuration.
By sysfs, I mean, not necessarily under /sys/bus/coreight/.../etmN/
but, under :
/sys/bus/event_source/.../cs_etm/
We already expose various different information there. e.g, nr_addr_filters, sinks/
So we could expose something like :
trcconfigr/
and could describe how each of the individual event configs affect the TRCCONFIGR register. So that the perf tool can construct the TRCCONFIGR register from scratch just by looking at the trcconfigr directory.
e.g,
# cd /sys/bus/event_source/devices/cs_etm/ # for f in $(find format -type f); do echo $(basename $f) - $(cat $f); done cycacc - config:12 sinkid - config2:0-31 contextid - config:14 retstack - config:29 timestamp - config:28
$ for f in $(find trcconfigr -type f); do echo $(basename $f) - $(cat $f); done cycacc - 0x10 // Bit 4 timestamp - 0x800 // Bit 11 contextid - 0x8080 // Bit15 | Bit 7 On EL2 kernels - 0x40 // Bit 6 on EL1 kernels retstack - 0x1000 // Bit12
So those are configurations that apply to specific trace sessions. When dealing with concurrent trace sessions there is no way to guarantee the information exported to sysfs applies to the trace session that would read the information.
I think there is a bit of confusion here; The idea is the perf tool will construct the value of the traceconfigr based on the "options" selected by the user, just like it does now for "config1"/"config2"
i.e,
perf record -e cs_etm/cycacc,timestamp/ blah-blah
perf tool will :
map cycacc -> config1 |= BIT(12) and trcconfigr |= BIT($cat /sys/.../trcconfigr/cycacc)
and similarly for all the options. The values under the "trcconfigr" are static and only maps a given attribute to the traceconfig register bit.
Thus, any session can create the TRCCONFIGR accurately based on the events now, by dynamically looking up under the sysfs, rather than "compile time" static assumption of the the ETM model.
So how does this work when I copy the captured data off my target and try to decode offline?
Just like it works now. There is no change in flow for the perf tool. The only difference, how the perf tool would emit the TRCCONFIGR values in the perf.data samples. Instead of assuming that perf_event.attr.config1.CID => TRCCONFIGR[bit6], use the sysfs exported values to figure out the TRCCONFIGR setting.
Suzuki
CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight