This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
Add a 'principal sink' flag to the CoreSight device structure and the routine to search for it.
In cases where no sink is defined, the coresight_find_principal_sink routine can search from a given source, through the child connections until a sink marked as a principal sink is located.
This allows for default sink to be discovered were none is specified (e.g. perf command line)
Signed-off-by: Mike Leach mike.leach@linaro.org --- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight.c | 34 ++++++++++++++++++++ include/linux/coresight.h | 7 ++++ 3 files changed, 43 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-priv.h b/drivers/hwtracing/coresight/coresight-priv.h index 890f9a5c97c6..5cce7a2a8125 100644 --- a/drivers/hwtracing/coresight/coresight-priv.h +++ b/drivers/hwtracing/coresight/coresight-priv.h @@ -150,6 +150,8 @@ int coresight_enable_path(struct list_head *path, u32 mode, void *sink_data); struct coresight_device *coresight_get_sink(struct list_head *path); struct coresight_device *coresight_get_enabled_sink(bool reset); struct coresight_device *coresight_get_sink_by_id(u32 id); +struct coresight_device * +coresight_find_principal_sink(struct coresight_device *csdev); struct list_head *coresight_build_path(struct coresight_device *csdev, struct coresight_device *sink); void coresight_release_path(struct list_head *path); diff --git a/drivers/hwtracing/coresight/coresight.c b/drivers/hwtracing/coresight/coresight.c index c71553c09f8e..9a25bceb4766 100644 --- a/drivers/hwtracing/coresight/coresight.c +++ b/drivers/hwtracing/coresight/coresight.c @@ -769,6 +769,40 @@ void coresight_release_path(struct list_head *path) path = NULL; }
+/** + * coresight_get_principal_sink - walk trace connections from source to find + * a principal sink. + * @csdev: source / current device to check. + * + * Checks if the current device is flagged as a principal sink, if not + * recursively checks the children of the device. + * + * This will walk the connection path from a source (etm) till a flagged + * sink is encountered and return that sink to the original caller. + */ +struct coresight_device * +coresight_find_principal_sink(struct coresight_device *csdev) +{ + int i; + + /* A default sink has been found. */ + if (csdev->principal_sink) + return csdev; + + /* Not a sink - recursively explore each port found on this element */ + for (i = 0; i < csdev->pdata->nr_outport; i++) { + struct coresight_device *child_dev, *sink; + + child_dev = csdev->pdata->conns[i].child_dev; + if (child_dev) { + sink = coresight_find_principal_sink(child_dev); + if (sink) + return sink; + } + } + return NULL; +} + /** coresight_validate_source - make sure a source has the right credentials * @csdev: the device structure for a source. * @function: the function this was called from. diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 193cc9dbf448..e361081e5695 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -163,6 +163,11 @@ struct coresight_connection { * activated but not yet enabled. Enabling for a _sink_ * appens when a source has been selected for that it. * @ea: Device attribute for sink representation under PMU directory. + * @principal_sink: 'true' if this sink is the principle sink on a trace path + * from a source. Each source can have a single principle sink on + * a path, which can be chosen by perf if no other sink is + * specified. A device can have multiple principle sinks in + * topologies were not all sources are routed to a single sink. * @ect_dev: Associated cross trigger device. Not part of the trace data * path or connections. */ @@ -180,6 +185,8 @@ struct coresight_device { struct dev_ext_attribute *ea; /* cross trigger handling */ struct coresight_device *ect_dev; + /* mark as a principal sink for a trace path */ + bool principal_sink; };
/*
Add detection of the principal sink fwnode attribute to mark a sink instance as the principal sink for a trace path.
Signed-off-by: Mike Leach mike.leach@linaro.org --- drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++++++ 1 file changed, 9 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-tmc.c b/drivers/hwtracing/coresight/coresight-tmc.c index 1cf82fa58289..b9a8e6815915 100644 --- a/drivers/hwtracing/coresight/coresight-tmc.c +++ b/drivers/hwtracing/coresight/coresight-tmc.c @@ -31,6 +31,9 @@ DEFINE_CORESIGHT_DEVLIST(etb_devs, "tmc_etb"); DEFINE_CORESIGHT_DEVLIST(etf_devs, "tmc_etf"); DEFINE_CORESIGHT_DEVLIST(etr_devs, "tmc_etr");
+/* Property keywords for principal sink */ +#define TMC_PROP_PRINCIPAL_SINK "arm,principal-sink" + void tmc_wait_for_tmcready(struct tmc_drvdata *drvdata) { /* Ensure formatter, unformatter and hardware fifo are empty */ @@ -372,6 +375,11 @@ static inline bool tmc_etr_can_use_sg(struct device *dev) return fwnode_property_present(dev->fwnode, "arm,scatter-gather"); }
+static inline bool tmc_is_principal_sink(struct device *dev) +{ + return fwnode_property_present(dev->fwnode, TMC_PROP_PRINCIPAL_SINK); +} + static inline bool tmc_etr_has_non_secure_access(struct tmc_drvdata *drvdata) { u32 auth = readl_relaxed(drvdata->base + TMC_AUTHSTATUS); @@ -525,6 +533,7 @@ static int tmc_probe(struct amba_device *adev, const struct amba_id *id) ret = PTR_ERR(drvdata->csdev); goto out; } + drvdata->csdev->principal_sink = tmc_is_principal_sink(dev);
drvdata->miscdev.name = desc.name; drvdata->miscdev.minor = MISC_DYNAMIC_MINOR;
Add default sink selection to the perf trace handling in the etm driver. Uses the principal sink infrastructure to select a sink for the perf session, if no other sink is specified.
Signed-off-by: Mike Leach mike.leach@linaro.org --- .../hwtracing/coresight/coresight-etm-perf.c | 20 ++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c index 84f1dcb69827..fd369a2fbce7 100644 --- a/drivers/hwtracing/coresight/coresight-etm-perf.c +++ b/drivers/hwtracing/coresight/coresight-etm-perf.c @@ -226,9 +226,6 @@ static void *etm_setup_aux(struct perf_event *event, void **pages, sink = coresight_get_enabled_sink(true); }
- if (!sink) - goto err; - mask = &event_data->mask;
/* @@ -253,6 +250,19 @@ static void *etm_setup_aux(struct perf_event *event, void **pages, continue; }
+ /* + * No sink provided or found - now search the path from the + * etm to find a principal sink to use as default. + */ + if (!sink) { + sink = coresight_find_principal_sink(csdev); + if (!sink) { + /* couldn't find a sink - remove this cpu */ + cpumask_clear_cpu(cpu, mask); + continue; + } + } + /* * Building a path doesn't enable it, it simply builds a * list of devices from source to sink that can be @@ -267,6 +277,10 @@ static void *etm_setup_aux(struct perf_event *event, void **pages, *etm_event_cpu_path_ptr(event_data, cpu) = path; }
+ /* no sink found - cannot trace */ + if (!sink) + goto err; + /* If we don't have any CPUs ready for tracing, abort */ cpu = cpumask_first(mask); if (cpu >= nr_cpu_ids)
Adjust the handling of the session sink selection to allow no sink to be selected on the command line. This then forwards the sink selection to the CoreSight infrastructure which will attempt to select a sink based on the principal sink labelling.
Signed-off-by: Mike Leach mike.leach@linaro.org --- tools/perf/arch/arm/util/cs-etm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c index 941f814820b8..ed9ea2c60f27 100644 --- a/tools/perf/arch/arm/util/cs-etm.c +++ b/tools/perf/arch/arm/util/cs-etm.c @@ -242,10 +242,10 @@ static int cs_etm_set_sink_attr(struct perf_pmu *pmu, }
/* - * No sink was provided on the command line - for _now_ treat - * this as an error. + * No sink was provided on the command line - allow the CoreSight + * system to look for a default */ - return ret; + return 0; }
static int cs_etm_recording_options(struct auxtrace_record *itr,
On Thu, Apr 16, 2020 at 04:39:23PM +0100, Mike Leach wrote:
Adjust the handling of the session sink selection to allow no sink to be selected on the command line. This then forwards the sink selection to the CoreSight infrastructure which will attempt to select a sink based on the principal sink labelling.
Signed-off-by: Mike Leach mike.leach@linaro.org
tools/perf/arch/arm/util/cs-etm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c index 941f814820b8..ed9ea2c60f27 100644 --- a/tools/perf/arch/arm/util/cs-etm.c +++ b/tools/perf/arch/arm/util/cs-etm.c @@ -242,10 +242,10 @@ static int cs_etm_set_sink_attr(struct perf_pmu *pmu, } /*
* No sink was provided on the command line - for _now_ treat* this as an error.
* No sink was provided on the command line - allow the CoreSight* system to look for a default
- return ret;
- return 0;
I understand this patchset is a proof of concept. When we reach a consensus on how to move forward and you are ready for a respin, please do two sets - one for kernel changes and another one for the perf tools. Normally changes to the kernel need to be merged first.
Regards, Mathieu
} static int cs_etm_recording_options(struct auxtrace_record *itr, -- 2.17.1
On Tue, 21 Apr 2020 at 18:43, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Thu, Apr 16, 2020 at 04:39:23PM +0100, Mike Leach wrote:
Adjust the handling of the session sink selection to allow no sink to be selected on the command line. This then forwards the sink selection to the CoreSight infrastructure which will attempt to select a sink based on the principal sink labelling.
Signed-off-by: Mike Leach mike.leach@linaro.org
tools/perf/arch/arm/util/cs-etm.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tools/perf/arch/arm/util/cs-etm.c b/tools/perf/arch/arm/util/cs-etm.c index 941f814820b8..ed9ea2c60f27 100644 --- a/tools/perf/arch/arm/util/cs-etm.c +++ b/tools/perf/arch/arm/util/cs-etm.c @@ -242,10 +242,10 @@ static int cs_etm_set_sink_attr(struct perf_pmu *pmu, }
/*
* No sink was provided on the command line - for _now_ treat* this as an error.
* No sink was provided on the command line - allow the CoreSight* system to look for a default */
return ret;
return 0;I understand this patchset is a proof of concept. When we reach a consensus on how to move forward and you are ready for a respin, please do two sets - one for kernel changes and another one for the perf tools. Normally changes to the kernel need to be merged first.
Regards, Mathieu
Understood. The "real" patchset is going to need doc updates too.
Regards
Mike
}
static int cs_etm_recording_options(struct auxtrace_record *itr,
2.17.1
Trace sinks may have a parameter added marking them as a principal sink for automatic sink selection while tracing.
The best sink to be used in this way is topology and board dependent so a boolean device tree parameter is used.
This patch adds principal sink selection for: 1) Arm Juno platform 2) Qualcomm Dragonboard DB410c (MSM8916) platforms
Signed-off-by: Mike Leach mike.leach@linaro.org --- arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + 2 files changed, 2 insertions(+)
diff --git a/arch/arm64/boot/dts/arm/juno-base.dtsi b/arch/arm64/boot/dts/arm/juno-base.dtsi index d65b4e949ef0..0553af4486e8 100644 --- a/arch/arm64/boot/dts/arm/juno-base.dtsi +++ b/arch/arm64/boot/dts/arm/juno-base.dtsi @@ -205,6 +205,7 @@ clock-names = "apb_pclk"; power-domains = <&scpi_devpd 0>; arm,scatter-gather; + arm,principal-sink; in-ports { port { etr_in_port: endpoint { diff --git a/arch/arm64/boot/dts/qcom/msm8916.dtsi b/arch/arm64/boot/dts/qcom/msm8916.dtsi index 194d5e45f4e5..e0bdf378739e 100644 --- a/arch/arm64/boot/dts/qcom/msm8916.dtsi +++ b/arch/arm64/boot/dts/qcom/msm8916.dtsi @@ -1337,6 +1337,7 @@
clocks = <&rpmcc RPM_QDSS_CLK>, <&rpmcc RPM_QDSS_A_CLK>; clock-names = "apb_pclk", "atclk"; + arm,principal-sink;
in-ports { port {
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On 04/21/2020 11:41 AM, Mike Leach wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
I agree. I am leaning towards more of the automatic detection than the firmware specifying it. I understand that this is complex, given the possible configurations.
May be we could limit our logic to :
1) Principal sinks are always ETR, nothing else. (We don't have to support legacy systems without ETR).
2) Track how many ETMs can reach an ETR and this can be a good indicator to select the sink automatically given a choice of multiple ETRs. (This involves a bit of work). i.e, choose the "available" ETR with lowest number of ETMs connected.
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Suzuki,
On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com wrote:
On 04/21/2020 11:41 AM, Mike Leach wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
I agree. I am leaning towards more of the automatic detection than the firmware specifying it. I understand that this is complex, given the possible configurations.
May be we could limit our logic to :
- Principal sinks are always ETR, nothing else. (We don't have to
support legacy systems without ETR).
I don't think there is too much extra effort to prioritize ETR whenever present, but where there is no ETR, select what is available.
- Track how many ETMs can reach an ETR and this can be a good
indicator to select the sink automatically given a choice of multiple ETRs. (This involves a bit of work). i.e, choose the "available" ETR with lowest number of ETMs connected.
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
Regards
Mike
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Yes as long as you count a non-programmable funnel as a hop, so if
- replicator - per-core ETR - invisible funnel - shared ETR
and if you happen to stumble across the shared ETR first, you have two ETRs at the same level, but it's the per-core ETR you likely want to use. (I think we are going to be seeing a SoC like this.)
Perhaps the rule should be "least shared and closest" ETR which is generally also the closest, but acts as a tie-breaker if there are several at the same level. It also does the right thing when there's a per-core ETR (or TRBE) when you invariably want to be using that.
Al
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
Regards
Mike
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as
principal sink, one for each cluster.
The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.htm l [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.htm l
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote:
Hi Suzuki,
On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com wrote:
On 04/21/2020 11:41 AM, Mike Leach wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
I agree. I am leaning towards more of the automatic detection than the firmware specifying it. I understand that this is complex, given the possible configurations.
May be we could limit our logic to :
- Principal sinks are always ETR, nothing else. (We don't have to
support legacy systems without ETR).
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
I don't think there is too much extra effort to prioritize ETR whenever present, but where there is no ETR, select what is available.
- Track how many ETMs can reach an ETR and this can be a good
indicator to select the sink automatically given a choice of multiple ETRs. (This involves a bit of work). i.e, choose the "available" ETR with lowest number of ETMs connected.
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
Regards
Mike
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
Hi,
It seems we are moving towards consensus on a first ETR / first sink strategy - without the complication of tagging at present.
As Mathieu pointed out - even "invisible" funnels mentioned in Al's suggested topology, will have an entry in the connections graph, ensuring that the common ETR will always be one hop further out than the 1:1 ETR (given an algorithm that walks the connections graph from the source ETM).
I do think that when we select a sink automatically, there should be some record of this, for debug / expert use. An entry in Auxtrace might work for perf, but not sysfs. I propose therefore that it is logged in the same way as the registration sign-ons for each of the components. Alternative suggestions welcome.
Regards
Mike
On Tue, 21 Apr 2020 at 18:40, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote:
Hi Suzuki,
On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com wrote:
On 04/21/2020 11:41 AM, Mike Leach wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
I agree. I am leaning towards more of the automatic detection than the firmware specifying it. I understand that this is complex, given the possible configurations.
May be we could limit our logic to :
- Principal sinks are always ETR, nothing else. (We don't have to
support legacy systems without ETR).
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
I don't think there is too much extra effort to prioritize ETR whenever present, but where there is no ETR, select what is available.
- Track how many ETMs can reach an ETR and this can be a good
indicator to select the sink automatically given a choice of multiple ETRs. (This involves a bit of work). i.e, choose the "available" ETR with lowest number of ETMs connected.
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
Regards
Mike
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
On 04/22/2020 11:22 AM, Mike Leach wrote:
Hi,
It seems we are moving towards consensus on a first ETR / first sink strategy - without the complication of tagging at present.
As Mathieu pointed out - even "invisible" funnels mentioned in Al's suggested topology, will have an entry in the connections graph, ensuring that the common ETR will always be one hop further out than the 1:1 ETR (given an algorithm that walks the connections graph from the source ETM).
Agreed.
I do think that when we select a sink automatically, there should be some record of this, for debug / expert use. An entry in Auxtrace
We could add the "sinkid" of the sink selected. (which we now specify for sink selection), similar to the traceid for ETMs.
might work for perf, but not sysfs. I propose therefore that it is logged in the same way as the registration sign-ons for each of the components. Alternative suggestions welcome.
I think we could limit this to perf mode, as we don't know which ETMs are going to be used.
For sysfs mode, we know which ETMs are included and it should be left to the user to select the sinks and collate them as needed.
Suzuki
Regards
Mike
On Tue, 21 Apr 2020 at 18:40, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote:
Hi Suzuki,
On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com wrote:
On 04/21/2020 11:41 AM, Mike Leach wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
> This patchset provides a proposed infrastructure to allow for the automatic > selection of a sink during CoreSight tracing operations. > > Currently starting tracing using perf requires a sink selection on the command > line:- > > sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a > > After this set the infrastructure will be able to select a default sink:- > > sudo ./perf record -e cs_etm// --per-thread uname -a > > This matches with the default operation provided with perf and intelpt. > > The CoreSight infrastructure is updated to allow the concept of a "principal sink". > For any given source there could be multiple possible sinks available for > selection. However, there are no good programmatic ways to determine the > optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, > last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
I agree. I am leaning towards more of the automatic detection than the firmware specifying it. I understand that this is complex, given the possible configurations.
May be we could limit our logic to :
- Principal sinks are always ETR, nothing else. (We don't have to
support legacy systems without ETR).
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
I don't think there is too much extra effort to prioritize ETR whenever present, but where there is no ETR, select what is available.
- Track how many ETMs can reach an ETR and this can be a good
indicator to select the sink automatically given a choice of multiple ETRs. (This involves a bit of work). i.e, choose the "available" ETR with lowest number of ETMs connected.
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
Regards
Mike
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
Regards
Mike
> Therefore, we select the principal sink using a property in the device tree. > This allows the routine to select the best possible for the situation. > > The only rule for selecting such sinks, is that there should only be one principal > sink marked for any given source, across all possible paths from that source to > any sinks. (if this rule is violated then the first found will be used) > > Therefore to select a sink, we start with the source, and walk the child > connections until a sink marked as a principal sink is found. > > Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410 > > Note A: Moving forward to topologies which have groups of sources going to > dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a > sink, then multiple sinks can be marked as principal sink, one for each cluster. > The same applies to 1:1 ETM:sink topolgy were each sink will be marked. > > Note B: The current set does not auto-select when using sysfs, but I believe this > could easily be added. > > [1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html > [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html > > Mike Leach (5): > coresight: Add principal sink handling to CoreSight base > coresight: tmc: Add principal sink attribute detection > coresight: etm: perf: Add default sink selection to etm perf > perf: cs-etm: Allow no CoreSight sink to be specified on command line > dt-bindings: arm: qcom: Add CoreSight principal sink select parameters > > arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + > arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + > .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- > drivers/hwtracing/coresight/coresight-priv.h | 2 ++ > drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ > drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ > include/linux/coresight.h | 7 ++++ > tools/perf/arch/arm/util/cs-etm.c | 6 ++-- > 8 files changed, 74 insertions(+), 6 deletions(-) > > -- > 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
On Wed, 22 Apr 2020 at 04:43, Suzuki K Poulose suzuki.poulose@arm.com wrote:
On 04/22/2020 11:22 AM, Mike Leach wrote:
Hi,
It seems we are moving towards consensus on a first ETR / first sink strategy - without the complication of tagging at present.
As Mathieu pointed out - even "invisible" funnels mentioned in Al's suggested topology, will have an entry in the connections graph, ensuring that the common ETR will always be one hop further out than the 1:1 ETR (given an algorithm that walks the connections graph from the source ETM).
Agreed.
I do think that when we select a sink automatically, there should be some record of this, for debug / expert use. An entry in Auxtrace
We could add the "sinkid" of the sink selected. (which we now specify for sink selection), similar to the traceid for ETMs.
I think using the sinkID is the only possible solution. Regarding implementation, the version number in the private auxtrace header has to be increased. From there and for each CPU entry, the sinkID of the sink that was selected for the session is added. That way there is no confusion as to how the system was configured.
might work for perf, but not sysfs. I propose therefore that it is logged in the same way as the registration sign-ons for each of the components. Alternative suggestions welcome.
I think we could limit this to perf mode, as we don't know which ETMs are going to be used.
For sysfs mode, we know which ETMs are included and it should be left to the user to select the sinks and collate them as needed.
I agree.
Suzuki
Regards
Mike
On Tue, 21 Apr 2020 at 18:40, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote:
Hi Suzuki,
On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com wrote:
On 04/21/2020 11:41 AM, Mike Leach wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote: > >> This patchset provides a proposed infrastructure to allow for the automatic >> selection of a sink during CoreSight tracing operations. >> >> Currently starting tracing using perf requires a sink selection on the command >> line:- >> >> sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a >> >> After this set the infrastructure will be able to select a default sink:- >> >> sudo ./perf record -e cs_etm// --per-thread uname -a >> >> This matches with the default operation provided with perf and intelpt. >> >> The CoreSight infrastructure is updated to allow the concept of a "principal sink". >> For any given source there could be multiple possible sinks available for >> selection. However, there are no good programmatic ways to determine the >> optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, >> last ETR), there exists a possible platform / topology that will not be suitable. > > While "first ETR" may not be optimal or even possible, it nearly always > is both optimal and possible. So it's a good default rule and it would be > useful to have a default rule. That would allow -e cs_etm// to immediately > do something useful with most devices and with ACPI, in advance of an > ACPI binding for the "principal_sink" flag. >
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
I agree. I am leaning towards more of the automatic detection than the firmware specifying it. I understand that this is complex, given the possible configurations.
May be we could limit our logic to :
- Principal sinks are always ETR, nothing else. (We don't have to
support legacy systems without ETR).
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
I don't think there is too much extra effort to prioritize ETR whenever present, but where there is no ETR, select what is available.
- Track how many ETMs can reach an ETR and this can be a good
indicator to select the sink automatically given a choice of multiple ETRs. (This involves a bit of work). i.e, choose the "available" ETR with lowest number of ETMs connected.
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
Regards
Mike
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
Regards
Mike
>> Therefore, we select the principal sink using a property in the device tree. >> This allows the routine to select the best possible for the situation. >> >> The only rule for selecting such sinks, is that there should only be one principal >> sink marked for any given source, across all possible paths from that source to >> any sinks. (if this rule is violated then the first found will be used) >> >> Therefore to select a sink, we start with the source, and walk the child >> connections until a sink marked as a principal sink is found. >> >> Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410 >> >> Note A: Moving forward to topologies which have groups of sources going to >> dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a >> sink, then multiple sinks can be marked as principal sink, one for each cluster. >> The same applies to 1:1 ETM:sink topolgy were each sink will be marked. >> >> Note B: The current set does not auto-select when using sysfs, but I believe this >> could easily be added. >> >> [1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html >> [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html >> >> Mike Leach (5): >> coresight: Add principal sink handling to CoreSight base >> coresight: tmc: Add principal sink attribute detection >> coresight: etm: perf: Add default sink selection to etm perf >> perf: cs-etm: Allow no CoreSight sink to be specified on command line >> dt-bindings: arm: qcom: Add CoreSight principal sink select parameters >> >> arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + >> arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + >> .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- >> drivers/hwtracing/coresight/coresight-priv.h | 2 ++ >> drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ >> drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ >> include/linux/coresight.h | 7 ++++ >> tools/perf/arch/arm/util/cs-etm.c | 6 ++-- >> 8 files changed, 74 insertions(+), 6 deletions(-) >> >> -- >> 2.17.1 > > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
We could add the "sinkid" of the sink selected. (which we now specify for sink selection), similar to the traceid for ETMs.
I think using the sinkID is the only possible solution. Regarding implementation, the version number in the private auxtrace header has to be increased. From there and for each CPU entry, the sinkID of the sink that was selected for the session is added. That way there is no confusion as to how the system was configured.
It would be useful to have generic information as well e.g.
- a flag to indicate memory (i.e. ETR) vs. ETB - size - a flag saying whether the buffer is in circular buffer (wrapping) mode or is set up to trigger a trace stop before reaching its limit - a unique index
The kernel has all that information, and it could help when interpreting what we're seeing in the AUX buffers (truncation flags etc.) and setting strobe parameters.
Exactly how it's arranged is a detail, e.g. could be a list of sinks followed by a list of CPUs with each CPU having a sink index... or whatever.
Currently the perf.data files are self-contained and can (mostly) be processed on a different system from the one they are generated on, as all the info is in headers, info records etc. There is a whole set of headers with details on cache / NUMA topology etc. so that consumers can interpret the data. Sink ids unique to the current system may be useful but they shouldn't be the only way of finding properties of the sinks.
Al
might work for perf, but not sysfs. I propose therefore that it is logged in the same way as the registration sign-ons for each of the components. Alternative suggestions welcome.
I think we could limit this to perf mode, as we don't know which ETMs are going to be used.
For sysfs mode, we know which ETMs are included and it should be left to the user to select the sinks and collate them as needed.
I agree.
Suzuki
Regards
Mike
On Tue, 21 Apr 2020 at 18:40, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote:
Hi Suzuki,
On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com
wrote:
On 04/21/2020 11:41 AM, Mike Leach wrote: > Hi Al, > > On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote: >> >>> This patchset provides a proposed infrastructure to allow for >>> the automatic selection of a sink during CoreSight tracing operations. >>> >>> Currently starting tracing using perf requires a sink >>> selection on the command >>> line:- >>> >>> sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a >>> >>> After this set the infrastructure will be able to select a >>> default sink:- >>> >>> sudo ./perf record -e cs_etm// --per-thread uname -a >>> >>> This matches with the default operation provided with perf and
intelpt.
>>> >>> The CoreSight infrastructure is updated to allow the concept of a
"principal sink".
>>> For any given source there could be multiple possible sinks >>> available for selection. However, there are no good >>> programmatic ways to determine the optimal sink for a given >>> topolgy. For any given rule (e.g. first sink found, first ETR, last ETR),
there exists a possible platform / topology that will not be suitable.
>> >> While "first ETR" may not be optimal or even possible, it >> nearly always is both optimal and possible. So it's a good >> default rule and it would be useful to have a default rule. >> That would allow -e cs_etm// to immediately do something useful >> with most devices and with ACPI, in advance of an ACPI binding for the
"principal_sink" flag.
>> > > That's certainly feasible - and would potentially benefit any > system without updated bindings, not just ACPI
I agree. I am leaning towards more of the automatic detection than the firmware specifying it. I understand that this is complex, given the possible configurations.
May be we could limit our logic to :
- Principal sinks are always ETR, nothing else. (We don't have
to support legacy systems without ETR).
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
I don't think there is too much extra effort to prioritize ETR whenever present, but where there is no ETR, select what is available.
- Track how many ETMs can reach an ETR and this can be a good
indicator to select the sink automatically given a choice of multiple ETRs. (This involves a bit of work). i.e, choose the "available" ETR with lowest number of ETMs connected.
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
Regards
Mike
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
> > Regards > > Mike > >>> Therefore, we select the principal sink using a property in the device
tree.
>>> This allows the routine to select the best possible for the situation. >>> >>> The only rule for selecting such sinks, is that there should >>> only be one principal sink marked for any given source, across >>> all possible paths from that source to any sinks. (if this >>> rule is violated then the first found will be used) >>> >>> Therefore to select a sink, we start with the source, and walk >>> the child connections until a sink marked as a principal sink is found. >>> >>> Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on >>> Juno / DB410 >>> >>> Note A: Moving forward to topologies which have groups of >>> sources going to dedicated sinks e.g. multisocket / >>> multicluster where each socket / cluster has a sink, then multiple
sinks can be marked as principal sink, one for each cluster.
>>> The same applies to 1:1 ETM:sink topolgy were each sink will be
marked.
>>> >>> Note B: The current set does not auto-select when using sysfs, >>> but I believe this could easily be added. >>> >>> [1] >>> https://lists.linaro.org/pipermail/coresight/2020-April/003819 >>> .html [2] >>> https://lists.linaro.org/pipermail/coresight/2020-April/003821 >>> .html >>> >>> Mike Leach (5): >>> coresight: Add principal sink handling to CoreSight base >>> coresight: tmc: Add principal sink attribute detection >>> coresight: etm: perf: Add default sink selection to etm perf >>> perf: cs-etm: Allow no CoreSight sink to be specified on command
line
>>> dt-bindings: arm: qcom: Add CoreSight principal sink >>> select parameters >>> >>> arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + >>> arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + >>> .../hwtracing/coresight/coresight-etm-perf.c | 20 >>> +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ >>> drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ >>> drivers/hwtracing/coresight/coresight.c | 34
+++++++++++++++++++
>>> include/linux/coresight.h | 7 ++++ >>> tools/perf/arch/arm/util/cs-etm.c | 6 ++-- >>> 8 files changed, 74 insertions(+), 6 deletions(-) >>> >>> -- >>> 2.17.1 >> >> IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
> > >
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
hi Althe info you requested below is very helpful for the application that has to deal with the stream of traces. currently, the perf application is collecting the info from different sources (mostly sysfs ) and writes them in a "pseudo" perf event (AUX_INFO) in the traces recorded in a file, so that they can be processed later on.  it would be very helpful if the kernel can publish such an event itself with info needed to process the traces. this event can be provided either in the data area or in the aux area. the advantage is that this can allow tracing multiple processes at the same time. Kind RegardsZied Guermazi On Wednesday, April 22, 2020, 06:02:45 PM GMT+2, Al Grant al.grant@arm.com wrote:
We could add the "sinkid" of the sink selected. (which we now specify for sink selection), similar to the traceid for ETMs.
I think using the sinkID is the only possible solution. Regarding implementation, the version number in the private auxtrace header has to be increased. From there and for each CPU entry, the sinkID of the sink that was selected for the session is added. That way there is no confusion as to how the system was configured.
It would be useful to have generic information as well e.g.
- a flag to indicate memory (i.e. ETR) vs. ETB - size - a flag saying whether the buffer is in circular buffer (wrapping) mode or is set up to trigger a trace stop before reaching its limit - a unique index
The kernel has all that information, and it could help when interpreting what we're seeing in the AUX buffers (truncation flags etc.) and setting strobe parameters.
Exactly how it's arranged is a detail, e.g. could be a list of sinks followed by a list of CPUs with each CPU having a sink index... or whatever.
Currently the perf.data files are self-contained and can (mostly) be processed on a different system from the one they are generated on, as all the info is in headers, info records etc. There is a whole set of headers with details on cache / NUMA topology etc. so that consumers can interpret the data. Sink ids unique to the current system may be useful but they shouldn't be the only way of finding properties of the sinks.
Al
might work for perf, but not sysfs. I propose therefore that it is logged in the same way as the registration sign-ons for each of the components. Alternative suggestions welcome.
I think we could limit this to perf mode, as we don't know which ETMs are going to be used.
For sysfs mode, we know which ETMs are included and it should be left to the user to select the sinks and collate them as needed.
I agree.
Suzuki
Regards
Mike
On Tue, 21 Apr 2020 at 18:40, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote:
Hi Suzuki,
On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com
wrote:
On 04/21/2020 11:41 AM, Mike Leach wrote: > Hi Al, > > On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote: >> >>> This patchset provides a proposed infrastructure to allow for >>> the automatic selection of a sink during CoreSight tracing operations. >>> >>> Currently starting tracing using perf requires a sink >>> selection on the command >>> line:- >>> >>> sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a >>> >>> After this set the infrastructure will be able to select a >>> default sink:- >>> >>> sudo ./perf record -e cs_etm// --per-thread uname -a >>> >>> This matches with the default operation provided with perf and
intelpt.
>>> >>> The CoreSight infrastructure is updated to allow the concept of a
"principal sink".
>>> For any given source there could be multiple possible sinks >>> available for selection. However, there are no good >>> programmatic ways to determine the optimal sink for a given >>> topolgy. For any given rule (e.g. first sink found, first ETR, last ETR),
there exists a possible platform / topology that will not be suitable.
>> >> While "first ETR" may not be optimal or even possible, it >> nearly always is both optimal and possible. So it's a good >> default rule and it would be useful to have a default rule. >> That would allow -e cs_etm// to immediately do something useful >> with most devices and with ACPI, in advance of an ACPI binding for the
"principal_sink" flag.
>> > > That's certainly feasible - and would potentially benefit any > system without updated bindings, not just ACPI
I agree. I am leaning towards more of the automatic detection than the firmware specifying it. I understand that this is complex, given the possible configurations.
May be we could limit our logic to :
- Principal sinks are always ETR, nothing else. (We don't have
to support legacy systems without ETR).
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
I don't think there is too much extra effort to prioritize ETR whenever present, but where there is no ETR, select what is available.
- Track how many ETMs can reach an ETR and this can be a good
indicator to select the sink automatically given a choice of multiple ETRs. (This involves a bit of work). i.e, choose the "available" ETR with lowest number of ETMs connected.
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
Regards
Mike
And this should solve the 1:1 configurations and per cluster ETR and one ETR per system.
This can be a one time list, updated when there is a change in the system (ETR added, ETM registered).
Thoughts ?
Suzuki
> > Regards > > Mike > >>> Therefore, we select the principal sink using a property in the device
tree.
>>> This allows the routine to select the best possible for the situation. >>> >>> The only rule for selecting such sinks, is that there should >>> only be one principal sink marked for any given source, across >>> all possible paths from that source to any sinks. (if this >>> rule is violated then the first found will be used) >>> >>> Therefore to select a sink, we start with the source, and walk >>> the child connections until a sink marked as a principal sink is found. >>> >>> Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on >>> Juno / DB410 >>> >>> Note A: Moving forward to topologies which have groups of >>> sources going to dedicated sinks e.g. multisocket / >>> multicluster where each socket / cluster has a sink, then multiple
sinks can be marked as principal sink, one for each cluster.
>>> The same applies to 1:1 ETM:sink topolgy were each sink will be
marked.
>>> >>> Note B: The current set does not auto-select when using sysfs, >>> but I believe this could easily be added. >>> >>> [1] >>> https://lists.linaro.org/pipermail/coresight/2020-April/003819 >>> .html [2] >>> https://lists.linaro.org/pipermail/coresight/2020-April/003821 >>> .html >>> >>> Mike Leach (5): >>>Â Â coresight: Add principal sink handling to CoreSight base >>>Â Â coresight: tmc: Add principal sink attribute detection >>>Â Â coresight: etm: perf: Add default sink selection to etm perf >>>Â Â perf: cs-etm: Allow no CoreSight sink to be specified on command
line
>>>  dt-bindings: arm: qcom: Add CoreSight principal sink >>> select parameters >>> >>>  arch/arm64/boot/dts/arm/juno-base.dtsi    | 1 + >>>  arch/arm64/boot/dts/qcom/msm8916.dtsi    | 1 + >>>  .../hwtracing/coresight/coresight-etm-perf.c | 20 >>> +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ >>>  drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ >>>  drivers/hwtracing/coresight/coresight.c   | 34
+++++++++++++++++++
>>>  include/linux/coresight.h          | 7 ++++ >>>  tools/perf/arch/arm/util/cs-etm.c      | 6 ++-- >>>  8 files changed, 74 insertions(+), 6 deletions(-) >>> >>> -- >>> 2.17.1 >> >> IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
> > >
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
Hi,
On Thu, 23 Apr 2020 at 14:49, zied guermazi guermazi_zied@yahoo.com wrote:
hi Al the info you requested below is very helpful for the application that has to deal with the stream of traces. currently, the perf application is collecting the info from different sources (mostly sysfs ) and writes them in a "pseudo" perf event (AUX_INFO) in the traces recorded in a file, so that they can be processed later on. it would be very helpful if the kernel can publish such an event itself with info needed to process the traces. this event can be provided either in the data area or in the aux area. the advantage is that this can allow tracing multiple processes at the same time.
Kind Regards Zied Guermazi
On Wednesday, April 22, 2020, 06:02:45 PM GMT+2, Al Grant al.grant@arm.com wrote:
We could add the "sinkid" of the sink selected. (which we now specify for sink selection), similar to the traceid for ETMs.
I think using the sinkID is the only possible solution. Regarding implementation, the version number in the private auxtrace header has to be increased. From there and for each CPU entry, the sinkID of the sink that was selected for the session is added. That way there is no confusion as to how the system was configured.
This looks like a more serious modification than I was thinking about - but certainly feasible
It would be useful to have generic information as well e.g.
- a flag to indicate memory (i.e. ETR) vs. ETB
- size
- a flag saying whether the buffer is in circular buffer (wrapping) mode or
is set up to trigger a trace stop before reaching its limit
- a unique index
All the above is reasonable - but what does the "unique index" represent - is that the sinkid?
The kernel has all that information, and it could help when interpreting what we're seeing in the AUX buffers (truncation flags etc.) and setting strobe parameters.
Exactly how it's arranged is a detail, e.g. could be a list of sinks followed by a list of CPUs with each CPU having a sink index... or whatever.
Currently the perf.data files are self-contained and can (mostly) be processed on a different system from the one they are generated on, as all the info is in headers, info records etc. There is a whole set of headers with details on cache / NUMA topology etc. so that consumers can interpret the data. Sink ids unique to the current system may be useful but they shouldn't be the only way of finding properties of the sinks.
Al
might work for perf, but not sysfs. I propose therefore that it is logged in the same way as the registration sign-ons for each of the components. Alternative suggestions welcome.
I think we could limit this to perf mode, as we don't know which ETMs are going to be used.
For sysfs mode, we know which ETMs are included and it should be left to the user to select the sinks and collate them as needed.
I agree.
Seems to me in general sysfs mode leaves the sink enabled. The same will be true if this is auto-selected - so finding out which sink was selected takes care of itself. I think it is important to allow this feature in sysfs - for people who are exploring systems. and it eases writing generic test scripts if trace can be obtained purely by activating an ETM.
Given that Al's related mail on this list relating to EL2 kernel trace also seems to require information on kernel option selection, the best way forward seems to be:-
1) a respin of this set to use the ETR first algorithm - without the additional information at present. One motivation for producing this set is that automatic sink selection / default operation is useful in complex config. 2) An investigation is required to update the AUXTRACE headers to allow additional information to be passed from automated driver option selection / configuration information to the user side perf code.
Regards
Mike
Suzuki
Regards
Mike
On Tue, 21 Apr 2020 at 18:40, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote:
Hi Suzuki,
On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com
wrote:
> > On 04/21/2020 11:41 AM, Mike Leach wrote: >> Hi Al, >> >> On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote: >>> >>>> This patchset provides a proposed infrastructure to allow for >>>> the automatic selection of a sink during CoreSight tracing operations. >>>> >>>> Currently starting tracing using perf requires a sink >>>> selection on the command >>>> line:- >>>> >>>> sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a >>>> >>>> After this set the infrastructure will be able to select a >>>> default sink:- >>>> >>>> sudo ./perf record -e cs_etm// --per-thread uname -a >>>> >>>> This matches with the default operation provided with perf and
intelpt.
>>>> >>>> The CoreSight infrastructure is updated to allow the concept of a
"principal sink".
>>>> For any given source there could be multiple possible sinks >>>> available for selection. However, there are no good >>>> programmatic ways to determine the optimal sink for a given >>>> topolgy. For any given rule (e.g. first sink found, first ETR, last ETR),
there exists a possible platform / topology that will not be suitable.
>>> >>> While "first ETR" may not be optimal or even possible, it >>> nearly always is both optimal and possible. So it's a good >>> default rule and it would be useful to have a default rule. >>> That would allow -e cs_etm// to immediately do something useful >>> with most devices and with ACPI, in advance of an ACPI binding for the
"principal_sink" flag.
>>> >> >> That's certainly feasible - and would potentially benefit any >> system without updated bindings, not just ACPI > > I agree. I am leaning towards more of the automatic detection > than the firmware specifying it. I understand that this is > complex, given the possible configurations. > > May be we could limit our logic to : > > 1) Principal sinks are always ETR, nothing else. (We don't have > to support legacy systems without ETR). >
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
I don't think there is too much extra effort to prioritize ETR whenever present, but where there is no ETR, select what is available.
> 2) Track how many ETMs can reach an ETR and this can be a good > indicator to select the sink automatically given a choice of > multiple ETRs. (This involves a bit of work). i.e, choose the "available" > ETR with lowest number of ETMs connected. >
I am hard pressed to figure a topology where the first ETR found after an ETM is not also the least connected ETR. Even a 1:1 with an additional global ETR via replicator still needs a funnel before the global ETR, ensuring the global ETR is further away.
Perhaps the first approach could be to implement the 1st ETR / 1st Sink approach for principal / default sink, and revisit specific tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
Regards
Mike
> And this should solve the 1:1 configurations and per cluster ETR > and one ETR per system. > > This can be a one time list, updated when there is a change in > the system (ETR added, ETM registered). > > Thoughts ? > > Suzuki > >> >> Regards >> >> Mike >> >>>> Therefore, we select the principal sink using a property in the device
tree.
>>>> This allows the routine to select the best possible for the situation. >>>> >>>> The only rule for selecting such sinks, is that there should >>>> only be one principal sink marked for any given source, across >>>> all possible paths from that source to any sinks. (if this >>>> rule is violated then the first found will be used) >>>> >>>> Therefore to select a sink, we start with the source, and walk >>>> the child connections until a sink marked as a principal sink is found. >>>> >>>> Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on >>>> Juno / DB410 >>>> >>>> Note A: Moving forward to topologies which have groups of >>>> sources going to dedicated sinks e.g. multisocket / >>>> multicluster where each socket / cluster has a sink, then multiple
sinks can be marked as principal sink, one for each cluster.
>>>> The same applies to 1:1 ETM:sink topolgy were each sink will be
marked.
>>>> >>>> Note B: The current set does not auto-select when using sysfs, >>>> but I believe this could easily be added. >>>> >>>> [1] >>>> https://lists.linaro.org/pipermail/coresight/2020-April/003819 >>>> .html [2] >>>> https://lists.linaro.org/pipermail/coresight/2020-April/003821 >>>> .html >>>> >>>> Mike Leach (5): >>>> coresight: Add principal sink handling to CoreSight base >>>> coresight: tmc: Add principal sink attribute detection >>>> coresight: etm: perf: Add default sink selection to etm perf >>>> perf: cs-etm: Allow no CoreSight sink to be specified on command
line
>>>> dt-bindings: arm: qcom: Add CoreSight principal sink >>>> select parameters >>>> >>>> arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + >>>> arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + >>>> .../hwtracing/coresight/coresight-etm-perf.c | 20 >>>> +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ >>>> drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ >>>> drivers/hwtracing/coresight/coresight.c | 34
+++++++++++++++++++
>>>> include/linux/coresight.h | 7 ++++ >>>> tools/perf/arch/arm/util/cs-etm.c | 6 ++-- >>>> 8 files changed, 74 insertions(+), 6 deletions(-) >>>> >>>> -- >>>> 2.17.1 >>> >>> IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
>> >> >> >
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
This looks like a more serious modification than I was thinking about
- but certainly feasible
It would be useful to have generic information as well e.g.
- a flag to indicate memory (i.e. ETR) vs. ETB
- size
- a flag saying whether the buffer is in circular buffer (wrapping)
mode or is set up to trigger a trace stop before reaching its limit
- a unique index
All the above is reasonable - but what does the "unique index" represent - is that the sinkid?
Could be... there needs to be some way of distingiushing multiple sinks that are otherwise identical. Maybe it's enough to have "index into the list of sinks" as an implicit identifier (in the same way that events are indexed in perf.data by their position in the list of event selectors).
I think we could limit this to perf mode, as we don't know which ETMs are going to be used.
For sysfs mode, we know which ETMs are included and it should be left to the user to select the sinks and collate them as needed.
I agree.
Seems to me in general sysfs mode leaves the sink enabled. The same will be true if this is auto-selected - so finding out which sink was selected takes care of itself. I think it is important to allow this feature in sysfs - for people who are exploring systems. and it eases writing generic test scripts if trace can be obtained purely by activating an ETM.
Given that Al's related mail on this list relating to EL2 kernel trace also seems to require information on kernel option selection, the best way forward seems to be:-
- a respin of this set to use the ETR first algorithm - without the additional
information at present. One motivation for producing this set is that automatic sink selection / default operation is useful in complex config. 2) An investigation is required to update the AUXTRACE headers to allow additional information to be passed from automated driver option selection / configuration information to the user side perf code.
In a PERF_RECORD_AUX (which is the one you actually get from the kernel) there are 64 bits of flags, of which 4 are currently used.
If you're happy for the CONFIGIDR in AUXTRACE_INFO to stay as the value userspace thinks it is rather than the value it really is, then maybe the kernel could signal the use of VMID for thread tracing, using one of these flags.
Al
Regards
Mike
Suzuki
Regards
Mike
On Tue, 21 Apr 2020 at 18:40, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote: > Hi Suzuki, > > On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose > suzuki.poulose@arm.com
wrote:
>> >> On 04/21/2020 11:41 AM, Mike Leach wrote: >>> Hi Al, >>> >>> On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote: >>>> >>>>> This patchset provides a proposed infrastructure to allow >>>>> for the automatic selection of a sink during CoreSight tracing
operations.
>>>>> >>>>> Currently starting tracing using perf requires a sink >>>>> selection on the command >>>>> line:- >>>>> >>>>> sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname >>>>> -a >>>>> >>>>> After this set the infrastructure will be able to select a >>>>> default sink:- >>>>> >>>>> sudo ./perf record -e cs_etm// --per-thread uname -a >>>>> >>>>> This matches with the default operation provided with perf >>>>> and
intelpt.
>>>>> >>>>> The CoreSight infrastructure is updated to allow the >>>>> concept of a
"principal sink".
>>>>> For any given source there could be multiple possible >>>>> sinks available for selection. However, there are no good >>>>> programmatic ways to determine the optimal sink for a >>>>> given topolgy. For any given rule (e.g. first sink found, >>>>> first ETR, last ETR),
there exists a possible platform / topology that will not be suitable.
>>>> >>>> While "first ETR" may not be optimal or even possible, it >>>> nearly always is both optimal and possible. So it's a good >>>> default rule and it would be useful to have a default rule. >>>> That would allow -e cs_etm// to immediately do something >>>> useful with most devices and with ACPI, in advance of an >>>> ACPI binding for the
"principal_sink" flag.
>>>> >>> >>> That's certainly feasible - and would potentially benefit >>> any system without updated bindings, not just ACPI >> >> I agree. I am leaning towards more of the automatic detection >> than the firmware specifying it. I understand that this is >> complex, given the possible configurations. >> >> May be we could limit our logic to : >> >> 1) Principal sinks are always ETR, nothing else. (We don't >> have to support legacy systems without ETR). >>
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
> > I don't think there is too much extra effort to prioritize ETR > whenever present, but where there is no ETR, select what is available. > > >> 2) Track how many ETMs can reach an ETR and this can be a >> good indicator to select the sink automatically given a >> choice of multiple ETRs. (This involves a bit of work). i.e, choose the
"available"
>> ETR with lowest number of ETMs connected. >> > > I am hard pressed to figure a topology where the first ETR > found after an ETM is not also the least connected ETR. Even a > 1:1 with an additional global ETR via replicator still needs a > funnel before the global ETR, ensuring the global ETR is further away. > > Perhaps the first approach could be to implement the 1st ETR / > 1st Sink approach for principal / default sink, and revisit > specific tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
> > Regards > > Mike > >> And this should solve the 1:1 configurations and per cluster >> ETR and one ETR per system. >> >> This can be a one time list, updated when there is a change >> in the system (ETR added, ETM registered). >> >> Thoughts ? >> >> Suzuki >> >>> >>> Regards >>> >>> Mike >>> >>>>> Therefore, we select the principal sink using a property >>>>> in the device
tree.
>>>>> This allows the routine to select the best possible for the situation. >>>>> >>>>> The only rule for selecting such sinks, is that there >>>>> should only be one principal sink marked for any given >>>>> source, across all possible paths from that source to any >>>>> sinks. (if this rule is violated then the first found will >>>>> be used) >>>>> >>>>> Therefore to select a sink, we start with the source, and >>>>> walk the child connections until a sink marked as a principal sink is
found.
>>>>> >>>>> Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on >>>>> Juno / DB410 >>>>> >>>>> Note A: Moving forward to topologies which have groups of >>>>> sources going to dedicated sinks e.g. multisocket / >>>>> multicluster where each socket / cluster has a sink, then >>>>> multiple
sinks can be marked as principal sink, one for each cluster.
>>>>> The same applies to 1:1 ETM:sink topolgy were each sink >>>>> will be
marked.
>>>>> >>>>> Note B: The current set does not auto-select when using >>>>> sysfs, but I believe this could easily be added. >>>>> >>>>> [1] >>>>> https://lists.linaro.org/pipermail/coresight/2020-April/00 >>>>> 3819 >>>>> .html [2] >>>>> https://lists.linaro.org/pipermail/coresight/2020-April/00 >>>>> 3821 >>>>> .html >>>>> >>>>> Mike Leach (5): >>>>> coresight: Add principal sink handling to CoreSight base >>>>> coresight: tmc: Add principal sink attribute detection >>>>> coresight: etm: perf: Add default sink selection to etm perf >>>>> perf: cs-etm: Allow no CoreSight sink to be specified >>>>> on command
line
>>>>> dt-bindings: arm: qcom: Add CoreSight principal sink >>>>> select parameters >>>>> >>>>> arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + >>>>> arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + >>>>> .../hwtracing/coresight/coresight-etm-perf.c | 20 >>>>> +++++++++-- drivers/hwtracing/coresight/coresight-priv.h >>>>> +++++++++| 2 ++ >>>>> drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ >>>>> drivers/hwtracing/coresight/coresight.c | 34
+++++++++++++++++++
>>>>> include/linux/coresight.h | 7 ++++ >>>>> tools/perf/arch/arm/util/cs-etm.c | 6 ++-- >>>>> 8 files changed, 74 insertions(+), 6 deletions(-) >>>>> >>>>> -- >>>>> 2.17.1 >>>> >>>> IMPORTANT NOTICE: The contents of this email and any >>>> attachments
are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
>>> >>> >>> >> > > > -- > Mike Leach > Principal Engineer, ARM Ltd. > Manchester Design Centre. UK
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On Thu, 23 Apr 2020 at 08:14, Mike Leach mike.leach@linaro.org wrote:
Hi,
On Thu, 23 Apr 2020 at 14:49, zied guermazi guermazi_zied@yahoo.com wrote:
hi Al the info you requested below is very helpful for the application that has to deal with the stream of traces. currently, the perf application is collecting the info from different sources (mostly sysfs ) and writes them in a "pseudo" perf event (AUX_INFO) in the traces recorded in a file, so that they can be processed later on. it would be very helpful if the kernel can publish such an event itself with info needed to process the traces. this event can be provided either in the data area or in the aux area. the advantage is that this can allow tracing multiple processes at the same time.
Kind Regards Zied Guermazi
On Wednesday, April 22, 2020, 06:02:45 PM GMT+2, Al Grant al.grant@arm.com wrote:
We could add the "sinkid" of the sink selected. (which we now specify for sink selection), similar to the traceid for ETMs.
I think using the sinkID is the only possible solution. Regarding implementation, the version number in the private auxtrace header has to be increased. From there and for each CPU entry, the sinkID of the sink that was selected for the session is added. That way there is no confusion as to how the system was configured.
This looks like a more serious modification than I was thinking about
- but certainly feasible
It would be useful to have generic information as well e.g.
- a flag to indicate memory (i.e. ETR) vs. ETB
- size
- a flag saying whether the buffer is in circular buffer (wrapping) mode or
is set up to trigger a trace stop before reaching its limit
- a unique index
All the above is reasonable - but what does the "unique index" represent - is that the sinkid?
The kernel has all that information, and it could help when interpreting what we're seeing in the AUX buffers (truncation flags etc.) and setting strobe parameters.
Exactly how it's arranged is a detail, e.g. could be a list of sinks followed by a list of CPUs with each CPU having a sink index... or whatever.
Currently the perf.data files are self-contained and can (mostly) be processed on a different system from the one they are generated on, as all the info is in headers, info records etc. There is a whole set of headers with details on cache / NUMA topology etc. so that consumers can interpret the data. Sink ids unique to the current system may be useful but they shouldn't be the only way of finding properties of the sinks.
Al
might work for perf, but not sysfs. I propose therefore that it is logged in the same way as the registration sign-ons for each of the components. Alternative suggestions welcome.
I think we could limit this to perf mode, as we don't know which ETMs are going to be used.
For sysfs mode, we know which ETMs are included and it should be left to the user to select the sinks and collate them as needed.
I agree.
Seems to me in general sysfs mode leaves the sink enabled. The same will be true if this is auto-selected - so finding out which sink was selected takes care of itself. I think it is important to allow this feature in sysfs - for people who are exploring systems. and it eases writing generic test scripts if trace can be obtained purely by activating an ETM.
Given that Al's related mail on this list relating to EL2 kernel trace also seems to require information on kernel option selection, the best way forward seems to be:-
- a respin of this set to use the ETR first algorithm - without the
additional information at present. One motivation for producing this set is that automatic sink selection / default operation is useful in complex config.
Yes, please proceed this way. At the very least you can move forward on complex configuration and it gives us time to assess what the next steps should be.
- An investigation is required to update the AUXTRACE headers to
allow additional information to be passed from automated driver option selection / configuration information to the user side perf code.
Regards
Mike
Suzuki
Regards
Mike
On Tue, 21 Apr 2020 at 18:40, Mathieu Poirier mathieu.poirier@linaro.org wrote:
On Tue, Apr 21, 2020 at 01:51:41PM +0100, Mike Leach wrote: > Hi Suzuki, > > On Tue, 21 Apr 2020 at 12:09, Suzuki K Poulose suzuki.poulose@arm.com
wrote:
>> >> On 04/21/2020 11:41 AM, Mike Leach wrote: >>> Hi Al, >>> >>> On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote: >>>> >>>>> This patchset provides a proposed infrastructure to allow for >>>>> the automatic selection of a sink during CoreSight tracing operations. >>>>> >>>>> Currently starting tracing using perf requires a sink >>>>> selection on the command >>>>> line:- >>>>> >>>>> sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a >>>>> >>>>> After this set the infrastructure will be able to select a >>>>> default sink:- >>>>> >>>>> sudo ./perf record -e cs_etm// --per-thread uname -a >>>>> >>>>> This matches with the default operation provided with perf and
intelpt.
>>>>> >>>>> The CoreSight infrastructure is updated to allow the concept of a
"principal sink".
>>>>> For any given source there could be multiple possible sinks >>>>> available for selection. However, there are no good >>>>> programmatic ways to determine the optimal sink for a given >>>>> topolgy. For any given rule (e.g. first sink found, first ETR, last ETR),
there exists a possible platform / topology that will not be suitable.
>>>> >>>> While "first ETR" may not be optimal or even possible, it >>>> nearly always is both optimal and possible. So it's a good >>>> default rule and it would be useful to have a default rule. >>>> That would allow -e cs_etm// to immediately do something useful >>>> with most devices and with ACPI, in advance of an ACPI binding for the
"principal_sink" flag.
>>>> >>> >>> That's certainly feasible - and would potentially benefit any >>> system without updated bindings, not just ACPI >> >> I agree. I am leaning towards more of the automatic detection >> than the firmware specifying it. I understand that this is >> complex, given the possible configurations. >> >> May be we could limit our logic to : >> >> 1) Principal sinks are always ETR, nothing else. (We don't have >> to support legacy systems without ETR). >>
I definitely favour a kernel oriented approach to a FW specific one. I also like the idea of "ETR and nothing else". When I thought about this problem I envisioned assigning weights to each sink component. Only considering ETR is meaner but better.
> > I don't think there is too much extra effort to prioritize ETR > whenever present, but where there is no ETR, select what is available. > > >> 2) Track how many ETMs can reach an ETR and this can be a good >> indicator to select the sink automatically given a choice of >> multiple ETRs. (This involves a bit of work). i.e, choose the "available" >> ETR with lowest number of ETMs connected. >> > > I am hard pressed to figure a topology where the first ETR found > after an ETM is not also the least connected ETR. Even a 1:1 with > an additional global ETR via replicator still needs a funnel > before the global ETR, ensuring the global ETR is further away. > > Perhaps the first approach could be to implement the 1st ETR / 1st > Sink approach for principal / default sink, and revisit specific > tagging should a use case arise.
As Al pointed out we will find exotic topologies. Since all components, configurable or not, show up in the topology, a "select closest ETR" heuristic should do the trick.
> > Regards > > Mike > >> And this should solve the 1:1 configurations and per cluster ETR >> and one ETR per system. >> >> This can be a one time list, updated when there is a change in >> the system (ETR added, ETM registered). >> >> Thoughts ? >> >> Suzuki >> >>> >>> Regards >>> >>> Mike >>> >>>>> Therefore, we select the principal sink using a property in the device
tree.
>>>>> This allows the routine to select the best possible for the situation. >>>>> >>>>> The only rule for selecting such sinks, is that there should >>>>> only be one principal sink marked for any given source, across >>>>> all possible paths from that source to any sinks. (if this >>>>> rule is violated then the first found will be used) >>>>> >>>>> Therefore to select a sink, we start with the source, and walk >>>>> the child connections until a sink marked as a principal sink is found. >>>>> >>>>> Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on >>>>> Juno / DB410 >>>>> >>>>> Note A: Moving forward to topologies which have groups of >>>>> sources going to dedicated sinks e.g. multisocket / >>>>> multicluster where each socket / cluster has a sink, then multiple
sinks can be marked as principal sink, one for each cluster.
>>>>> The same applies to 1:1 ETM:sink topolgy were each sink will be
marked.
>>>>> >>>>> Note B: The current set does not auto-select when using sysfs, >>>>> but I believe this could easily be added. >>>>> >>>>> [1] >>>>> https://lists.linaro.org/pipermail/coresight/2020-April/003819 >>>>> .html [2] >>>>> https://lists.linaro.org/pipermail/coresight/2020-April/003821 >>>>> .html >>>>> >>>>> Mike Leach (5): >>>>> coresight: Add principal sink handling to CoreSight base >>>>> coresight: tmc: Add principal sink attribute detection >>>>> coresight: etm: perf: Add default sink selection to etm perf >>>>> perf: cs-etm: Allow no CoreSight sink to be specified on command
line
>>>>> dt-bindings: arm: qcom: Add CoreSight principal sink >>>>> select parameters >>>>> >>>>> arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + >>>>> arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + >>>>> .../hwtracing/coresight/coresight-etm-perf.c | 20 >>>>> +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ >>>>> drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ >>>>> drivers/hwtracing/coresight/coresight.c | 34
+++++++++++++++++++
>>>>> include/linux/coresight.h | 7 ++++ >>>>> tools/perf/arch/arm/util/cs-etm.c | 6 ++-- >>>>> 8 files changed, 74 insertions(+), 6 deletions(-) >>>>> >>>>> -- >>>>> 2.17.1 >>>> >>>> IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
>>> >>> >>> >> > > > -- > Mike Leach > Principal Engineer, ARM Ltd. > Manchester Design Centre. UK
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
Hi Al,coresight tracing support is also being integrated into gdb, this is done by using perf events at the system level. if my understanding is correct, with this patch, the default sink selection is done at the kernel level. - how can the application get this info back (current/default sink)?- if an application selects a different sink, what will happen after the application exists? will the next application still use the kernel selected default sink or the last selected sink (by previous application) thanks Zied Guermazi On Tuesday, April 21, 2020, 12:41:40 PM GMT+2, Mike Leach mike.leach@linaro.org wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5):  coresight: Add principal sink handling to CoreSight base  coresight: tmc: Add principal sink attribute detection  coresight: etm: perf: Add default sink selection to etm perf  perf: cs-etm: Allow no CoreSight sink to be specified on command line  dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi    | 1 +  arch/arm64/boot/dts/qcom/msm8916.dtsi    | 1 +  .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++  drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++  drivers/hwtracing/coresight/coresight.c   | 34 +++++++++++++++++++  include/linux/coresight.h          | 7 ++++  tools/perf/arch/arm/util/cs-etm.c      | 6 ++--  8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Zeid,
On Tue, 21 Apr 2020 at 12:19, zied guermazi guermazi_zied@yahoo.com wrote:
Hi Al, coresight tracing support is also being integrated into gdb, this is done by using perf events at the system level. if my understanding is correct, with this patch, the default sink selection is done at the kernel level.
- how can the application get this info back (current/default sink)?
The selection of a default sink will only occur if the user space perf / command line does not select a sink. The mechanisms for selecting a specific sink remain unchanged. So assuming GDB is currently specifying a sink for the perf event, then this will continue to work, the sink will be passed to the kernel drivers and that is the sink used.
- if an application selects a different sink, what will happen after the application exists? will the next application still use the kernel selected default sink or the last selected sink (by previous application)
My understanding is that a sink is chosen by perf on a per session basis - so again, this should continue to work in cases where a sink is specified for an application.
Regards
Mike
thanks Zied Guermazi
On Tuesday, April 21, 2020, 12:41:40 PM GMT+2, Mike Leach mike.leach@linaro.org wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
On Tue, 21 Apr 2020 at 12:19, zied guermazi guermazi_zied@yahoo.com wrote:
Hi Al, coresight tracing support is also being integrated into gdb, this is done by using
perf events at the system level. if my understanding is correct, with this patch, the default sink selection is done at the kernel level.
- how can the application get this info back (current/default sink)?
The selection of a default sink will only occur if the user space perf / command line does not select a sink. The mechanisms for selecting a specific sink remain unchanged. So assuming GDB is currently specifying a sink for the perf event, then this will continue to work, the sink will be passed to the kernel drivers and that is the sink used.
Right... the point of specifying the sink, is, firstly, to tell the kernel how to find a sink if it doesn't know how, and secondly, to influence whether it uses ETB or ETR (and perhaps the size of the ETR memory buffer) if it wants to trade off probe effect vs. buffer size. It should not affect the data returned and the client should not be sensitive to which sink was used. There should be no need for the client to know what sink was used.
An expert using ETM for performance analysis might be interested in which sinks were used, so it might be something to return in AUXTRACE_INFO... but that's just a bonus. It should never be necessary to know this, and the kernel might even want to withhold the information.
What would be reasonable is to have something in each AUX buffer that says the set of CPUs that the AUX buffer is for... this might be useful for decode, and is generic rather than exposing details of trace topology.
My understanding is that a sink is chosen by perf on a per session basis - so again, this should continue to work in cases where a sink is specified for an application.
That's my understanding too.
Al
Regards
Mike
thanks Zied Guermazi
On Tuesday, April 21, 2020, 12:41:40 PM GMT+2, Mike Leach
mike.leach@linaro.org wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal
sink".
For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible
platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as
principal sink, one for each cluster.
The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.htm l [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.htm l
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On Tue, Apr 21, 2020 at 12:34:43PM +0100, Mike Leach wrote:
Hi Zeid,
On Tue, 21 Apr 2020 at 12:19, zied guermazi guermazi_zied@yahoo.com wrote:
Hi Al, coresight tracing support is also being integrated into gdb, this is done by using perf events at the system level. if my understanding is correct, with this patch, the default sink selection is done at the kernel level.
- how can the application get this info back (current/default sink)?
The selection of a default sink will only occur if the user space perf / command line does not select a sink. The mechanisms for selecting a specific sink remain unchanged. So assuming GDB is currently specifying a sink for the perf event, then this will continue to work, the sink will be passed to the kernel drivers and that is the sink used.
- if an application selects a different sink, what will happen after the application exists? will the next application still use the kernel selected default sink or the last selected sink (by previous application)
My understanding is that a sink is chosen by perf on a per session basis - so again, this should continue to work in cases where a sink is specified for an application.
That is correct.
Regards
Mike
thanks Zied Guermazi
On Tuesday, April 21, 2020, 12:41:40 PM GMT+2, Mike Leach mike.leach@linaro.org wrote:
Hi Al,
On Thu, 16 Apr 2020 at 22:03, Al Grant Al.Grant@arm.com wrote:
This patchset provides a proposed infrastructure to allow for the automatic selection of a sink during CoreSight tracing operations.
Currently starting tracing using perf requires a sink selection on the command line:-
sudo ./perf record -e cs_etm/@tmc_etr0/ --per-thread uname -a
After this set the infrastructure will be able to select a default sink:-
sudo ./perf record -e cs_etm// --per-thread uname -a
This matches with the default operation provided with perf and intelpt.
The CoreSight infrastructure is updated to allow the concept of a "principal sink". For any given source there could be multiple possible sinks available for selection. However, there are no good programmatic ways to determine the optimal sink for a given topolgy. For any given rule (e.g. first sink found, first ETR, last ETR), there exists a possible platform / topology that will not be suitable.
While "first ETR" may not be optimal or even possible, it nearly always is both optimal and possible. So it's a good default rule and it would be useful to have a default rule. That would allow -e cs_etm// to immediately do something useful with most devices and with ACPI, in advance of an ACPI binding for the "principal_sink" flag.
That's certainly feasible - and would potentially benefit any system without updated bindings, not just ACPI
Regards
Mike
Therefore, we select the principal sink using a property in the device tree. This allows the routine to select the best possible for the situation.
The only rule for selecting such sinks, is that there should only be one principal sink marked for any given source, across all possible paths from that source to any sinks. (if this rule is violated then the first found will be used)
Therefore to select a sink, we start with the source, and walk the child connections until a sink marked as a principal sink is found.
Applied to Linux 5.7-rc1 (with dts updates 1,2), tested on Juno / DB410
Note A: Moving forward to topologies which have groups of sources going to dedicated sinks e.g. multisocket / multicluster where each socket / cluster has a sink, then multiple sinks can be marked as principal sink, one for each cluster. The same applies to 1:1 ETM:sink topolgy were each sink will be marked.
Note B: The current set does not auto-select when using sysfs, but I believe this could easily be added.
[1] https://lists.linaro.org/pipermail/coresight/2020-April/003819.html [2] https://lists.linaro.org/pipermail/coresight/2020-April/003821.html
Mike Leach (5): coresight: Add principal sink handling to CoreSight base coresight: tmc: Add principal sink attribute detection coresight: etm: perf: Add default sink selection to etm perf perf: cs-etm: Allow no CoreSight sink to be specified on command line dt-bindings: arm: qcom: Add CoreSight principal sink select parameters
arch/arm64/boot/dts/arm/juno-base.dtsi | 1 + arch/arm64/boot/dts/qcom/msm8916.dtsi | 1 + .../hwtracing/coresight/coresight-etm-perf.c | 20 +++++++++-- drivers/hwtracing/coresight/coresight-priv.h | 2 ++ drivers/hwtracing/coresight/coresight-tmc.c | 9 +++++ drivers/hwtracing/coresight/coresight.c | 34 +++++++++++++++++++ include/linux/coresight.h | 7 ++++ tools/perf/arch/arm/util/cs-etm.c | 6 ++-- 8 files changed, 74 insertions(+), 6 deletions(-)
-- 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK
CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-
Al Grant -
Mathieu Poirier -
Mike Leach -
Suzuki K Poulose -
zied guermazi