Any loaded configurations must be correctly unloaded on coresight module
exit, or issues can arise with nested locking in the configfs directory
code if built with CONFIG_LOCKDEP.
Prior to this patch, the preloaded configuration configfs directory entries
were being unloaded by the recursive code in
configfs_unregister_subsystem().
However, when built with CONFIG_LOCKDEP, this caused a nested lock warning,
which was not mitigated by the LOCKDEP dependent code in fs/configfs/dir.c
designed to prevent this, due to the different directory levels for the
root of the directory being removed.
As the preloaded (and all other) configurations are registered after
configfs_register_subsystem(), we now explicitly unload them before the
call to configfs_unregister_subsystem().
The new routine cscfg_unload_cfgs_on_exit() iterates through the load
owner list to unload any remaining configurations that were not unloaded
by the user before the module exits. This covers both the
CSCFG_OWNER_PRELOAD and CSCFG_OWNER_MODULE owner types, and will be
extended to cover future load owner types for CoreSight configurations.
Applies to coresight/next
Fixes: eb2ec49606c2 ("coresight: syscfg: Update load API for config loadable modules")
Reported-by: Suzuki Poulose <suzuki.poulose(a)arm.com>
Signed-off-by: Mike Leach <mike.leach(a)linaro.org>
---
.../hwtracing/coresight/coresight-syscfg.c | 57 +++++++++++++++++--
1 file changed, 52 insertions(+), 5 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-syscfg.c b/drivers/hwtracing/coresight/coresight-syscfg.c
index 11850fd8c3b5..75df0f865355 100644
--- a/drivers/hwtracing/coresight/coresight-syscfg.c
+++ b/drivers/hwtracing/coresight/coresight-syscfg.c
@@ -1056,14 +1056,61 @@ static int cscfg_create_device(void)
return err;
}
-static void cscfg_clear_device(void)
+/*
+ * Loading and unloading is generally on user discretion.
+ * If exiting due to coresight module unload, we need to unload any configurations that remain,
+ * before we unregister the configfs intrastructure.
+ *
+ * Do this by walking the load_owner list and taking appropriate action, depending on the load
+ * owner type.
+ *
+ * called with the cscfg_mutex held
+ */
+
+#define LOADABLE_MOD_ERR "cscfg: ERROR - a loadable module failed to unload configs on exit\n"
+
+static void cscfg_unload_cfgs_on_exit(void)
{
- struct cscfg_config_desc *cfg_desc;
+ struct cscfg_load_owner_info *owner_info = NULL;
- mutex_lock(&cscfg_mutex);
- list_for_each_entry(cfg_desc, &cscfg_mgr->config_desc_list, item) {
- etm_perf_del_symlink_cscfg(cfg_desc);
+ while (!list_empty(&cscfg_mgr->load_order_list)) {
+
+ /* remove in reverse order of loading */
+ owner_info = list_last_entry(&cscfg_mgr->load_order_list,
+ struct cscfg_load_owner_info, item);
+
+ /* action according to type */
+ switch (owner_info->type) {
+ case CSCFG_OWNER_PRELOAD:
+ /*
+ * preloaded descriptors are statically allocated in
+ * this module - just need to unload dynamic items from
+ * csdev lists, and remove from configfs directories.
+ */
+ pr_info("cscfg: unloading preloaded configurations\n");
+ cscfg_unload_owned_cfgs_feats(owner_info);
+ break;
+
+ case CSCFG_OWNER_MODULE:
+ /*
+ * this is an error - the loadable module must have been unloaded prior
+ * to the coresight module unload. Therefore that module has not
+ * correctly unloaded configs in its own exit code.
+ * Nothing to do other than emit an error string.
+ */
+ pr_err(LOADABLE_MOD_ERR);
+ break;
+ }
+
+ /* remove from load order list */
+ list_del(&owner_info->item);
}
+}
+
+static void cscfg_clear_device(void)
+{
+ mutex_lock(&cscfg_mutex);
+ cscfg_unload_cfgs_on_exit();
cscfg_configfs_release(cscfg_mgr);
device_unregister(cscfg_device());
mutex_unlock(&cscfg_mutex);
--
2.17.1
Apologies for the delay.
Changes since v2:
* Pick up some of Mike's review tags
* Add a comment explaining rationale for not opening the event when BB isn't
supported
* Extend docs to say that Perf doesn't support decode when binaries are
modified
* Drop Perf side patches that were already merged
Thanks
James
James Clark (4):
coresight: Add config flag to enable branch broadcast
Documentation: coresight: Turn numbered subsections into real
subsections
Documentation: coresight: Link config options to existing
documentation
Documentation: coresight: Expand branch broadcast documentation
.../coresight/coresight-etm4x-reference.rst | 17 +++++-
Documentation/trace/coresight/coresight.rst | 56 +++++++++++++++++--
.../hwtracing/coresight/coresight-etm-perf.c | 2 +
.../coresight/coresight-etm4x-core.c | 14 +++++
include/linux/coresight-pmu.h | 2 +
5 files changed, 85 insertions(+), 6 deletions(-)
--
2.28.0
Hi Chenkai,
I am taking the liberty of CC'ing the coresight mailing list so that
people with more experience with the HW implementation can chime in.
On Tue, 24 May 2022 at 05:07, 陈凯 <chenkai.system(a)bytedance.com> wrote:
>
> Dear Poirier,
> I‘m recently read the Linux driver of coresight, I have a question for you about the coresight driver ;
That is a lot of reading.
> I have a SOC(without arm core) ,in which we integrate the ARM CoreSight;
Interesting.
> Is the CoreSight driver still compatible for the soc without ETM,beacuse SOC donot include arm core;
> and look forwarding to your replay.
>
The coresight framework does not specifically need an ETM device to
work and will transport traces generated by any kind of source that
comply with the specification. But you will need to supply your own
decoder in the perf tools if you want to take advantage of the perf
infrastructure.
>
> Best regards,
> ChenKai
The current method for allocating trace source ID values to sources is
to use a fixed algorithm for CPU based sources of (cpu_num * 2 + 0x10).
The STM is allocated ID 0x1.
This fixed algorithm is used in both the CoreSight driver code, and by
perf when writing the trace metadata in the AUXTRACE_INFO record.
The method needs replacing as currently:-
1. It is inefficient in using available IDs.
2. Does not scale to larger systems with many cores and the algorithm
has no limits so will generate invalid trace IDs for cpu number > 44.
Additionally requirements to allocate additional system IDs on some
systems have been seen.
This patch set introduces an API that allows the allocation of trace IDs
in a dynamic manner.
Architecturally reserved IDs are never allocated, and the system is
limited to allocating only valid IDs.
Each of the current trace sources ETM3.x, ETM4.x and STM is updated to use
the new API.
perf handling is changed so that the ID associated with the CPU is read
from sysfs. The ID allocator is notified when perf events start and stop
so CPU based IDs are kept constant throughout any perf session.
For the ETMx.x devices IDs are allocated on certain events
a) When using sysfs, an ID will be allocated on hardware enable, and freed
when the sysfs reset is written.
b) When using perf, ID is allocated on hardware enable, and freed on
hardware disable.
For both cases the ID is allocated when sysfs is read to get the current
trace ID. This ensures that consistent decode metadata can be extracted
from the system where this read occurs before device enable.
Note: This patchset breaks backward compatibility for perf record.
Because the method for generating the AUXTRACE_INFO meta data has
changed, using an older perf record will result in metadata that
does not match the trace IDs used in the recorded trace data.
This mismatch will cause subsequent decode to fail. Older versions of
perf will still be able to decode data generated by the updated system.
Applies to coresight/next [b54f53bc11a5]
Tested on DB410c
Mike Leach (10):
coresight: trace-id: Add API to dynamically assign trace ID values
coresight: trace-id: Set up source trace ID map for system
coresight: stm: Update STM driver to use Trace ID api
coresight: etm4x: Use trace ID API to dynamically allocate trace ID
coresight: etm3x: Use trace ID API to allocate IDs
coresight: perf: traceid: Add perf notifiers for trace ID
perf: cs-etm: Update event to read trace ID from sysfs
coresight: Remove legacy Trace ID allocation mechanism
coresight: etmX.X: stm: Remove unused legacy source trace ID ops
coresight: trace-id: Add debug & test macros to trace id allocation
drivers/hwtracing/coresight/Makefile | 2 +-
drivers/hwtracing/coresight/coresight-core.c | 64 ++---
.../hwtracing/coresight/coresight-etm-perf.c | 16 +-
drivers/hwtracing/coresight/coresight-etm.h | 3 +-
.../coresight/coresight-etm3x-core.c | 93 ++++---
.../coresight/coresight-etm3x-sysfs.c | 28 +-
.../coresight/coresight-etm4x-core.c | 63 ++++-
.../coresight/coresight-etm4x-sysfs.c | 32 ++-
drivers/hwtracing/coresight/coresight-etm4x.h | 3 +
drivers/hwtracing/coresight/coresight-priv.h | 1 +
drivers/hwtracing/coresight/coresight-stm.c | 49 +---
.../hwtracing/coresight/coresight-trace-id.c | 255 ++++++++++++++++++
.../hwtracing/coresight/coresight-trace-id.h | 69 +++++
include/linux/coresight-pmu.h | 12 -
include/linux/coresight.h | 3 -
tools/perf/arch/arm/util/cs-etm.c | 12 +-
16 files changed, 530 insertions(+), 175 deletions(-)
create mode 100644 drivers/hwtracing/coresight/coresight-trace-id.c
create mode 100644 drivers/hwtracing/coresight/coresight-trace-id.h
--
2.17.1
Hi all,
My colleague Junhao He noticed this issue when tracing CPU48 on
Kunpeng920 platform, log as follows:
[root@localhost ~]# perf record -e cs_etm/@sink_smb1/ -C 48 -o perf.data
taskset -c 48 uname -a
[root@localhost ~]# perf report -D --stdio -i perf.data > perf_48.log
0x270 [0xc8]: failed to process type: 70 [Invalid argument]
Error:
failed to process sample
[root@localhost ~]# perf -v
perf version 5.17.rc4.gdeea22e4af29
[root@localhost ~]# ldd /usr/bin/perf | grep opencsd
libopencsd_c_api.so.1 => /root/lib/libopencsd_c_api.so.1
libopencsd.so.1 => /root/lib/libopencsd.so.1
As (CORESIGHT_ETM_PMU_SEED + (cpu * 2)) is used in
coresight_get_trace_id() to cacualate trace_id, if there are more than
48 CPUs on chip, we will have some ETM device which trace id is
invalid(trace_id = 0 or trace_id > 0x6F). In this situation, we cannot
parse trace data using perf tool.
Perhaps we should make trace_id in the range of 1 to 0x6F in
coresight_get_trace_id()? But there also might be parsing problem if
duplicate trace ID is used during collection.
Any response will be highly appreciated.
Thanks,
Qi
On Sun, May 15, 2022 at 02:18:53PM -0700, Ian Rogers wrote:
[...]
> This looks good to me and will run on python 2. The code would be more
> idiomatic in python3 using f-strings. I'd rather the code was
> idiomatic from the beginning, but others may disagree and prefer
> python 2 compatibility (python 2 is now end of life). f-strings are
> python 3.6+ and so have been supported for 5 years.
Using f-string is the right thing for me, will update.
Thanks for reviewing and suggestion!
Leo