This series addresses CPU power management issues in the CoreSight drivers. For easier review, the patches are organized into two categories:
o Patches 01 ~ 10 focus on CPU power management within the ETM drivers. These patches fix SMP-safe access to the mode, correct context synchronization, and refactor the CPU suspend/resume flows.
o Patches 11 ~ 31 extend CPU power management to cover activated paths, including helpers, links, and sinks. These changes move CPU PM and hotplug notifiers from the ETMv4 driver into the CoreSight core layer.
Summary:
- Patches 01 ~ 03: Fix device mode access in the SMP mode. - Patch 04 : A minor fix for polling bit. - Patches 05 ~ 07: Improve the context synchronization based on the ETM specification (IHI0064H.b) and Arm ARM (ARM DDI 0487 L.a). - Patches 08 ~ 10: Refactor the context save/restore flow in the ETMv4 driver, in the end, the CPU PM callbacks reuse the normal enabling and disabling flows. - Patches 11 ~ 17: Move CPU PM code from ETMv4 driver to the core layer. - Patches 18 ~ 23: Enhance the CTI driver for preparation dynamically controlling CTI devices in PM flows. This includes device mode handling in the CTI driver and distinguishing trace modes (Perf or SysFS). Refactor the PM notifier and improve locking usage (including for syscfg). - Patches 24 ~ 31: Support save and restore context for TRBE and manages activated paths during CPU idle and CPU hotplug.
Verification:
This series has been verified on Juno-r0 and r2 platform.
Stress test script:
#!/usr/bin/bash
echo 1 > /sys/bus/coresight/devices/tmc_etr0/enable_sink
while true; do echo 0 > /sys/devices/system/cpu/cpu2/online; echo 1 > /sys/devices/system/cpu/cpu2/online; done &
while true; do echo 1 > /sys/bus/coresight/devices/etm2/enable_source; echo 0 > /sys/bus/coresight/devices/etm2/enable_source; done &
Test script with kernel module insmod / rmmod:
#!/bin/bash
modules=( "coresight" "coresight_dummy" "coresight_replicator" "coresight_tpiu" "coresight_cpu_debug" "coresight_etb10" "coresight_stm" "coresight_trbe" "coresight_tmc" "coresight_ctcu" "coresight_catu" "coresight_etm4x" "coresight_cti" "coresight_funnel" "coresight_tnoc" ) module_dir="/mnt/build/drivers/hwtracing/coresight/"
if [[ "$1" == "in" ]]; then echo "Inserting modules..." for mod in "${modules[@]}"; do modprobe $mod && echo "Loaded $mod" || echo "Failed to load $mod" done elif [[ "$1" == "rm" ]]; then echo "Removing modules..." # Remove in reverse order to respect dependencies for (( idx=${#modules[@]}-1 ; idx>=0 ; idx-- )); do mod="${modules[$idx]}" echo "Removing $mod..." rmmod "$mod" && echo "Removed $mod" || echo "Failed to remove $mod" done fi
--- Changes in v3: - Fixed building failure in ETMv3 driver (kernel test robot). - Refactoring ETMv3 change for checking CPU ID (Levi). - Fixed NULL pointer issue during CPU idle (James). - Fixed lockdep complaint for HARDIRQ-safe and HARDIRA-unsafe (James). - Fixed acquiring mutex in atomic context (James). - Rebased on the latest coresight-next branch. - Link to v2: https://lore.kernel.org/r/20250701-arm_cs_pm_fix_v3-v2-0-23ebb864fcc1@arm.co...
Changes in v2: - Refactored ETMv4 suspend and resume for reusing the normal enabling and disabling flows (James). - Used a per-CPU structure to maintain path pointers (James). - Supported helpers in CPU PM flows (James). - Fixed the SMP-safe access to device mode. - Fixed the context synchronization in ETMv4x driver. - Link to v1: https://lore.kernel.org/linux-arm-kernel/20250516160742.1200904-1-leo.yan@ar...
Signed-off-by: Leo Yan leo.yan@arm.com
--- Leo Yan (30): coresight: Change device mode to atomic type coresight: etm4x: Always set tracer's device mode on target CPU coresight: etm3x: Always set tracer's device mode on target CPU coresight: etm4x: Correct polling IDLE bit coresight: etm4x: Ensure context synchronization is not ignored coresight: etm4x: Add context synchronization before enabling trace coresight: etm4x: Properly control filter in CPU idle with FEAT_TRF coresight: etm4x: Remove the state_needs_restore flag coresight: etm4x: Add flag to control single-shot restart coresight: etm4x: Reuse normal enable and disable logic in CPU idle coresight: Populate CPU ID into the coresight_device structure coresight: sysfs: Validate CPU online status for per-CPU sources coresight: Set per CPU source pointer coresight: Register CPU PM notifier in core layer coresight: etm4x: Hook CPU PM callbacks coresight: Add callback to determine if context save/restore is needed coresight: etm4x: Remove redundant condition checks in save and restore coresight: cti: Fix race condition by using device mode coresight: cti: Introduce CS_MODE_DEBUG mode coresight: cti: Register PM notifier after data initialization coresight: cti: Properly handle modes in CPU PM notifiers coresight: cti: Make spin lock usage consistent coresight: syscfg: Use spinlock to protect active variables coresight: Add per-CPU path pointer coresight: Add 'in_idle' argument to path enable/disable functions coresight: Control path during CPU idle coresight: Add PM callbacks for percpu sink coresight: Take hotplug lock in enable_source_store() for Sysfs mode coresight: Move CPU hotplug callbacks to core layer coresight: Manage activated path during CPU hotplug
Yabin Cui (1): coresight: trbe: Save and restore state across CPU low power state
drivers/hwtracing/coresight/coresight-catu.c | 1 + drivers/hwtracing/coresight/coresight-core.c | 337 ++++++++++++-- drivers/hwtracing/coresight/coresight-ctcu-core.c | 1 + drivers/hwtracing/coresight/coresight-cti-core.c | 69 ++- drivers/hwtracing/coresight/coresight-cti-sysfs.c | 2 +- drivers/hwtracing/coresight/coresight-dummy.c | 1 + drivers/hwtracing/coresight/coresight-etb10.c | 1 + drivers/hwtracing/coresight/coresight-etm3x-core.c | 55 ++- drivers/hwtracing/coresight/coresight-etm4x-core.c | 491 ++++++--------------- drivers/hwtracing/coresight/coresight-etm4x.h | 62 --- drivers/hwtracing/coresight/coresight-funnel.c | 1 + drivers/hwtracing/coresight/coresight-replicator.c | 1 + drivers/hwtracing/coresight/coresight-stm.c | 1 + drivers/hwtracing/coresight/coresight-syscfg.c | 22 +- drivers/hwtracing/coresight/coresight-syscfg.h | 2 + drivers/hwtracing/coresight/coresight-sysfs.c | 10 + drivers/hwtracing/coresight/coresight-tmc-core.c | 1 + drivers/hwtracing/coresight/coresight-tpda.c | 1 + drivers/hwtracing/coresight/coresight-tpdm.c | 1 + drivers/hwtracing/coresight/coresight-tpiu.c | 1 + drivers/hwtracing/coresight/coresight-trbe.c | 85 ++++ drivers/hwtracing/coresight/ultrasoc-smb.c | 1 + include/linux/coresight.h | 55 ++- 23 files changed, 687 insertions(+), 515 deletions(-) --- base-commit: 559d6c380ea0a27e71d0269410301303515e4179 change-id: 20250909-arm_coresight_power_management_fix-139873f942e8
Best regards,
The device mode is defined as local type. This type cannot promise SMP-safe access.
Change to atomic type and impose relax ordering, which ensures the SMP-safe synchronisation and the ordering between the mode setting and relevant operations.
Fixes: 22fd532eaa0c ("coresight: etm3x: adding operation mode for etm_enable()") Signed-off-by: Leo Yan leo.yan@arm.com --- include/linux/coresight.h | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-)
diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 6de59ce8ef8ca45c29e2f09c1b979eb7686b685f..3e5e5acd0c7fcde7d312d440da4355faaf682c7b 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -251,15 +251,11 @@ struct coresight_trace_id_map { * by @coresight_ops. * @access: Device i/o access abstraction for this device. * @dev: The device entity associated to this component. - * @mode: This tracer's mode, i.e sysFS, Perf or disabled. This is - * actually an 'enum cs_mode', but is stored in an atomic type. - * This is always accessed through local_read() and local_set(), - * but wherever it's done from within the Coresight device's lock, - * a non-atomic read would also work. This is the main point of - * synchronisation between code happening inside the sysfs mode's - * coresight_mutex and outside when running in Perf mode. A compare - * and exchange swap is done to atomically claim one mode or the - * other. + * @mode: The device mode, i.e sysFS, Perf or disabled. This is actually + * an 'enum cs_mode' but stored in an atomic type. Access is always + * through atomic APIs, ensuring SMP-safe synchronisation between + * racing from sysFS and Perf mode. A compare-and-exchange + * operation is done to atomically claim one mode or the other. * @refcnt: keep track of what is in use. Only access this outside of the * device's spinlock when the coresight_mutex held and mode == * CS_MODE_SYSFS. Otherwise it must be accessed from inside the @@ -288,7 +284,7 @@ struct coresight_device { const struct coresight_ops *ops; struct csdev_access access; struct device dev; - local_t mode; + atomic_t mode; int refcnt; bool orphan; /* sink specific fields */ @@ -621,13 +617,14 @@ static inline bool coresight_is_percpu_sink(struct coresight_device *csdev) static inline bool coresight_take_mode(struct coresight_device *csdev, enum cs_mode new_mode) { - return local_cmpxchg(&csdev->mode, CS_MODE_DISABLED, new_mode) == - CS_MODE_DISABLED; + int curr = CS_MODE_DISABLED; + + return atomic_try_cmpxchg_acquire(&csdev->mode, &curr, new_mode); }
static inline enum cs_mode coresight_get_mode(struct coresight_device *csdev) { - return local_read(&csdev->mode); + return atomic_read_acquire(&csdev->mode); }
static inline void coresight_set_mode(struct coresight_device *csdev, @@ -643,7 +640,7 @@ static inline void coresight_set_mode(struct coresight_device *csdev, WARN(new_mode != CS_MODE_DISABLED && current_mode != CS_MODE_DISABLED && current_mode != new_mode, "Device already in use\n");
- local_set(&csdev->mode, new_mode); + atomic_set_release(&csdev->mode, new_mode); }
struct coresight_device *coresight_register(struct coresight_desc *desc);
When enabling a tracer via SysFS interface, the device mode may be set by any CPU - not necessarily the target CPU. This can lead to race condition in SMP, and may result in incorrect mode values being read.
Consider the following example, where CPU0 attempts to enable the tracer on CPU1 (the target CPU):
CPU0 CPU1 etm4_enable() ` coresight_take_mode(SYSFS) ` etm4_enable_sysfs() ` smp_call_function_single() ----> etm4_enable_hw_smp_call() / / CPU idle: / etm4_cpu_save() / ` coresight_get_mode() Failed to enable h/w / ^^^ ` coresight_set_mode(DISABLED) <-' Read the intermediate SYSFS mode
In this case, CPU0 initiates the operation by taking the SYSFS mode to avoid conflicts with the Perf mode. It then sends an IPI to CPU1 to configure the tracer registers. If any error occurs during this process, CPU0 rolls back by setting the mode to DISABLED.
However, if CPU1 enters an idle state during this time, it might read the intermediate SYSFS mode. As a result, the CPU PM flow could wrongly save and restore tracer context that is actually disabled.
To resolve the issue, this commit moves the device mode setting logic on the target CPU. This ensures that the device mode is only modified by the target CPU, eliminating race condition between mode writes and reads across CPUs.
An additional change introduces the etm4_disable_hw_smp_call() function for SMP calls, which disables the tracer and explicitly set the mode to DISABLED during SysFS operations.
The flow is updated with this change:
CPU0 CPU1 etm4_enable() ` etm4_enable_sysfs() ` smp_call_function_single() ----> etm4_enable_hw_smp_call() ` coresight_take_mode(SYSFS) Failed, set back to DISABLED ` coresight_set_mode(DISABLED)
CPU idle: etm4_cpu_save() ` coresight_get_mode() ^^^ Read out the DISABLED mode
Fixes: c38a9ec2b2c1 ("coresight: etm4x: moving etm_drvdata::enable to atomic field") Reviewed-by: Yeoreum Yun yeoreum.yun@arm.com Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 48 +++++++++++++++------- 1 file changed, 33 insertions(+), 15 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 020f070bf17dc0557dfb3ae4f282b0a0c1778bd8..02ad41da7356547a67c53ff0a9146aec844f89da 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -592,10 +592,23 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata) static void etm4_enable_hw_smp_call(void *info) { struct etm4_enable_arg *arg = info; + struct coresight_device *csdev;
if (WARN_ON(!arg)) return; + + csdev = arg->drvdata->csdev; + if (!coresight_take_mode(csdev, CS_MODE_SYSFS)) { + /* Someone is already using the tracer */ + arg->rc = -EBUSY; + return; + } + arg->rc = etm4_enable_hw(arg->drvdata); + + /* The tracer didn't start */ + if (arg->rc) + coresight_set_mode(csdev, CS_MODE_DISABLED); }
/* @@ -811,6 +824,9 @@ static int etm4_enable_perf(struct coresight_device *csdev, int ret = 0; struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
+ if (!coresight_take_mode(csdev, CS_MODE_PERF)) + return -EBUSY; + if (WARN_ON_ONCE(drvdata->cpu != smp_processor_id())) { ret = -EINVAL; goto out; @@ -830,6 +846,9 @@ static int etm4_enable_perf(struct coresight_device *csdev, ret = etm4_enable_hw(drvdata);
out: + /* The tracer didn't start */ + if (ret) + coresight_set_mode(csdev, CS_MODE_DISABLED); return ret; }
@@ -882,11 +901,6 @@ static int etm4_enable(struct coresight_device *csdev, struct perf_event *event, { int ret;
- if (!coresight_take_mode(csdev, mode)) { - /* Someone is already using the tracer */ - return -EBUSY; - } - switch (mode) { case CS_MODE_SYSFS: ret = etm4_enable_sysfs(csdev, path); @@ -898,10 +912,6 @@ static int etm4_enable(struct coresight_device *csdev, struct perf_event *event, ret = -EINVAL; }
- /* The tracer didn't start */ - if (ret) - coresight_set_mode(csdev, CS_MODE_DISABLED); - return ret; }
@@ -953,10 +963,9 @@ static void etm4_disable_trace_unit(struct etmv4_drvdata *drvdata) isb(); }
-static void etm4_disable_hw(void *info) +static void etm4_disable_hw(struct etmv4_drvdata *drvdata) { u32 control; - struct etmv4_drvdata *drvdata = info; struct etmv4_config *config = &drvdata->config; struct coresight_device *csdev = drvdata->csdev; struct csdev_access *csa = &csdev->access; @@ -993,6 +1002,15 @@ static void etm4_disable_hw(void *info) "cpu: %d disable smp call done\n", drvdata->cpu); }
+static void etm4_disable_hw_smp_call(void *info) +{ + struct etmv4_drvdata *drvdata = info; + + etm4_disable_hw(drvdata); + + coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED); +} + static int etm4_disable_perf(struct coresight_device *csdev, struct perf_event *event) { @@ -1022,6 +1040,8 @@ static int etm4_disable_perf(struct coresight_device *csdev, /* TRCVICTLR::SSSTATUS, bit[9] */ filters->ssstatus = (control & BIT(9));
+ coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED); + /* * perf will release trace ids when _free_aux() is * called at the end of the session. @@ -1047,7 +1067,8 @@ static void etm4_disable_sysfs(struct coresight_device *csdev) * Executing etm4_disable_hw on the cpu whose ETM is being disabled * ensures that register writes occur when cpu is powered. */ - smp_call_function_single(drvdata->cpu, etm4_disable_hw, drvdata, 1); + smp_call_function_single(drvdata->cpu, etm4_disable_hw_smp_call, + drvdata, 1);
raw_spin_unlock(&drvdata->spinlock);
@@ -1087,9 +1108,6 @@ static void etm4_disable(struct coresight_device *csdev, etm4_disable_perf(csdev, event); break; } - - if (mode) - coresight_set_mode(csdev, CS_MODE_DISABLED); }
static int etm4_resume_perf(struct coresight_device *csdev)
On 15/09/2025 11:33, Leo Yan wrote:
When enabling a tracer via SysFS interface, the device mode may be set by any CPU - not necessarily the target CPU. This can lead to race condition in SMP, and may result in incorrect mode values being read.
Consider the following example, where CPU0 attempts to enable the tracer on CPU1 (the target CPU):
CPU0 CPU1 etm4_enable() ` coresight_take_mode(SYSFS) ` etm4_enable_sysfs() ` smp_call_function_single() ----> etm4_enable_hw_smp_call() / / CPU idle: / etm4_cpu_save() / ` coresight_get_mode() Failed to enable h/w / ^^^ ` coresight_set_mode(DISABLED) <-' Read the intermediate SYSFS mode
In this case, CPU0 initiates the operation by taking the SYSFS mode to avoid conflicts with the Perf mode. It then sends an IPI to CPU1 to configure the tracer registers. If any error occurs during this process, CPU0 rolls back by setting the mode to DISABLED.
However, if CPU1 enters an idle state during this time, it might read the intermediate SYSFS mode. As a result, the CPU PM flow could wrongly save and restore tracer context that is actually disabled.
To resolve the issue, this commit moves the device mode setting logic on the target CPU. This ensures that the device mode is only modified by the target CPU, eliminating race condition between mode writes and reads across CPUs.
An additional change introduces the etm4_disable_hw_smp_call() function for SMP calls, which disables the tracer and explicitly set the mode to DISABLED during SysFS operations.
The flow is updated with this change:
CPU0 CPU1 etm4_enable() ` etm4_enable_sysfs() ` smp_call_function_single() ----> etm4_enable_hw_smp_call() ` coresight_take_mode(SYSFS) Failed, set back to DISABLED ` coresight_set_mode(DISABLED)
CPU idle: etm4_cpu_save() ` coresight_get_mode() ^^^ Read out the DISABLED modeFixes: c38a9ec2b2c1 ("coresight: etm4x: moving etm_drvdata::enable to atomic field") Reviewed-by: Yeoreum Yun yeoreum.yun@arm.com Signed-off-by: Leo Yan leo.yan@arm.com
drivers/hwtracing/coresight/coresight-etm4x-core.c | 48 +++++++++++++++------- 1 file changed, 33 insertions(+), 15 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 020f070bf17dc0557dfb3ae4f282b0a0c1778bd8..02ad41da7356547a67c53ff0a9146aec844f89da 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -592,10 +592,23 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata) static void etm4_enable_hw_smp_call(void *info)
While at this, please could you rename this function to make it explicit that this is only for sysfs mode ?
e.g., etm4_enable_hw_sysfs_smp_call()
{ struct etm4_enable_arg *arg = info;
- struct coresight_device *csdev;
if (WARN_ON(!arg)) return;
- csdev = arg->drvdata->csdev;
- if (!coresight_take_mode(csdev, CS_MODE_SYSFS)) {
/* Someone is already using the tracer */arg->rc = -EBUSY;return;- }
- arg->rc = etm4_enable_hw(arg->drvdata);
- /* The tracer didn't start */
- if (arg->rc)
coresight_set_mode(csdev, CS_MODE_DISABLED);/* @@ -811,6 +824,9 @@ static int etm4_enable_perf(struct coresight_device *csdev, int ret = 0; struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
- if (!coresight_take_mode(csdev, CS_MODE_PERF))
return -EBUSY;
Should be done after the CPU check below ? Otherwise you are undoing the fix on sysfs side.
if (WARN_ON_ONCE(drvdata->cpu != smp_processor_id())) { ret = -EINVAL; goto out; @@ -830,6 +846,9 @@ static int etm4_enable_perf(struct coresight_device *csdev, ret = etm4_enable_hw(drvdata); out:
- /* The tracer didn't start */
- if (ret)
coresight_set_mode(csdev, CS_MODE_DISABLED);@@ -882,11 +901,6 @@ static int etm4_enable(struct coresight_device *csdev, struct perf_event *event, { int ret;
- if (!coresight_take_mode(csdev, mode)) {
/* Someone is already using the tracer */return -EBUSY;- }
- switch (mode) { case CS_MODE_SYSFS: ret = etm4_enable_sysfs(csdev, path);
@@ -898,10 +912,6 @@ static int etm4_enable(struct coresight_device *csdev, struct perf_event *event, ret = -EINVAL; }
- /* The tracer didn't start */
- if (ret)
coresight_set_mode(csdev, CS_MODE_DISABLED);- return ret; }
@@ -953,10 +963,9 @@ static void etm4_disable_trace_unit(struct etmv4_drvdata *drvdata) isb(); } -static void etm4_disable_hw(void *info) +static void etm4_disable_hw(struct etmv4_drvdata *drvdata) { u32 control;
- struct etmv4_drvdata *drvdata = info; struct etmv4_config *config = &drvdata->config; struct coresight_device *csdev = drvdata->csdev; struct csdev_access *csa = &csdev->access;
@@ -993,6 +1002,15 @@ static void etm4_disable_hw(void *info) "cpu: %d disable smp call done\n", drvdata->cpu); } +static void etm4_disable_hw_smp_call(void *info)
sysfs mode again.
Rest looks fine
Suzuki
+{
- struct etmv4_drvdata *drvdata = info;
- etm4_disable_hw(drvdata);
- coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED);
+}
- static int etm4_disable_perf(struct coresight_device *csdev, struct perf_event *event) {
@@ -1022,6 +1040,8 @@ static int etm4_disable_perf(struct coresight_device *csdev, /* TRCVICTLR::SSSTATUS, bit[9] */ filters->ssstatus = (control & BIT(9));
- coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED);
- /*
- perf will release trace ids when _free_aux() is
- called at the end of the session.
@@ -1047,7 +1067,8 @@ static void etm4_disable_sysfs(struct coresight_device *csdev) * Executing etm4_disable_hw on the cpu whose ETM is being disabled * ensures that register writes occur when cpu is powered. */
- smp_call_function_single(drvdata->cpu, etm4_disable_hw, drvdata, 1);
- smp_call_function_single(drvdata->cpu, etm4_disable_hw_smp_call,
drvdata, 1);raw_spin_unlock(&drvdata->spinlock); @@ -1087,9 +1108,6 @@ static void etm4_disable(struct coresight_device *csdev, etm4_disable_perf(csdev, event); break; }
- if (mode)
coresight_set_mode(csdev, CS_MODE_DISABLED);static int etm4_resume_perf(struct coresight_device *csdev)
On Mon, Sep 15, 2025 at 11:47:43AM +0100, Suzuki Kuruppassery Poulose wrote:
[...]
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 020f070bf17dc0557dfb3ae4f282b0a0c1778bd8..02ad41da7356547a67c53ff0a9146aec844f89da 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -592,10 +592,23 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata) static void etm4_enable_hw_smp_call(void *info)
While at this, please could you rename this function to make it explicit that this is only for sysfs mode ?
e.g., etm4_enable_hw_sysfs_smp_call()
If no objection, I will update pair functions to
etm4_enable_sysfs_smp_call() / etm4_disable_sysfs_smp_call()
[...]
@@ -811,6 +824,9 @@ static int etm4_enable_perf(struct coresight_device *csdev, int ret = 0; struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
- if (!coresight_take_mode(csdev, CS_MODE_PERF))
return -EBUSY;Should be done after the CPU check below ? Otherwise you are undoing the fix on sysfs side.
ETMv3 driver has done in the suggested way for checking CPU first. ETMv4 driver should align with it. I will update in next version.
Thanks, Leo
The ETMv3 driver shares the same issue as ETMv4 regarding race conditions when accessing the device mode.
This commit applies the same fix: ensuring that the device mode is modified only by the target CPU to eliminate race conditions across CPUs.
Fixes: 22fd532eaa0c ("coresight: etm3x: adding operation mode for etm_enable()") Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm3x-core.c | 54 +++++++++++++++------- 1 file changed, 37 insertions(+), 17 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm3x-core.c b/drivers/hwtracing/coresight/coresight-etm3x-core.c index 45630a1cd32fbd05ec8b2a6979f0174cacce365e..f759a47eaed9e804e30530a232370c9504912ac6 100644 --- a/drivers/hwtracing/coresight/coresight-etm3x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm3x-core.c @@ -442,10 +442,23 @@ struct etm_enable_arg { static void etm_enable_hw_smp_call(void *info) { struct etm_enable_arg *arg = info; + struct coresight_device *csdev;
if (WARN_ON(!arg)) return; + + csdev = arg->drvdata->csdev; + if (!coresight_take_mode(csdev, CS_MODE_SYSFS)) { + /* Someone is already using the tracer */ + arg->rc = -EBUSY; + return; + } + arg->rc = etm_enable_hw(arg->drvdata); + + /* The tracer didn't start */ + if (arg->rc) + coresight_set_mode(csdev, CS_MODE_DISABLED); }
static int etm_cpu_id(struct coresight_device *csdev) @@ -464,17 +477,26 @@ static int etm_enable_perf(struct coresight_device *csdev, struct perf_event *event, struct coresight_path *path) { + int ret = 0; struct etm_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
if (WARN_ON_ONCE(drvdata->cpu != smp_processor_id())) return -EINVAL;
+ if (!coresight_take_mode(csdev, CS_MODE_PERF)) + return -EBUSY; + /* Configure the tracer based on the session's specifics */ etm_parse_event_config(drvdata, event); drvdata->traceid = path->trace_id;
/* And enable it */ - return etm_enable_hw(drvdata); + ret = etm_enable_hw(drvdata); + /* The tracer didn't start */ + if (ret) + coresight_set_mode(csdev, CS_MODE_DISABLED); + + return ret; }
static int etm_enable_sysfs(struct coresight_device *csdev, struct coresight_path *path) @@ -517,12 +539,6 @@ static int etm_enable(struct coresight_device *csdev, struct perf_event *event, enum cs_mode mode, struct coresight_path *path) { int ret; - struct etm_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); - - if (!coresight_take_mode(csdev, mode)) { - /* Someone is already using the tracer */ - return -EBUSY; - }
switch (mode) { case CS_MODE_SYSFS: @@ -535,17 +551,12 @@ static int etm_enable(struct coresight_device *csdev, struct perf_event *event, ret = -EINVAL; }
- /* The tracer didn't start */ - if (ret) - coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED); - return ret; }
-static void etm_disable_hw(void *info) +static void etm_disable_hw(struct etm_drvdata *drvdata) { int i; - struct etm_drvdata *drvdata = info; struct etm_config *config = &drvdata->config; struct coresight_device *csdev = drvdata->csdev;
@@ -567,6 +578,15 @@ static void etm_disable_hw(void *info) "cpu: %d disable smp call done\n", drvdata->cpu); }
+static void etm_disable_hw_smp_call(void *info) +{ + struct etm_drvdata *drvdata = info; + + etm_disable_hw(drvdata); + + coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED); +} + static void etm_disable_perf(struct coresight_device *csdev) { struct etm_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); @@ -588,6 +608,8 @@ static void etm_disable_perf(struct coresight_device *csdev)
CS_LOCK(drvdata->csa.base);
+ coresight_set_mode(drvdata->csdev, CS_MODE_DISABLED); + /* * perf will release trace ids when _free_aux() * is called at the end of the session @@ -612,7 +634,8 @@ static void etm_disable_sysfs(struct coresight_device *csdev) * Executing etm_disable_hw on the cpu whose ETM is being disabled * ensures that register writes occur when cpu is powered. */ - smp_call_function_single(drvdata->cpu, etm_disable_hw, drvdata, 1); + smp_call_function_single(drvdata->cpu, etm_disable_hw_smp_call, + drvdata, 1);
spin_unlock(&drvdata->spinlock); cpus_read_unlock(); @@ -652,9 +675,6 @@ static void etm_disable(struct coresight_device *csdev, WARN_ON_ONCE(mode); return; } - - if (mode) - coresight_set_mode(csdev, CS_MODE_DISABLED); }
static const struct coresight_ops_source etm_source_ops = {
Since commit 4ff6039ffb79 ("coresight-etm4x: add isb() before reading the TRCSTATR"), the code has incorrectly been polling the PMSTABLE bit instead of the IDLE bit.
This commit corrects the typo.
Fixes: 4ff6039ffb79 ("coresight-etm4x: add isb() before reading the TRCSTATR") Reviewed-by: Yeoreum Yun yeoreum.yun@arm.com Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 02ad41da7356547a67c53ff0a9146aec844f89da..382ec1a1cdd035e76d451d0112afd3f029ecbdb6 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1926,7 +1926,7 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) state->trcpdcr = etm4x_read32(csa, TRCPDCR);
/* wait for TRCSTATR.IDLE to go up */ - if (etm4x_wait_status(csa, TRCSTATR_PMSTABLE_BIT, 1)) { + if (etm4x_wait_status(csa, TRCSTATR_IDLE_BIT, 1)) { dev_err(etm_dev, "timeout while waiting for Idle Trace Status\n"); etm4_os_unlock(drvdata);
As recommended in section 4.3.7 "Synchronization of register updates" of ARM IHI0064H.b, a self-hosted trace analyzer should always executes an ISB instruction after programming the trace unit registers.
An ISB works as a context synchronization event; a DSB is not required. Removes the redundant barrier in the enabling flow.
The ISB was placed at the end of the enable and disable functions. However, this does not guarantee a context synchronization event in the calling code, which may speculatively execute across function boundaries.
ISB instructions are moved into callers to ensure that a context synchronization is imposed immediately after enabling or disabling trace unit.
Fixes: 40f682ae5086 ("coresight: etm4x: Extract the trace unit controlling") Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 38 +++++++++++++++------- 1 file changed, 26 insertions(+), 12 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 382ec1a1cdd035e76d451d0112afd3f029ecbdb6..31d4838b47ae4a3a96c6a06774707214943268fc 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -460,13 +460,6 @@ static int etm4_enable_trace_unit(struct etmv4_drvdata *drvdata) return -ETIME; }
- /* - * As recommended by section 4.3.7 ("Synchronization when using the - * memory-mapped interface") of ARM IHI 0064D - */ - dsb(sy); - isb(); - return 0; }
@@ -581,6 +574,13 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata)
if (!drvdata->paused) rc = etm4_enable_trace_unit(drvdata); + + /* + * As recommended by section 4.3.7 (Synchronization of register updates) + * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an + * ISB instruction after programming the trace unit registers. + */ + isb(); done: etm4_cs_lock(drvdata, csa);
@@ -956,11 +956,6 @@ static void etm4_disable_trace_unit(struct etmv4_drvdata *drvdata) if (etm4x_wait_status(csa, TRCSTATR_PMSTABLE_BIT, 1)) dev_err(etm_dev, "timeout while waiting for PM stable Trace Status\n"); - /* - * As recommended by section 4.3.7 (Synchronization of register updates) - * of ARM IHI 0064H.b. - */ - isb(); }
static void etm4_disable_hw(struct etmv4_drvdata *drvdata) @@ -983,6 +978,13 @@ static void etm4_disable_hw(struct etmv4_drvdata *drvdata)
etm4_disable_trace_unit(drvdata);
+ /* + * As recommended by section 4.3.7 (Synchronization of register updates) + * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an + * ISB instruction after programming the trace unit registers. + */ + isb(); + /* read the status of the single shot comparators */ for (i = 0; i < drvdata->nr_ss_cmp; i++) { config->ss_status[i] = @@ -1120,6 +1122,12 @@ static int etm4_resume_perf(struct coresight_device *csdev)
etm4_cs_unlock(drvdata, csa); etm4_enable_trace_unit(drvdata); + /* + * As recommended by section 4.3.7 (Synchronization of register updates) + * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an + * ISB instruction after programming the trace unit registers. + */ + isb(); etm4_cs_lock(drvdata, csa);
drvdata->paused = false; @@ -1136,6 +1144,12 @@ static void etm4_pause_perf(struct coresight_device *csdev)
etm4_cs_unlock(drvdata, csa); etm4_disable_trace_unit(drvdata); + /* + * As recommended by section 4.3.7 (Synchronization of register updates) + * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an + * ISB instruction after programming the trace unit registers. + */ + isb(); etm4_cs_lock(drvdata, csa);
drvdata->paused = true;
According to the software usage PKLXF in Arm ARM (ARM DDI 0487 L.a), a Context synchronization event is required before enabling the trace unit. An ISB is added to meet this requirement.
Improved the barrier comments to provide more accurate information by specifying which section of the document the requirement comes from and clarifying its intended purpose.
Fixes: 1ab3bb9df5e3 ("coresight: etm4x: Add necessary synchronization for sysreg access") Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 37 +++++++++++++++++++--- 1 file changed, 33 insertions(+), 4 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 31d4838b47ae4a3a96c6a06774707214943268fc..999bceef4cd771e46aec891f4c814a95e0921c2f 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -446,13 +446,37 @@ static int etm4_enable_trace_unit(struct etmv4_drvdata *drvdata) etm4x_relaxed_write32(csa, TRCRSR_TA, TRCRSR);
etm4x_allow_trace(drvdata); + + /* + * According to software usage PKLXF in Arm ARM (ARM DDI 0487 L.a), + * execute a Context synchronization event to guarantee the trace unit + * will observe the new values of the System registers. + */ + if (!csa->io_mem) + isb(); + /* Enable the trace unit */ etm4x_relaxed_write32(csa, 1, TRCPRGCTLR);
- /* Synchronize the register updates for sysreg access */ + /* + * As recommended by section 4.3.7 ("Synchronization when using system + * instructions to progrom the trace unit") of ARM IHI 0064H.b, the + * self-hosted trace analyzer must perform a Context synchronization + * event between writing to the TRCPRGCTLR and reading the TRCSTATR. + */ if (!csa->io_mem) isb();
+ /* + * For the memory-mapped interface, the registers are mapped as Device + * type (Device-nGnRE). As described in section 4.3.7 ("Synchronization + * of register updates") of ARM IHI0064H.b, read back the value of any + * register in the trace unit to ensure writes have completed. + * + * Therefore, polling on TRCSTATR ensures that the writing TRCPRGCTLR + * is complete. + */ + /* wait for TRCSTATR.IDLE to go back down to '0' */ if (etm4x_wait_status(csa, TRCSTATR_IDLE_BIT, 0)) { dev_err(etm_dev, @@ -933,11 +957,16 @@ static void etm4_disable_trace_unit(struct etmv4_drvdata *drvdata) */ etm4x_prohibit_trace(drvdata); /* - * Make sure everything completes before disabling, as recommended - * by section 7.3.77 ("TRCVICTLR, ViewInst Main Control Register, - * SSTATUS") of ARM IHI 0064D + * Prevent being speculative at the point of disabling the trace unit, + * as recommended by section 7.3.77 ("TRCVICTLR, ViewInst Main Control + * Register, SSTATUS") of ARM IHI 0064D */ dsb(sy); + /* + * According to software usage VKHHY in Arm ARM (ARM DDI 0487 L.a), + * execute a Context synchronization event to guarantee no new + * program-flow trace is generated. + */ isb(); /* Trace synchronization barrier, is a nop if not supported */ tsb_csync();
If a CPU supports FEAT_TRF, as described in the section K5.5 "Context switching", Arm ARM (ARM DDI 0487 L.a), it defines a flow to prohibit program-flow trace, execute a TSB CSYNC instruction for flushing, followed by clearing TRCPRGCTLR.EN bit.
To restore the state, the reverse sequence is required.
This differs from the procedure described in the section 3.4.1 "The procedure when powering down the PE" of ARM IHI0064H.b, which involves the OS Lock to prevent external debugger accesses and implicitly disables trace.
To be compatible with different ETM versions, explicitly control trace unit using etm4_disable_trace_unit() and etm4_enable_trace_unit() during CPU idle to comply with FEAT_TRF.
As a result, the save states for TRFCR_ELx and trcprgctlr are redundant, remove them.
Fixes: f188b5e76aae ("coresight: etm4x: Save/restore state across CPU low power states") Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 28 ++++++++++++++++------ drivers/hwtracing/coresight/coresight-etm4x.h | 3 --- 2 files changed, 21 insertions(+), 10 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 999bceef4cd771e46aec891f4c814a95e0921c2f..2d2ef58393db10d058b3e29c5b270dd2d62e1a49 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1884,9 +1884,18 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) goto out; }
+ if (!drvdata->paused) + etm4_disable_trace_unit(drvdata); + + /* + * As recommended by section 4.3.7 (Synchronization of register updates) + * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an + * ISB instruction after programming the trace unit registers. + */ + isb(); + state = drvdata->save_state;
- state->trcprgctlr = etm4x_read32(csa, TRCPRGCTLR); if (drvdata->nr_pe) state->trcprocselr = etm4x_read32(csa, TRCPROCSELR); state->trcconfigr = etm4x_read32(csa, TRCCONFIGR); @@ -1996,9 +2005,6 @@ static int etm4_cpu_save(struct etmv4_drvdata *drvdata) { int ret = 0;
- /* Save the TRFCR irrespective of whether the ETM is ON */ - if (drvdata->trfcr) - drvdata->save_trfcr = read_trfcr(); /* * Save and restore the ETM Trace registers only if * the ETM is active. @@ -2020,7 +2026,6 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata) etm4_cs_unlock(drvdata, csa); etm4x_relaxed_write32(csa, state->trcclaimset, TRCCLAIMSET);
- etm4x_relaxed_write32(csa, state->trcprgctlr, TRCPRGCTLR); if (drvdata->nr_pe) etm4x_relaxed_write32(csa, state->trcprocselr, TRCPROCSELR); etm4x_relaxed_write32(csa, state->trcconfigr, TRCCONFIGR); @@ -2105,13 +2110,22 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata)
/* Unlock the OS lock to re-enable trace and external debug access */ etm4_os_unlock(drvdata); + + if (!drvdata->paused) + etm4_enable_trace_unit(drvdata); + + /* + * As recommended by section 4.3.7 (Synchronization of register updates) + * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an + * ISB instruction after programming the trace unit registers. + */ + isb(); + etm4_cs_lock(drvdata, csa); }
static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) { - if (drvdata->trfcr) - write_trfcr(drvdata->save_trfcr); if (drvdata->state_needs_restore) __etm4_cpu_restore(drvdata); } diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h index 13ec9ecef46f5b60a38c464e8ebde3ef72e7d50b..b8796b4271025f6aba79fe3cd6e4d17d0b56952d 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.h +++ b/drivers/hwtracing/coresight/coresight-etm4x.h @@ -866,7 +866,6 @@ struct etmv4_config { * struct etm4_save_state - state to be preserved when ETM is without power */ struct etmv4_save_state { - u32 trcprgctlr; u32 trcprocselr; u32 trcconfigr; u32 trcauxctlr; @@ -980,7 +979,6 @@ struct etmv4_save_state { * at runtime, due to the additional setting of TRFCR_CX when * in EL2. Otherwise, 0. * @config: structure holding configuration parameters. - * @save_trfcr: Saved TRFCR_EL1 register during a CPU PM event. * @save_state: State to be preserved across power loss * @state_needs_restore: True when there is context to restore after PM exit * @skip_power_up: Indicates if an implementation can skip powering up @@ -1037,7 +1035,6 @@ struct etmv4_drvdata { bool lpoverride; u64 trfcr; struct etmv4_config config; - u64 save_trfcr; struct etmv4_save_state *save_state; bool state_needs_restore; bool skip_power_up;
When the restore flow is invoked, it means no error occurred during the save phase. Otherwise, if any errors happened while saving the context, the function would return an error and abort the suspend sequence.
Therefore, the state_needs_restore flag is unnecessary. The save and restore functions are changed to check two conditions:
1) The global flag pm_save_enable is SELF_HOSTED mode; 2) The device is in active mode (non DISABLED).
Reviewed-by: Yeoreum Yun yeoreum.yun@arm.com Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 14 ++++++++------ drivers/hwtracing/coresight/coresight-etm4x.h | 2 -- 2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 2d2ef58393db10d058b3e29c5b270dd2d62e1a49..fbcd0fcd1992cd600d2eb8c95ae9a5adf0c287da 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1986,8 +1986,6 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) goto out; }
- drvdata->state_needs_restore = true; - /* * Power can be removed from the trace unit now. We do this to * potentially save power on systems that respect the TRCPDCR_PU @@ -2005,11 +2003,14 @@ static int etm4_cpu_save(struct etmv4_drvdata *drvdata) { int ret = 0;
+ if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) + return 0; + /* * Save and restore the ETM Trace registers only if * the ETM is active. */ - if (coresight_get_mode(drvdata->csdev) && drvdata->save_state) + if (coresight_get_mode(drvdata->csdev)) ret = __etm4_cpu_save(drvdata); return ret; } @@ -2099,8 +2100,6 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata) if (!drvdata->skip_power_up) etm4x_relaxed_write32(csa, state->trcpdcr, TRCPDCR);
- drvdata->state_needs_restore = false; - /* * As recommended by section 4.3.7 ("Synchronization when using the * memory-mapped interface") of ARM IHI 0064D @@ -2126,7 +2125,10 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata)
static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) { - if (drvdata->state_needs_restore) + if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) + return; + + if (coresight_get_mode(drvdata->csdev)) __etm4_cpu_restore(drvdata); }
diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h index b8796b4271025f6aba79fe3cd6e4d17d0b56952d..012c52fd19338133129752d35523dc102df24604 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.h +++ b/drivers/hwtracing/coresight/coresight-etm4x.h @@ -980,7 +980,6 @@ struct etmv4_save_state { * in EL2. Otherwise, 0. * @config: structure holding configuration parameters. * @save_state: State to be preserved across power loss - * @state_needs_restore: True when there is context to restore after PM exit * @skip_power_up: Indicates if an implementation can skip powering up * the trace unit. * @paused: Indicates if the trace unit is paused. @@ -1036,7 +1035,6 @@ struct etmv4_drvdata { u64 trfcr; struct etmv4_config config; struct etmv4_save_state *save_state; - bool state_needs_restore; bool skip_power_up; bool paused; DECLARE_BITMAP(arch_features, ETM4_IMPDEF_FEATURE_MAX);
Add a new argument restart_ss to etm4_enable_hw(). When passed as true, it resets the TRCSSCSRn.STATUS bit to 0 to re-enable single-shot control.
No functional change. This is a preparation for a subsequent change.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index fbcd0fcd1992cd600d2eb8c95ae9a5adf0c287da..f61c9924ad0496be2e41c84172c2bbe348df7b60 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -487,7 +487,7 @@ static int etm4_enable_trace_unit(struct etmv4_drvdata *drvdata) return 0; }
-static int etm4_enable_hw(struct etmv4_drvdata *drvdata) +static int etm4_enable_hw(struct etmv4_drvdata *drvdata, bool restart_ss) { int i, rc; struct etmv4_config *config = &drvdata->config; @@ -562,9 +562,11 @@ static int etm4_enable_hw(struct etmv4_drvdata *drvdata) etm4x_relaxed_write32(csa, config->res_ctrl[i], TRCRSCTLRn(i));
for (i = 0; i < drvdata->nr_ss_cmp; i++) { - /* always clear status bit on restart if using single-shot */ - if (config->ss_ctrl[i] || config->ss_pe_cmp[i]) - config->ss_status[i] &= ~TRCSSCSRn_STATUS; + if (restart_ss) { + /* always clear status bit on restart if using single-shot */ + if (config->ss_ctrl[i] || config->ss_pe_cmp[i]) + config->ss_status[i] &= ~TRCSSCSRn_STATUS; + } etm4x_relaxed_write32(csa, config->ss_ctrl[i], TRCSSCCRn(i)); etm4x_relaxed_write32(csa, config->ss_status[i], TRCSSCSRn(i)); if (etm4x_sspcicrn_present(drvdata, i)) @@ -628,7 +630,7 @@ static void etm4_enable_hw_smp_call(void *info) return; }
- arg->rc = etm4_enable_hw(arg->drvdata); + arg->rc = etm4_enable_hw(arg->drvdata, true);
/* The tracer didn't start */ if (arg->rc) @@ -867,7 +869,7 @@ static int etm4_enable_perf(struct coresight_device *csdev, drvdata->paused = !!READ_ONCE(event->hw.aux_paused);
/* And enable it */ - ret = etm4_enable_hw(drvdata); + ret = etm4_enable_hw(drvdata, true);
out: /* The tracer didn't start */ @@ -1833,7 +1835,7 @@ static int etm4_starting_cpu(unsigned int cpu) etm4_os_unlock(etmdrvdata[cpu]);
if (coresight_get_mode(etmdrvdata[cpu]->csdev)) - etm4_enable_hw(etmdrvdata[cpu]); + etm4_enable_hw(etmdrvdata[cpu], true); raw_spin_unlock(&etmdrvdata[cpu]->spinlock); return 0; }
The etm4_enable_trace_unit() function is almost identical to the restore flow, with a few differences listed below:
1) TRCQCTLR register
The TRCQCTLR register is saved and restored during CPU idle, but it is never touched in the normal flow. Given the Q element is not enabled (TRCCONFIGR.QE bits), it is acceptable to omit saving and restoring this register during idle.
2) TRCSSCSRn.STATUS bit
The restore flow does not explicitly clear the TRCSSCSRn.STATUS bit but instead directly loads the saved value. In contrast, the normal flow clears this bit to 0 when re-enabling single-shot control.
Therefore, the restore function passes the restart_ss argument as false to etm4_enable_hw() to avoid re-enabling single-shot mode.
3) Claim Tag Handling
The claim tag is acquired and released in normal flow, on the other hand, the claim protocol is not applied in CPU idle flow - it simply restores the saved value.
The claim bits serve two purpose:
* Exclusive access between the kernel driver and an external debugger. During CPU idle, the kernel driver locks the OS Lock, ensuring that the external debugger cannot access the trace unit. Therefore, it is safe to release the claim tag during idle.
* Notification to supervisory software to save/restore context for external debuggers. The kernel driver does not touch the external debugger's claim bit (ETMCLAIM[0]).
Based on this, it is safe to acquire and release claim tag in the idle sequence.
4) OS Lock Behavior
The OS Lock should be locked during CPU idle states. This differs from the normal flow, which unlock it. This special handling remains in the CPU save path.
This commit reuses the normal enable and disable logic in the CPU idle path. The only addition is locking the OS Lock upon entering idle to ensure exclusive access.
The save context in the driver data is no longer needed and has been removed.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 215 +-------------------- drivers/hwtracing/coresight/coresight-etm4x.h | 57 ------ 2 files changed, 5 insertions(+), 267 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index f61c9924ad0496be2e41c84172c2bbe348df7b60..4a75c76bb44fc284034e25f8eb8e7a04e9d35885 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1854,8 +1854,7 @@ static int etm4_dying_cpu(unsigned int cpu)
static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) { - int i, ret = 0; - struct etmv4_save_state *state; + int ret = 0; struct coresight_device *csdev = drvdata->csdev; struct csdev_access *csa; struct device *etm_dev; @@ -1885,100 +1884,11 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) ret = -EBUSY; goto out; } + etm4_cs_lock(drvdata, csa);
- if (!drvdata->paused) - etm4_disable_trace_unit(drvdata); - - /* - * As recommended by section 4.3.7 (Synchronization of register updates) - * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an - * ISB instruction after programming the trace unit registers. - */ - isb(); - - state = drvdata->save_state; - - if (drvdata->nr_pe) - state->trcprocselr = etm4x_read32(csa, TRCPROCSELR); - state->trcconfigr = etm4x_read32(csa, TRCCONFIGR); - state->trcauxctlr = etm4x_read32(csa, TRCAUXCTLR); - state->trceventctl0r = etm4x_read32(csa, TRCEVENTCTL0R); - state->trceventctl1r = etm4x_read32(csa, TRCEVENTCTL1R); - if (drvdata->stallctl) - state->trcstallctlr = etm4x_read32(csa, TRCSTALLCTLR); - state->trctsctlr = etm4x_read32(csa, TRCTSCTLR); - state->trcsyncpr = etm4x_read32(csa, TRCSYNCPR); - state->trcccctlr = etm4x_read32(csa, TRCCCCTLR); - state->trcbbctlr = etm4x_read32(csa, TRCBBCTLR); - state->trctraceidr = etm4x_read32(csa, TRCTRACEIDR); - if (drvdata->q_filt) - state->trcqctlr = etm4x_read32(csa, TRCQCTLR); - - state->trcvictlr = etm4x_read32(csa, TRCVICTLR); - state->trcviiectlr = etm4x_read32(csa, TRCVIIECTLR); - state->trcvissctlr = etm4x_read32(csa, TRCVISSCTLR); - if (drvdata->nr_pe_cmp) - state->trcvipcssctlr = etm4x_read32(csa, TRCVIPCSSCTLR); - - for (i = 0; i < drvdata->nrseqstate - 1; i++) - state->trcseqevr[i] = etm4x_read32(csa, TRCSEQEVRn(i)); - - if (drvdata->nrseqstate) { - state->trcseqrstevr = etm4x_read32(csa, TRCSEQRSTEVR); - state->trcseqstr = etm4x_read32(csa, TRCSEQSTR); - } - - if (drvdata->numextinsel) - state->trcextinselr = etm4x_read32(csa, TRCEXTINSELR); - - for (i = 0; i < drvdata->nr_cntr; i++) { - state->trccntrldvr[i] = etm4x_read32(csa, TRCCNTRLDVRn(i)); - state->trccntctlr[i] = etm4x_read32(csa, TRCCNTCTLRn(i)); - state->trccntvr[i] = etm4x_read32(csa, TRCCNTVRn(i)); - } - - /* Resource selector pair 0 is reserved */ - for (i = 2; i < drvdata->nr_resource * 2; i++) - state->trcrsctlr[i] = etm4x_read32(csa, TRCRSCTLRn(i)); - - for (i = 0; i < drvdata->nr_ss_cmp; i++) { - state->trcssccr[i] = etm4x_read32(csa, TRCSSCCRn(i)); - state->trcsscsr[i] = etm4x_read32(csa, TRCSSCSRn(i)); - if (etm4x_sspcicrn_present(drvdata, i)) - state->trcsspcicr[i] = etm4x_read32(csa, TRCSSPCICRn(i)); - } - - for (i = 0; i < drvdata->nr_addr_cmp * 2; i++) { - state->trcacvr[i] = etm4x_read64(csa, TRCACVRn(i)); - state->trcacatr[i] = etm4x_read64(csa, TRCACATRn(i)); - } - - /* - * Data trace stream is architecturally prohibited for A profile cores - * so we don't save (or later restore) trcdvcvr and trcdvcmr - As per - * section 1.3.4 ("Possible functional configurations of an ETMv4 trace - * unit") of ARM IHI 0064D. - */ - - for (i = 0; i < drvdata->numcidc; i++) - state->trccidcvr[i] = etm4x_read64(csa, TRCCIDCVRn(i)); - - for (i = 0; i < drvdata->numvmidc; i++) - state->trcvmidcvr[i] = etm4x_read64(csa, TRCVMIDCVRn(i)); - - state->trccidcctlr0 = etm4x_read32(csa, TRCCIDCCTLR0); - if (drvdata->numcidc > 4) - state->trccidcctlr1 = etm4x_read32(csa, TRCCIDCCTLR1); - - state->trcvmidcctlr0 = etm4x_read32(csa, TRCVMIDCCTLR0); - if (drvdata->numvmidc > 4) - state->trcvmidcctlr0 = etm4x_read32(csa, TRCVMIDCCTLR1); - - state->trcclaimset = etm4x_read32(csa, TRCCLAIMCLR); - - if (!drvdata->skip_power_up) - state->trcpdcr = etm4x_read32(csa, TRCPDCR); + etm4_disable_hw(drvdata);
+ etm4_cs_unlock(drvdata, csa); /* wait for TRCSTATR.IDLE to go up */ if (etm4x_wait_status(csa, TRCSTATR_IDLE_BIT, 1)) { dev_err(etm_dev, @@ -1988,14 +1898,6 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) goto out; }
- /* - * Power can be removed from the trace unit now. We do this to - * potentially save power on systems that respect the TRCPDCR_PU - * despite requesting software to save/restore state. - */ - if (!drvdata->skip_power_up) - etm4x_relaxed_write32(csa, (state->trcpdcr & ~TRCPDCR_PU), - TRCPDCR); out: etm4_cs_lock(drvdata, csa); return ret; @@ -2019,110 +1921,10 @@ static int etm4_cpu_save(struct etmv4_drvdata *drvdata)
static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata) { - int i; - struct etmv4_save_state *state = drvdata->save_state; - struct csdev_access *csa = &drvdata->csdev->access; - if (WARN_ON(!drvdata->csdev)) return;
- etm4_cs_unlock(drvdata, csa); - etm4x_relaxed_write32(csa, state->trcclaimset, TRCCLAIMSET); - - if (drvdata->nr_pe) - etm4x_relaxed_write32(csa, state->trcprocselr, TRCPROCSELR); - etm4x_relaxed_write32(csa, state->trcconfigr, TRCCONFIGR); - etm4x_relaxed_write32(csa, state->trcauxctlr, TRCAUXCTLR); - etm4x_relaxed_write32(csa, state->trceventctl0r, TRCEVENTCTL0R); - etm4x_relaxed_write32(csa, state->trceventctl1r, TRCEVENTCTL1R); - if (drvdata->stallctl) - etm4x_relaxed_write32(csa, state->trcstallctlr, TRCSTALLCTLR); - etm4x_relaxed_write32(csa, state->trctsctlr, TRCTSCTLR); - etm4x_relaxed_write32(csa, state->trcsyncpr, TRCSYNCPR); - etm4x_relaxed_write32(csa, state->trcccctlr, TRCCCCTLR); - etm4x_relaxed_write32(csa, state->trcbbctlr, TRCBBCTLR); - etm4x_relaxed_write32(csa, state->trctraceidr, TRCTRACEIDR); - if (drvdata->q_filt) - etm4x_relaxed_write32(csa, state->trcqctlr, TRCQCTLR); - - etm4x_relaxed_write32(csa, state->trcvictlr, TRCVICTLR); - etm4x_relaxed_write32(csa, state->trcviiectlr, TRCVIIECTLR); - etm4x_relaxed_write32(csa, state->trcvissctlr, TRCVISSCTLR); - if (drvdata->nr_pe_cmp) - etm4x_relaxed_write32(csa, state->trcvipcssctlr, TRCVIPCSSCTLR); - - for (i = 0; i < drvdata->nrseqstate - 1; i++) - etm4x_relaxed_write32(csa, state->trcseqevr[i], TRCSEQEVRn(i)); - - if (drvdata->nrseqstate) { - etm4x_relaxed_write32(csa, state->trcseqrstevr, TRCSEQRSTEVR); - etm4x_relaxed_write32(csa, state->trcseqstr, TRCSEQSTR); - } - if (drvdata->numextinsel) - etm4x_relaxed_write32(csa, state->trcextinselr, TRCEXTINSELR); - - for (i = 0; i < drvdata->nr_cntr; i++) { - etm4x_relaxed_write32(csa, state->trccntrldvr[i], TRCCNTRLDVRn(i)); - etm4x_relaxed_write32(csa, state->trccntctlr[i], TRCCNTCTLRn(i)); - etm4x_relaxed_write32(csa, state->trccntvr[i], TRCCNTVRn(i)); - } - - /* Resource selector pair 0 is reserved */ - for (i = 2; i < drvdata->nr_resource * 2; i++) - etm4x_relaxed_write32(csa, state->trcrsctlr[i], TRCRSCTLRn(i)); - - for (i = 0; i < drvdata->nr_ss_cmp; i++) { - etm4x_relaxed_write32(csa, state->trcssccr[i], TRCSSCCRn(i)); - etm4x_relaxed_write32(csa, state->trcsscsr[i], TRCSSCSRn(i)); - if (etm4x_sspcicrn_present(drvdata, i)) - etm4x_relaxed_write32(csa, state->trcsspcicr[i], TRCSSPCICRn(i)); - } - - for (i = 0; i < drvdata->nr_addr_cmp * 2; i++) { - etm4x_relaxed_write64(csa, state->trcacvr[i], TRCACVRn(i)); - etm4x_relaxed_write64(csa, state->trcacatr[i], TRCACATRn(i)); - } - - for (i = 0; i < drvdata->numcidc; i++) - etm4x_relaxed_write64(csa, state->trccidcvr[i], TRCCIDCVRn(i)); - - for (i = 0; i < drvdata->numvmidc; i++) - etm4x_relaxed_write64(csa, state->trcvmidcvr[i], TRCVMIDCVRn(i)); - - etm4x_relaxed_write32(csa, state->trccidcctlr0, TRCCIDCCTLR0); - if (drvdata->numcidc > 4) - etm4x_relaxed_write32(csa, state->trccidcctlr1, TRCCIDCCTLR1); - - etm4x_relaxed_write32(csa, state->trcvmidcctlr0, TRCVMIDCCTLR0); - if (drvdata->numvmidc > 4) - etm4x_relaxed_write32(csa, state->trcvmidcctlr0, TRCVMIDCCTLR1); - - etm4x_relaxed_write32(csa, state->trcclaimset, TRCCLAIMSET); - - if (!drvdata->skip_power_up) - etm4x_relaxed_write32(csa, state->trcpdcr, TRCPDCR); - - /* - * As recommended by section 4.3.7 ("Synchronization when using the - * memory-mapped interface") of ARM IHI 0064D - */ - dsb(sy); - isb(); - - /* Unlock the OS lock to re-enable trace and external debug access */ - etm4_os_unlock(drvdata); - - if (!drvdata->paused) - etm4_enable_trace_unit(drvdata); - - /* - * As recommended by section 4.3.7 (Synchronization of register updates) - * of ARM IHI 0064H.b, the self-hosted trace analyzer always executes an - * ISB instruction after programming the trace unit registers. - */ - isb(); - - etm4_cs_lock(drvdata, csa); + etm4_enable_hw(drvdata, false); }
static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) @@ -2309,13 +2111,6 @@ static int etm4_probe(struct device *dev) pm_save_enable = coresight_loses_context_with_cpu(dev) ? PARAM_PM_SAVE_SELF_HOSTED : PARAM_PM_SAVE_NEVER;
- if (pm_save_enable != PARAM_PM_SAVE_NEVER) { - drvdata->save_state = devm_kmalloc(dev, - sizeof(struct etmv4_save_state), GFP_KERNEL); - if (!drvdata->save_state) - return -ENOMEM; - } - raw_spin_lock_init(&drvdata->spinlock);
drvdata->cpu = coresight_get_cpu(dev); diff --git a/drivers/hwtracing/coresight/coresight-etm4x.h b/drivers/hwtracing/coresight/coresight-etm4x.h index 012c52fd19338133129752d35523dc102df24604..9dfdcfc2b16748bedc278ba2ceaa7e39b29cc351 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.h +++ b/drivers/hwtracing/coresight/coresight-etm4x.h @@ -862,61 +862,6 @@ struct etmv4_config { u8 s_ex_level; };
-/** - * struct etm4_save_state - state to be preserved when ETM is without power - */ -struct etmv4_save_state { - u32 trcprocselr; - u32 trcconfigr; - u32 trcauxctlr; - u32 trceventctl0r; - u32 trceventctl1r; - u32 trcstallctlr; - u32 trctsctlr; - u32 trcsyncpr; - u32 trcccctlr; - u32 trcbbctlr; - u32 trctraceidr; - u32 trcqctlr; - - u32 trcvictlr; - u32 trcviiectlr; - u32 trcvissctlr; - u32 trcvipcssctlr; - - u32 trcseqevr[ETM_MAX_SEQ_STATES]; - u32 trcseqrstevr; - u32 trcseqstr; - u32 trcextinselr; - u32 trccntrldvr[ETMv4_MAX_CNTR]; - u32 trccntctlr[ETMv4_MAX_CNTR]; - u32 trccntvr[ETMv4_MAX_CNTR]; - - u32 trcrsctlr[ETM_MAX_RES_SEL]; - - u32 trcssccr[ETM_MAX_SS_CMP]; - u32 trcsscsr[ETM_MAX_SS_CMP]; - u32 trcsspcicr[ETM_MAX_SS_CMP]; - - u64 trcacvr[ETM_MAX_SINGLE_ADDR_CMP]; - u64 trcacatr[ETM_MAX_SINGLE_ADDR_CMP]; - u64 trccidcvr[ETMv4_MAX_CTXID_CMP]; - u64 trcvmidcvr[ETM_MAX_VMID_CMP]; - u32 trccidcctlr0; - u32 trccidcctlr1; - u32 trcvmidcctlr0; - u32 trcvmidcctlr1; - - u32 trcclaimset; - - u32 cntr_val[ETMv4_MAX_CNTR]; - u32 seq_state; - u32 vinst_ctrl; - u32 ss_status[ETM_MAX_SS_CMP]; - - u32 trcpdcr; -}; - /** * struct etm4_drvdata - specifics associated to an ETM component * @pclk: APB clock if present, otherwise NULL @@ -979,7 +924,6 @@ struct etmv4_save_state { * at runtime, due to the additional setting of TRFCR_CX when * in EL2. Otherwise, 0. * @config: structure holding configuration parameters. - * @save_state: State to be preserved across power loss * @skip_power_up: Indicates if an implementation can skip powering up * the trace unit. * @paused: Indicates if the trace unit is paused. @@ -1034,7 +978,6 @@ struct etmv4_drvdata { bool lpoverride; u64 trfcr; struct etmv4_config config; - struct etmv4_save_state *save_state; bool skip_power_up; bool paused; DECLARE_BITMAP(arch_features, ETM4_IMPDEF_FEATURE_MAX);
The coresight_desc structure is a temporary structure for passing the information to the coresight_device structure during registration.
Introduces a new "cpu" field in both structures to store the CPU ID. For components that are not CPU bound, set this field to -1.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-catu.c | 1 + drivers/hwtracing/coresight/coresight-core.c | 1 + drivers/hwtracing/coresight/coresight-ctcu-core.c | 1 + drivers/hwtracing/coresight/coresight-cti-core.c | 1 + drivers/hwtracing/coresight/coresight-dummy.c | 1 + drivers/hwtracing/coresight/coresight-etb10.c | 1 + drivers/hwtracing/coresight/coresight-etm3x-core.c | 1 + drivers/hwtracing/coresight/coresight-etm4x-core.c | 1 + drivers/hwtracing/coresight/coresight-funnel.c | 1 + drivers/hwtracing/coresight/coresight-replicator.c | 1 + drivers/hwtracing/coresight/coresight-stm.c | 1 + drivers/hwtracing/coresight/coresight-tmc-core.c | 1 + drivers/hwtracing/coresight/coresight-tpda.c | 1 + drivers/hwtracing/coresight/coresight-tpdm.c | 1 + drivers/hwtracing/coresight/coresight-tpiu.c | 1 + drivers/hwtracing/coresight/coresight-trbe.c | 1 + drivers/hwtracing/coresight/ultrasoc-smb.c | 1 + include/linux/coresight.h | 4 ++++ 18 files changed, 21 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-catu.c b/drivers/hwtracing/coresight/coresight-catu.c index a3ccb7034ae14d7339bc2549bccadf11e28c45e2..2685e3a883b1f8067de643519b5fc91153123c57 100644 --- a/drivers/hwtracing/coresight/coresight-catu.c +++ b/drivers/hwtracing/coresight/coresight-catu.c @@ -575,6 +575,7 @@ static int __catu_probe(struct device *dev, struct resource *res) catu_desc.type = CORESIGHT_DEV_TYPE_HELPER; catu_desc.subtype.helper_subtype = CORESIGHT_DEV_SUBTYPE_HELPER_CATU; catu_desc.ops = &catu_ops; + catu_desc.cpu = -1;
coresight_clear_self_claim_tag(&catu_desc.access); drvdata->csdev = coresight_register(&catu_desc); diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 3267192f0c1c667b0570b9100c3c449064e7fb5e..db325dff884d21cc8d93f32bfe3be5c0491ffc8d 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -1336,6 +1336,7 @@ struct coresight_device *coresight_register(struct coresight_desc *desc) csdev->ops = desc->ops; csdev->access = desc->access; csdev->orphan = true; + csdev->cpu = desc->cpu;
csdev->dev.type = &coresight_dev_type[desc->type]; csdev->dev.groups = desc->groups; diff --git a/drivers/hwtracing/coresight/coresight-ctcu-core.c b/drivers/hwtracing/coresight/coresight-ctcu-core.c index c586495e9a088a63cec481a82fd9f4ec7c645160..05497a5114785397a98e408466df0b937c87ab61 100644 --- a/drivers/hwtracing/coresight/coresight-ctcu-core.c +++ b/drivers/hwtracing/coresight/coresight-ctcu-core.c @@ -231,6 +231,7 @@ static int ctcu_probe(struct platform_device *pdev) desc.dev = dev; desc.ops = &ctcu_ops; desc.access = CSDEV_ACCESS_IOMEM(base); + desc.cpu = -1;
drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) diff --git a/drivers/hwtracing/coresight/coresight-cti-core.c b/drivers/hwtracing/coresight/coresight-cti-core.c index 8fb30dd73fd20ae613a45b1a467f457a046a9412..2c8bf5dbe8b8206c92ae5ea64a26c947ef5b9582 100644 --- a/drivers/hwtracing/coresight/coresight-cti-core.c +++ b/drivers/hwtracing/coresight/coresight-cti-core.c @@ -931,6 +931,7 @@ static int cti_probe(struct amba_device *adev, const struct amba_id *id) cti_desc.ops = &cti_ops; cti_desc.groups = drvdata->ctidev.con_groups; cti_desc.dev = dev; + cti_desc.cpu = drvdata->ctidev.cpu;
coresight_clear_self_claim_tag(&cti_desc.access); drvdata->csdev = coresight_register(&cti_desc); diff --git a/drivers/hwtracing/coresight/coresight-dummy.c b/drivers/hwtracing/coresight/coresight-dummy.c index aaa92b5081e3d2bb85d57f90ab68a1dc6a1f0dd8..653d37cd0bd12a57c78e2615f839e0d8af25d7f0 100644 --- a/drivers/hwtracing/coresight/coresight-dummy.c +++ b/drivers/hwtracing/coresight/coresight-dummy.c @@ -179,6 +179,7 @@ static int dummy_probe(struct platform_device *pdev)
desc.pdata = pdev->dev.platform_data; desc.dev = &pdev->dev; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) { ret = PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-etb10.c b/drivers/hwtracing/coresight/coresight-etb10.c index 35db1b6093d154d67dc567df42f838e2ba3d1d58..8e0bbffdf36215fe82c89a34767e425e03f4a744 100644 --- a/drivers/hwtracing/coresight/coresight-etb10.c +++ b/drivers/hwtracing/coresight/coresight-etb10.c @@ -770,6 +770,7 @@ static int etb_probe(struct amba_device *adev, const struct amba_id *id) desc.pdata = pdata; desc.dev = dev; desc.groups = coresight_etb_groups; + desc.cpu = -1;
coresight_clear_self_claim_tag(&desc.access); drvdata->csdev = coresight_register(&desc); diff --git a/drivers/hwtracing/coresight/coresight-etm3x-core.c b/drivers/hwtracing/coresight/coresight-etm3x-core.c index f759a47eaed9e804e30530a232370c9504912ac6..2cce5f4dd0e168cab066e882c0856886b30aaf91 100644 --- a/drivers/hwtracing/coresight/coresight-etm3x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm3x-core.c @@ -885,6 +885,7 @@ static int etm_probe(struct amba_device *adev, const struct amba_id *id) desc.pdata = pdata; desc.dev = dev; desc.groups = coresight_etm_groups; + desc.cpu = drvdata->cpu; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 4a75c76bb44fc284034e25f8eb8e7a04e9d35885..7e66d094269de4a1344f5d962bc320b2603b7627 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -2062,6 +2062,7 @@ static int etm4_add_coresight_dev(struct etm4_init_arg *init_arg) desc.pdata = pdata; desc.dev = dev; desc.groups = coresight_etmv4_groups; + desc.cpu = drvdata->cpu; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-funnel.c b/drivers/hwtracing/coresight/coresight-funnel.c index 3b248e54471a38f501777fe162fea850d1c851b3..457291bb24212c502de9aadecb41da218c733c2f 100644 --- a/drivers/hwtracing/coresight/coresight-funnel.c +++ b/drivers/hwtracing/coresight/coresight-funnel.c @@ -263,6 +263,7 @@ static int funnel_probe(struct device *dev, struct resource *res) desc.ops = &funnel_cs_ops; desc.pdata = pdata; desc.dev = dev; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-replicator.c b/drivers/hwtracing/coresight/coresight-replicator.c index e6472658235dc479cec91ac18f3737f76f8c74f0..38518ba862928ea904c953c8303559e349e45200 100644 --- a/drivers/hwtracing/coresight/coresight-replicator.c +++ b/drivers/hwtracing/coresight/coresight-replicator.c @@ -273,6 +273,7 @@ static int replicator_probe(struct device *dev, struct resource *res) desc.ops = &replicator_cs_ops; desc.pdata = dev->platform_data; desc.dev = dev; + desc.cpu = -1;
drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) diff --git a/drivers/hwtracing/coresight/coresight-stm.c b/drivers/hwtracing/coresight/coresight-stm.c index e68529bf89c9815a8118955bf3114ad1ed4fb346..e2197ae0a7c8412785ce8b1161311048cbaa3f58 100644 --- a/drivers/hwtracing/coresight/coresight-stm.c +++ b/drivers/hwtracing/coresight/coresight-stm.c @@ -901,6 +901,7 @@ static int __stm_probe(struct device *dev, struct resource *res) desc.pdata = pdata; desc.dev = dev; desc.groups = coresight_stm_groups; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) { ret = PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-tmc-core.c b/drivers/hwtracing/coresight/coresight-tmc-core.c index 36599c431be6203e871fdcb8de569cc6701c52bb..bd783c9dcb56afa0adfa9724e45fcd0c4cfe367b 100644 --- a/drivers/hwtracing/coresight/coresight-tmc-core.c +++ b/drivers/hwtracing/coresight/coresight-tmc-core.c @@ -868,6 +868,7 @@ static int __tmc_probe(struct device *dev, struct resource *res) } dev->platform_data = pdata; desc.pdata = pdata; + desc.cpu = -1;
coresight_clear_self_claim_tag(&desc.access); drvdata->csdev = coresight_register(&desc); diff --git a/drivers/hwtracing/coresight/coresight-tpda.c b/drivers/hwtracing/coresight/coresight-tpda.c index 333b3cb236859f0feb1498f4ab81037c772143fd..6739deda402ed5024e17add349314d6db793dd13 100644 --- a/drivers/hwtracing/coresight/coresight-tpda.c +++ b/drivers/hwtracing/coresight/coresight-tpda.c @@ -324,6 +324,7 @@ static int tpda_probe(struct amba_device *adev, const struct amba_id *id) desc.pdata = adev->dev.platform_data; desc.dev = &adev->dev; desc.access = CSDEV_ACCESS_IOMEM(base); + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-tpdm.c b/drivers/hwtracing/coresight/coresight-tpdm.c index 7214e65097ec9ac69f6c7c9278bcd28d25945c9e..ec1e9a585f90a02f06db27d6dfb22091eb66cbd9 100644 --- a/drivers/hwtracing/coresight/coresight-tpdm.c +++ b/drivers/hwtracing/coresight/coresight-tpdm.c @@ -1392,6 +1392,7 @@ static int tpdm_probe(struct amba_device *adev, const struct amba_id *id) desc.dev = &adev->dev; desc.access = CSDEV_ACCESS_IOMEM(base); desc.groups = tpdm_attr_grps; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) return PTR_ERR(drvdata->csdev); diff --git a/drivers/hwtracing/coresight/coresight-tpiu.c b/drivers/hwtracing/coresight/coresight-tpiu.c index 9463afdbda8ad74eee78c72185fe7603f81b7888..3ade05e268e31dd2572b84e91f9448fcf317f2ab 100644 --- a/drivers/hwtracing/coresight/coresight-tpiu.c +++ b/drivers/hwtracing/coresight/coresight-tpiu.c @@ -171,6 +171,7 @@ static int __tpiu_probe(struct device *dev, struct resource *res) desc.ops = &tpiu_cs_ops; desc.pdata = pdata; desc.dev = dev; + desc.cpu = -1; drvdata->csdev = coresight_register(&desc);
if (!IS_ERR(drvdata->csdev)) diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c index 7a226316c48fa516293bf4e88e4009c8606485d5..3fd379f327eb4f82665b216856d43ec39e4e7d10 100644 --- a/drivers/hwtracing/coresight/coresight-trbe.c +++ b/drivers/hwtracing/coresight/coresight-trbe.c @@ -1288,6 +1288,7 @@ static void arm_trbe_register_coresight_cpu(struct trbe_drvdata *drvdata, int cp desc.ops = &arm_trbe_cs_ops; desc.groups = arm_trbe_groups; desc.dev = dev; + desc.cpu = cpu; trbe_csdev = coresight_register(&desc); if (IS_ERR(trbe_csdev)) goto cpu_clear; diff --git a/drivers/hwtracing/coresight/ultrasoc-smb.c b/drivers/hwtracing/coresight/ultrasoc-smb.c index 26cfc939e5bd810295a336f392ac282ccf316f9f..e5897767d9ae2c21ef39e9280dea211c110bd168 100644 --- a/drivers/hwtracing/coresight/ultrasoc-smb.c +++ b/drivers/hwtracing/coresight/ultrasoc-smb.c @@ -483,6 +483,7 @@ static int smb_register_sink(struct platform_device *pdev, return -ENOMEM; } desc.access = CSDEV_ACCESS_IOMEM(drvdata->base); + desc.cpu = -1;
drvdata->csdev = coresight_register(&desc); if (IS_ERR(drvdata->csdev)) diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 3e5e5acd0c7fcde7d312d440da4355faaf682c7b..222597ec7a089e10ad763df206917e90f34bb5c2 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -153,6 +153,7 @@ struct csdev_access { * in the component's sysfs sub-directory. * @name: name for the coresight device, also shown under sysfs. * @access: Describe access to the device + * @cpu: The CPU this component is affined to (-1 for not CPU bound). */ struct coresight_desc { enum coresight_dev_type type; @@ -163,6 +164,7 @@ struct coresight_desc { const struct attribute_group **groups; const char *name; struct csdev_access access; + int cpu; };
/** @@ -261,6 +263,7 @@ struct coresight_trace_id_map { * CS_MODE_SYSFS. Otherwise it must be accessed from inside the * spinlock. * @orphan: true if the component has connections that haven't been linked. + * @cpu: The CPU this component is affined to (-1 for not CPU bound). * @sysfs_sink_activated: 'true' when a sink has been selected for use via sysfs * by writing a 1 to the 'enable_sink' file. A sink can be * activated but not yet enabled. Enabling for a _sink_ happens @@ -287,6 +290,7 @@ struct coresight_device { atomic_t mode; int refcnt; bool orphan; + int cpu; /* sink specific fields */ bool sysfs_sink_activated; struct dev_ext_attribute *ea;
The current SysFS flow first enables the links and sink, then rolls back to disable them if the source fails to enable. This failure can occur if the associated CPU is offline, which causes the SMP call to fail.
Validate whether the associated CPU is online for a per-CPU tracer. If the CPU is offline, return -ENODEV and bail out.
Reviewed-by: Yeoreum Yun yeoreum.yun@arm.com Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-sysfs.c | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c index 5e52324aa9ac7b3de9379bc3f2b349a2cdea83c2..cb02f1bfaf068ec3f50f997bcb94f7d5215ccea8 100644 --- a/drivers/hwtracing/coresight/coresight-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-sysfs.c @@ -162,6 +162,9 @@ static int coresight_validate_source_sysfs(struct coresight_device *csdev, return -EINVAL; }
+ if (coresight_is_percpu_source(csdev) && !cpu_online(csdev->cpu)) + return -ENODEV; + return 0; }
Introduce coresight_set_percpu_source() for setting CPU source device. The sources are maintained in a per CPU structure 'csdev_source'.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index db325dff884d21cc8d93f32bfe3be5c0491ffc8d..25a530cde8938b366bbb144fdcc107271ebae276 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -34,6 +34,7 @@ */ DEFINE_MUTEX(coresight_mutex); static DEFINE_PER_CPU(struct coresight_device *, csdev_sink); +static DEFINE_PER_CPU(struct coresight_device *, csdev_source);
/** * struct coresight_node - elements of a path, from source to sink @@ -79,6 +80,11 @@ struct coresight_device *coresight_get_percpu_sink(int cpu) } EXPORT_SYMBOL_GPL(coresight_get_percpu_sink);
+static void coresight_set_percpu_source(int cpu, struct coresight_device *csdev) +{ + per_cpu(csdev_source, cpu) = csdev; +} + static struct coresight_device *coresight_get_source(struct coresight_path *path) { struct coresight_device *csdev; @@ -1393,6 +1399,10 @@ struct coresight_device *coresight_register(struct coresight_desc *desc) goto out_unlock; } } + + if (coresight_is_percpu_source(csdev)) + coresight_set_percpu_source(csdev->cpu, csdev); + /* Device is now registered */ registered = true;
@@ -1424,6 +1434,8 @@ EXPORT_SYMBOL_GPL(coresight_register);
void coresight_unregister(struct coresight_device *csdev) { + if (coresight_is_percpu_source(csdev)) + coresight_set_percpu_source(csdev->cpu, NULL); etm_perf_del_symlink_sink(csdev); /* Remove references of that device in the topology */ if (cti_assoc_ops && cti_assoc_ops->remove)
The current implementation only saves and restores the context for ETM sources while ignoring the context of links. However, if funnels or replicators on a linked path resides in a CPU or cluster power domain, the hardware context for the link will be lost after resuming from low power states.
To support context management for links during CPU low power modes, a better way is to implement CPU PM callbacks in the Arm CoreSight core layer. As the core layer has sufficient information for linked paths, from tracers to links, which can be used for power management.
As a first step, this patch registers CPU PM notifier in the core layer. If a source driver provides callbacks for saving and restoring context, these callbacks will be invoked in CPU suspend and resume.
Further changes will extend for controlling path.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 62 ++++++++++++++++++++++++++++ include/linux/coresight.h | 4 ++ 2 files changed, 66 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 25a530cde8938b366bbb144fdcc107271ebae276..f944a610c85e9e3fce43497543ccc48ab7c2c0d9 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -6,6 +6,7 @@ #include <linux/acpi.h> #include <linux/bitfield.h> #include <linux/build_bug.h> +#include <linux/cpu_pm.h> #include <linux/kernel.h> #include <linux/init.h> #include <linux/types.h> @@ -422,6 +423,21 @@ int coresight_resume_source(struct coresight_device *csdev) } EXPORT_SYMBOL_GPL(coresight_resume_source);
+static int coresight_save_source(struct coresight_device *csdev) +{ + if (csdev && source_ops(csdev)->save) + return source_ops(csdev)->save(csdev); + + /* Return success if callback is not supported */ + return 0; +} + +static void coresight_restore_source(struct coresight_device *csdev) +{ + if (csdev && source_ops(csdev)->restore) + source_ops(csdev)->restore(csdev); +} + /* * coresight_disable_path_from : Disable components in the given path beyond * @nd in the list. If @nd is NULL, all the components, except the SOURCE are @@ -1575,6 +1591,45 @@ char *coresight_alloc_device_name(struct coresight_dev_list *dict, } EXPORT_SYMBOL_GPL(coresight_alloc_device_name);
+static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, + void *v) +{ + unsigned int cpu = smp_processor_id(); + struct coresight_device *source = per_cpu(csdev_source, cpu); + + if (!source) + return NOTIFY_OK; + + switch (cmd) { + case CPU_PM_ENTER: + if (coresight_save_source(source)) + return NOTIFY_BAD; + break; + case CPU_PM_EXIT: + case CPU_PM_ENTER_FAILED: + coresight_restore_source(source); + break; + default: + return NOTIFY_DONE; + } + + return NOTIFY_OK; +} + +static struct notifier_block coresight_cpu_pm_nb = { + .notifier_call = coresight_cpu_pm_notify, +}; + +static int __init coresight_pm_setup(void) +{ + return cpu_pm_register_notifier(&coresight_cpu_pm_nb); +} + +static void coresight_pm_cleanup(void) +{ + cpu_pm_unregister_notifier(&coresight_cpu_pm_nb); +} + const struct bus_type coresight_bustype = { .name = "coresight", }; @@ -1629,9 +1684,15 @@ static int __init coresight_init(void)
/* initialise the coresight syscfg API */ ret = cscfg_init(); + if (ret) + goto exit_notifier; + + ret = coresight_pm_setup(); if (!ret) return 0;
+ cscfg_exit(); +exit_notifier: atomic_notifier_chain_unregister(&panic_notifier_list, &coresight_notifier); exit_perf: @@ -1643,6 +1704,7 @@ static int __init coresight_init(void)
static void __exit coresight_exit(void) { + coresight_pm_cleanup(); cscfg_exit(); atomic_notifier_chain_unregister(&panic_notifier_list, &coresight_notifier); diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 222597ec7a089e10ad763df206917e90f34bb5c2..d0764a9c7692181d2619c277acb83701c6b0115e 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -400,6 +400,8 @@ struct coresight_ops_link { * @disable: disables tracing for a source. * @resume_perf: resumes tracing for a source in perf session. * @pause_perf: pauses tracing for a source in perf session. + * @save: save context for a source. + * @restore: restore context for a source. */ struct coresight_ops_source { int (*cpu_id)(struct coresight_device *csdev); @@ -409,6 +411,8 @@ struct coresight_ops_source { struct perf_event *event); int (*resume_perf)(struct coresight_device *csdev); void (*pause_perf)(struct coresight_device *csdev); + int (*save)(struct coresight_device *csdev); + void (*restore)(struct coresight_device *csdev); };
/**
Since the CoreSight core layer has registered CPU PM notifiers, this patch hooks CPU save and restore callbacks to be invoked from the core layer.
The CPU PM notifier in the ETMv4 driver is no longer needed, remove it along with its registration and unregistration code.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 73 ++++++---------------- 1 file changed, 18 insertions(+), 55 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 7e66d094269de4a1344f5d962bc320b2603b7627..ec7e8d09ce58d62dac826fba9edafc967504a1a5 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1186,19 +1186,6 @@ static void etm4_pause_perf(struct coresight_device *csdev) drvdata->paused = true; }
-static const struct coresight_ops_source etm4_source_ops = { - .cpu_id = etm4_cpu_id, - .enable = etm4_enable, - .disable = etm4_disable, - .resume_perf = etm4_resume_perf, - .pause_perf = etm4_pause_perf, -}; - -static const struct coresight_ops etm4_cs_ops = { - .trace_id = coresight_etm_get_trace_id, - .source_ops = &etm4_source_ops, -}; - static bool cpu_supports_sysreg_trace(void) { u64 dfr0 = read_sysreg_s(SYS_ID_AA64DFR0_EL1); @@ -1903,8 +1890,9 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) return ret; }
-static int etm4_cpu_save(struct etmv4_drvdata *drvdata) +static int etm4_cpu_save(struct coresight_device *csdev) { + struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); int ret = 0;
if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) @@ -1927,8 +1915,10 @@ static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata) etm4_enable_hw(drvdata, false); }
-static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) +static void etm4_cpu_restore(struct coresight_device *csdev) { + struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); + if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) return;
@@ -1936,38 +1926,19 @@ static void etm4_cpu_restore(struct etmv4_drvdata *drvdata) __etm4_cpu_restore(drvdata); }
-static int etm4_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, - void *v) -{ - struct etmv4_drvdata *drvdata; - unsigned int cpu = smp_processor_id(); - - if (!etmdrvdata[cpu]) - return NOTIFY_OK; - - drvdata = etmdrvdata[cpu]; - - if (WARN_ON_ONCE(drvdata->cpu != cpu)) - return NOTIFY_BAD; - - switch (cmd) { - case CPU_PM_ENTER: - if (etm4_cpu_save(drvdata)) - return NOTIFY_BAD; - break; - case CPU_PM_EXIT: - case CPU_PM_ENTER_FAILED: - etm4_cpu_restore(drvdata); - break; - default: - return NOTIFY_DONE; - } - - return NOTIFY_OK; -} +static const struct coresight_ops_source etm4_source_ops = { + .cpu_id = etm4_cpu_id, + .enable = etm4_enable, + .disable = etm4_disable, + .resume_perf = etm4_resume_perf, + .pause_perf = etm4_pause_perf, + .save = etm4_cpu_save, + .restore = etm4_cpu_restore, +};
-static struct notifier_block etm4_cpu_pm_nb = { - .notifier_call = etm4_cpu_pm_notify, +static const struct coresight_ops etm4_cs_ops = { + .trace_id = coresight_etm_get_trace_id, + .source_ops = &etm4_source_ops, };
/* Setup PM. Deals with error conditions and counts */ @@ -1975,16 +1946,12 @@ static int __init etm4_pm_setup(void) { int ret;
- ret = cpu_pm_register_notifier(&etm4_cpu_pm_nb); - if (ret) - return ret; - ret = cpuhp_setup_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING, "arm/coresight4:starting", etm4_starting_cpu, etm4_dying_cpu);
if (ret) - goto unregister_notifier; + return ret;
ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "arm/coresight4:online", @@ -1998,15 +1965,11 @@ static int __init etm4_pm_setup(void)
/* failed dyn state - remove others */ cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); - -unregister_notifier: - cpu_pm_unregister_notifier(&etm4_cpu_pm_nb); return ret; }
static void etm4_pm_clear(void) { - cpu_pm_unregister_notifier(&etm4_cpu_pm_nb); cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); if (hp_online) { cpuhp_remove_state_nocalls(hp_online);
Add a callback in the source device that returns a boolean indicating whether context save and restore operations are required. The save and restore flow is skipped if the callback returns false.
The ETMv4 driver implements its own version's callback.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 16 +++++++++++- drivers/hwtracing/coresight/coresight-etm4x-core.c | 30 +++++++++++++++++----- include/linux/coresight.h | 2 ++ 3 files changed, 40 insertions(+), 8 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index f944a610c85e9e3fce43497543ccc48ab7c2c0d9..94803cacf1a374ae8a3b9513d7d029d5c77d5925 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -423,6 +423,20 @@ int coresight_resume_source(struct coresight_device *csdev) } EXPORT_SYMBOL_GPL(coresight_resume_source);
+static bool coresight_need_save_restore_source(struct coresight_device *csdev) +{ + if (!csdev) + return false; + + if (source_ops(csdev)->need_context_save_restore) + return source_ops(csdev)->need_context_save_restore(csdev); + + if (coresight_get_mode(csdev)) + return true; + + return false; +} + static int coresight_save_source(struct coresight_device *csdev) { if (csdev && source_ops(csdev)->save) @@ -1597,7 +1611,7 @@ static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, unsigned int cpu = smp_processor_id(); struct coresight_device *source = per_cpu(csdev_source, cpu);
- if (!source) + if (!coresight_need_save_restore_source(source)) return NOTIFY_OK;
switch (cmd) { diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index ec7e8d09ce58d62dac826fba9edafc967504a1a5..66aefdb002f075446572df9c4eed9093da19a066 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1839,6 +1839,21 @@ static int etm4_dying_cpu(unsigned int cpu) return 0; }
+static bool etm4_need_context_save_restore(struct coresight_device *csdev) +{ + if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) + return false; + + /* + * Save and restore the ETM Trace registers only if + * the ETM is active. + */ + if (coresight_get_mode(csdev)) + return true; + + return false; +} + static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) { int ret = 0; @@ -1927,13 +1942,14 @@ static void etm4_cpu_restore(struct coresight_device *csdev) }
static const struct coresight_ops_source etm4_source_ops = { - .cpu_id = etm4_cpu_id, - .enable = etm4_enable, - .disable = etm4_disable, - .resume_perf = etm4_resume_perf, - .pause_perf = etm4_pause_perf, - .save = etm4_cpu_save, - .restore = etm4_cpu_restore, + .cpu_id = etm4_cpu_id, + .enable = etm4_enable, + .disable = etm4_disable, + .resume_perf = etm4_resume_perf, + .pause_perf = etm4_pause_perf, + .save = etm4_cpu_save, + .restore = etm4_cpu_restore, + .need_context_save_restore = etm4_need_context_save_restore, };
static const struct coresight_ops etm4_cs_ops = { diff --git a/include/linux/coresight.h b/include/linux/coresight.h index d0764a9c7692181d2619c277acb83701c6b0115e..39781ba24ee2ad9f3065770409aafb0e5af57fa8 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -400,6 +400,7 @@ struct coresight_ops_link { * @disable: disables tracing for a source. * @resume_perf: resumes tracing for a source in perf session. * @pause_perf: pauses tracing for a source in perf session. + * @need_context_save_restore: if need to save and restore context. * @save: save context for a source. * @restore: restore context for a source. */ @@ -411,6 +412,7 @@ struct coresight_ops_source { struct perf_event *event); int (*resume_perf)(struct coresight_device *csdev); void (*pause_perf)(struct coresight_device *csdev); + bool (*need_context_save_restore)(struct coresight_device *csdev); int (*save)(struct coresight_device *csdev); void (*restore)(struct coresight_device *csdev); };
Since a dedicated callback is now used to determine whether context save and restore operations are needed, performing the same check within the save and restore callbacks is redundant.
The save and restore flows currently use two-level functions: the first level handles the condition check, while the second level performs the low level operations. As the checks are no longer necessary, simplify the logic into single-level functions.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 35 ++-------------------- 1 file changed, 3 insertions(+), 32 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 66aefdb002f075446572df9c4eed9093da19a066..b63433e80d305588c18b084408496690480a7af0 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1854,10 +1854,10 @@ static bool etm4_need_context_save_restore(struct coresight_device *csdev) return false; }
-static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) +static int etm4_cpu_save(struct coresight_device *csdev) { int ret = 0; - struct coresight_device *csdev = drvdata->csdev; + struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); struct csdev_access *csa; struct device *etm_dev;
@@ -1905,40 +1905,11 @@ static int __etm4_cpu_save(struct etmv4_drvdata *drvdata) return ret; }
-static int etm4_cpu_save(struct coresight_device *csdev) -{ - struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent); - int ret = 0; - - if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) - return 0; - - /* - * Save and restore the ETM Trace registers only if - * the ETM is active. - */ - if (coresight_get_mode(drvdata->csdev)) - ret = __etm4_cpu_save(drvdata); - return ret; -} - -static void __etm4_cpu_restore(struct etmv4_drvdata *drvdata) -{ - if (WARN_ON(!drvdata->csdev)) - return; - - etm4_enable_hw(drvdata, false); -} - static void etm4_cpu_restore(struct coresight_device *csdev) { struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
- if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) - return; - - if (coresight_get_mode(drvdata->csdev)) - __etm4_cpu_restore(drvdata); + etm4_enable_hw(drvdata, false); }
static const struct coresight_ops_source etm4_source_ops = {
A CTI device can be enabled as a helper in a CoreSight path or directly via the SysFS knob. Currently, the driver lacks protection when multiple flows contend for access to the same CTI device.
Use the device mode to mark the CTI as in use, and reset it to the DISABLED state when the device is no longer needed. Any conflict will cause the attempt to acquire the mode to fail, ensuring the device can only be enabled exclusively.
Fixes: 1b5b1646e63d ("coresight: Fix CTI module refcount leak by making it a helper device") Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-cti-core.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-cti-core.c b/drivers/hwtracing/coresight/coresight-cti-core.c index 2c8bf5dbe8b8206c92ae5ea64a26c947ef5b9582..ba4635dfc2e30b4e9ae4972f91bdc6647975b719 100644 --- a/drivers/hwtracing/coresight/coresight-cti-core.c +++ b/drivers/hwtracing/coresight/coresight-cti-core.c @@ -87,20 +87,31 @@ void cti_write_all_hw_regs(struct cti_drvdata *drvdata) }
/* write regs to hardware and enable */ -static int cti_enable_hw(struct cti_drvdata *drvdata) +static int cti_enable_hw(struct cti_drvdata *drvdata, enum cs_mode mode) { struct cti_config *config = &drvdata->config; + struct coresight_device *csdev = drvdata->csdev; unsigned long flags; int rc = 0;
raw_spin_lock_irqsave(&drvdata->spinlock, flags);
+ if (!drvdata->config.enable_req_count) { + coresight_set_mode(csdev, mode); + } else { + /* The device has been configured with an incompatible mode */ + if (coresight_get_mode(csdev) != mode) { + rc = -EBUSY; + goto cti_err_not_enabled; + } + } + /* no need to do anything if enabled or unpowered*/ if (config->hw_enabled || !config->hw_powered) goto cti_state_unchanged;
/* claim the device */ - rc = coresight_claim_device(drvdata->csdev); + rc = coresight_claim_device(csdev); if (rc) goto cti_err_not_enabled;
@@ -116,6 +127,8 @@ static int cti_enable_hw(struct cti_drvdata *drvdata)
/* cannot enable due to error */ cti_err_not_enabled: + if (!drvdata->config.enable_req_count) + coresight_set_mode(csdev, CS_MODE_DISABLED); raw_spin_unlock_irqrestore(&drvdata->spinlock, flags); return rc; } @@ -165,6 +178,8 @@ static int cti_disable_hw(struct cti_drvdata *drvdata) if (--drvdata->config.enable_req_count > 0) goto cti_not_disabled;
+ coresight_set_mode(csdev, CS_MODE_DISABLED); + /* no need to do anything if disabled or cpu unpowered */ if (!config->hw_enabled || !config->hw_powered) goto cti_not_disabled; @@ -803,7 +818,7 @@ int cti_enable(struct coresight_device *csdev, enum cs_mode mode, void *data) { struct cti_drvdata *drvdata = csdev_to_cti_drvdata(csdev);
- return cti_enable_hw(drvdata); + return cti_enable_hw(drvdata, mode); }
int cti_disable(struct coresight_device *csdev, void *data)
Introduce a new CS_MODE_DEBUG mode to indicate that a device is being used for debugging purposes, e.g, if a system enables CTI for debugging but not for hardware trace
Update ETM4x driver to mute compiler warnings for the newly added mode.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-cti-sysfs.c | 2 +- drivers/hwtracing/coresight/coresight-etm4x-core.c | 2 ++ include/linux/coresight.h | 9 +++++---- 3 files changed, 8 insertions(+), 5 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-cti-sysfs.c b/drivers/hwtracing/coresight/coresight-cti-sysfs.c index 572b80ee96fbf18ec8cf9abc30d109a676dfbc5d..3e0039f75f5aad104cb86561bf37971c7dc4e408 100644 --- a/drivers/hwtracing/coresight/coresight-cti-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-cti-sysfs.c @@ -112,7 +112,7 @@ static ssize_t enable_store(struct device *dev, ret = pm_runtime_resume_and_get(dev->parent); if (ret) return ret; - ret = cti_enable(drvdata->csdev, CS_MODE_SYSFS, NULL); + ret = cti_enable(drvdata->csdev, CS_MODE_DEBUG, NULL); if (ret) pm_runtime_put(dev->parent); } else { diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index b63433e80d305588c18b084408496690480a7af0..4218cefca63d7b2f3b102e74d7f007ebc7b08861 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -934,6 +934,7 @@ static int etm4_enable(struct coresight_device *csdev, struct perf_event *event, case CS_MODE_PERF: ret = etm4_enable_perf(csdev, event, path); break; + case CS_MODE_DEBUG: default: ret = -EINVAL; } @@ -1132,6 +1133,7 @@ static void etm4_disable(struct coresight_device *csdev, mode = coresight_get_mode(csdev);
switch (mode) { + case CS_MODE_DEBUG: case CS_MODE_DISABLED: break; case CS_MODE_SYSFS: diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 39781ba24ee2ad9f3065770409aafb0e5af57fa8..462876b5a870f3e93ad2a1807a9045d5eb115ccd 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -253,10 +253,10 @@ struct coresight_trace_id_map { * by @coresight_ops. * @access: Device i/o access abstraction for this device. * @dev: The device entity associated to this component. - * @mode: The device mode, i.e sysFS, Perf or disabled. This is actually - * an 'enum cs_mode' but stored in an atomic type. Access is always - * through atomic APIs, ensuring SMP-safe synchronisation between - * racing from sysFS and Perf mode. A compare-and-exchange + * @mode: The device mode, i.e sysFS, Perf, debug or disabled. This is + * actually an 'enum cs_mode' but stored in an atomic type. Access + * is always through atomic APIs, ensuring SMP-safe synchronisation + * between racing from sysFS and Perf mode. A compare-and-exchange * operation is done to atomically claim one mode or the other. * @refcnt: keep track of what is in use. Only access this outside of the * device's spinlock when the coresight_mutex held and mode == @@ -344,6 +344,7 @@ enum cs_mode { CS_MODE_DISABLED, CS_MODE_SYSFS, CS_MODE_PERF, + CS_MODE_DEBUG, };
#define coresight_ops(csdev) csdev->ops
The PM notifier needs to access the driver data, so it is only safe to register PM notifier after data initialization.
For this reason, move cti_pm_setup() to a later point, specifically after 'drvdata->csdev' has been set up.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-cti-core.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-cti-core.c b/drivers/hwtracing/coresight/coresight-cti-core.c index ba4635dfc2e30b4e9ae4972f91bdc6647975b719..b2ddbe12be8d6140a3098797d6d1cc4393259b3a 100644 --- a/drivers/hwtracing/coresight/coresight-cti-core.c +++ b/drivers/hwtracing/coresight/coresight-cti-core.c @@ -926,17 +926,12 @@ static int cti_probe(struct amba_device *adev, const struct amba_id *id) if (!cti_desc.name) return -ENOMEM;
- /* setup CPU power management handling for CPU bound CTI devices. */ - ret = cti_pm_setup(drvdata); - if (ret) - return ret; - /* create dynamic attributes for connections */ ret = cti_create_cons_sysfs(dev, drvdata); if (ret) { dev_err(dev, "%s: create dynamic sysfs entries failed\n", cti_desc.name); - goto pm_release; + return ret; }
/* set up coresight component description */ @@ -950,9 +945,14 @@ static int cti_probe(struct amba_device *adev, const struct amba_id *id)
coresight_clear_self_claim_tag(&cti_desc.access); drvdata->csdev = coresight_register(&cti_desc); - if (IS_ERR(drvdata->csdev)) { - ret = PTR_ERR(drvdata->csdev); - goto pm_release; + if (IS_ERR(drvdata->csdev)) + return PTR_ERR(drvdata->csdev); + + /* setup CPU power management handling for CPU bound CTI devices. */ + ret = cti_pm_setup(drvdata); + if (ret) { + coresight_unregister(drvdata->csdev); + return ret; }
/* add to list of CTI devices */ @@ -970,10 +970,6 @@ static int cti_probe(struct amba_device *adev, const struct amba_id *id) pm_runtime_put(&adev->dev); dev_info(&drvdata->csdev->dev, "CTI initialized\n"); return 0; - -pm_release: - cti_pm_release(drvdata); - return ret; }
static struct amba_cs_uci_id uci_id_cti[] = {
This commit distinguishes CPU PM flows based on the mode.
The CTI driver retains its existing behavior for the CS_MODE_DEBUG mode.
For modes other than DEBUG, a future change will be made to manage CTI devices by iterating through the CoreSight path in the core layer. To avoid conflicts, the CTI driver no longer controls CTI hardware in CPU PM notifiers for non DEBUG modes.
However, the CTI driver continues to update the hw_powered flag for all modes to reflect the device's power state.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-cti-core.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-cti-core.c b/drivers/hwtracing/coresight/coresight-cti-core.c index b2ddbe12be8d6140a3098797d6d1cc4393259b3a..206a46fd4efd05165c56567289928295c5e35d4f 100644 --- a/drivers/hwtracing/coresight/coresight-cti-core.c +++ b/drivers/hwtracing/coresight/coresight-cti-core.c @@ -141,6 +141,9 @@ static void cti_cpuhp_enable_hw(struct cti_drvdata *drvdata) raw_spin_lock(&drvdata->spinlock); config->hw_powered = true;
+ if (coresight_get_mode(drvdata->csdev) != CS_MODE_DEBUG) + goto cti_hp_not_enabled; + /* no need to do anything if no enable request */ if (!drvdata->config.enable_req_count) goto cti_hp_not_enabled; @@ -697,21 +700,27 @@ static int cti_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, case CPU_PM_ENTER: /* CTI regs all static - we have a copy & nothing to save */ drvdata->config.hw_powered = false; - if (drvdata->config.hw_enabled) + if ((coresight_get_mode(drvdata->csdev) == CS_MODE_DEBUG) && + drvdata->config.hw_enabled) coresight_disclaim_device(csdev); break;
case CPU_PM_ENTER_FAILED: drvdata->config.hw_powered = true; - if (drvdata->config.hw_enabled) { + if ((coresight_get_mode(drvdata->csdev) == CS_MODE_DEBUG) && + drvdata->config.hw_enabled) { if (coresight_claim_device(csdev)) drvdata->config.hw_enabled = false; } break;
case CPU_PM_EXIT: - /* write hardware registers to re-enable. */ drvdata->config.hw_powered = true; + + if (coresight_get_mode(drvdata->csdev) != CS_MODE_DEBUG) + break; + + /* write hardware registers to re-enable. */ drvdata->config.hw_enabled = false;
/* check enable reference count to enable HW */ @@ -760,7 +769,8 @@ static int cti_dying_cpu(unsigned int cpu)
raw_spin_lock(&drvdata->spinlock); drvdata->config.hw_powered = false; - if (drvdata->config.hw_enabled) + if ((coresight_get_mode(drvdata->csdev) == CS_MODE_DEBUG) && + drvdata->config.hw_enabled) coresight_disclaim_device(drvdata->csdev); raw_spin_unlock(&drvdata->spinlock); return 0;
The spin lock is currently acquired sometimes with IRQs disabled and sometimes without. This leads to inconsistent semantics: the lock can be either HARDIRQ-safe or HARDIRQ-unsafe, which may trigger lockdep complaints.
Make the spin lock usage consistent by always without disabling IRQ.
Note: the CTI driver currently has a flaw that it does not use SMP call for CPU-bound devices. Once the SMP call is supported in the CTI driver, the locking scheme should consider to disable IRQs when taking the lock.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-cti-core.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-cti-core.c b/drivers/hwtracing/coresight/coresight-cti-core.c index 206a46fd4efd05165c56567289928295c5e35d4f..081b645987c7c7276361609bdf437949f62b5964 100644 --- a/drivers/hwtracing/coresight/coresight-cti-core.c +++ b/drivers/hwtracing/coresight/coresight-cti-core.c @@ -91,10 +91,9 @@ static int cti_enable_hw(struct cti_drvdata *drvdata, enum cs_mode mode) { struct cti_config *config = &drvdata->config; struct coresight_device *csdev = drvdata->csdev; - unsigned long flags; int rc = 0;
- raw_spin_lock_irqsave(&drvdata->spinlock, flags); + raw_spin_lock(&drvdata->spinlock);
if (!drvdata->config.enable_req_count) { coresight_set_mode(csdev, mode); @@ -119,7 +118,7 @@ static int cti_enable_hw(struct cti_drvdata *drvdata, enum cs_mode mode)
config->hw_enabled = true; drvdata->config.enable_req_count++; - raw_spin_unlock_irqrestore(&drvdata->spinlock, flags); + raw_spin_unlock(&drvdata->spinlock); return rc;
cti_state_unchanged: @@ -129,7 +128,7 @@ static int cti_enable_hw(struct cti_drvdata *drvdata, enum cs_mode mode) cti_err_not_enabled: if (!drvdata->config.enable_req_count) coresight_set_mode(csdev, CS_MODE_DISABLED); - raw_spin_unlock_irqrestore(&drvdata->spinlock, flags); + raw_spin_unlock(&drvdata->spinlock); return rc; }
To support dynamically enabling and disabling the Coresight path during CPU idle, cscfg_config_sysfs_get_active_cfg() may be called by the ETM driver in an atomic context. However, the function currently acquires a mutex, which is a sleepable lock, and this is not allowed in atomic context.
To make cscfg_config_sysfs_get_active_cfg() called in the idle flow, replace the mutex with a raw spinlock to protect the active variables.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-syscfg.c | 22 +++++++++++++--------- drivers/hwtracing/coresight/coresight-syscfg.h | 2 ++ 2 files changed, 15 insertions(+), 9 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-syscfg.c b/drivers/hwtracing/coresight/coresight-syscfg.c index 6836b05986e8091b4f8ad7d082d6a016f8cf7d74..53aa1a44f9ac8f65b4e3cce06a33255abfc17351 100644 --- a/drivers/hwtracing/coresight/coresight-syscfg.c +++ b/drivers/hwtracing/coresight/coresight-syscfg.c @@ -957,15 +957,19 @@ int cscfg_config_sysfs_activate(struct cscfg_config_desc *config_desc, bool acti
cfg_hash = (unsigned long)config_desc->event_ea->var;
+ raw_spin_lock(&cscfg_mgr->spinlock); + if (activate) { + /* cannot be a current active value to activate this */ if (cscfg_mgr->sysfs_active_config) { err = -EBUSY; - goto exit_unlock; + } else { + err = _cscfg_activate_config(cfg_hash); + if (!err) + cscfg_mgr->sysfs_active_config = cfg_hash; } - err = _cscfg_activate_config(cfg_hash); - if (!err) - cscfg_mgr->sysfs_active_config = cfg_hash; + } else { /* disable if matching current value */ if (cscfg_mgr->sysfs_active_config == cfg_hash) { @@ -975,7 +979,8 @@ int cscfg_config_sysfs_activate(struct cscfg_config_desc *config_desc, bool acti err = -EINVAL; }
-exit_unlock: + raw_spin_unlock(&cscfg_mgr->spinlock); + mutex_unlock(&cscfg_mutex); return err; } @@ -983,9 +988,8 @@ int cscfg_config_sysfs_activate(struct cscfg_config_desc *config_desc, bool acti /* set the sysfs preset value */ void cscfg_config_sysfs_set_preset(int preset) { - mutex_lock(&cscfg_mutex); + guard(raw_spinlock)(&cscfg_mgr->spinlock); cscfg_mgr->sysfs_active_preset = preset; - mutex_unlock(&cscfg_mutex); }
/* @@ -994,10 +998,9 @@ void cscfg_config_sysfs_set_preset(int preset) */ void cscfg_config_sysfs_get_active_cfg(unsigned long *cfg_hash, int *preset) { - mutex_lock(&cscfg_mutex); + guard(raw_spinlock)(&cscfg_mgr->spinlock); *preset = cscfg_mgr->sysfs_active_preset; *cfg_hash = cscfg_mgr->sysfs_active_config; - mutex_unlock(&cscfg_mutex); } EXPORT_SYMBOL_GPL(cscfg_config_sysfs_get_active_cfg);
@@ -1201,6 +1204,7 @@ static int cscfg_create_device(void) INIT_LIST_HEAD(&cscfg_mgr->load_order_list); atomic_set(&cscfg_mgr->sys_active_cnt, 0); cscfg_mgr->load_state = CSCFG_NONE; + raw_spin_lock_init(&cscfg_mgr->spinlock);
/* setup the device */ dev = cscfg_device(); diff --git a/drivers/hwtracing/coresight/coresight-syscfg.h b/drivers/hwtracing/coresight/coresight-syscfg.h index 66e2db890d8203853a0c3c907b48aa66dd8014e6..f52aba4608dadc1c7666e0e28c9b43dc1c33d1eb 100644 --- a/drivers/hwtracing/coresight/coresight-syscfg.h +++ b/drivers/hwtracing/coresight/coresight-syscfg.h @@ -42,6 +42,7 @@ enum cscfg_load_ops { * @sysfs_active_config:Active config hash used if CoreSight controlled from sysfs. * @sysfs_active_preset:Active preset index used if CoreSight controlled from sysfs. * @load_state: A multi-stage load/unload operation is in progress. + * @spinlock: Exclusive access sysfs_active_* variables. */ struct cscfg_manager { struct device dev; @@ -54,6 +55,7 @@ struct cscfg_manager { u32 sysfs_active_config; int sysfs_active_preset; enum cscfg_load_ops load_state; + raw_spinlock_t spinlock; };
/* get reference to dev in cscfg_manager */
Introduce per-CPU path pointers, save activated paths into the structure. This will be used by later changes for controlling the path during CPU idle.
The path pointer is assigned before setting the source device mode to active, and it is cleared after the device is changed to an inactive mode. So safe access to path pointers is guaranteed when the device is in an active mode.
The path enabling has been refactored: a new _coresight_enable_path() function is introduced to enable components in the path. The existing coresight_enable_path() function now calls this new helper and updates the per-CPU pointers accordingly.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 48 ++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 94803cacf1a374ae8a3b9513d7d029d5c77d5925..904123279ee70531787feea98e5f9fc502470f88 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -37,6 +37,27 @@ DEFINE_MUTEX(coresight_mutex); static DEFINE_PER_CPU(struct coresight_device *, csdev_sink); static DEFINE_PER_CPU(struct coresight_device *, csdev_source);
+/* + * Activated path pointer for a per-CPU source device. When enabling a path, + * the path pointer is first assigned, followed by a synchronous SMP call on + * the target CPU to transition the device mode from DISABLED to an enabled + * state. Conversely, during the disable flow, an SMP call on the target CPU + * transitions the device mode to DISABLED, after which the path pointer is + * cleared. + * + * per_cpu(csdev_cpu_path, csdev->cpu) = path + * coresight_take_mode(csdev, CS_MODE_SYSFS or CS_MODE_PERF) + * + * // Safe to access per_cpu(csdev_cpu_path, cpu); + * + * coresight_set_mode(csdev, CS_MODE_DISABLED) + * per_cpu(csdev_cpu_path, csdev->cpu) = NULL + * + * As a result, the device mode is used to determine whether it is safe + * to access the path pointer. + */ +static DEFINE_PER_CPU(struct coresight_path *, csdev_cpu_path); + /** * struct coresight_node - elements of a path, from source to sink * @csdev: Address of an element. @@ -510,6 +531,12 @@ static void coresight_disable_path_from(struct coresight_path *path,
void coresight_disable_path(struct coresight_path *path) { + struct coresight_device *source; + + source = coresight_get_source(path); + if (coresight_is_percpu_source(source)) + per_cpu(csdev_cpu_path, source->cpu) = NULL; + coresight_disable_path_from(path, NULL); } EXPORT_SYMBOL_GPL(coresight_disable_path); @@ -533,8 +560,8 @@ static int coresight_enable_helpers(struct coresight_device *csdev, return 0; }
-int coresight_enable_path(struct coresight_path *path, enum cs_mode mode, - void *sink_data) +static int _coresight_enable_path(struct coresight_path *path, + enum cs_mode mode, void *sink_data) { int ret = 0; u32 type; @@ -601,6 +628,23 @@ int coresight_enable_path(struct coresight_path *path, enum cs_mode mode, goto out; }
+int coresight_enable_path(struct coresight_path *path, enum cs_mode mode, + void *sink_data) +{ + int ret; + struct coresight_device *source; + + ret = _coresight_enable_path(path, mode, sink_data); + if (ret) + return ret; + + source = coresight_get_source(path); + if (coresight_is_percpu_source(source)) + per_cpu(csdev_cpu_path, source->cpu) = path; + + return 0; +} + struct coresight_device *coresight_get_sink(struct coresight_path *path) { struct coresight_device *csdev;
Introduce an in_idle argument to the path enable and disable functions. When set to true, it skips to touch the sink device. To avoid invoking sink related helpers, the condition check is moved before the enable helpers are called.
This is a preparation for managing the path during CPU idle.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 29 +++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 904123279ee70531787feea98e5f9fc502470f88..0e8506d5eaa01d1cc4846324a64eae5f0f442da3 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -479,7 +479,8 @@ static void coresight_restore_source(struct coresight_device *csdev) * disabled. */ static void coresight_disable_path_from(struct coresight_path *path, - struct coresight_node *nd) + struct coresight_node *nd, + bool in_idle) { u32 type; struct coresight_device *csdev, *parent, *child; @@ -502,6 +503,10 @@ static void coresight_disable_path_from(struct coresight_path *path, CORESIGHT_DEV_TYPE_SINK : CORESIGHT_DEV_TYPE_LINK;
+ /* To reduce latency, CPU idle does not touch the sink */ + if (in_idle && type == CORESIGHT_DEV_TYPE_SINK) + continue; + switch (type) { case CORESIGHT_DEV_TYPE_SINK: coresight_disable_sink(csdev); @@ -537,7 +542,7 @@ void coresight_disable_path(struct coresight_path *path) if (coresight_is_percpu_source(source)) per_cpu(csdev_cpu_path, source->cpu) = NULL;
- coresight_disable_path_from(path, NULL); + coresight_disable_path_from(path, NULL, false); } EXPORT_SYMBOL_GPL(coresight_disable_path);
@@ -561,7 +566,8 @@ static int coresight_enable_helpers(struct coresight_device *csdev, }
static int _coresight_enable_path(struct coresight_path *path, - enum cs_mode mode, void *sink_data) + enum cs_mode mode, void *sink_data, + bool in_idle) { int ret = 0; u32 type; @@ -574,10 +580,6 @@ static int _coresight_enable_path(struct coresight_path *path, csdev = nd->csdev; type = csdev->type;
- /* Enable all helpers adjacent to the path first */ - ret = coresight_enable_helpers(csdev, mode, path); - if (ret) - goto err_disable_path; /* * ETF devices are tricky... They can be a link or a sink, * depending on how they are configured. If an ETF has been @@ -589,6 +591,15 @@ static int _coresight_enable_path(struct coresight_path *path, CORESIGHT_DEV_TYPE_SINK : CORESIGHT_DEV_TYPE_LINK;
+ /* To reduce latency, CPU idle does not touch the sink */ + if (in_idle && type == CORESIGHT_DEV_TYPE_SINK) + continue; + + /* Enable all helpers adjacent to the path first */ + ret = coresight_enable_helpers(csdev, mode, path); + if (ret) + goto err_disable_path; + switch (type) { case CORESIGHT_DEV_TYPE_SINK: ret = coresight_enable_sink(csdev, mode, sink_data); @@ -624,7 +635,7 @@ static int _coresight_enable_path(struct coresight_path *path, err_disable_helpers: coresight_disable_helpers(csdev, path); err_disable_path: - coresight_disable_path_from(path, nd); + coresight_disable_path_from(path, nd, false); goto out; }
@@ -634,7 +645,7 @@ int coresight_enable_path(struct coresight_path *path, enum cs_mode mode, int ret; struct coresight_device *source;
- ret = _coresight_enable_path(path, mode, sink_data); + ret = _coresight_enable_path(path, mode, sink_data, false); if (ret) return ret;
Control links and helpers on an activated path during CPU idle. Since coresight_disable_path_from() does not handle a source device's helpers, explicitly disable them alongside the source device.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 0e8506d5eaa01d1cc4846324a64eae5f0f442da3..04e8c3b862494abb2a5a46d385f0ff4cad93b71f 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -460,8 +460,15 @@ static bool coresight_need_save_restore_source(struct coresight_device *csdev)
static int coresight_save_source(struct coresight_device *csdev) { - if (csdev && source_ops(csdev)->save) - return source_ops(csdev)->save(csdev); + int ret; + + if (csdev && source_ops(csdev)->save) { + ret = source_ops(csdev)->save(csdev); + if (ret) + return ret; + + coresight_disable_helpers(csdev, NULL); + }
/* Return success if callback is not supported */ return 0; @@ -1665,17 +1672,33 @@ static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, { unsigned int cpu = smp_processor_id(); struct coresight_device *source = per_cpu(csdev_source, cpu); + struct coresight_path *path;
if (!coresight_need_save_restore_source(source)) return NOTIFY_OK;
+ /* + * When run at here, the source device mode is enabled. + * The activated path pointer must not be NULL. + */ + path = per_cpu(csdev_cpu_path, source->cpu); + if (WARN_ON(!path)) + return NOTIFY_BAD; + switch (cmd) { case CPU_PM_ENTER: if (coresight_save_source(source)) return NOTIFY_BAD; + + coresight_disable_path_from(path, NULL, true); break; case CPU_PM_EXIT: case CPU_PM_ENTER_FAILED: + if (_coresight_enable_path(path, + coresight_get_mode(source), + NULL, true)) + return NOTIFY_BAD; + coresight_restore_source(source); break; default:
Unlike a system level's sink, the per-CPU sink may lose power during CPU idle states. Currently, this refers specifically to TRBE as the sink. This commit registers save and restore callbacks for the per-CPU sink via the PM notifier.
There are no changes to the coresight_enable_helpers() function; the code movement is solely for compilation purposes.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 101 +++++++++++++++++++++------ include/linux/coresight.h | 4 ++ 2 files changed, 82 insertions(+), 23 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index 04e8c3b862494abb2a5a46d385f0ff4cad93b71f..ef9dbba285710fe508580ee2726e22bcc654014e 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -393,6 +393,25 @@ static void coresight_disable_helper(struct coresight_device *csdev, void *data) helper_ops(csdev)->disable(csdev, data); }
+static int coresight_enable_helpers(struct coresight_device *csdev, + enum cs_mode mode, void *data) +{ + int i, ret = 0; + struct coresight_device *helper; + + for (i = 0; i < csdev->pdata->nr_outconns; ++i) { + helper = csdev->pdata->out_conns[i]->dest_dev; + if (!helper || !coresight_is_helper(helper)) + continue; + + ret = coresight_enable_helper(helper, mode, data); + if (ret) + return ret; + } + + return 0; +} + static void coresight_disable_helpers(struct coresight_device *csdev, void *data) { int i; @@ -480,6 +499,43 @@ static void coresight_restore_source(struct coresight_device *csdev) source_ops(csdev)->restore(csdev); }
+static int coresight_save_percpu_sink(struct coresight_device *csdev) +{ + int ret; + + if (csdev && sink_ops(csdev)->save) { + ret = sink_ops(csdev)->save(csdev); + if (ret) + return ret; + + coresight_disable_helpers(csdev, NULL); + } + + /* Return success if callback is not supported */ + return 0; +} + +static int coresight_restore_percpu_sink(struct coresight_device *csdev, + struct coresight_path *path, + enum cs_mode mode) +{ + int ret = 0; + + if (csdev && sink_ops(csdev)->restore) { + ret = coresight_enable_helpers(csdev, mode, path); + if (ret) + return ret; + + ret = sink_ops(csdev)->restore(csdev); + if (ret) { + coresight_disable_helpers(csdev, path); + return ret; + } + } + + return ret; +} + /* * coresight_disable_path_from : Disable components in the given path beyond * @nd in the list. If @nd is NULL, all the components, except the SOURCE are @@ -553,25 +609,6 @@ void coresight_disable_path(struct coresight_path *path) } EXPORT_SYMBOL_GPL(coresight_disable_path);
-static int coresight_enable_helpers(struct coresight_device *csdev, - enum cs_mode mode, void *data) -{ - int i, ret = 0; - struct coresight_device *helper; - - for (i = 0; i < csdev->pdata->nr_outconns; ++i) { - helper = csdev->pdata->out_conns[i]->dest_dev; - if (!helper || !coresight_is_helper(helper)) - continue; - - ret = coresight_enable_helper(helper, mode, data); - if (ret) - return ret; - } - - return 0; -} - static int _coresight_enable_path(struct coresight_path *path, enum cs_mode mode, void *sink_data, bool in_idle) @@ -1670,9 +1707,12 @@ EXPORT_SYMBOL_GPL(coresight_alloc_device_name); static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, void *v) { + int ret; unsigned int cpu = smp_processor_id(); struct coresight_device *source = per_cpu(csdev_source, cpu); struct coresight_path *path; + struct coresight_device *sink; + enum cs_mode mode;
if (!coresight_need_save_restore_source(source)) return NOTIFY_OK; @@ -1685,18 +1725,33 @@ static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, if (WARN_ON(!path)) return NOTIFY_BAD;
+ sink = coresight_get_sink(path); + mode = coresight_get_mode(source); + switch (cmd) { case CPU_PM_ENTER: if (coresight_save_source(source)) return NOTIFY_BAD;
- coresight_disable_path_from(path, NULL, true); + ret = 0; + if (coresight_is_percpu_sink(sink)) + ret = coresight_save_percpu_sink(sink); + else + coresight_disable_path_from(path, NULL, true); + + if (ret) { + coresight_restore_source(source); + return NOTIFY_BAD; + } break; case CPU_PM_EXIT: case CPU_PM_ENTER_FAILED: - if (_coresight_enable_path(path, - coresight_get_mode(source), - NULL, true)) + if (coresight_is_percpu_sink(sink)) + ret = coresight_restore_percpu_sink(sink, path, mode); + else + ret = _coresight_enable_path(path, mode, NULL, true); + + if (ret) return NOTIFY_BAD;
coresight_restore_source(source); diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 462876b5a870f3e93ad2a1807a9045d5eb115ccd..21bb99f9684747d7a610aa7185e1ba5f987248d9 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -363,6 +363,8 @@ enum cs_mode { * @alloc_buffer: initialises perf's ring buffer for trace collection. * @free_buffer: release memory allocated in @get_config. * @update_buffer: update buffer pointers after a trace session. + * @save: save context for a sink. + * @restore: restore context for a sink. */ struct coresight_ops_sink { int (*enable)(struct coresight_device *csdev, enum cs_mode mode, @@ -375,6 +377,8 @@ struct coresight_ops_sink { unsigned long (*update_buffer)(struct coresight_device *csdev, struct perf_output_handle *handle, void *sink_config); + int (*save)(struct coresight_device *csdev); + int (*restore)(struct coresight_device *csdev); };
/**
From: Yabin Cui yabinc@google.com
Similar to ETE, TRBE may lose its context when a CPU enters low power state. To make things worse, if ETE is restored without TRBE being restored, an enabled source device with no enabled sink devices can cause CPU hang on some devices (e.g., Pixel 9).
The save and restore flows are described in the section K5.5 "Context switching" of Arm ARM (ARM DDI 0487 L.a). This commit adds save and restore callbacks with following the software usages defined in the architecture manual.
Signed-off-by: Yabin Cui yabinc@google.com Co-developed-by: Leo Yan leo.yan@arm.com Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-trbe.c | 84 ++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+)
diff --git a/drivers/hwtracing/coresight/coresight-trbe.c b/drivers/hwtracing/coresight/coresight-trbe.c index 3fd379f327eb4f82665b216856d43ec39e4e7d10..5d103f2bb0a9bc851e4d3e9d13a93412bb71b8ed 100644 --- a/drivers/hwtracing/coresight/coresight-trbe.c +++ b/drivers/hwtracing/coresight/coresight-trbe.c @@ -115,6 +115,20 @@ static int trbe_errata_cpucaps[] = { */ #define TRBE_WORKAROUND_OVERWRITE_FILL_MODE_SKIP_BYTES 256
+/* + * struct trbe_save_state: Register values representing TRBE state + * @trblimitr - Trace Buffer Limit Address Register value + * @trbbaser - Trace Buffer Base Register value + * @trbptr - Trace Buffer Write Pointer Register value + * @trbsr - Trace Buffer Status Register value + */ +struct trbe_save_state { + u64 trblimitr; + u64 trbbaser; + u64 trbptr; + u64 trbsr; +}; + /* * struct trbe_cpudata: TRBE instance specific data * @trbe_flag - TRBE dirty/access flag support @@ -133,6 +147,7 @@ struct trbe_cpudata { enum cs_mode mode; struct trbe_buf *buf; struct trbe_drvdata *drvdata; + struct trbe_save_state save_state; DECLARE_BITMAP(errata, TRBE_ERRATA_MAX); };
@@ -1188,12 +1203,81 @@ static irqreturn_t arm_trbe_irq_handler(int irq, void *dev) return IRQ_HANDLED; }
+static int arm_trbe_save(struct coresight_device *csdev) +{ + struct trbe_cpudata *cpudata = dev_get_drvdata(&csdev->dev); + struct trbe_save_state *state = &cpudata->save_state; + + if (cpudata->mode == CS_MODE_DISABLED) + return 0; + + /* + * According to the section K5.5 Context switching, Arm ARM (ARM DDI + * 0487 L.a), the software usage VKHHY requires a TSB CSYNC instruction + * to ensure the program-flow trace is flushed, which has been executed + * in ETM driver. + */ + + /* Disable trace buffer unit */ + state->trblimitr = read_sysreg_s(SYS_TRBLIMITR_EL1); + write_sysreg_s(state->trblimitr & ~TRBLIMITR_EL1_E, SYS_TRBLIMITR_EL1); + + /* + * Execute a further Context synchronization event. Ensure the writes to + * memory are complete. + */ + trbe_drain_buffer(); + + /* Synchronize the TRBE disabling */ + isb(); + + state->trbbaser = read_sysreg_s(SYS_TRBBASER_EL1); + state->trbptr = read_sysreg_s(SYS_TRBPTR_EL1); + state->trbsr = read_sysreg_s(SYS_TRBSR_EL1); + return 0; +} + +static int arm_trbe_restore(struct coresight_device *csdev) +{ + struct trbe_cpudata *cpudata = dev_get_drvdata(&csdev->dev); + struct trbe_save_state *state = &cpudata->save_state; + + if (cpudata->mode == CS_MODE_DISABLED) + return 0; + + write_sysreg_s(state->trbbaser, SYS_TRBBASER_EL1); + write_sysreg_s(state->trbptr, SYS_TRBPTR_EL1); + write_sysreg_s(state->trbsr, SYS_TRBSR_EL1); + write_sysreg_s(state->trblimitr & ~TRBLIMITR_EL1_E, SYS_TRBLIMITR_EL1); + + /* + * According to the section K5.5 Context switching, Arm ARM (ARM DDI + * 0487 L.a), the software usage PKLXF requires a Context + * synchronization event to guarantee the Trace Buffer Unit will observe + * the new values of the System registers. + */ + isb(); + + /* Enable the Trace Buffer Unit */ + write_sysreg_s(state->trblimitr, SYS_TRBLIMITR_EL1); + + /* Synchronize the TRBE enable event */ + isb(); + + if (trbe_needs_ctxt_sync_after_enable(cpudata)) + isb(); + + return 0; +} + static const struct coresight_ops_sink arm_trbe_sink_ops = { .enable = arm_trbe_enable, .disable = arm_trbe_disable, .alloc_buffer = arm_trbe_alloc_buffer, .free_buffer = arm_trbe_free_buffer, .update_buffer = arm_trbe_update_buffer, + .save = arm_trbe_save, + .restore = arm_trbe_restore, };
static const struct coresight_ops arm_trbe_cs_ops = {
The hotplug lock is acquired and released in the etm4_disable_sysfs() function, which is a low-level function located in the ETM4 driver. This prevents us from a new solution for hotplug.
Firstly, hotplug callbacks cannot invoke etm4_disable_sysfs() to disable the source; otherwise, a deadlock issue occurs. The reason is that, in the hotplug flow, the kernel acquires the hotplug lock before calling callbacks. Subsequently, if coresight_disable_source() is invoked and it calls etm4_disable_sysfs(), the hotplug lock will be acquired twice, leading to a double lock issue.
Secondly, when hotplugging a CPU on or off, if we want to manipulate all components on a path attached to the CPU, we need to maintain atomicity for the entire path. Otherwise, a race condition may occur with users setting the same path via the Sysfs knobs, ultimately causing mess states in CoreSight components.
This patch moves the hotplug locking from etm4_disable_sysfs() into enable_source_store(). As a result, when users control the Sysfs knobs, the whole flow is protected by hotplug locking, ensuring it is mutual exclusive with hotplug callbacks.
Note, the paired function etm4_enable_sysfs() does not use hotplug locking, which is why this patch does not modify it.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 9 --------- drivers/hwtracing/coresight/coresight-sysfs.c | 7 +++++++ 2 files changed, 7 insertions(+), 9 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 4218cefca63d7b2f3b102e74d7f007ebc7b08861..920c817de8c33751760a1ed7ab1df8437620bb4e 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1088,13 +1088,6 @@ static void etm4_disable_sysfs(struct coresight_device *csdev) { struct etmv4_drvdata *drvdata = dev_get_drvdata(csdev->dev.parent);
- /* - * Taking hotplug lock here protects from clocks getting disabled - * with tracing being left on (crash scenario) if user disable occurs - * after cpu online mask indicates the cpu is offline but before the - * DYING hotplug callback is serviced by the ETM driver. - */ - cpus_read_lock(); raw_spin_lock(&drvdata->spinlock);
/* @@ -1108,8 +1101,6 @@ static void etm4_disable_sysfs(struct coresight_device *csdev)
cscfg_csdev_disable_active_config(csdev);
- cpus_read_unlock(); - /* * we only release trace IDs when resetting sysfs. * This permits sysfs users to read the trace ID after the trace diff --git a/drivers/hwtracing/coresight/coresight-sysfs.c b/drivers/hwtracing/coresight/coresight-sysfs.c index cb02f1bfaf068ec3f50f997bcb94f7d5215ccea8..a4c5a334703f0bb35db5eeda87ae5a9756f6a453 100644 --- a/drivers/hwtracing/coresight/coresight-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-sysfs.c @@ -363,6 +363,13 @@ static ssize_t enable_source_store(struct device *dev, if (ret) return ret;
+ /* + * CoreSight hotplug callbacks in core layer control a activated path + * from its source to sink. Taking hotplug lock here protects a race + * condition with hotplug callbacks. + */ + guard(cpus_read_lock)(); + if (val) { ret = coresight_enable_sysfs(csdev); if (ret)
This commit moves CPU hotplug callbacks from ETMv4 driver to core layer. The motivation is the core layer can control all components on an activated path rather but not only managing tracer in ETMv4 driver.
The perf event layer will disable CoreSight PMU event 'cs_etm' when hotplug off a CPU. That means a perf mode will be always converted to disabled mode in CPU hotplug. Arm CoreSight CPU hotplug callbacks only need to handle the Sysfs mode and ignore the perf mode.
The core layer invokes a high level API coresight_disable_source() to disable a source when hotplug-off a CPU. It disables a tracer and changes the tracer's mode to CS_MODE_DISABLED.
When hotplug-in a CPU, if a activated path is detected - when the activated path pointer is not NULL - in this case, the tracer will be re-enabled.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 61 +++++++++++++++++++++- drivers/hwtracing/coresight/coresight-etm4x-core.c | 37 ------------- include/linux/coresight.h | 17 +++--- 3 files changed, 70 insertions(+), 45 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index ef9dbba285710fe508580ee2726e22bcc654014e..e39d1f835d77c302d475f93e125dd4f19d313864 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -1704,6 +1704,52 @@ char *coresight_alloc_device_name(struct coresight_dev_list *dict, } EXPORT_SYMBOL_GPL(coresight_alloc_device_name);
+static int coresight_starting_cpu(unsigned int cpu) +{ + struct coresight_device *source = per_cpu(csdev_source, cpu); + struct coresight_path *path; + + if (!source) + return 0; + + /* Re-enable components on an activated path */ + path = per_cpu(csdev_cpu_path, cpu); + if (!path) + return 0; + + if (path->saved_mode != CS_MODE_SYSFS) + return 0; + + source_ops(source)->enable(source, NULL, path->saved_mode, path); + return 0; +} + +static int coresight_dying_cpu(unsigned int cpu) +{ + struct coresight_device *source = per_cpu(csdev_source, cpu); + struct coresight_path *path; + + if (!source) + return 0; + + /* Don't proceed if no path is activated */ + path = per_cpu(csdev_cpu_path, cpu); + if (!path) + return 0; + + path->saved_mode = coresight_get_mode(source); + + /* + * The perf event layer will disable PMU events in the CPU hotplug. + * CoreSight driver should never handle the CS_MODE_PERF case. + */ + if (WARN_ON(path->saved_mode != CS_MODE_SYSFS)) + return 0; + + source_ops(source)->disable(source, NULL); + return 0; +} + static int coresight_cpu_pm_notify(struct notifier_block *nb, unsigned long cmd, void *v) { @@ -1769,11 +1815,24 @@ static struct notifier_block coresight_cpu_pm_nb = {
static int __init coresight_pm_setup(void) { - return cpu_pm_register_notifier(&coresight_cpu_pm_nb); + int ret; + + ret = cpu_pm_register_notifier(&coresight_cpu_pm_nb); + if (ret) + return ret; + + ret = cpuhp_setup_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING, + "arm/coresight-core:starting", + coresight_starting_cpu, coresight_dying_cpu); + if (ret) + cpu_pm_unregister_notifier(&coresight_cpu_pm_nb); + + return ret; }
static void coresight_pm_cleanup(void) { + cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); cpu_pm_unregister_notifier(&coresight_cpu_pm_nb); }
diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index 920c817de8c33751760a1ed7ab1df8437620bb4e..e865d3fda10e10d7861cdb07ac3606e649b13ace 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1805,33 +1805,6 @@ static int etm4_online_cpu(unsigned int cpu) return 0; }
-static int etm4_starting_cpu(unsigned int cpu) -{ - if (!etmdrvdata[cpu]) - return 0; - - raw_spin_lock(&etmdrvdata[cpu]->spinlock); - if (!etmdrvdata[cpu]->os_unlock) - etm4_os_unlock(etmdrvdata[cpu]); - - if (coresight_get_mode(etmdrvdata[cpu]->csdev)) - etm4_enable_hw(etmdrvdata[cpu], true); - raw_spin_unlock(&etmdrvdata[cpu]->spinlock); - return 0; -} - -static int etm4_dying_cpu(unsigned int cpu) -{ - if (!etmdrvdata[cpu]) - return 0; - - raw_spin_lock(&etmdrvdata[cpu]->spinlock); - if (coresight_get_mode(etmdrvdata[cpu]->csdev)) - etm4_disable_hw(etmdrvdata[cpu]); - raw_spin_unlock(&etmdrvdata[cpu]->spinlock); - return 0; -} - static bool etm4_need_context_save_restore(struct coresight_device *csdev) { if (pm_save_enable != PARAM_PM_SAVE_SELF_HOSTED) @@ -1926,13 +1899,6 @@ static int __init etm4_pm_setup(void) { int ret;
- ret = cpuhp_setup_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING, - "arm/coresight4:starting", - etm4_starting_cpu, etm4_dying_cpu); - - if (ret) - return ret; - ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "arm/coresight4:online", etm4_online_cpu, NULL); @@ -1943,14 +1909,11 @@ static int __init etm4_pm_setup(void) return 0; }
- /* failed dyn state - remove others */ - cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); return ret; }
static void etm4_pm_clear(void) { - cpuhp_remove_state_nocalls(CPUHP_AP_ARM_CORESIGHT_STARTING); if (hp_online) { cpuhp_remove_state_nocalls(hp_online); hp_online = 0; diff --git a/include/linux/coresight.h b/include/linux/coresight.h index 21bb99f9684747d7a610aa7185e1ba5f987248d9..08a2ea9b843405f179f1748b35827502faba7472 100644 --- a/include/linux/coresight.h +++ b/include/linux/coresight.h @@ -330,21 +330,24 @@ static struct coresight_dev_list (var) = { \
#define to_coresight_device(d) container_of(d, struct coresight_device, dev)
+enum cs_mode { + CS_MODE_DISABLED, + CS_MODE_SYSFS, + CS_MODE_PERF, + CS_MODE_DEBUG, +}; + /** * struct coresight_path - data needed by enable/disable path * @path_list: path from source to sink. * @trace_id: trace_id of the whole path. + * @saved_mode: The saved device mode. It stores the source device's mode + * to represent the path mode during CPU hotplug off. */ struct coresight_path { struct list_head path_list; u8 trace_id; -}; - -enum cs_mode { - CS_MODE_DISABLED, - CS_MODE_SYSFS, - CS_MODE_PERF, - CS_MODE_DEBUG, + enum cs_mode saved_mode; };
#define coresight_ops(csdev) csdev->ops
This commit handles activated path during the CPU hotplug process.
When a CPU is hotplug off or hotplug in, and if an activated path is associated with it, the CPU PM notifier disables and enables the path, including the sink and helpers.
When disabling a path, the sink's disable() callback updates its buffer in SysFS mode.
Signed-off-by: Leo Yan leo.yan@arm.com --- drivers/hwtracing/coresight/coresight-core.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/hwtracing/coresight/coresight-core.c b/drivers/hwtracing/coresight/coresight-core.c index e39d1f835d77c302d475f93e125dd4f19d313864..bbad58335129968f7d0f5f22084a93c3cb114ae7 100644 --- a/drivers/hwtracing/coresight/coresight-core.c +++ b/drivers/hwtracing/coresight/coresight-core.c @@ -1720,6 +1720,7 @@ static int coresight_starting_cpu(unsigned int cpu) if (path->saved_mode != CS_MODE_SYSFS) return 0;
+ _coresight_enable_path(path, path->saved_mode, NULL, false); source_ops(source)->enable(source, NULL, path->saved_mode, path); return 0; } @@ -1746,7 +1747,8 @@ static int coresight_dying_cpu(unsigned int cpu) if (WARN_ON(path->saved_mode != CS_MODE_SYSFS)) return 0;
- source_ops(source)->disable(source, NULL); + coresight_disable_source(source, NULL); + coresight_disable_path_from(path, NULL, false); return 0; }
-
Leo Yan -
Suzuki K Poulose