The current ETMx configuration via sysfs can lead to the following
inconsistencies:
- If a configuration is modified via sysfs while a perf session is
active, the running configuration may differ between before
a sched-out and after a subsequent sched-in.
- If a perf session and sysfs session tries to enable concurrently,
configuration from configfs could be corrupted (etm4).
- There is chance to corrupt drvdata->config if perf session tries
to enabled among handling cscfg_csdev_disable_active_config()
in etm4_disable_sysfs() (etm4).
To resolve these inconsistencies, the configuration should be separated into:
- active_config, which is applied configuration for the current session
- config, which stores the settings configured via sysfs.
and apply configuration from configfs after taking a mode.
Patch History
=============
from v10 to 11:
- replace direct register read from etm3 sysfs with IPI.
- fix issue cntr_val and others field which required to be shown after
sysfs session disable.
- prohibit the write for some etm3 sysfs while sysfs session is
enabled.
- drop locktype change in etm3
- Link to v10: https://lore.kernel.org/r/20260911-separate_etm_cfg_v2-v10-0-1b715d95927a@a…
from v9 to v10:
- rebase to coresight/next
- https://lore.kernel.org/all/20260725113645.57519-1-yeoreum.yun@arm.com/
from v8 to v9:
- add feat_csdev_lock guard interface.
- set feat_csdev->drv_spinlock as NULL for etmv4_drvdata.
- https://lore.kernel.org/all/20260629090007.1718746-1-yeoreum.yun@arm.com/
from v7 to v8:
- accept @Leo Yan' suggestion to handle error.
- small minor fixes following @Suzuki' suggestion.
- https://lore.kernel.org/all/20260519154812.254884-1-yeoreum.yun@arm.com/
from v6 to v7:
- rebase on coresight/next
- add ETM_MAX_SEQ_TRANSITIONS define
- remove redundant patch relavent cpu-hotplug as coresight-pm patch
merged.
- https://lore.kernel.org/all/20260422132203.977549-1-yeoreum.yun@arm.com/
from v5 to v6:
- fix missing of calling cscfg_csdev_disable_active_config()
- add rb & fixes tags.
- add ss_status field in etm4x_drvdata to expose STATUS and PENDING bits.
- https://lore.kernel.org/all/20260415165528.3369607-1-yeoreum.yun@arm.com/
from v4 to v5:
- add rb-tag.
- fix underflow issue for nrseqstate.
- fix wrong check in etm4_sspcicrn_present().
- remove redundant fields on etmv4_save_state.
- rename caps->ss_status to ss_cmp.
- fix wrong location of etm4_release_trace_id.
- https://lore.kernel.org/all/20260413142003.3549310-1-yeoreum.yun@arm.com/
from v3 to v4:
- change etm_drvdata->spinlock type to raw_spin_lock_t
- remove redundant call etmX_enable_hw() with starting_cpu() callsback.
- fix missing trace id release.
- add missing docs.
- https://lore.kernel.org/all/20260412175506.412301-1-yeoreum.yun@arm.com/
from v2 to v3:
- fix build error for etm3x.
- fix checkpatch warning.
- https://lore.kernel.org/all/20260410074310.2693385-1-yeoreum.yun@arm.com/
from v1 to v2
- rebased to v7.0-rc7.
- introduce etmX_caps structure to save etmX's capabilities.
- remove ss_status from etmv4_config.
- modify active_config after taking a mode (perf/sysfs).
- https://lore.kernel.org/all/20260317181705.2456271-1-yeoreum.yun@arm.com/
---
Yeoreum Yun (9):
coresight: etm4x: prohibit modifying ss_status and cntr_val while session is enabled
coresight: etm3x: prohibit modifying cntr_val and reset while session is enabled
coresight: etm4x: fix inconsistencies with sysfs configuration
coresight: etm3x: fix inconsistencies with sysfs configuration
coresight: etm3x: remove redundant cpu online check on etm_enable_sysfs()
coresight: etm4x: introduce struct etm4_caps
coresight: etm4x: exclude ss_status from drvdata->config
coresight: etm4x: remove s_ex_level from config
coresight: etm3x: introduce struct etm_caps
drivers/hwtracing/coresight/coresight-config.c | 18 +-
drivers/hwtracing/coresight/coresight-config.h | 22 ++
drivers/hwtracing/coresight/coresight-etm.h | 46 ++-
drivers/hwtracing/coresight/coresight-etm3x-core.c | 126 +++---
.../hwtracing/coresight/coresight-etm3x-sysfs.c | 104 +++--
drivers/hwtracing/coresight/coresight-etm4x-cfg.c | 14 +-
drivers/hwtracing/coresight/coresight-etm4x-core.c | 426 +++++++++++----------
.../hwtracing/coresight/coresight-etm4x-sysfs.c | 210 ++++++----
drivers/hwtracing/coresight/coresight-etm4x.h | 196 +++++-----
9 files changed, 676 insertions(+), 486 deletions(-)
---
base-commit: 9e3604d7369cfc0110100eb1a0acab1865ee2d18
change-id: 20260911-separate_etm_cfg_v2-3518a168cbff
Best regards,
--
Sincerely,
Yeoreum Yun
The current ETMx configuration via sysfs can lead to the following
inconsistencies:
- If a configuration is modified via sysfs while a perf session is
active, the running configuration may differ between before
a sched-out and after a subsequent sched-in.
- If a perf session and sysfs session tries to enable concurrently,
configuration from configfs could be corrupted (etm4).
- There is chance to corrupt drvdata->config if perf session tries
to enabled among handling cscfg_csdev_disable_active_config()
in etm4_disable_sysfs() (etm4).
To resolve these inconsistencies, the configuration should be separated into:
- active_config, which is applied configuration for the current session
- config, which stores the settings configured via sysfs.
and apply configuration from configfs after taking a mode.
Patch History
=============
from v9 to v10:
- rebase to coresight/next
- https://lore.kernel.org/all/20260725113645.57519-1-yeoreum.yun@arm.com/
from v8 to v9:
- add feat_csdev_lock guard interface.
- set feat_csdev->drv_spinlock as NULL for etmv4_drvdata.
- https://lore.kernel.org/all/20260629090007.1718746-1-yeoreum.yun@arm.com/
from v7 to v8:
- accept @Leo Yan' suggestion to handle error.
- small minor fixes following @Suzuki' suggestion.
- https://lore.kernel.org/all/20260519154812.254884-1-yeoreum.yun@arm.com/
from v6 to v7:
- rebase on coresight/next
- add ETM_MAX_SEQ_TRANSITIONS define
- remove redundant patch relavent cpu-hotplug as coresight-pm patch
merged.
- https://lore.kernel.org/all/20260422132203.977549-1-yeoreum.yun@arm.com/
from v5 to v6:
- fix missing of calling cscfg_csdev_disable_active_config()
- add rb & fixes tags.
- add ss_status field in etm4x_drvdata to expose STATUS and PENDING bits.
- https://lore.kernel.org/all/20260415165528.3369607-1-yeoreum.yun@arm.com/
from v4 to v5:
- add rb-tag.
- fix underflow issue for nrseqstate.
- fix wrong check in etm4_sspcicrn_present().
- remove redundant fields on etmv4_save_state.
- rename caps->ss_status to ss_cmp.
- fix wrong location of etm4_release_trace_id.
- https://lore.kernel.org/all/20260413142003.3549310-1-yeoreum.yun@arm.com/
from v3 to v4:
- change etm_drvdata->spinlock type to raw_spin_lock_t
- remove redundant call etmX_enable_hw() with starting_cpu() callsback.
- fix missing trace id release.
- add missing docs.
- https://lore.kernel.org/all/20260412175506.412301-1-yeoreum.yun@arm.com/
from v2 to v3:
- fix build error for etm3x.
- fix checkpatch warning.
- https://lore.kernel.org/all/20260410074310.2693385-1-yeoreum.yun@arm.com/
from v1 to v2
- rebased to v7.0-rc7.
- introduce etmX_caps structure to save etmX's capabilities.
- remove ss_status from etmv4_config.
- modify active_config after taking a mode (perf/sysfs).
- https://lore.kernel.org/all/20260317181705.2456271-1-yeoreum.yun@arm.com/
---
Yeoreum Yun (8):
coresight: etm4x: fix inconsistencies with sysfs configuration
coresight: etm3x: fix inconsistencies with sysfs configuration
coresight: etm3x: change drvdata->spinlock type to raw_spin_lock_t
coresight: etm3x: remove redundant cpu online check on etm_enable_sysfs()
coresight: etm4x: introduce struct etm4_caps
coresight: etm4x: exclude ss_status from drvdata->config
coresight: etm4x: remove s_ex_level from config
coresight: etm3x: introduce struct etm_caps
drivers/hwtracing/coresight/coresight-config.c | 18 +-
drivers/hwtracing/coresight/coresight-config.h | 22 ++
drivers/hwtracing/coresight/coresight-etm.h | 48 ++-
drivers/hwtracing/coresight/coresight-etm3x-core.c | 96 ++---
.../hwtracing/coresight/coresight-etm3x-sysfs.c | 155 ++++----
drivers/hwtracing/coresight/coresight-etm4x-cfg.c | 14 +-
drivers/hwtracing/coresight/coresight-etm4x-core.c | 413 +++++++++++----------
.../hwtracing/coresight/coresight-etm4x-sysfs.c | 204 +++++-----
drivers/hwtracing/coresight/coresight-etm4x.h | 196 +++++-----
9 files changed, 645 insertions(+), 521 deletions(-)
---
base-commit: 9e3604d7369cfc0110100eb1a0acab1865ee2d18
change-id: 20260911-separate_etm_cfg_v2-3518a168cbff
Best regards,
--
Sincerely,
Yeoreum Yun
On Wed, Sep 16, 2026 at 11:04:20AM +0800, Jie Gan wrote:
[...]
> > From: Min Chen <min.chen(a)siengine.com>
> >
> > The flat ETR buffer comes from dma_alloc_noncoherent(), which zeroes it
> > with CPU stores. The DMA API requires the caller to sync the buffer for
> > the device before the device writes into it, but the TMC driver only
> > ever syncs for the CPU afterwards. On a non-coherent sink the zero fill
> > is therefore still dirty in cache when the ETR starts writing, and its
> > write-back lands on top of the trace data.
Good catch! I'm curious how you observed the dirty cache lines
overwriting trace data in DDR and causing corruption.
> Agree, without the sync, the dirty data may overwrites the trace data.
> > Add a sync_for_device() buffer operation and call it from
> > __tmc_etr_enable_hw() just before the TMC is enabled.
I don't think __tmc_etr_enable_hw() is the best place for the sync, as
it can be called frequently when an event is enabled, e.g. when a task
is scheduled in or migrated between CPUs. We should be able to sync
once after dma_alloc_noncoherent() instead.
The issue is not limited to buffer init. The driver also injects barrier
packets into the bounce buffer, which can race with the sink. Even
worse, the barrier packet write may collide with trace data when they
share a cache line.
I think we should consider writing barrier packets directly into the
AUX buffer. This would avoid stale cache data from barrier packet writes
and simplify the flow without additional sync operations.
Would you mind if I pick up this patch (keeping you as the author) and
add a second patch to address the barrier packet issue? That part may
need some several rounds refactoring so can have better shape, I think
it would be easier to consolidate the fixes on my side.
Thanks,
Leo
P.s. Please CC me on future CoreSight patches. If you're using the
mainline ./scripts/get_maintainer.pl, it should add me automatically.
I didn't receive this patch directly, which is why I'm replying to
Jie's email (also thanks Jie's review).
This series adds AUX sampling support to Arm CoreSight. A PMU sample can
carry a window of recent AUX trace, which perf generates callchain and
branch stack for the sampled thread. This provides execution history for
context-based profiling.
The series can be divided into four parts:
1. Patches 01 ~ 03 for event core:
The perf event core changes warn on non-positive AUX snapshot
returns before checking alignment padding. Space for a non-zero AUX
payload has already been reserved, so a zero return leaves that
payload unwritten. CoreSight fills unavailable trace with zeros and
returns the requested size.
It also prevents AUX sampling and pause/resume from nesting through
an NMI. A driver guard alone can leave perf's pause state
(aux_paused) inconsistent with the hardware state. The event core
ensures only one AUX callback is exclusively invoked.
Export the output copy helpers so CoreSight can fill sample payloads
when the driver is built as a module.
2. Patches 04 ~ 09 for ETM perf:
The ETM perf changes separate the lifetime of the published CoreSight
context from that of the AUX output handle.
Centralize buffer updates, and end AUX output when throttling stops
an event without PERF_EF_UPDATE so a later restart can acquire a
fresh handle.
Make hardware-state transitions reflect completed operations.
3. Patches 10 ~ 13 for TRBE driver:
TRBE snapshot buffers need padding to meet the CPU's alignment and
wrap constraints. Use circular buffer mode for overwrite snapshots,
retaining Fill mode on CPUs that require the write out-of-range
workaround.
TRBE faults can still raise interrupts in circular mode. Track when
the sink is handling an interrupt so an AUX sampling NMI can skip a
snapshot while the interrupted handler is updating the buffer. Make
repeated sink disable safe for deferred cleanup after a snapshot
failure.
4. Patches 14 ~ 25 for AUX sampling and decoding:
Shared atomic STOP and AUX action bits prevent an NMI stop from
tearing down the context while pause, resume or snapshot accesses it.
The AUX operation finishes and then completes the deferred stop,
publishing the stopped state only after hardware teardown.
Add snapshot_aux() to the CoreSight driver to pause the source, update
the sink buffer, disable the path, copy the recent trace into the PMU
sample, and then re-enable the path and resume the source.
Patch 17 fixes history collection for zero-IP PMU samples. Perf clears
the IP when a user-only sampling interrupt skids into the kernel, but
the timestamp and TID remain valid for matching the trace history.
In perf, decode each embedded payload as an independent trace window,
selecting the decoder with the sampled CPU and using the sample's
PID/TID and traced context IDs to attribute history to the sampled
thread. Refactor common timeless decoding and history collection, and
drain pending OpenCSD output so buffered history is not lost.
Attach reconstructed callchains and branch stacks to the original PMU
samples, preserving histories already present in the samples. Add a
test, and document the recording and decoding workflow and its
limitations.
The initial support targets unformatted trace from per-CPU sinks such as
TRBE. Shared formatted sinks such as ETR can mix trace from multiple CPUs,
consuming the limited sample window with execution unrelated to the
sampled thread.
This series is based on Amir's series "perf: Add CoreSight branch
history to existing samples" [1] for perf tool's branch / callchain
generating.
[1] https://lore.kernel.org/linux-perf-users/cover.1787005265.git.aaupov@fb.com/
Signed-off-by: Leo Yan <leo.yan(a)arm.com>
---
Leo Yan (25):
perf/core: Reject non-positive AUX snapshot sizes
perf/core: Prevent AUX sampling from racing with pause/resume
perf/core: Export output copy helpers for CoreSight
coresight: perf: End AUX output when an event is throttled
coresight: perf: Extract AUX buffer update helper
coresight: perf: Simplify flow for CPUs without a path
coresight: perf: Validate the live context through its path
coresight: perf: Validate context before resuming trace
coresight: perf: Improve hardware state transitions
coresight: trbe: Pad snapshot buffers
coresight: trbe: Use circular buffer mode for snapshots
coresight: trbe: Track per-CPU sink interrupt handling
coresight: trbe: Handle an already disabled sink
coresight: perf: Look up the trace path in etm_event_pause()
coresight: perf: Serialize AUX pause and resume with event stops
coresight: perf: Support AUX sampling with per-CPU sinks
perf cs-etm: Allow history collection for samples with zero IP
perf cs-etm: Unify timeless buffer decoding
perf cs-etm: Propagate errors from trace queue flushing
perf cs-etm: Drain pending packets before finishing trace blocks
perf cs-etm: Complete packet draining with end of trace
perf cs-etm: Centralize sample history collection
perf cs-etm: Decode AUX samples into callchains and branch stacks
perf test: Add CoreSight AUX sample decoding test
Documentation: coresight: Document AUX sample decoding
Documentation/trace/coresight/coresight-perf.rst | 39 ++
drivers/hwtracing/coresight/coresight-etm-perf.c | 396 +++++++++++++-----
drivers/hwtracing/coresight/coresight-etm-perf.h | 2 +
drivers/hwtracing/coresight/coresight-trbe.c | 122 ++++--
include/linux/coresight.h | 2 +
kernel/events/core.c | 26 +-
kernel/events/ring_buffer.c | 2 +
tools/perf/arch/arm/util/cs-etm.c | 6 +
tools/perf/tests/shell/coresight/aux_sample.sh | 189 +++++++++
tools/perf/util/cs-etm-decoder/cs-etm-decoder.c | 39 ++
tools/perf/util/cs-etm-decoder/cs-etm-decoder.h | 7 +
tools/perf/util/cs-etm.c | 491 ++++++++++++++++-------
12 files changed, 1040 insertions(+), 281 deletions(-)
---
base-commit: 134e1295eedfc9a3c6e4e63d8ef45dc63749449b
change-id: 20260826-arm_cs_support_aux_sample-275664545588
Best regards,
--
Leo Yan <leo.yan(a)arm.com>
Fix a few issues with per-thread mode:
- With TRBE, thread migrations hit a queue without a decoder assert
- On nVHE, everything looks like guest trace instead of host
- Tracing multiple threads didn't really work
After adding a tests for those, it turns out there were some snapshot
bugs not picked up by the other snapshot tests. That showed that the
different snapshot searches for Intel BTS and Arm SPE were inconsistent
and I think all tracers can benefit from using the same pointer fixup
logic, so that gets refactored into the common auxtrace.c. Then SPE can
just share the Intel BTS search as it was obviously copied from it
originally. IntelPT keeps the more advanced duplicate data search, but
in the future that could probably be made the common one and all tracers
would benefit from using it (except maybe BTS if it always has a very
small buffer, but I doubt the overhead of the duplicate search would be
an issue). For now this is more of a refactor rather than behavioral
change so don't do that yet.
Signed-off-by: James Clark <james.clark(a)linaro.org>
---
James Clark (14):
perf cs-etm: Fix nVHE per-thread decoding
perf cs-etm: Warn for invalid timestamp option
perf cs-etm: Turn on context packet timestamps in per-thread mode
perf cs-etm: Use per-CPU queues for per-thread mode
perf cs-etm: Increase default timestamp generation period
perf auxtrace: Turn Intel BTS snapshot search into a generic one
perf arm-spe: Use generic snapshot search
perf auxtrace: intel-pt: Use new snapshot_has_wrapped callback
perf cs-etm: Queue partial AUX records
perf cs-etm: Don't print missing buffers in snapshot mode
perf auxtrace: cs-etm: Capture wrapped snapshots
perf test: Allow infinite named_thread loops
perf test: Add test for per-thread mode
perf cs-etm: Test multiple per-thread threads
Documentation/userspace-api/perf_ring_buffer.rst | 6 +-
tools/perf/Documentation/perf-test.txt | 2 +-
tools/perf/arch/arm/util/cs-etm.c | 30 ++++-
tools/perf/arch/arm64/util/arm-spe.c | 147 +--------------------
tools/perf/arch/x86/util/intel-bts.c | 115 +---------------
tools/perf/arch/x86/util/intel-pt.c | 58 +++-----
.../perf/tests/shell/coresight/per-thread-multi.sh | 78 +++++++++++
tools/perf/tests/shell/coresight/per-thread.sh | 48 +++++++
.../perf/tests/shell/coresight/raw_dump_stress.sh | 5 -
tools/perf/tests/workloads/named_threads.c | 7 +-
tools/perf/util/auxtrace.c | 136 +++++++++++++++++--
tools/perf/util/auxtrace.h | 20 ++-
tools/perf/util/cs-etm.c | 98 ++++++++++----
13 files changed, 382 insertions(+), 368 deletions(-)
---
base-commit: bf10e6ee2ac3034c9068e03eed418fd16961984e
change-id: 20260605-james-cs-unformatted-per-thread-fix-50e723aa7f0e
Best regards,
--
James Clark <james.clark(a)linaro.org>
All CoreSight compliant components have an implementation defined number
of 0 to 8 claim tag bits in the claim tag registers.
These are used to claim the CoreSight resources by system agents.
ARM recommends implementions have 4 claim tag bits, though a valid
implementation can have 0 claim tags bits.
The CoreSight drivers implement a 2 claim tag bit protocol to allow
self hosted and external debug agents to manage access to the hardware.
However, if there are less than 2 claim tags available the protocol
incorrectly returns an error on device claim, as no checks are made.
If insufficient claim tags are present in a component then the protocol
must return success on claim / disclaim to allow components to be used
normally.
Add initialisation to read the CLAIMSET bits to establish the number of
available claim tag bits, and adjust the claim returns accordingly.
Cache the claimtag protocol availablity in the coresight_device to reduce
reads for the main claim/disclaim api.
changes since v3:
1) removed unneeded bool casts (Leo)
2) rebased to kernel 7.2 on coresight/next
changes since v2:
1) consolidated API to remove the API calls using just cs_access, which were
used purely to clear down stale self claim tags, replace with a normal
coresight_device API for initialisation, to match the claim/disclaim API.
This does both the check on availability and the stale tag clearance.
Updated all drivers to use the new init functionality
2) Added option for drivers to skip claim tag checking completely for devices
with no-compliant hardware, that do not implement registers at the claim tag
location, or do not operate correctly to indicate the correct number of
claim tags for the device.
changes since v1:
1) Added claim tag availability cache into coresight_device when using the
main coresight_claim_device() / coresight_disclaim_device() API.
Applies to coresight/next
Mike Leach (1):
coresight: fix issue where coresight component has no claimtags
drivers/hwtracing/coresight/coresight-catu.c | 6 +-
drivers/hwtracing/coresight/coresight-core.c | 139 ++++++++++++++++--
.../hwtracing/coresight/coresight-cti-core.c | 7 +-
drivers/hwtracing/coresight/coresight-etb10.c | 9 +-
.../coresight/coresight-etm3x-core.c | 8 +-
.../coresight/coresight-etm4x-core.c | 8 +-
.../hwtracing/coresight/coresight-funnel.c | 7 +-
drivers/hwtracing/coresight/coresight-priv.h | 7 +
.../coresight/coresight-replicator.c | 9 +-
.../hwtracing/coresight/coresight-tmc-core.c | 7 +-
include/linux/coresight.h | 23 ++-
11 files changed, 205 insertions(+), 25 deletions(-)
--
2.43.0