CoreSight September 2023

coresight@lists.linaro.org

10 participants
39 discussions

[RFC PATCH v3 0/8] Coresight for Kernel panic and watchdog reset

by Linu Cherian

This RFC v3 patch series is rebased on v6.5-rc7 and is dependent on the below two patches. - coresight: tmc: Make etr buffer mode user configurable from sysfs[1] - coresight: Fix run time warnings while reusing ETR buffer[2] Changelog from v2: * ETR reserved buffer mode can be selected only through the new sysfs buffer mode and not by default. This would avoid any conflicts with normal usage. * ETR buffer size in reserved mode is now always fixed to the maximum size of the reserved buffer and not user configurable. This avoids any conflicts with the default buffer size used in other ETR buffer modes. * Introduced new ops called prevboot_ops to factor out common code in tmc_etr_prepare_prevboot() and tmc_etb_prepare_prevboot(). spin_lock/unlock invocations tmc_read_prepare_* are now in a single function. * Added more stringent checks for selecting READ_PREVBOOT mode Other misc changes: * Added more details to DT bindings documentation * Fixed unhandled case error in etm4_disable with CONFIG_WERROR * TMC register saving now uses standard accessor functions * Added panic notifier unregistration * memremap of reserved and metadata buffers are now with _WB attributes * Cover letter title has been shortened. Changelog from v1: * V2 is a complete patchset with kernel panic trace tested on Linux 6.4. Details on testing with relevant console logs has been added for reference. * Two additional patches(patch 6 & 7) has been included to manage stopping of trace at the time of kernel panic. * Few bug fixes. RFC v1 is posted here: https://lists.linaro.org/archives/list/coresight@lists.linaro.org/thread/6F… Using Coresight for Kernel panic and Watchdog reset =================================================== This RFC is about extending Linux coresight driver support to address kernel panic and watchdog reset scenarios. This would help coresight users to debug kernel panic and watchdog reset with the help of coresight trace data. Coresight trace capture: Kernel panic ------------------------------------- From the coresight driver point of view, addressing the kernel panic situation has four main requirements. a. Support for allocation of trace buffer pages from reserved memory area. Platform can advertise this using a new device tree property added to relevant coresight nodes. b. Support for stopping coresight blocks at the time of panic c. Saving required metadata in the specified format d. Support for reading trace data captured at the time of panic Allocation of trace buffer pages from reserved RAM ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A new optional device tree property "memory-region" is added to the ETR/ETF device nodes, that would give the base address and size of trace buffer. Static allocation of trace buffers would ensure that both IOMMU enabled and disabled cases are handled. Also, platforms that support persistent RAM will allow users to read trace data in the subsequent boot without booting the crashdump kernel. Note: For ETR sink devices, this reserved region will be used for both trace capture and trace data retrieval. For ETF sink devices, internal SRAM would be used for trace capture, and they would be synced to reserved region for retrieval. Note: Patches 1 & 2 adds support for this. Disabling coresight blocks at the time of panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order to avoid the situation of losing relevant trace data after a kernel panic, it would be desirable to stop the coresight blocks at the time of panic. This can be achieved by configuring the comparator, CTI and sink devices as below, Comparator(triggers on kernel panic) --->External out --->CTI -- | ETR/ETF stop <------External In <-------------- Note: * Patch 6 provides the necessary ETR configuration. * Patch 7 provides the necessary ETM configuration. Saving metadata at the time of kernel panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Coresight metadata involves all additional data that are required for a successful trace decode in addition to the trace data. This involves ETR/ETF, ETE register snapshot etc. A new optional device property "memory-region" is added to the ETR/ETF/ETE device nodes for this. Note: Patches 3 & 4 adds support for this. Reading trace data captured at the time of panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Trace data captured at the time of panic, can be read from rebooted kernel or from crashdump kernel using the below mentioned interface. Note: Patch 5 adds support for this. Steps for reading trace data captured in previous boot ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1. cd /sys/bus/coresight/devices/tmc_etrXX/ 2. Change to special mode called, read_prevboot. #echo 1 > read_prevboot 3. Dump trace buffer data to a file, #dd if=/dev/tmc_etrXX of=~/cstrace.bin 4. Reset back to normal mode #echo 0 > read_prevboot General flow of trace capture and decode incase of kernel panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Enable source and sink on all the cores using the sysfs interface. ETR sink will have trace buffers allocated from reserved memory, by selecting "resrv" buffer mode from sysfs. 2. Run relevant tests. 3. On a kernel panic, all coresight blocks are disabled, necessary metadata is synced by kernel panic handler. System would eventually reboot or boot a crashdump kernel. 4. For platforms that supports crashdump kernel, raw trace data can be dumped using the coresight sysfs interface from the crashdump kernel itself. Persistent RAM is not a requirement in this case. 5. For platforms that supports persistent RAM, trace data can be dumped using the coresight sysfs interface in the subsequent Linux boot. Crashdump kernel is not a requirement in this case. Persistent RAM ensures that trace data is intact across reboot. Coresight trace capture: Watchdog reset --------------------------------------- The main difference between addressing the watchdog reset and kernel panic case are below, a. Saving coresight metadata need to be taken care by the SCP(system control processor) firmware in the specified format, instead of kernel. b. Reserved memory region given by firmware for trace buffer and metadata has to be in persistent RAM. Note: This is a requirement for watchdog reset case but optional in kernel panic case. Watchdog reset can be supported only on platforms that meet the above two requirements. Testing Kernel panic on Linux 6.5 --------------------------------- 1. Configure CTI using sysfs interface #./cti_setup.sh #cat cti_setup.sh cd /sys/bus/coresight/devices/ ap_cti_config () { #ETM trig out[0] trigger to Channel 0 echo 0 4 > channels/trigin_attach } etf_cti_config () { #ETF Flush in trigger from Channel 0 echo 0 1 > channels/trigout_attach echo 1 > channels/trig_filter_enable } etr_cti_config () { #ETR Flush in from Channel 0 echo 0 1 > channels/trigout_attach echo 1 > channels/trig_filter_enable } ctidevs=`find . -name "cti*"` for i in $ctidevs do cd $i connection=`find . -name "ete*"` if [ ! -z "$connection" ] then echo "AP CTI config for $i" ap_cti_config fi connection=`find . -name "tmc_etf*"` if [ ! -z "$connection" ] then echo "ETF CTI config for $i" etf_cti_config fi connection=`find . -name "tmc_etr*"` if [ ! -z "$connection" ] then echo "ETR CTI config for $i" etr_cti_config fi cd .. done Note: CTI connections are SOC specific and hence the above script is added just for reference. 2. Start Coresight tracing on cores 1 and 2 using sysfs interface 3. Run some application on core 1 #taskset -c 1 dd if=/dev/urandom of=/dev/null & 4. Invoke kernel panic on core 2 #echo 1 > /proc/sys/kernel/panic #taskset -c 2 echo c > /proc/sysrq-trigger 5. From rebooted kernel, enable previous boot mode #echo 1 > /sys/bus/coresight/devices/tmc_etr0/read_prevboot 6. Read trace data #dd if=/dev/tmc_etr0 of=/trace/cstrace.bin 7. Run opencsd decoder tools/scripts to generate the instruction trace. Core 1 instruction trace dump: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> A etm4_enable_hw: ffff800008ae1dd4 CONTEXT EL2 etm4_enable_hw: ffff800008ae1dd4 I etm4_enable_hw: ffff800008ae1dd4: d503201f nop I etm4_enable_hw: ffff800008ae1dd8: d503201f nop I etm4_enable_hw: ffff800008ae1ddc: d503201f nop I etm4_enable_hw: ffff800008ae1de0: d503201f nop I etm4_enable_hw: ffff800008ae1de4: d503201f nop I etm4_enable_hw: ffff800008ae1de8: d503233f paciasp I etm4_enable_hw: ffff800008ae1dec: a9be7bfd stp x29, x30, [sp, #-32]! I etm4_enable_hw: ffff800008ae1df0: 910003fd mov x29, sp I etm4_enable_hw: ffff800008ae1df4: a90153f3 stp x19, x20, [sp, #16] I etm4_enable_hw: ffff800008ae1df8: 2a0003f4 mov w20, w0 I etm4_enable_hw: ffff800008ae1dfc: 900085b3 adrp x19, ffff800009b95000 <reserved_mem+0xc48> I etm4_enable_hw: ffff800008ae1e00: 910f4273 add x19, x19, #0x3d0 I etm4_enable_hw: ffff800008ae1e04: f8747a60 ldr x0, [x19, x20, lsl #3] E etm4_enable_hw: ffff800008ae1e08: b4000140 cbz x0, ffff800008ae1e30 <etm4_starting_cpu+0x50> I 149.039572921 etm4_enable_hw: ffff800008ae1e30: a94153f3 ldp x19, x20, [sp, #16] I 149.039572921 etm4_enable_hw: ffff800008ae1e34: 52800000 mov w0, #0x0 // #0 I 149.039572921 etm4_enable_hw: ffff800008ae1e38: a8c27bfd ldp x29, x30, [sp], #32 ..snip 149.052324811 chacha_block_generic: ffff800008642d80: 9100a3e0 add x0, I 149.052324811 chacha_block_generic: ffff800008642d84: b86178a2 ldr w2, [x5, x1, lsl #2] I 149.052324811 chacha_block_generic: ffff800008642d88: 8b010803 add x3, x0, x1, lsl #2 I 149.052324811 chacha_block_generic: ffff800008642d8c: b85fc063 ldur w3, [x3, #-4] I 149.052324811 chacha_block_generic: ffff800008642d90: 0b030042 add w2, w2, w3 I 149.052324811 chacha_block_generic: ffff800008642d94: b8217882 str w2, [x4, x1, lsl #2] I 149.052324811 chacha_block_generic: ffff800008642d98: 91000421 add x1, x1, #0x1 I 149.052324811 chacha_block_generic: ffff800008642d9c: f100443f cmp x1, #0x11 Core 2 instruction trace dump(kernel panic triggered core): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A etm4_enable_hw: ffff800008ae1dd4 CONTEXT EL2 etm4_enable_hw: ffff800008ae1dd4 I etm4_enable_hw: ffff800008ae1dd4: d503201f nop I etm4_enable_hw: ffff800008ae1dd8: d503201f nop I etm4_enable_hw: ffff800008ae1ddc: d503201f nop I etm4_enable_hw: ffff800008ae1de0: d503201f nop I etm4_enable_hw: ffff800008ae1de4: d503201f nop I etm4_enable_hw: ffff800008ae1de8: d503233f paciasp I etm4_enable_hw: ffff800008ae1dec: a9be7bfd stp x29, x30, [sp, #-32]! I etm4_enable_hw: ffff800008ae1df0: 910003fd mov x29, sp I etm4_enable_hw: ffff800008ae1df4: a90153f3 stp x19, x20, [sp, #16] I etm4_enable_hw: ffff800008ae1df8: 2a0003f4 mov w20, w0 I etm4_enable_hw: ffff800008ae1dfc: 900085b3 adrp x19, ffff800009b95000 <reserved_mem+0xc48> I etm4_enable_hw: ffff800008ae1e00: 910f4273 add x19, x19, #0x3d0 I etm4_enable_hw: ffff800008ae1e04: f8747a60 ldr x0, [x19, x20, lsl #3] E etm4_enable_hw: ffff800008ae1e08: b4000140 cbz x0, ffff800008ae1e30 <etm4_starting_cpu+0x50> I 149.046243445 etm4_enable_hw: ffff800008ae1e30: a94153f3 ldp x19, x20, [sp, #16] I 149.046243445 etm4_enable_hw: ffff800008ae1e34: 52800000 mov w0, #0x0 // #0 I 149.046243445 etm4_enable_hw: ffff800008ae1e38: a8c27bfd ldp x29, x30, [sp], #32 I 149.046243445 etm4_enable_hw: ffff800008ae1e3c: d50323bf autiasp E 149.046243445 etm4_enable_hw: ffff800008ae1e40: d65f03c0 ret A ete_sysreg_write: ffff800008adfa18 ..snip I 149.05422547 panic: ffff800008096300: a90363f7 stp x23, x24, [sp, #48] I 149.05422547 panic: ffff800008096304: 6b00003f cmp w1, w0 I 149.05422547 panic: ffff800008096308: 3a411804 ccmn w0, #0x1, #0x4, ne // ne = any N 149.05422547 panic: ffff80000809630c: 540001e0 b.eq ffff800008096348 <panic+0xe0> // b.none I 149.05422547 panic: ffff800008096310: f90023f9 str x25, [sp, #64] E 149.05422547 panic: ffff800008096314: 97fe44ef bl ffff8000080276d0 <panic_smp_self_stop> A panic: ffff80000809634c I 149.05422547 panic: ffff80000809634c: 910102d5 add x21, x22, #0x40 I 149.05422547 panic: ffff800008096350: 52800020 mov w0, #0x1 // #1 E 149.05422547 panic: ffff800008096354: 94166b8b bl ffff800008631180 <bust_spinlocks> N 149.054225518 bust_spinlocks: ffff800008631180: 340000c0 cbz w0, ffff800008631198 <bust_spinlocks+0x18> I 149.054225518 bust_spinlocks: ffff800008631184: f000a321 adrp x1, ffff800009a98000 <pbufs.0+0xbb8> I 149.054225518 bust_spinlocks: ffff800008631188: b9405c20 ldr w0, [x1, #92] I 149.054225518 bust_spinlocks: ffff80000863118c: 11000400 add w0, w0, #0x1 I 149.054225518 bust_spinlocks: ffff800008631190: b9005c20 str w0, [x1, #92] E 149.054225518 bust_spinlocks: ffff800008631194: d65f03c0 ret A panic: ffff800008096358 TODO ---- * Change ETM configuration done in patch #7 to new system configuration manager profile * Change CTI sysfs script to system configuration manager profile * Reading tracedata from crashdump kernel is not tested. * Perf based trace capture and decode is not tested. Linu Cherian (8): dt-bindings: arm: coresight-tmc: Add "memory-region" property coresight: tmc-etr: Add support to use reserved trace memory coresight: core: Add provision for panic callbacks coresight: tmc: Enable panic sync handling coresight: tmc: Add support for reading tracedata from previous boot coresight: tmc: Stop trace capture on FlIn coresight: etm4x: Configure ETM to trigger on panic coresight: cti: Add CTI id for Neoverse N2 core CTI .../bindings/arm/arm,coresight-tmc.yaml | 13 + drivers/hwtracing/coresight/coresight-core.c | 32 ++ .../hwtracing/coresight/coresight-cti-core.c | 1 + .../coresight/coresight-etm4x-core.c | 18 +- drivers/hwtracing/coresight/coresight-etm4x.h | 26 ++ .../hwtracing/coresight/coresight-tmc-core.c | 146 +++++++++- .../hwtracing/coresight/coresight-tmc-etf.c | 126 +++++++- .../hwtracing/coresight/coresight-tmc-etr.c | 274 +++++++++++++++++- drivers/hwtracing/coresight/coresight-tmc.h | 48 +++ include/linux/coresight.h | 25 ++ 10 files changed, 701 insertions(+), 8 deletions(-) Links: 1. https://lore.kernel.org/linux-arm-kernel/20230818082112.554638-1-anshuman.k… 2. https://lore.kernel.org/linux-arm-kernel/20230823042948.12879-1-lcherian@ma… -- 2.41.0

7 months, 2 weeks

Re: [PATCH] perf cs-etm: Fix missing decoder for per-process trace

by James Clark

On 19/09/2023 23:45, Besar Wicaksono wrote: > The decoder creation for raw trace uses metadata from the first CPU. > On per-process/per-thread traces, the first CPU is CPU0. If CPU0 trace > is not enabled, its metadata will be marked unused and the decoder is > not created. Perf report dump skips the decoding part because the > decoder is missing. > Hi Besar, It's not just per-process trace, the bug is also in per-cpu mode but it means that the metadata from CPU 0 is used for every decoder which is wrong. Although your change also fixes this issue. > To fix this, use metadata of the CPU associated with sample object. > > Signed-off-by: Besar Wicaksono <bwicaksono(a)nvidia.com> > --- > tools/perf/util/cs-etm.c | 130 +++++++++++++++++++++++---------------- > 1 file changed, 77 insertions(+), 53 deletions(-) > [...] > + if (!formatted) { > + /* > + * There is only one decoder when unformatted. Use metadata of > + * sample AUX cpu. > + */ > + t_param = t_params; > + metadata = get_cpu_data(etm, sample_cpu); > + if (!metadata) { > + pr_err("CS_ETM: invalid sample CPU: %d\n", sample_cpu); > return -EINVAL; > } Apart from Mike's comments, this looks ok. Thanks for fixing this it has been on our list for a while. One issue with calling get_cpu_data() with the sample CPU ID is that it won't work with old files that don't have the CPU sample flag set. Mike added the sample flag fairly recently, and I don't think that was a breaking change for old files at that time. It should be easy to avoid that by still returning the metadata from CPU 0 when CPU = -1 (Which isn't correct but is 99% likely to work). I checked the Coresight tests and they're all passing, at least on a system without ETE. If you could make sure they're all passing for you as well that would be great: sudo ./perf test coresight I think they currently only work from an in source build, if you get stuck there. Thanks James

7 months, 2 weeks

Re: [PATCH 2/2] dt-bindings: arm,coresight-cti: Add missing additionalProperties on child nodes

by Mike Leach

On Mon, 25 Sept 2023 at 23:05, Rob Herring <robh(a)kernel.org> wrote: > > Just as unevaluatedProperties or additionalProperties are required at > the top level of schemas, they should (and will) also be required for > child node schemas. That ensures only documented properties are > present for any node. > > Adding additionalProperties constraint on 'trig-conns' nodes results in > warnings that 'cpu' and 'arm,cs-dev-assoc' are not allowed. These are > already defined for the parent node, but need to be duplicated for the > child node. Drop the free form description that the properties also apply > to the child nodes. > > Signed-off-by: Rob Herring <robh(a)kernel.org> > --- > .../bindings/arm/arm,coresight-cti.yaml | 33 ++++++++++++++----- > 1 file changed, 24 insertions(+), 9 deletions(-) > > diff --git a/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml b/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > index b9bdfc8969cd..2d5545a2b49c 100644 > --- a/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > +++ b/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > @@ -93,9 +93,7 @@ properties: > > cpu: > description: > - Handle to cpu this device is associated with. This must appear in the > - base cti node if compatible string arm,coresight-cti-v8-arch is used, > - or may appear in a trig-conns child node when appropriate. > + Handle to cpu this CTI is associated with. > > power-domains: > maxItems: 1 > @@ -112,12 +110,12 @@ properties: > description: > defines a phandle reference to an associated CoreSight trace device. > When the associated trace device is enabled, then the respective CTI > - will be enabled. Use in a trig-conns node, or in CTI base node when > - compatible string arm,coresight-cti-v8-arch used. If the associated > - device has not been registered then the node name will be stored as > - the connection name for later resolution. If the associated device is > - not a CoreSight device or not registered then the node name will remain > - the connection name and automatic enabling will not occur. > + will be enabled. Use in CTI base node when compatible string > + arm,coresight-cti-v8-arch used. If the associated device has not been > + registered then the node name will be stored as the connection name for > + later resolution. If the associated device is not a CoreSight device or > + not registered then the node name will remain the connection name and > + automatic enabling will not occur. > > # size cells and address cells required if trig-conns node present. > "#size-cells": > @@ -129,6 +127,8 @@ properties: > patternProperties: > '^trig-conns@([0-9]+)$': > type: object > + additionalProperties: false > + > description: > A trigger connections child node which describes the trigger signals > between this CTI and another hardware device. This device may be a CPU, > @@ -140,6 +140,21 @@ patternProperties: > reg: > maxItems: 1 > > + cpu: > + description: > + Handle to cpu this trigger connection is associated with. > + > + arm,cs-dev-assoc: > + $ref: /schemas/types.yaml#/definitions/phandle > + description: > + defines a phandle reference to an associated CoreSight trace device. > + When the associated trace device is enabled, then the respective CTI > + will be enabled. If the associated device has not been registered > + then the node name will be stored as the connection name for later > + resolution. If the associated device is not a CoreSight device or > + not registered then the node name will remain the connection name > + and automatic enabling will not occur. > + > arm,trig-in-sigs: > $ref: /schemas/types.yaml#/definitions/uint32-array > minItems: 1 > -- > 2.40.1 > Reviewed-by: Mike Leach <mike.leach(a)linaro.org> -- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

7 months, 2 weeks

Re: [PATCH 1/2] dt-bindings: arm,coresight-cti: Drop type for 'cpu' property

by Mike Leach

On Mon, 25 Sept 2023 at 23:05, Rob Herring <robh(a)kernel.org> wrote: > > 'cpu' has been added as a single phandle type to dtschema, so drop the > type here. > > Signed-off-by: Rob Herring <robh(a)kernel.org> > --- > Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml b/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > index 6216cfb0a188..b9bdfc8969cd 100644 > --- a/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > +++ b/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > @@ -92,7 +92,6 @@ properties: > maxItems: 1 > > cpu: > - $ref: /schemas/types.yaml#/definitions/phandle > description: > Handle to cpu this device is associated with. This must appear in the > base cti node if compatible string arm,coresight-cti-v8-arch is used, > -- > 2.40.1 > Reviewed-by: Mike Leach <mike.leach(a)linaro.org> -- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

7 months, 2 weeks

Re: [PATCH v3 1/2] arm64/sysreg: Move TRFCR definitions to sysreg

by James Clark

On 25/09/2023 16:59, Catalin Marinas wrote: > On Tue, Sep 05, 2023 at 11:21:14AM +0100, James Clark wrote: >> Add separate definitions for ELx and EL2 as TRFCR_EL1 doesn't have CX. >> This also mirrors the previous definition so no code change is required. >> >> Signed-off-by: James Clark <james.clark(a)arm.com> > > Acked-by: Catalin Marinas <catalin.marinas(a)arm.com> > > I assume these patches would go in via the coresight tree. Possibly, but I'm working on a V4 that combines the nVHE version and Suzuki's comment from here [1]. There will be more kvm patches in that one so I'm not sure. [1]: https://lists.linaro.org/archives/list/coresight@lists.linaro.org/message/A…

7 months, 2 weeks

[RFC RESEND 0/7] coresight: Move remaining AMBA ACPI devices into platform driver

by Anshuman Khandual

This moves remaining AMBA ACPI devices into respective platform drivers for enabling ACPI based power management support. This might still require some further changes but presented here just for some initial review & feedback. This series applies on coresight/next coresight/for-next/queue and has been built tested. This series has also been boot tested on a DT based coresight device latform. Although it still requires testing on ACPI platforms. Cc: Suzuki Poulose <suzuki.poulose(a)arm.com> Cc: James Clark <james.clark(a)arm.com> Cc: Mike Leach <mike.leach(a)linaro.org> Cc: coresight(a)lists.linaro.org Cc: linux-arm-kernel(a)lists.infradead.org Cc: linux-kernel(a)vger.kernel.org Anshuman Khandual (7): coresight: replicator: Move ACPI support from AMBA driver to platform driver coresight: funnel: Move ACPI support from AMBA driver to platform driver coresight: catu: Move ACPI support from AMBA driver to platform driver coresight: tpiu: Move ACPI support from AMBA driver to platform driver coresight: tmc: Move ACPI support from AMBA driver to platform driver coresight: stm: Move ACPI support from AMBA driver to platform driver coresight: debug: Move ACPI support from AMBA driver to platform driver drivers/acpi/arm64/amba.c | 8 -- drivers/hwtracing/coresight/coresight-catu.c | 136 ++++++++++++++++-- drivers/hwtracing/coresight/coresight-catu.h | 1 + .../hwtracing/coresight/coresight-cpu-debug.c | 130 +++++++++++++++-- .../hwtracing/coresight/coresight-funnel.c | 49 ++++--- .../coresight/coresight-replicator.c | 44 +++--- drivers/hwtracing/coresight/coresight-stm.c | 80 +++++++++-- .../hwtracing/coresight/coresight-tmc-core.c | 127 ++++++++++++++-- drivers/hwtracing/coresight/coresight-tmc.h | 1 + drivers/hwtracing/coresight/coresight-tpiu.c | 76 +++++++++- 10 files changed, 549 insertions(+), 103 deletions(-) -- 2.25.1

7 months, 2 weeks

[PATCH V5 - RESEND 0/3] coresight: etm: Make cycle count threshold user configurable

by Anshuman Khandual

This series makes ETM TRCCCCTRL based 'cc_threshold' user configurable via the perf event attribute. But first, this implements an errata work around affecting ETM TRCIDR3.CCITMIN value on certain cpus, overriding the field. This series applies on coresight/for-next/queue. Cc: Catalin Marinas <catalin.marinas(a)arm.com> Cc: Will Deacon <will(a)kernel.org> Cc: Suzuki K Poulose <suzuki.poulose(a)arm.com> Cc: Mike Leach <mike.leach(a)linaro.org> Cc: James Clark <james.clark(a)arm.com> Cc: Leo Yan <leo.yan(a)linaro.org> Cc: Jonathan Corbet <corbet(a)lwn.net> Cc: linux-doc(a)vger.kernel.org Cc: coresight(a)lists.linaro.org Cc: linux-arm-kernel(a)lists.infradead.org Cc: linux-kernel(a)vger.kernel.org Changes in V5: https://lore.kernel.org/all/20230821045216.641499-1-anshuman.khandual@arm.c… - Replaced 'where as' with single word 'whereas' - Reworked 'cc_threshold' fallback to ETM_CYC_THRESHOLD_DEFAULT Changes in V4: https://lore.kernel.org/all/20230818112051.594986-1-anshuman.khandual@arm.c… - Fixed a typo s/rangess/ranges, - Renamed etm4_work_around_wrong_ccitmin() as etm4_core_reads_wrong_ccitmin() - Moved drvdata->ccitmin value check for 256 inside etm4_core_reads_wrong_ccitmin() - Moved the comment inside etm4_core_reads_wrong_ccitmin() Changes in V3: https://lore.kernel.org/all/20230811034600.944386-1-anshuman.khandual@arm.c… - Added errata work around affecting TRCIDR3.CCITMIN - Split the document update into a separate patch Changes in V2: https://lore.kernel.org/all/20230808074533.380537-1-anshuman.khandual@arm.c… - s/treshhold/threshold Changes in V1: https://lore.kernel.org/all/20230804044720.1478900-1-anshuman.khandual@arm.… Anshuman Khandual (3): coresight: etm: Override TRCIDR3.CCITMIN on errata affected cpus coresight: etm: Make cycle count threshold user configurable Documentation: coresight: Add cc_threshold tunable Documentation/arch/arm64/silicon-errata.rst | 10 +++++ Documentation/trace/coresight/coresight.rst | 4 ++ .../hwtracing/coresight/coresight-etm-perf.c | 2 + .../coresight/coresight-etm4x-core.c | 45 ++++++++++++++++++- 4 files changed, 59 insertions(+), 2 deletions(-) -- 2.25.1

7 months, 3 weeks

[PATCH V6 0/3] coresight: etm: Make cycle count threshold user configurable

by Anshuman Khandual

This series makes ETM TRCCCCTRL based 'cc_threshold' user configurable via the perf event attribute. But first, this implements an errata work around affecting ETM TRCIDR3.CCITMIN value on certain cpus, overriding the field. This series applies on coresight/for-next/queue. Cc: Catalin Marinas <catalin.marinas(a)arm.com> Cc: Will Deacon <will(a)kernel.org> Cc: Suzuki K Poulose <suzuki.poulose(a)arm.com> Cc: Mike Leach <mike.leach(a)linaro.org> Cc: James Clark <james.clark(a)arm.com> Cc: Leo Yan <leo.yan(a)linaro.org> Cc: Jonathan Corbet <corbet(a)lwn.net> Cc: linux-doc(a)vger.kernel.org Cc: coresight(a)lists.linaro.org Cc: linux-arm-kernel(a)lists.infradead.org Cc: linux-kernel(a)vger.kernel.org Changes in V6: - Renamed etm4_core_reads_wrong_ccitmin() as etm4_fixup_wrong_ccitmin() - Moved drvdata->ccitmin fixup inside etm4_fixup_wrong_ccitmin() Changes in V5: https://lore.kernel.org/all/20230821045216.641499-1-anshuman.khandual@arm.c… https://lore.kernel.org/all/20230915093649.435163-1-anshuman.khandual@arm.c… - Replaced 'where as' with single word 'whereas' - Reworked 'cc_threshold' fallback to ETM_CYC_THRESHOLD_DEFAULT Changes in V4: https://lore.kernel.org/all/20230818112051.594986-1-anshuman.khandual@arm.c… - Fixed a typo s/rangess/ranges, - Renamed etm4_work_around_wrong_ccitmin() as etm4_core_reads_wrong_ccitmin() - Moved drvdata->ccitmin value check for 256 inside etm4_core_reads_wrong_ccitmin() - Moved the comment inside etm4_core_reads_wrong_ccitmin() Changes in V3: https://lore.kernel.org/all/20230811034600.944386-1-anshuman.khandual@arm.c… - Added errata work around affecting TRCIDR3.CCITMIN - Split the document update into a separate patch Changes in V2: https://lore.kernel.org/all/20230808074533.380537-1-anshuman.khandual@arm.c… - s/treshhold/threshold Changes in V1: https://lore.kernel.org/all/20230804044720.1478900-1-anshuman.khandual@arm.… Anshuman Khandual (3): coresight: etm: Override TRCIDR3.CCITMIN on errata affected cpus coresight: etm: Make cycle count threshold user configurable Documentation: coresight: Add cc_threshold tunable Documentation/arch/arm64/silicon-errata.rst | 10 ++++ Documentation/trace/coresight/coresight.rst | 4 ++ .../hwtracing/coresight/coresight-etm-perf.c | 2 + .../coresight/coresight-etm4x-core.c | 46 ++++++++++++++++++- 4 files changed, 60 insertions(+), 2 deletions(-) -- 2.25.1

7 months, 3 weeks

Re: [PATCH] perf cs-etm: Fix missing decoder for per-process trace

by Mike Leach

Hi, Can you provide a perf command line for both the record and decode phases that demonstrates the problem you are having? Also note the cpu / trace protocol types On Tue, 19 Sept 2023 at 23:47, Besar Wicaksono <bwicaksono(a)nvidia.com> wrote: > > The decoder creation for raw trace uses metadata from the first CPU. > On per-process/per-thread traces, the first CPU is CPU0. If CPU0 trace > is not enabled, its metadata will be marked unused and the decoder is > not created. Perf report dump skips the decoding part because the > decoder is missing. > > To fix this, use metadata of the CPU associated with sample object. > > Signed-off-by: Besar Wicaksono <bwicaksono(a)nvidia.com> > --- > tools/perf/util/cs-etm.c | 130 +++++++++++++++++++++++---------------- > 1 file changed, 77 insertions(+), 53 deletions(-) > > diff --git a/tools/perf/util/cs-etm.c b/tools/perf/util/cs-etm.c > index 9729d006550d..3c3179a5eac6 100644 > --- a/tools/perf/util/cs-etm.c > +++ b/tools/perf/util/cs-etm.c > @@ -640,71 +640,94 @@ static void cs_etm__packet_dump(const char *pkt_string) > fflush(stdout); > } > > -static void cs_etm__set_trace_param_etmv3(struct cs_etm_trace_params *t_params, > - struct cs_etm_auxtrace *etm, int idx, > - u32 etmidr) > +static void cs_etm__set_trace_param_etmv3(struct cs_etm_trace_params *t_param, > + u64 *metadata, u32 etmidr) > { > - u64 **metadata = etm->metadata; > - > - t_params[idx].protocol = cs_etm__get_v7_protocol_version(etmidr); > - t_params[idx].etmv3.reg_ctrl = metadata[idx][CS_ETM_ETMCR]; > - t_params[idx].etmv3.reg_trc_id = metadata[idx][CS_ETM_ETMTRACEIDR]; > + t_param->protocol = cs_etm__get_v7_protocol_version(etmidr); > + t_param->etmv3.reg_ctrl = metadata[CS_ETM_ETMCR]; > + t_param->etmv3.reg_trc_id = metadata[CS_ETM_ETMTRACEIDR]; > } > This is inherently incorrect - you are assuming that all trace devices on a given system are the same and have the same parameters. This does not have to be the case. > -static void cs_etm__set_trace_param_etmv4(struct cs_etm_trace_params *t_params, > - struct cs_etm_auxtrace *etm, int idx) > +static void cs_etm__set_trace_param_etmv4(struct cs_etm_trace_params *t_param, > + u64 *metadata) > { > - u64 **metadata = etm->metadata; > + t_param->protocol = CS_ETM_PROTO_ETMV4i; > + t_param->etmv4.reg_idr0 = metadata[CS_ETMV4_TRCIDR0]; > + t_param->etmv4.reg_idr1 = metadata[CS_ETMV4_TRCIDR1]; > + t_param->etmv4.reg_idr2 = metadata[CS_ETMV4_TRCIDR2]; > + t_param->etmv4.reg_idr8 = metadata[CS_ETMV4_TRCIDR8]; > + t_param->etmv4.reg_configr = metadata[CS_ETMV4_TRCCONFIGR]; > + t_param->etmv4.reg_traceidr = metadata[CS_ETMV4_TRCTRACEIDR]; > +} > > - t_params[idx].protocol = CS_ETM_PROTO_ETMV4i; > - t_params[idx].etmv4.reg_idr0 = metadata[idx][CS_ETMV4_TRCIDR0]; > - t_params[idx].etmv4.reg_idr1 = metadata[idx][CS_ETMV4_TRCIDR1]; > - t_params[idx].etmv4.reg_idr2 = metadata[idx][CS_ETMV4_TRCIDR2]; > - t_params[idx].etmv4.reg_idr8 = metadata[idx][CS_ETMV4_TRCIDR8]; > - t_params[idx].etmv4.reg_configr = metadata[idx][CS_ETMV4_TRCCONFIGR]; > - t_params[idx].etmv4.reg_traceidr = metadata[idx][CS_ETMV4_TRCTRACEIDR]; > +static void cs_etm__set_trace_param_ete(struct cs_etm_trace_params *t_param, > + u64 *metadata) > +{ > + t_param->protocol = CS_ETM_PROTO_ETE; > + t_param->ete.reg_idr0 = metadata[CS_ETE_TRCIDR0]; > + t_param->ete.reg_idr1 = metadata[CS_ETE_TRCIDR1]; > + t_param->ete.reg_idr2 = metadata[CS_ETE_TRCIDR2]; > + t_param->ete.reg_idr8 = metadata[CS_ETE_TRCIDR8]; > + t_param->ete.reg_configr = metadata[CS_ETE_TRCCONFIGR]; > + t_param->ete.reg_traceidr = metadata[CS_ETE_TRCTRACEIDR]; > + t_param->ete.reg_devarch = metadata[CS_ETE_TRCDEVARCH]; > } > as above for ETE & ETMv4 > -static void cs_etm__set_trace_param_ete(struct cs_etm_trace_params *t_params, > - struct cs_etm_auxtrace *etm, int idx) > +static int cs_etm__set_trace_param(struct cs_etm_trace_params *t_param, > + u64 *metadata) > { > - u64 **metadata = etm->metadata; > + u32 etmidr; > + u64 architecture = metadata[CS_ETM_MAGIC]; > + > + switch (architecture) { > + case __perf_cs_etmv3_magic: > + etmidr = metadata[CS_ETM_ETMIDR]; > + cs_etm__set_trace_param_etmv3(t_param, metadata, etmidr); > + break; > + case __perf_cs_etmv4_magic: > + cs_etm__set_trace_param_etmv4(t_param, metadata); > + break; > + case __perf_cs_ete_magic: > + cs_etm__set_trace_param_ete(t_param, metadata); > + break; > + default: > + return -EINVAL; > + } > > - t_params[idx].protocol = CS_ETM_PROTO_ETE; > - t_params[idx].ete.reg_idr0 = metadata[idx][CS_ETE_TRCIDR0]; > - t_params[idx].ete.reg_idr1 = metadata[idx][CS_ETE_TRCIDR1]; > - t_params[idx].ete.reg_idr2 = metadata[idx][CS_ETE_TRCIDR2]; > - t_params[idx].ete.reg_idr8 = metadata[idx][CS_ETE_TRCIDR8]; > - t_params[idx].ete.reg_configr = metadata[idx][CS_ETE_TRCCONFIGR]; > - t_params[idx].ete.reg_traceidr = metadata[idx][CS_ETE_TRCTRACEIDR]; > - t_params[idx].ete.reg_devarch = metadata[idx][CS_ETE_TRCDEVARCH]; > + return 0; > } > > static int cs_etm__init_trace_params(struct cs_etm_trace_params *t_params, > struct cs_etm_auxtrace *etm, > + bool formatted, > + int sample_cpu, > int decoders) > { > - int i; > - u32 etmidr; > - u64 architecture; > - > - for (i = 0; i < decoders; i++) { > - architecture = etm->metadata[i][CS_ETM_MAGIC]; > + int i, ret; > + struct cs_etm_trace_params *t_param; > + u64 *metadata; > > - switch (architecture) { > - case __perf_cs_etmv3_magic: > - etmidr = etm->metadata[i][CS_ETM_ETMIDR]; > - cs_etm__set_trace_param_etmv3(t_params, etm, i, etmidr); > - break; > - case __perf_cs_etmv4_magic: > - cs_etm__set_trace_param_etmv4(t_params, etm, i); > - break; > - case __perf_cs_ete_magic: > - cs_etm__set_trace_param_ete(t_params, etm, i); > - break; > - default: > + if (!formatted) { > + /* > + * There is only one decoder when unformatted. Use metadata of > + * sample AUX cpu. > + */ Unformatted ETE data should not only have a single decoder - there must be a decoder per CPU - as per all trace decode flows. The key difference with ETE is that it usually has its own trace sink (TRBE), so no formatting is needed, but per cpu decoder is still required. Regards Mike > + t_param = t_params; > + metadata = get_cpu_data(etm, sample_cpu); > + if (!metadata) { > + pr_err("CS_ETM: invalid sample CPU: %d\n", sample_cpu); > return -EINVAL; > } > + > + return cs_etm__set_trace_param(t_param, metadata); > + } > + > + for (i = 0; i < decoders; i++) { > + t_param = &t_params[i]; > + metadata = etm->metadata[i]; > + ret = cs_etm__set_trace_param(t_param, metadata); > + if (ret) > + return ret; > } > > return 0; > @@ -1016,7 +1039,7 @@ static u32 cs_etm__mem_access(struct cs_etm_queue *etmq, u8 trace_chan_id, > } > > static struct cs_etm_queue *cs_etm__alloc_queue(struct cs_etm_auxtrace *etm, > - bool formatted) > + bool formatted, int sample_cpu) > { > struct cs_etm_decoder_params d_params; > struct cs_etm_trace_params *t_params = NULL; > @@ -1041,7 +1064,7 @@ static struct cs_etm_queue *cs_etm__alloc_queue(struct cs_etm_auxtrace *etm, > if (!t_params) > goto out_free; > > - if (cs_etm__init_trace_params(t_params, etm, decoders)) > + if (cs_etm__init_trace_params(t_params, etm, formatted, sample_cpu, decoders)) > goto out_free; > > /* Set decoder parameters to decode trace packets */ > @@ -1081,14 +1104,15 @@ static struct cs_etm_queue *cs_etm__alloc_queue(struct cs_etm_auxtrace *etm, > static int cs_etm__setup_queue(struct cs_etm_auxtrace *etm, > struct auxtrace_queue *queue, > unsigned int queue_nr, > - bool formatted) > + bool formatted, > + int sample_cpu) > { > struct cs_etm_queue *etmq = queue->priv; > > if (list_empty(&queue->head) || etmq) > return 0; > > - etmq = cs_etm__alloc_queue(etm, formatted); > + etmq = cs_etm__alloc_queue(etm, formatted, sample_cpu); > > if (!etmq) > return -ENOMEM; > @@ -2816,7 +2840,7 @@ static int cs_etm__process_auxtrace_event(struct perf_session *session, > * formatted in piped mode (true). > */ > err = cs_etm__setup_queue(etm, &etm->queues.queue_array[idx], > - idx, true); > + idx, true, -1); > if (err) > return err; > > @@ -3022,7 +3046,7 @@ static int cs_etm__queue_aux_fragment(struct perf_session *session, off_t file_o > idx = auxtrace_event->idx; > formatted = !(aux_event->flags & PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW); > return cs_etm__setup_queue(etm, &etm->queues.queue_array[idx], > - idx, formatted); > + idx, formatted, sample->cpu); > } > > /* Wasn't inside this buffer, but there were no parse errors. 1 == 'not found' */ > -- > 2.34.1 > -- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

7 months, 3 weeks

[RFC PATCH v2 0/7]Extending Coresight for Kernel panic and watchdog reset

by Linu Cherian

Changelog from v1: * V2 is a complete patchset with kernel panic trace tested on Linux 6.4. Details on testing with relevant console logs has been added for reference. * Two additional patches(patch 6 & 7) has been included to manage stopping of trace at the time of kernel panic. * Few bug fixes. TODO: * Add support to prevent overwriting of trace data captured in previous boot. (Suggested by James) * DTS properties for reserved memory might need some refinements, since Linux arm64 kernel has limitation on the number of reserved regions it supports(ie. 64). * ETM & CTI configuration using system configuration manager is a work progress. Currently ETM configuration is done in the driver(patch 7) and CTI configuration is done using CTI sysfs interface. * Reading tracedata from crashdump kernel is not tested. * Perf based trace capture is not tested. Introduction ============ This RFC is about extending Linux coresight driver support to address kernel panic and watchdog reset scenarios. This would help coresight users to debug kernel panic and watchdog reset with the help of coresight trace data. For simplicity, watchdog and kernel panic are addressed in separate sections. Coresight trace capture: Kernel panic ------------------------------------- From the coresight driver point of view, addressing the kernel panic situation has four main requirements. a. Support for allocation of trace buffer pages from reserved memory area. Platform can advertise this using a new device tree property added to relevant coresight nodes. b. Support for stopping coresight blocks at the time of panic c. Saving required metadata in the specified format d. Support for reading trace data captured at the time of panic Allocation of trace buffer pages from reserved RAM ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A new optional device tree property "memory-region" is added to the ETR/ETF device nodes, that would give the base address and size of trace buffer. Static allocation of trace buffers would ensure that both IOMMU enabled and disabled cases are handled. Also, platforms that support persistent RAM will allow users to read trace data in the subsequent boot without booting the crashdump kernel. Note: For ETR sink devices, this reserved region will be used for both trace capture and trace data retrieval. For ETF sink devices, internal SRAM would be used for trace capture, and they would be synced to reserved region for retrieval. Note: Patches 1 & 2 adds support for this. Disabling coresight blocks at the time of panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order to avoid the situation of losing relevant trace data after a kernel panic, it would be desirable to stop the coresight blocks at the time of panic. This can be achieved by configuring the comparator, CTI and sink devices as below, Comparator(triggers on kernel panic) --->External out --->CTI -- | ETR/ETF stop <------External In <-------------- Note: * Patch 6 provides the necessary ETR configuration. * Patch 7 provides the necessary ETM configuration. Saving metadata at the time of kernel panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Coresight metadata involves all additional data that are required for a successful trace decode in addition to the trace data. This involves ETR/ETF, ETE register snapshot etc. A new optional device property "memory-region" is added to the ETR/ETF/ETE device nodes for this. Note: Patches 3 & 4 adds support for this. Reading trace data captured at the time of panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Trace data captured at the time of panic, can be read from rebooted kernel or from crashdump kernel using the below mentioned interface. Note: Patch 5 adds support for this. Steps for reading trace data captured in previous boot ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1. cd /sys/bus/coresight/devices/tmc_etrXX/ 2. Change to special mode called, read_prevboot. #echo 1 > read_prevboot 3. Dump trace buffer data to a file, #dd if=/dev/tmc_etrXX of=~/cstrace.bin 4. Reset back to normal mode #echo 0 > read_prevboot General flow of trace capture and decode incase of kernel panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Enable source and sink on all the cores using the sysfs interface. ETR sink will have trace buffers allocated from reserved memory. 2. Run relevant tests. 3. On a kernel panic, all coresight blocks are disabled, necessary metadata is synced by kernel panic handler. System would eventually reboot or boot a crashdump kernel. 4. For platforms that supports crashdump kernel, raw trace data can be dumped using the coresight sysfs interface from the crashdump kernel itself. Persistent RAM is not a requirement in this case. 5. For platforms that supports persistent RAM, trace data can be dumped using the the coresight sysfs interface in the subsequent Linux boot. Crashdump kernel is not a requirement in this case. Persistent RAM ensures that trace data is intact across reboot. Coresight trace capture: Watchdog reset --------------------------------------- The main difference between addressing the watchdog reset and kernel panic case are below, a. Saving coresight metadata need to be taken care by the SCP(system control processor) firmware in the specified format, instead of kernel. b. Reserved memory region given by firmware for trace buffer and metadata has to be in persistent RAM. Note: This is a requirement for watchdog reset case but optional in kernel panic case. Watchdog reset can be supported only on platforms that meet the above two requirements. Testing Kernel panic on Linux 6.4 --------------------------------- 1. Configure CTI using sysfs interface #./cti_setup.sh #cat cti_setup.sh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cd /sys/bus/coresight/devices/ ap_cti_config () { #ETM trig out[0] trigger to Channel 0 echo 0 4 > channels/trigin_attach } etf_cti_config () { #ETF Flush in trigger from Channel 0 echo 0 1 > channels/trigout_attach echo 1 > channels/trig_filter_enable } etr_cti_config () { #ETR Flush in from Channel 0 echo 0 1 > channels/trigout_attach echo 1 > channels/trig_filter_enable } ctidevs=`find . -name "cti*"` for i in $ctidevs do cd $i connection=`find . -name "ete*"` if [ ! -z "$connection" ] then echo "AP CTI config for $i" ap_cti_config fi connection=`find . -name "tmc_etf*"` if [ ! -z "$connection" ] then echo "ETF CTI config for $i" etf_cti_config fi connection=`find . -name "tmc_etr*"` if [ ! -z "$connection" ] then echo "ETR CTI config for $i" etr_cti_config fi cd .. done <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Note: CTI connections are SOC specific and hence the above script is just for reference. 2. Start Coresight tracing on cores 1 and 2 using sysfs interface 3. Run some application on core 1 #taskset -c 1 dd if=/dev/urandom of=/dev/null & 4. Invoke kernel panic on core 2 #echo 1 > /proc/sys/kernel/panic #taskset -c 2 echo c > /proc/sysrq-trigger 5. From rebooted kernel, enable previous boot mode #echo 1 > /sys/bus/coresight/devices/tmc_etr0/read_prevboot 6. Read trace data #dd if=/dev/tmc_etr0 of=/trace/cstrace.bin 7. Run opencsd decoder tools/scripts to generate the instruction trace. Core 1 instruction trace dump: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> A etm4_enable_hw: ffff800008ae1dd4 CONTEXT EL2 etm4_enable_hw: ffff800008ae1dd4 I etm4_enable_hw: ffff800008ae1dd4: d503201f nop I etm4_enable_hw: ffff800008ae1dd8: d503201f nop I etm4_enable_hw: ffff800008ae1ddc: d503201f nop I etm4_enable_hw: ffff800008ae1de0: d503201f nop I etm4_enable_hw: ffff800008ae1de4: d503201f nop I etm4_enable_hw: ffff800008ae1de8: d503233f paciasp I etm4_enable_hw: ffff800008ae1dec: a9be7bfd stp x29, x30, [sp, #-32]! I etm4_enable_hw: ffff800008ae1df0: 910003fd mov x29, sp I etm4_enable_hw: ffff800008ae1df4: a90153f3 stp x19, x20, [sp, #16] I etm4_enable_hw: ffff800008ae1df8: 2a0003f4 mov w20, w0 I etm4_enable_hw: ffff800008ae1dfc: 900085b3 adrp x19, ffff800009b95000 <reserved_mem+0xc48> I etm4_enable_hw: ffff800008ae1e00: 910f4273 add x19, x19, #0x3d0 I etm4_enable_hw: ffff800008ae1e04: f8747a60 ldr x0, [x19, x20, lsl #3] E etm4_enable_hw: ffff800008ae1e08: b4000140 cbz x0, ffff800008ae1e30 <etm4_starting_cpu+0x50> I 149.039572921 etm4_enable_hw: ffff800008ae1e30: a94153f3 ldp x19, x20, [sp, #16] I 149.039572921 etm4_enable_hw: ffff800008ae1e34: 52800000 mov w0, #0x0 // #0 I 149.039572921 etm4_enable_hw: ffff800008ae1e38: a8c27bfd ldp x29, x30, [sp], #32 ..snip 149.052324811 chacha_block_generic: ffff800008642d80: 9100a3e0 add x0, I 149.052324811 chacha_block_generic: ffff800008642d84: b86178a2 ldr w2, [x5, x1, lsl #2] I 149.052324811 chacha_block_generic: ffff800008642d88: 8b010803 add x3, x0, x1, lsl #2 I 149.052324811 chacha_block_generic: ffff800008642d8c: b85fc063 ldur w3, [x3, #-4] I 149.052324811 chacha_block_generic: ffff800008642d90: 0b030042 add w2, w2, w3 I 149.052324811 chacha_block_generic: ffff800008642d94: b8217882 str w2, [x4, x1, lsl #2] I 149.052324811 chacha_block_generic: ffff800008642d98: 91000421 add x1, x1, #0x1 I 149.052324811 chacha_block_generic: ffff800008642d9c: f100443f cmp x1, #0x11 Core 2 instruction trace dump(kernel panic triggered core): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A etm4_enable_hw: ffff800008ae1dd4 CONTEXT EL2 etm4_enable_hw: ffff800008ae1dd4 I etm4_enable_hw: ffff800008ae1dd4: d503201f nop I etm4_enable_hw: ffff800008ae1dd8: d503201f nop I etm4_enable_hw: ffff800008ae1ddc: d503201f nop I etm4_enable_hw: ffff800008ae1de0: d503201f nop I etm4_enable_hw: ffff800008ae1de4: d503201f nop I etm4_enable_hw: ffff800008ae1de8: d503233f paciasp I etm4_enable_hw: ffff800008ae1dec: a9be7bfd stp x29, x30, [sp, #-32]! I etm4_enable_hw: ffff800008ae1df0: 910003fd mov x29, sp I etm4_enable_hw: ffff800008ae1df4: a90153f3 stp x19, x20, [sp, #16] I etm4_enable_hw: ffff800008ae1df8: 2a0003f4 mov w20, w0 I etm4_enable_hw: ffff800008ae1dfc: 900085b3 adrp x19, ffff800009b95000 <reserved_mem+0xc48> I etm4_enable_hw: ffff800008ae1e00: 910f4273 add x19, x19, #0x3d0 I etm4_enable_hw: ffff800008ae1e04: f8747a60 ldr x0, [x19, x20, lsl #3] E etm4_enable_hw: ffff800008ae1e08: b4000140 cbz x0, ffff800008ae1e30 <etm4_starting_cpu+0x50> I 149.046243445 etm4_enable_hw: ffff800008ae1e30: a94153f3 ldp x19, x20, [sp, #16] I 149.046243445 etm4_enable_hw: ffff800008ae1e34: 52800000 mov w0, #0x0 // #0 I 149.046243445 etm4_enable_hw: ffff800008ae1e38: a8c27bfd ldp x29, x30, [sp], #32 I 149.046243445 etm4_enable_hw: ffff800008ae1e3c: d50323bf autiasp E 149.046243445 etm4_enable_hw: ffff800008ae1e40: d65f03c0 ret A ete_sysreg_write: ffff800008adfa18 ..snip I 149.05422547 panic: ffff800008096300: a90363f7 stp x23, x24, [sp, #48] I 149.05422547 panic: ffff800008096304: 6b00003f cmp w1, w0 I 149.05422547 panic: ffff800008096308: 3a411804 ccmn w0, #0x1, #0x4, ne // ne = any N 149.05422547 panic: ffff80000809630c: 540001e0 b.eq ffff800008096348 <panic+0xe0> // b.none I 149.05422547 panic: ffff800008096310: f90023f9 str x25, [sp, #64] E 149.05422547 panic: ffff800008096314: 97fe44ef bl ffff8000080276d0 <panic_smp_self_stop> A panic: ffff80000809634c I 149.05422547 panic: ffff80000809634c: 910102d5 add x21, x22, #0x40 I 149.05422547 panic: ffff800008096350: 52800020 mov w0, #0x1 // #1 E 149.05422547 panic: ffff800008096354: 94166b8b bl ffff800008631180 <bust_spinlocks> N 149.054225518 bust_spinlocks: ffff800008631180: 340000c0 cbz w0, ffff800008631198 <bust_spinlocks+0x18> I 149.054225518 bust_spinlocks: ffff800008631184: f000a321 adrp x1, ffff800009a98000 <pbufs.0+0xbb8> I 149.054225518 bust_spinlocks: ffff800008631188: b9405c20 ldr w0, [x1, #92] I 149.054225518 bust_spinlocks: ffff80000863118c: 11000400 add w0, w0, #0x1 I 149.054225518 bust_spinlocks: ffff800008631190: b9005c20 str w0, [x1, #92] E 149.054225518 bust_spinlocks: ffff800008631194: d65f03c0 ret A panic: ffff800008096358 Linu Cherian (7): dt-bindings: arm: coresight-tmc: Add "memory-region" property ccoresight: tmc-etr: Add support to use reserved trace memory coresight: core: Add provision for panic callbacks coresight: tmc: Enable panic sync handling coresight: tmc: Add support for reading tracedata from previous boot coresight: tmc: Stop trace capture on FlIn coresight: etm4x: Configure ETM to trigger on panic .../bindings/arm/arm,coresight-tmc.yaml | 9 + drivers/hwtracing/coresight/coresight-core.c | 31 ++ .../coresight/coresight-etm4x-core.c | 17 +- drivers/hwtracing/coresight/coresight-etm4x.h | 26 ++ drivers/hwtracing/coresight/coresight-priv.h | 1 + .../hwtracing/coresight/coresight-tmc-core.c | 125 +++++++- .../hwtracing/coresight/coresight-tmc-etf.c | 151 ++++++++- .../hwtracing/coresight/coresight-tmc-etr.c | 286 +++++++++++++++++- drivers/hwtracing/coresight/coresight-tmc.h | 39 +++ include/linux/coresight.h | 11 + 10 files changed, 688 insertions(+), 8 deletions(-) -- 2.40.1

7 months, 3 weeks

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CoreSight September 2023