This patch set is to explore Coresight tracing data for postmortem debugging. When kernel panic happens, the Coresight panic kdump can help to save on-chip tracing data and tracer metadata into DRAM, later relies on kdump and crash/perf tools to recovery tracing data for "offline" analysis.
Comparing the patch series v4 and previous series, this patch series has heavily refactored the implementation after investigated Intel PT for kdump support. Intel PT calls one function for emergency stopping trace when kernel panic occurs, in the function it reuses perf operation to dump trace data into ring buffer, later crash tool extracts trace data from perf ring buffer.
This patch series takes Intel PT as an example to use the same way to stop ETM trace with perf mode. So far the related work is primarily to focus on to support Coresight kdump with perf mode and we can add support SysFS mode if later there have more clear requirement.
Comparing to previous series, this patch series also simplifies the handling for tracer metadata. The old series introduced extra data structure and two double link lists to maintain CoreSignt kdump components; in the old implementation, one list was used to track tracer metadata and another list was used to trace dump buffers, later these two lists can be used to retrieve metadata and trace data buffer from vmcore file. In this patch series it directly relies on CoreSight driver global variables to retrieve related info, e.g. for perf mode we can rely on per CPU pointer 'ctx_handle' to get perf ring buffer related info and 'csdev_src' is for per CPU tracer device structure for metadata.
The crash extension program now has been enhanced to parse the data structures in the kernel and use them to extract metadata and dump trace data [1]; the crash extension program is updated to build with OpenCSD decoder so this can simplize the decoding process, rather than before needs to use perf to help decoding trace data.
This patch series has been verified on 96boards DB410c with below steps, the 'long_loop' is a pretty simple program to only execute big number loops so can generate big amount number of branch instructions.
Enable trace on the target board:
$ perf record -e cs_etm/@825000.etf/ --per-thread ./long_loop & $ sleep 3 $ echo c > /proc/sysrq-trigger
Use crash tool for post analysis:
$ crash vmcore vmlinux crash> extend arm_cs_dump.so crash> arm_cs_dump -o out
[1] https://git.linaro.org/people/leo.yan/crash.git/log/?h=arm_cs_dump_etm_perf
Changes from v4: * Support for CoreSight ETM with perf mode; * Add API for crash stop; * Simplized implementation with removing kdump dedicated data structures and functions;
Changes from v3: * Following Mathieu suggestion, reworked the panic kdump framework, used kdump array to maintain source and sink device handlers; * According to Mathieu suggestion, optimized panic notifier to firstly dump panic CPU tracing data and then dump other CPUs tracing data; * Refined doc to reflect these implementation changes; * Changed ETMv4 driver to add source device handler at probe phase; * Refactored crash extension program to reflect kernel changes.
Changes from v2: * Add the two patches for documentation. * Following Mathieu suggestion, reworked the panic kdump framework, removed the useless flag "PRE_PANIC". * According to comment, changed to add and delete kdump node operations in sink enable/disable functions; * According to Mathieu suggestion, handle kdump node addition/deletion/updating separately for sysFS interface and perf method.
Changes from v1: * Add support to dump ETMv4 meta data. * Wrote 'crash' extension csdump.so so rely on it to generate 'perf' format compatible file. * Refactored panic dump driver to support pre & post panic dump.
Changes from RFC: * Follow Mathieu's suggestion, use general framework to support dump functionality. * Changed to use perf to analyse trace data.
Leo Yan (6): doc: Add Coresight documentation directory doc: Add documentation for Coresight panic kdump coresight: etm4x: Save ID values in config structure coresight: tmc: Update latest value for page index and offset coresight: etm-perf: Add interface to stop etm trace arm64: smp: Stop CoreSight trace for kdump
.../trace/{ => coresight}/coresight-cpu-debug.txt | 0 .../trace/coresight/coresight-panic-kdump.txt | 99 ++++++++++++++++++++++ Documentation/trace/{ => coresight}/coresight.txt | 0 MAINTAINERS | 5 +- arch/arm64/kernel/smp.c | 5 ++ drivers/hwtracing/coresight/Kconfig | 10 +++ drivers/hwtracing/coresight/coresight-etm-perf.c | 10 +++ drivers/hwtracing/coresight/coresight-etm4x.c | 7 ++ drivers/hwtracing/coresight/coresight-etm4x.h | 8 ++ drivers/hwtracing/coresight/coresight-tmc-etf.c | 8 ++ include/linux/coresight.h | 6 ++ 11 files changed, 156 insertions(+), 2 deletions(-) rename Documentation/trace/{ => coresight}/coresight-cpu-debug.txt (100%) create mode 100644 Documentation/trace/coresight/coresight-panic-kdump.txt rename Documentation/trace/{ => coresight}/coresight.txt (100%)