CoreSight

coresight@lists.linaro.org

1 participants
2095 discussions

Re: [PATCH] coresight: etm4x: Remove bogous __exit annotation for some functions

by James Clark

On 29/09/2023 09:16, Uwe Kleine-König wrote: > etm4_platform_driver (which lives in ".data" contains a reference to > etm4_remove_platform_dev(). So the latter must not be marked with __exit > which results in the function being discarded for a build with > CONFIG_CORESIGHT_SOURCE_ETM4X=y which in turn makes the remove pointer > contain invalid data. > > etm4x_amba_driver referencing etm4_remove_amba() has the same issue. > > Drop the __exit annotations for the two affected functions and a third > one that is called by the other two. > Makes sense to me, I can't see any other struct platform_driver callbacks marked with __init or __exit. Reviewed-by: James Clark <james.clark(a)arm.com> > For reasons I don't understand this isn't catched by building with > CONFIG_DEBUG_SECTION_MISMATCH=y. > > Fixes: c23bc382ef0e ("coresight: etm4x: Refactor probing routine") > Fixes: 5214b563588e ("coresight: etm4x: Add support for sysreg only devices") > Signed-off-by: Uwe Kleine-König <u.kleine-koenig(a)pengutronix.de> > --- > drivers/hwtracing/coresight/coresight-etm4x-core.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c > index 77b0271ce6eb..34aee59dd147 100644 > --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c > +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c > @@ -2224,7 +2224,7 @@ static void clear_etmdrvdata(void *info) > per_cpu(delayed_probe, cpu) = NULL; > } > > -static void __exit etm4_remove_dev(struct etmv4_drvdata *drvdata) > +static void etm4_remove_dev(struct etmv4_drvdata *drvdata) > { > bool had_delayed_probe; > /* > @@ -2253,7 +2253,7 @@ static void __exit etm4_remove_dev(struct etmv4_drvdata *drvdata) > } > } > > -static void __exit etm4_remove_amba(struct amba_device *adev) > +static void etm4_remove_amba(struct amba_device *adev) > { > struct etmv4_drvdata *drvdata = dev_get_drvdata(&adev->dev); > > @@ -2261,7 +2261,7 @@ static void __exit etm4_remove_amba(struct amba_device *adev) > etm4_remove_dev(drvdata); > } > > -static int __exit etm4_remove_platform_dev(struct platform_device *pdev) > +static int etm4_remove_platform_dev(struct platform_device *pdev) > { > struct etmv4_drvdata *drvdata = dev_get_drvdata(&pdev->dev); > > > base-commit: 0bb80ecc33a8fb5a682236443c1e740d5c917d1d

1 year, 1 month

[PATCH v1 0/5] kvm/coresight: Support exclude guest and exclude host

by James Clark

This is a combination of the RFC for nVHE here [1] and v3 of VHE version here [2]. After a few of the review comments it seemed much simpler for both versions to use the same interface and be in the same patchset. FEAT_TRF is a Coresight feature that allows trace capture to be completely filtered at different exception levels, unlike the existing TRCVICTLR controls which may still emit target addresses of branches, even if the following trace is filtered. Without FEAT_TRF, it was possible to start a trace session on a host and also collect trace from the guest as TRCVICTLR was never programmed to exclude guests (and it could still emit target addresses even if it was). With FEAT_TRF, the current behavior of trace in guests exists depends on whether nVHE or VHE are being used. Both of the examples below are from the host's point of view, as Coresight isn't accessible from guests. This patchset is only relevant to when FEAT_TRF exists, otherwise there is no change. nVHE: Because the host and the guest are both using TRFCR_EL1, trace will be generated in guests depending on the same filter rules the host is using. For example if the host is tracing userspace only, then guest userspace trace will also be collected. (This is further limited by whether TRBE is used because an issue with TRBE means that it's completely disabled in nVHE guests, but it's possible to have other tracing components.) VHE: With VHE, the host filters will be in TRFCR_EL2, but the filters in TRFCR_EL1 will be active when the guest is running. Because we don't write to TRFCR_EL1, guest trace will be completely disabled. With this change, the guest filtering rules from the Perf session are honored for both nVHE and VHE modes. This is done by either writing to TRFCR_EL12 at the start of the Perf session and doing nothing else further, or caching the guest value and writing it at guest switch for nVHE. The first commit moves the register to sysreg because I add the EL12 version in a later commit. --- Changes since nVHE RFC [1]: * Re-write just in terms of the register value to be written for the host and the guest. This removes some logic from the hyp code and a value of kvm_vcpu_arch:trfcr_el1 = 0 no longer means "don't restore". * Remove all the conditional compilation and new files. * Change the kvm_etm_update_vcpu_events macro to a function. * Re-use DEBUG_STATE_SAVE_TRFCR so iflags don't need to be expanded anymore. * Expand the cover letter. Changes since VHE v3 [2]: * Use the same interface as nVHE mode so TRFCR_EL12 is now written by kvm. [1]: https://lore.kernel.org/kvmarm/20230804101317.460697-1-james.clark@arm.com/ [2]: https://lore.kernel.org/kvmarm/20230905102117.2011094-1-james.clark@arm.com/ James Clark (5): arm64/sysreg: Move TRFCR definitions to sysreg arm64: KVM: Rename DEBUG_STATE_SAVE_TRBE to DEBUG_STATE_SAVE_TRFCR arm64: KVM: Add interface to set guest value for TRFCR register arm64: KVM: Write TRFCR value on guest switch with nVHE coresight: Pass guest TRFCR value to KVM arch/arm64/include/asm/kvm_host.h | 10 +++- arch/arm64/include/asm/sysreg.h | 12 ---- arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/debug.c | 38 +++++++++++- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 60 +++++++++++-------- arch/arm64/tools/sysreg | 41 +++++++++++++ .../coresight/coresight-etm4x-core.c | 42 +++++++++++-- drivers/hwtracing/coresight/coresight-etm4x.h | 2 +- drivers/hwtracing/coresight/coresight-priv.h | 3 + 9 files changed, 160 insertions(+), 49 deletions(-) -- 2.34.1

1 year, 1 month

[PATCH v9 00/13] Add support to configure TPDM DSB subunit

by Tao Zhang

Introduction of TPDM DSB subunit DSB subunit is responsible for creating a dataset element, and is also optionally responsible for packing it to fit multiple elements on a single ATB transfer if possible in the configuration. The TPDM Core Datapath requests timestamps be stored by the TPDA and then delivering ATB sized data (depending on ATB width and element size, this could be smaller or larger than a dataset element) to the ATB Mast FSM. The DSB subunit must be configured prior to enablement. This series adds support for TPDM to configure the configure DSB subunit. Once this series patches are applied properly, the new tpdm nodes for should be observed at the tpdm path /sys/bus/coresight/devices/tpdm* which supports DSB subunit. e.g. root@qemuarm64:/sys/devices/platform/soc@0/6c08000.tpdm/tpdm1# ls -l drwxr-xr-x 2 root root 0 Jan 1 00:00 connections drwxr-xr-x 2 root root 0 Jan 1 00:00 dsb_edge -rw-r--r-- 1 root root 4096 Jan 1 00:00 dsb_mode drwxr-xr-x 2 root root 0 Jan 1 00:00 dsb_msr drwxr-xr-x 2 root root 0 Jan 1 00:00 dsb_patt -rw-r--r-- 1 root root 4096 Jan 1 00:00 dsb_patt_ts -rw-r--r-- 1 root root 4096 Jan 1 00:00 dsb_patt_type drwxr-xr-x 2 root root 0 Jan 1 00:00 dsb_trig_patt -rw-r--r-- 1 root root 4096 Jan 1 00:00 dsb_trig_ts -rw-r--r-- 1 root root 4096 Jan 1 00:00 dsb_trig_type -rw-r--r-- 1 root root 4096 Jan 1 00:02 enable_source --w------- 1 root root 4096 Jan 1 00:00 integration_test drwxr-xr-x 2 root root 0 Jan 1 00:00 power --w------- 1 root root 4096 Jan 1 00:02 reset_dataset lrwxrwxrwx 1 root root 0 Apr 5 2021 subsystem -> ../../../../../bus/coresight -rw-r--r-- 1 root root 4096 Apr 5 2021 uevent -r--r--r-- 1 root root 4096 Jan 1 00:00 waiting_for_supplier We can use the commands are similar to the below to configure the TPDMs which support DSB subunit. Enable coresight sink first. echo 1 > /sys/bus/coresight/devices/tmc_etf0/enable_sink echo 1 > /sys/bus/coresight/devices/tpdm1/reset_dataset echo 0x3 > /sys/bus/coresight/devices/tpdm1/dsb_edge/ctrl_idx echo 0x1 > /sys/bus/coresight/devices/tpdm1/dsb_edge/ctrl_mask echo 0x0 > /sys/bus/coresight/devices/tpdm1/dsb_edge/ctrl_val echo 1 > /sys/bus/coresight/devices/tpdm1/dsb_patt/enable_ts echo 1 > /sys/bus/coresight/devices/tpdm1/dsb_patt/set_type echo 0 > /sys/bus/coresight/devices/tpdm1/dsb_trig_ts echo 0xFFFFFFFF > /sys/bus/coresight/devices/tpdm1/dsb_patt/tpmr5 echo 0xFFFFFFFF > /sys/bus/coresight/devices/tpdm1/dsb_trig_patt/xpr2 echo 1 > /sys/bus/coresight/devices/tpdm1/enable_source TPDM_DSB commit tree: https://git.codelinaro.org/clo/linux-kernel/coresight/-/tree/tpdm-dsb-v9 https://git.codelinaro.org/clo/linux-kernel/coresight/-/commits/tpdm-dsb-v9 Changes in V9: 1. Adjust the alignment for all the patches in this series. -- Suzuki K Poulose 2. Initialize the variable "ret" to 0 in the patch#4. -- Suzuki K Poulose 3. Change the kernel version to 6.7 for all the sysfs files in the documents for this patch series. -- Suzuki K Poulose 4. Remove the parameter "max" from the general read/write functions in the patch#9 and patch#10. -- Suzuki K Poulose 5. Avoid releasing the lock in two different places in the patch#10. -- Suzuki K Poulose 6. Rename "dsb_patt_ts" to "enable_ts", and rename "dsb_patt_type" to "set_type" in the patch#11. Move these two sysfs files to the folder "dsb_patt". -- Suzuki K Poulose 7. Clear the relevant fields first in the function "set_dsb_tier". -- Suzuki K Poulose 8. Reading/writing the TIER register in the function "set_dsb_tier". -- Suzuki K Poulose 9. Read the property "qcom,dsb_msr_num" during probe. -- Suzuki K Poulose 10. Move the variable "dsb_msr_num" to "drvdata". -- Suzuki K Poulose 11. Move the msr number check to simple tpdm sysfs API. -- Suzuki K Poulose 12. Remove the needless "msr number" check in the function "tpdm_reset_datasets" -- Suzuki K Poulose Changes in V8: 1. Refine the function "tpda_set_element_size" and rename it to "tpda_get_element_size" in the patch#4. -- Suzuki K Poulose 2. Refine the functioin "tpda_enable_port" in the patch#4. -- Suzuki K Poulose 3. Write a helper to check if the TPDM has DSB dataset in the patch#5. -- Suzuki K Poulose 4. Move the function "tpdm_reset_datasets" to "datasets_setup" to call in the patch#5. -- Suzuki K Poulose 5. Refine the comment of DSB in "tpdm_drvdata" in the patch#5. -- Suzuki K Poulose 6. Refine the comments in the documents for this patch series. -- Suzuki K Poulose 7. Adjust the code alignment in this patch series. -- Suzuki K Poulose 8. Combine the mode related functions to one in the patch#8. -- Suzuki K Poulose 9. Refine the R/W functions of "dsb_mode" in the patch#8. -- Suzuki K Poulose 10. Adjust the macros of mode in the TPDM header file in the patch#8. -- Suzuki K Poulose 11. Remove the unused code and fix the warnings in compiling for the patch#9. -- kernel test robot 12. Use the following sysfs nodes to read/set edge control related value in the patch#9. dsb_edge/ \- ctrl_idx -> Set the index number \- ctrl_val -> Set the edge control value \- ctrl_mask -> Set the edge control mask \- edcr0 ... edcr15 -> Read the edge control value \- edcmr0 ... edcmr7 -> Read the edge control mask -- Suzuki K Poulose 13. Use the following sysfs nodes to read/set DSB trigger pattern value and mask in the patch#10. dsb_trig_patt/ \- xpr0 ... xpr15 -> (RW) Set/Get the value \- xpmr0 ... xpmr7 -> (RW) Set/Get the mask -- Suzuki K Poulose 14. Use the following sysfs nodes to read/set DSB pattern value and mask in the patch#11. dsb_patt/ \- tpr0 ... tpr15 -> (RW) Set/Get the value \- tpmr0 ... tpmr7 -> (RW) Set/Get the mask -- Suzuki K Poulose 15. Add "Acked-by" tag to the patch#12. -- Rob Herring 16. Use the following sysfs nodes to read/set DSB MSR in the patch#13. dsb_msr/ \- msr0 ... msr31 -> (RW) Set/Get the value -- Suzuki K Poulose 17. Create the maximal number of DSB MSR sysfs nodes if the TPDM supports DSB MSR. Write the values set by user space to the DSB MSR according to the number of MSR supported by the TPDM. -- Suzuki K Poulose Changes in V7: 1. Since the "One value" limitation on SysFs file usage, add the nodes to read/write the index number for configuring the DSB TPDM. The following index number nodes are added. "dsb_edge_ctrl_idx" in the patch #9 "dsb_trig_patt_idx" in the patch #10 "dsb_patt_idx" in the patch #11 "dsb_msr_idx" in the patch #13 -- Suzuki K Poulose Changes in V6: 1. Align the code to fix the styling issue. -- Suzuki K Poulose Changes in V5: 1. Correct data type for DSB element size in dt-bindings patch. 2. Refine the recursive function "tpda_set_element_size". -- Suzuki K Poulose 3. Get return value of the function "__tpda_enable" in "tpda_enable". -- Suzuki K Poulose 4. Refine the comments on "dsb_esize". -- Suzuki K Poulose 5. Split the chage that introduce the subtype "SUBTYPE_SOURCE_TPDM" to Coresight driver. -- Suzuki K Poulose 6. Inline the trigger type setting to "tpdm_enable_dsb" simply. -- Suzuki K Poulose 7. Split the change that remove the needless CS_{UN,}LOCK in the function "tpdm_datasets_setup". -- Suzuki K Poulose 8. Remove the disablement step in the reset node. -- Suzuki K Poulose 9. Update the kernel version to 6.5 in the sysfs document. -- Suzuki K Poulose 10. Remove the needless check in "tpdm_dsb_is_visible". -- Suzuki K Poulose 11. Change the macro to mask the mode of DSB TPDM. -- Suzuki K Poulose 12. Add a check to make sure "sysfs_emit_at" calling will not cause overflow. -- Suzuki K Poulose 13. Change the macro to get "edge_ctrl" value. -- Suzuki K Poulose 14. Remove the needless comments in the sysfs document. -- Suzuki K Poulose 15. Replace "TPDM_DSB_MAX_PATT" with "drvdata->dsb->msr_num" in "dsb_msr_show". -- Suzuki K Poulose 16. Update the check of MSR number in "dsb_msr_store". -- Suzuki K Poulose 17. Write data to the MSR registers in the DSB TPDM enablement function. -- Suzuki K Poulose Changes in V4: 1. Change the range of the property "qcom,dsb-element-size", and change the type to enumeration. -- Suzuki K Poulose, Krzysztof Kozlowski 2. Change dsb_esize from 32 bits to 8 bits. -- Suzuki K Poulose 3. Update the function tpda_set_element_size since James has updated the dependency series. Meanwhile, it will send out a warning if it detects more than one TPDM from the same TPDA input port. -- Suzuki K Poulose 4. Add a source_sub_type for TPDM to distinguish TPDM from the other coresight source. -- Suzuki K Poulose 5. Return error if the element size is not configured on devicetree in TPDA enablement. -- Suzuki K Poulose 6. Move memory allocation from "tpdm_init_datasets" to "tpdm_datasets_setup". Rename "tpdm_init_datasets" as "tpdm_reset_datasets". -- Suzuki K Poulose 7. Replace "coresight_disable" with "coresight_disable_source" to disable the TPDM in resetting. -- Suzuki K Poulose 8. Make sure "drvdata" is not NULL pointer before using it. -- Suzuki K Poulose 9. Change "set_dsb_cycacc_mode" to "set_dsb_test_mode" since cycle accurate mode is not supported on the current targets. It is replaced by test mode. 10. Document the value of "dsb_mode". -- Suzuki K Poulose 11. Macros are used to replace the formulas on dsb edge control nodes. -- Suzuki K Poulose 12. Document the values of "dsb_trig_patt_val" and "dsb_trig_patt_mask". -- Suzuki K Poulose 13. Combine two pattern related loops to one. And move DSB TIER register configurations to the new function "set_dsb_tier". -- Suzuki K Poulose 14. Rename the property "qcom,dsb_msr_num" to "qcom,dsb-msrs-num". -- Suzuki K Poulose, Krzysztof Kozlowski Changes in V3: 1. Move the property "qcom,dsb-element-size" to TPDM devicetree and update the TPDM yaml file for this item. -- Suzuki K Poulose 2. Add the error message when the DSB element size is not set to 32-bit or 64-bit. -- Suzuki K Poulose 3. Add more information to the comments of patch #3 -- Suzuki K Poulose 4. Combine the value updates to the TPDM_DSB_CR for TPDM. -- Suzuki K Poulose 5. Remove the function "tpdm_datasets_alloc", and fold its code to a new function "tpdm_init_datasets". It will complete the initialization of TPDM. -- Suzuki K Poulose 6. Change the method of qualifying input values. -- Suzuki K Poulose 7. Add the documentation of the new sysfs handles. -- Suzuki K Poulose 8. Provide the separate handles for the "mode bits". -- Suzuki K Poulose Changes in V2: 1. Change the name of the property "qcom,dsb-elem-size" to "qcom,dsb-element-size" -- Suzuki K Poulose 2. Update the TPDA yaml file for the item "qcom,dsb-elem-size". -- Krzysztof Kozlowski 3. Add the full name of DSB in the description of the item "qcom,dsb-elem-size". -- Rob Herring Changes in V1: 1. Change the definition of the property "qcom,dsb-elem-size" from "uint32-array" to "uint32-matrix". -- Krzysztof Kozlowski 2. Add the full name of DSB. -- Rob Herring 3. Deal with 2 entries in an iteration in TPDA driver. -- Suzuki K Poulose 4. Divide the function "tpdm_datasets_alloc" into two functions, "tpdm_datasets_setup" and "tpdm_datasets_alloc". 5. Detecte the input string with the conventional semantics automatically, and constrain the size of the input value. -- Suzuki K Poulose 6. Use the hook function "is_visible()" to hide the DSB related knobs if the data sets are missing. -- Suzuki K Poulose 7. Use the macros "FIELD_GET" and "FIELD_PREP" to set the values. -- Suzuki K Poulose 8. Update the definition of the macros in TPDM driver. 9. Update the comments of the values for the nodes which are for DSB element creation and onfigure pattern match output. -- Suzuki K Poulose 10. Use API "sysfs_emit" to "replace scnprintf". -- Suzuki K Poulose Tao Zhang (13): coresight-tpdm: Remove the unnecessary lock dt-bindings: arm: Add support for DSB element size coresight-tpdm: Introduce TPDM subtype to TPDM driver coresight-tpda: Add DSB dataset support coresight-tpdm: Initialize DSB subunit configuration coresight-tpdm: Add reset node to TPDM node coresight-tpdm: Add nodes to set trigger timestamp and type coresight-tpdm: Add node to set dsb programming mode coresight-tpdm: Add nodes for dsb edge control coresight-tpdm: Add nodes to configure pattern match output coresight-tpdm: Add nodes for timestamp request dt-bindings: arm: Add support for DSB MSR register coresight-tpdm: Add nodes for dsb msr support .../ABI/testing/sysfs-bus-coresight-devices-tpdm | 159 +++++ .../bindings/arm/qcom,coresight-tpdm.yaml | 20 + drivers/hwtracing/coresight/coresight-core.c | 3 + drivers/hwtracing/coresight/coresight-tpda.c | 126 +++- drivers/hwtracing/coresight/coresight-tpda.h | 2 + drivers/hwtracing/coresight/coresight-tpdm.c | 714 ++++++++++++++++++++- drivers/hwtracing/coresight/coresight-tpdm.h | 161 +++++ include/linux/coresight.h | 1 + 8 files changed, 1164 insertions(+), 22 deletions(-) -- 2.7.4

1 year, 1 month

[RFC PATCH v3 0/8] Coresight for Kernel panic and watchdog reset

by Linu Cherian

This RFC v3 patch series is rebased on v6.5-rc7 and is dependent on the below two patches. - coresight: tmc: Make etr buffer mode user configurable from sysfs[1] - coresight: Fix run time warnings while reusing ETR buffer[2] Changelog from v2: * ETR reserved buffer mode can be selected only through the new sysfs buffer mode and not by default. This would avoid any conflicts with normal usage. * ETR buffer size in reserved mode is now always fixed to the maximum size of the reserved buffer and not user configurable. This avoids any conflicts with the default buffer size used in other ETR buffer modes. * Introduced new ops called prevboot_ops to factor out common code in tmc_etr_prepare_prevboot() and tmc_etb_prepare_prevboot(). spin_lock/unlock invocations tmc_read_prepare_* are now in a single function. * Added more stringent checks for selecting READ_PREVBOOT mode Other misc changes: * Added more details to DT bindings documentation * Fixed unhandled case error in etm4_disable with CONFIG_WERROR * TMC register saving now uses standard accessor functions * Added panic notifier unregistration * memremap of reserved and metadata buffers are now with _WB attributes * Cover letter title has been shortened. Changelog from v1: * V2 is a complete patchset with kernel panic trace tested on Linux 6.4. Details on testing with relevant console logs has been added for reference. * Two additional patches(patch 6 & 7) has been included to manage stopping of trace at the time of kernel panic. * Few bug fixes. RFC v1 is posted here: https://lists.linaro.org/archives/list/coresight@lists.linaro.org/thread/6F… Using Coresight for Kernel panic and Watchdog reset =================================================== This RFC is about extending Linux coresight driver support to address kernel panic and watchdog reset scenarios. This would help coresight users to debug kernel panic and watchdog reset with the help of coresight trace data. Coresight trace capture: Kernel panic ------------------------------------- From the coresight driver point of view, addressing the kernel panic situation has four main requirements. a. Support for allocation of trace buffer pages from reserved memory area. Platform can advertise this using a new device tree property added to relevant coresight nodes. b. Support for stopping coresight blocks at the time of panic c. Saving required metadata in the specified format d. Support for reading trace data captured at the time of panic Allocation of trace buffer pages from reserved RAM ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A new optional device tree property "memory-region" is added to the ETR/ETF device nodes, that would give the base address and size of trace buffer. Static allocation of trace buffers would ensure that both IOMMU enabled and disabled cases are handled. Also, platforms that support persistent RAM will allow users to read trace data in the subsequent boot without booting the crashdump kernel. Note: For ETR sink devices, this reserved region will be used for both trace capture and trace data retrieval. For ETF sink devices, internal SRAM would be used for trace capture, and they would be synced to reserved region for retrieval. Note: Patches 1 & 2 adds support for this. Disabling coresight blocks at the time of panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order to avoid the situation of losing relevant trace data after a kernel panic, it would be desirable to stop the coresight blocks at the time of panic. This can be achieved by configuring the comparator, CTI and sink devices as below, Comparator(triggers on kernel panic) --->External out --->CTI -- | ETR/ETF stop <------External In <-------------- Note: * Patch 6 provides the necessary ETR configuration. * Patch 7 provides the necessary ETM configuration. Saving metadata at the time of kernel panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Coresight metadata involves all additional data that are required for a successful trace decode in addition to the trace data. This involves ETR/ETF, ETE register snapshot etc. A new optional device property "memory-region" is added to the ETR/ETF/ETE device nodes for this. Note: Patches 3 & 4 adds support for this. Reading trace data captured at the time of panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Trace data captured at the time of panic, can be read from rebooted kernel or from crashdump kernel using the below mentioned interface. Note: Patch 5 adds support for this. Steps for reading trace data captured in previous boot ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1. cd /sys/bus/coresight/devices/tmc_etrXX/ 2. Change to special mode called, read_prevboot. #echo 1 > read_prevboot 3. Dump trace buffer data to a file, #dd if=/dev/tmc_etrXX of=~/cstrace.bin 4. Reset back to normal mode #echo 0 > read_prevboot General flow of trace capture and decode incase of kernel panic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Enable source and sink on all the cores using the sysfs interface. ETR sink will have trace buffers allocated from reserved memory, by selecting "resrv" buffer mode from sysfs. 2. Run relevant tests. 3. On a kernel panic, all coresight blocks are disabled, necessary metadata is synced by kernel panic handler. System would eventually reboot or boot a crashdump kernel. 4. For platforms that supports crashdump kernel, raw trace data can be dumped using the coresight sysfs interface from the crashdump kernel itself. Persistent RAM is not a requirement in this case. 5. For platforms that supports persistent RAM, trace data can be dumped using the coresight sysfs interface in the subsequent Linux boot. Crashdump kernel is not a requirement in this case. Persistent RAM ensures that trace data is intact across reboot. Coresight trace capture: Watchdog reset --------------------------------------- The main difference between addressing the watchdog reset and kernel panic case are below, a. Saving coresight metadata need to be taken care by the SCP(system control processor) firmware in the specified format, instead of kernel. b. Reserved memory region given by firmware for trace buffer and metadata has to be in persistent RAM. Note: This is a requirement for watchdog reset case but optional in kernel panic case. Watchdog reset can be supported only on platforms that meet the above two requirements. Testing Kernel panic on Linux 6.5 --------------------------------- 1. Configure CTI using sysfs interface #./cti_setup.sh #cat cti_setup.sh cd /sys/bus/coresight/devices/ ap_cti_config () { #ETM trig out[0] trigger to Channel 0 echo 0 4 > channels/trigin_attach } etf_cti_config () { #ETF Flush in trigger from Channel 0 echo 0 1 > channels/trigout_attach echo 1 > channels/trig_filter_enable } etr_cti_config () { #ETR Flush in from Channel 0 echo 0 1 > channels/trigout_attach echo 1 > channels/trig_filter_enable } ctidevs=`find . -name "cti*"` for i in $ctidevs do cd $i connection=`find . -name "ete*"` if [ ! -z "$connection" ] then echo "AP CTI config for $i" ap_cti_config fi connection=`find . -name "tmc_etf*"` if [ ! -z "$connection" ] then echo "ETF CTI config for $i" etf_cti_config fi connection=`find . -name "tmc_etr*"` if [ ! -z "$connection" ] then echo "ETR CTI config for $i" etr_cti_config fi cd .. done Note: CTI connections are SOC specific and hence the above script is added just for reference. 2. Start Coresight tracing on cores 1 and 2 using sysfs interface 3. Run some application on core 1 #taskset -c 1 dd if=/dev/urandom of=/dev/null & 4. Invoke kernel panic on core 2 #echo 1 > /proc/sys/kernel/panic #taskset -c 2 echo c > /proc/sysrq-trigger 5. From rebooted kernel, enable previous boot mode #echo 1 > /sys/bus/coresight/devices/tmc_etr0/read_prevboot 6. Read trace data #dd if=/dev/tmc_etr0 of=/trace/cstrace.bin 7. Run opencsd decoder tools/scripts to generate the instruction trace. Core 1 instruction trace dump: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> A etm4_enable_hw: ffff800008ae1dd4 CONTEXT EL2 etm4_enable_hw: ffff800008ae1dd4 I etm4_enable_hw: ffff800008ae1dd4: d503201f nop I etm4_enable_hw: ffff800008ae1dd8: d503201f nop I etm4_enable_hw: ffff800008ae1ddc: d503201f nop I etm4_enable_hw: ffff800008ae1de0: d503201f nop I etm4_enable_hw: ffff800008ae1de4: d503201f nop I etm4_enable_hw: ffff800008ae1de8: d503233f paciasp I etm4_enable_hw: ffff800008ae1dec: a9be7bfd stp x29, x30, [sp, #-32]! I etm4_enable_hw: ffff800008ae1df0: 910003fd mov x29, sp I etm4_enable_hw: ffff800008ae1df4: a90153f3 stp x19, x20, [sp, #16] I etm4_enable_hw: ffff800008ae1df8: 2a0003f4 mov w20, w0 I etm4_enable_hw: ffff800008ae1dfc: 900085b3 adrp x19, ffff800009b95000 <reserved_mem+0xc48> I etm4_enable_hw: ffff800008ae1e00: 910f4273 add x19, x19, #0x3d0 I etm4_enable_hw: ffff800008ae1e04: f8747a60 ldr x0, [x19, x20, lsl #3] E etm4_enable_hw: ffff800008ae1e08: b4000140 cbz x0, ffff800008ae1e30 <etm4_starting_cpu+0x50> I 149.039572921 etm4_enable_hw: ffff800008ae1e30: a94153f3 ldp x19, x20, [sp, #16] I 149.039572921 etm4_enable_hw: ffff800008ae1e34: 52800000 mov w0, #0x0 // #0 I 149.039572921 etm4_enable_hw: ffff800008ae1e38: a8c27bfd ldp x29, x30, [sp], #32 ..snip 149.052324811 chacha_block_generic: ffff800008642d80: 9100a3e0 add x0, I 149.052324811 chacha_block_generic: ffff800008642d84: b86178a2 ldr w2, [x5, x1, lsl #2] I 149.052324811 chacha_block_generic: ffff800008642d88: 8b010803 add x3, x0, x1, lsl #2 I 149.052324811 chacha_block_generic: ffff800008642d8c: b85fc063 ldur w3, [x3, #-4] I 149.052324811 chacha_block_generic: ffff800008642d90: 0b030042 add w2, w2, w3 I 149.052324811 chacha_block_generic: ffff800008642d94: b8217882 str w2, [x4, x1, lsl #2] I 149.052324811 chacha_block_generic: ffff800008642d98: 91000421 add x1, x1, #0x1 I 149.052324811 chacha_block_generic: ffff800008642d9c: f100443f cmp x1, #0x11 Core 2 instruction trace dump(kernel panic triggered core): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A etm4_enable_hw: ffff800008ae1dd4 CONTEXT EL2 etm4_enable_hw: ffff800008ae1dd4 I etm4_enable_hw: ffff800008ae1dd4: d503201f nop I etm4_enable_hw: ffff800008ae1dd8: d503201f nop I etm4_enable_hw: ffff800008ae1ddc: d503201f nop I etm4_enable_hw: ffff800008ae1de0: d503201f nop I etm4_enable_hw: ffff800008ae1de4: d503201f nop I etm4_enable_hw: ffff800008ae1de8: d503233f paciasp I etm4_enable_hw: ffff800008ae1dec: a9be7bfd stp x29, x30, [sp, #-32]! I etm4_enable_hw: ffff800008ae1df0: 910003fd mov x29, sp I etm4_enable_hw: ffff800008ae1df4: a90153f3 stp x19, x20, [sp, #16] I etm4_enable_hw: ffff800008ae1df8: 2a0003f4 mov w20, w0 I etm4_enable_hw: ffff800008ae1dfc: 900085b3 adrp x19, ffff800009b95000 <reserved_mem+0xc48> I etm4_enable_hw: ffff800008ae1e00: 910f4273 add x19, x19, #0x3d0 I etm4_enable_hw: ffff800008ae1e04: f8747a60 ldr x0, [x19, x20, lsl #3] E etm4_enable_hw: ffff800008ae1e08: b4000140 cbz x0, ffff800008ae1e30 <etm4_starting_cpu+0x50> I 149.046243445 etm4_enable_hw: ffff800008ae1e30: a94153f3 ldp x19, x20, [sp, #16] I 149.046243445 etm4_enable_hw: ffff800008ae1e34: 52800000 mov w0, #0x0 // #0 I 149.046243445 etm4_enable_hw: ffff800008ae1e38: a8c27bfd ldp x29, x30, [sp], #32 I 149.046243445 etm4_enable_hw: ffff800008ae1e3c: d50323bf autiasp E 149.046243445 etm4_enable_hw: ffff800008ae1e40: d65f03c0 ret A ete_sysreg_write: ffff800008adfa18 ..snip I 149.05422547 panic: ffff800008096300: a90363f7 stp x23, x24, [sp, #48] I 149.05422547 panic: ffff800008096304: 6b00003f cmp w1, w0 I 149.05422547 panic: ffff800008096308: 3a411804 ccmn w0, #0x1, #0x4, ne // ne = any N 149.05422547 panic: ffff80000809630c: 540001e0 b.eq ffff800008096348 <panic+0xe0> // b.none I 149.05422547 panic: ffff800008096310: f90023f9 str x25, [sp, #64] E 149.05422547 panic: ffff800008096314: 97fe44ef bl ffff8000080276d0 <panic_smp_self_stop> A panic: ffff80000809634c I 149.05422547 panic: ffff80000809634c: 910102d5 add x21, x22, #0x40 I 149.05422547 panic: ffff800008096350: 52800020 mov w0, #0x1 // #1 E 149.05422547 panic: ffff800008096354: 94166b8b bl ffff800008631180 <bust_spinlocks> N 149.054225518 bust_spinlocks: ffff800008631180: 340000c0 cbz w0, ffff800008631198 <bust_spinlocks+0x18> I 149.054225518 bust_spinlocks: ffff800008631184: f000a321 adrp x1, ffff800009a98000 <pbufs.0+0xbb8> I 149.054225518 bust_spinlocks: ffff800008631188: b9405c20 ldr w0, [x1, #92] I 149.054225518 bust_spinlocks: ffff80000863118c: 11000400 add w0, w0, #0x1 I 149.054225518 bust_spinlocks: ffff800008631190: b9005c20 str w0, [x1, #92] E 149.054225518 bust_spinlocks: ffff800008631194: d65f03c0 ret A panic: ffff800008096358 TODO ---- * Change ETM configuration done in patch #7 to new system configuration manager profile * Change CTI sysfs script to system configuration manager profile * Reading tracedata from crashdump kernel is not tested. * Perf based trace capture and decode is not tested. Linu Cherian (8): dt-bindings: arm: coresight-tmc: Add "memory-region" property coresight: tmc-etr: Add support to use reserved trace memory coresight: core: Add provision for panic callbacks coresight: tmc: Enable panic sync handling coresight: tmc: Add support for reading tracedata from previous boot coresight: tmc: Stop trace capture on FlIn coresight: etm4x: Configure ETM to trigger on panic coresight: cti: Add CTI id for Neoverse N2 core CTI .../bindings/arm/arm,coresight-tmc.yaml | 13 + drivers/hwtracing/coresight/coresight-core.c | 32 ++ .../hwtracing/coresight/coresight-cti-core.c | 1 + .../coresight/coresight-etm4x-core.c | 18 +- drivers/hwtracing/coresight/coresight-etm4x.h | 26 ++ .../hwtracing/coresight/coresight-tmc-core.c | 146 +++++++++- .../hwtracing/coresight/coresight-tmc-etf.c | 126 +++++++- .../hwtracing/coresight/coresight-tmc-etr.c | 274 +++++++++++++++++- drivers/hwtracing/coresight/coresight-tmc.h | 48 +++ include/linux/coresight.h | 25 ++ 10 files changed, 701 insertions(+), 8 deletions(-) Links: 1. https://lore.kernel.org/linux-arm-kernel/20230818082112.554638-1-anshuman.k… 2. https://lore.kernel.org/linux-arm-kernel/20230823042948.12879-1-lcherian@ma… -- 2.41.0

1 year, 1 month

Re: [PATCH] perf cs-etm: Fix missing decoder for per-process trace

by James Clark

On 19/09/2023 23:45, Besar Wicaksono wrote: > The decoder creation for raw trace uses metadata from the first CPU. > On per-process/per-thread traces, the first CPU is CPU0. If CPU0 trace > is not enabled, its metadata will be marked unused and the decoder is > not created. Perf report dump skips the decoding part because the > decoder is missing. > Hi Besar, It's not just per-process trace, the bug is also in per-cpu mode but it means that the metadata from CPU 0 is used for every decoder which is wrong. Although your change also fixes this issue. > To fix this, use metadata of the CPU associated with sample object. > > Signed-off-by: Besar Wicaksono <bwicaksono(a)nvidia.com> > --- > tools/perf/util/cs-etm.c | 130 +++++++++++++++++++++++---------------- > 1 file changed, 77 insertions(+), 53 deletions(-) > [...] > + if (!formatted) { > + /* > + * There is only one decoder when unformatted. Use metadata of > + * sample AUX cpu. > + */ > + t_param = t_params; > + metadata = get_cpu_data(etm, sample_cpu); > + if (!metadata) { > + pr_err("CS_ETM: invalid sample CPU: %d\n", sample_cpu); > return -EINVAL; > } Apart from Mike's comments, this looks ok. Thanks for fixing this it has been on our list for a while. One issue with calling get_cpu_data() with the sample CPU ID is that it won't work with old files that don't have the CPU sample flag set. Mike added the sample flag fairly recently, and I don't think that was a breaking change for old files at that time. It should be easy to avoid that by still returning the metadata from CPU 0 when CPU = -1 (Which isn't correct but is 99% likely to work). I checked the Coresight tests and they're all passing, at least on a system without ETE. If you could make sure they're all passing for you as well that would be great: sudo ./perf test coresight I think they currently only work from an in source build, if you get stuck there. Thanks James

1 year, 1 month

Re: [PATCH 2/2] dt-bindings: arm,coresight-cti: Add missing additionalProperties on child nodes

by Mike Leach

On Mon, 25 Sept 2023 at 23:05, Rob Herring <robh(a)kernel.org> wrote: > > Just as unevaluatedProperties or additionalProperties are required at > the top level of schemas, they should (and will) also be required for > child node schemas. That ensures only documented properties are > present for any node. > > Adding additionalProperties constraint on 'trig-conns' nodes results in > warnings that 'cpu' and 'arm,cs-dev-assoc' are not allowed. These are > already defined for the parent node, but need to be duplicated for the > child node. Drop the free form description that the properties also apply > to the child nodes. > > Signed-off-by: Rob Herring <robh(a)kernel.org> > --- > .../bindings/arm/arm,coresight-cti.yaml | 33 ++++++++++++++----- > 1 file changed, 24 insertions(+), 9 deletions(-) > > diff --git a/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml b/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > index b9bdfc8969cd..2d5545a2b49c 100644 > --- a/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > +++ b/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > @@ -93,9 +93,7 @@ properties: > > cpu: > description: > - Handle to cpu this device is associated with. This must appear in the > - base cti node if compatible string arm,coresight-cti-v8-arch is used, > - or may appear in a trig-conns child node when appropriate. > + Handle to cpu this CTI is associated with. > > power-domains: > maxItems: 1 > @@ -112,12 +110,12 @@ properties: > description: > defines a phandle reference to an associated CoreSight trace device. > When the associated trace device is enabled, then the respective CTI > - will be enabled. Use in a trig-conns node, or in CTI base node when > - compatible string arm,coresight-cti-v8-arch used. If the associated > - device has not been registered then the node name will be stored as > - the connection name for later resolution. If the associated device is > - not a CoreSight device or not registered then the node name will remain > - the connection name and automatic enabling will not occur. > + will be enabled. Use in CTI base node when compatible string > + arm,coresight-cti-v8-arch used. If the associated device has not been > + registered then the node name will be stored as the connection name for > + later resolution. If the associated device is not a CoreSight device or > + not registered then the node name will remain the connection name and > + automatic enabling will not occur. > > # size cells and address cells required if trig-conns node present. > "#size-cells": > @@ -129,6 +127,8 @@ properties: > patternProperties: > '^trig-conns@([0-9]+)$': > type: object > + additionalProperties: false > + > description: > A trigger connections child node which describes the trigger signals > between this CTI and another hardware device. This device may be a CPU, > @@ -140,6 +140,21 @@ patternProperties: > reg: > maxItems: 1 > > + cpu: > + description: > + Handle to cpu this trigger connection is associated with. > + > + arm,cs-dev-assoc: > + $ref: /schemas/types.yaml#/definitions/phandle > + description: > + defines a phandle reference to an associated CoreSight trace device. > + When the associated trace device is enabled, then the respective CTI > + will be enabled. If the associated device has not been registered > + then the node name will be stored as the connection name for later > + resolution. If the associated device is not a CoreSight device or > + not registered then the node name will remain the connection name > + and automatic enabling will not occur. > + > arm,trig-in-sigs: > $ref: /schemas/types.yaml#/definitions/uint32-array > minItems: 1 > -- > 2.40.1 > Reviewed-by: Mike Leach <mike.leach(a)linaro.org> -- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

1 year, 1 month

Re: [PATCH 1/2] dt-bindings: arm,coresight-cti: Drop type for 'cpu' property

by Mike Leach

On Mon, 25 Sept 2023 at 23:05, Rob Herring <robh(a)kernel.org> wrote: > > 'cpu' has been added as a single phandle type to dtschema, so drop the > type here. > > Signed-off-by: Rob Herring <robh(a)kernel.org> > --- > Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml b/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > index 6216cfb0a188..b9bdfc8969cd 100644 > --- a/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > +++ b/Documentation/devicetree/bindings/arm/arm,coresight-cti.yaml > @@ -92,7 +92,6 @@ properties: > maxItems: 1 > > cpu: > - $ref: /schemas/types.yaml#/definitions/phandle > description: > Handle to cpu this device is associated with. This must appear in the > base cti node if compatible string arm,coresight-cti-v8-arch is used, > -- > 2.40.1 > Reviewed-by: Mike Leach <mike.leach(a)linaro.org> -- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

1 year, 1 month

Re: [PATCH v3 1/2] arm64/sysreg: Move TRFCR definitions to sysreg

by James Clark

On 25/09/2023 16:59, Catalin Marinas wrote: > On Tue, Sep 05, 2023 at 11:21:14AM +0100, James Clark wrote: >> Add separate definitions for ELx and EL2 as TRFCR_EL1 doesn't have CX. >> This also mirrors the previous definition so no code change is required. >> >> Signed-off-by: James Clark <james.clark(a)arm.com> > > Acked-by: Catalin Marinas <catalin.marinas(a)arm.com> > > I assume these patches would go in via the coresight tree. Possibly, but I'm working on a V4 that combines the nVHE version and Suzuki's comment from here [1]. There will be more kvm patches in that one so I'm not sure. [1]: https://lists.linaro.org/archives/list/coresight@lists.linaro.org/message/A…

1 year, 1 month

[RFC RESEND 0/7] coresight: Move remaining AMBA ACPI devices into platform driver

by Anshuman Khandual

This moves remaining AMBA ACPI devices into respective platform drivers for enabling ACPI based power management support. This might still require some further changes but presented here just for some initial review & feedback. This series applies on coresight/next coresight/for-next/queue and has been built tested. This series has also been boot tested on a DT based coresight device latform. Although it still requires testing on ACPI platforms. Cc: Suzuki Poulose <suzuki.poulose(a)arm.com> Cc: James Clark <james.clark(a)arm.com> Cc: Mike Leach <mike.leach(a)linaro.org> Cc: coresight(a)lists.linaro.org Cc: linux-arm-kernel(a)lists.infradead.org Cc: linux-kernel(a)vger.kernel.org Anshuman Khandual (7): coresight: replicator: Move ACPI support from AMBA driver to platform driver coresight: funnel: Move ACPI support from AMBA driver to platform driver coresight: catu: Move ACPI support from AMBA driver to platform driver coresight: tpiu: Move ACPI support from AMBA driver to platform driver coresight: tmc: Move ACPI support from AMBA driver to platform driver coresight: stm: Move ACPI support from AMBA driver to platform driver coresight: debug: Move ACPI support from AMBA driver to platform driver drivers/acpi/arm64/amba.c | 8 -- drivers/hwtracing/coresight/coresight-catu.c | 136 ++++++++++++++++-- drivers/hwtracing/coresight/coresight-catu.h | 1 + .../hwtracing/coresight/coresight-cpu-debug.c | 130 +++++++++++++++-- .../hwtracing/coresight/coresight-funnel.c | 49 ++++--- .../coresight/coresight-replicator.c | 44 +++--- drivers/hwtracing/coresight/coresight-stm.c | 80 +++++++++-- .../hwtracing/coresight/coresight-tmc-core.c | 127 ++++++++++++++-- drivers/hwtracing/coresight/coresight-tmc.h | 1 + drivers/hwtracing/coresight/coresight-tpiu.c | 76 +++++++++- 10 files changed, 549 insertions(+), 103 deletions(-) -- 2.25.1

1 year, 1 month

[PATCH V5 - RESEND 0/3] coresight: etm: Make cycle count threshold user configurable

by Anshuman Khandual

This series makes ETM TRCCCCTRL based 'cc_threshold' user configurable via the perf event attribute. But first, this implements an errata work around affecting ETM TRCIDR3.CCITMIN value on certain cpus, overriding the field. This series applies on coresight/for-next/queue. Cc: Catalin Marinas <catalin.marinas(a)arm.com> Cc: Will Deacon <will(a)kernel.org> Cc: Suzuki K Poulose <suzuki.poulose(a)arm.com> Cc: Mike Leach <mike.leach(a)linaro.org> Cc: James Clark <james.clark(a)arm.com> Cc: Leo Yan <leo.yan(a)linaro.org> Cc: Jonathan Corbet <corbet(a)lwn.net> Cc: linux-doc(a)vger.kernel.org Cc: coresight(a)lists.linaro.org Cc: linux-arm-kernel(a)lists.infradead.org Cc: linux-kernel(a)vger.kernel.org Changes in V5: https://lore.kernel.org/all/20230821045216.641499-1-anshuman.khandual@arm.c… - Replaced 'where as' with single word 'whereas' - Reworked 'cc_threshold' fallback to ETM_CYC_THRESHOLD_DEFAULT Changes in V4: https://lore.kernel.org/all/20230818112051.594986-1-anshuman.khandual@arm.c… - Fixed a typo s/rangess/ranges, - Renamed etm4_work_around_wrong_ccitmin() as etm4_core_reads_wrong_ccitmin() - Moved drvdata->ccitmin value check for 256 inside etm4_core_reads_wrong_ccitmin() - Moved the comment inside etm4_core_reads_wrong_ccitmin() Changes in V3: https://lore.kernel.org/all/20230811034600.944386-1-anshuman.khandual@arm.c… - Added errata work around affecting TRCIDR3.CCITMIN - Split the document update into a separate patch Changes in V2: https://lore.kernel.org/all/20230808074533.380537-1-anshuman.khandual@arm.c… - s/treshhold/threshold Changes in V1: https://lore.kernel.org/all/20230804044720.1478900-1-anshuman.khandual@arm.… Anshuman Khandual (3): coresight: etm: Override TRCIDR3.CCITMIN on errata affected cpus coresight: etm: Make cycle count threshold user configurable Documentation: coresight: Add cc_threshold tunable Documentation/arch/arm64/silicon-errata.rst | 10 +++++ Documentation/trace/coresight/coresight.rst | 4 ++ .../hwtracing/coresight/coresight-etm-perf.c | 2 + .../coresight/coresight-etm4x-core.c | 45 ++++++++++++++++++- 4 files changed, 59 insertions(+), 2 deletions(-) -- 2.25.1

1 year, 1 month

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CoreSight