Re: [PATCH v2 0/5] coresight: ctcu: Enable byte-cntr function for TMC ETR

Hi,

I can see that this patchset has fixed some of the issues raised from v1 - using the existing file handle for read, and stopping the ETR to read the RWP.

However the fundamental problem that it is still attempting to read memory without stopping the ETR has not been addressed.

As mentioned in mine and Suzuki's comments for v1, this means:-

1) you cannot guarantee that the buffer has not wrapped when reading data back, which will corrupt the trace decode process. 2) The DMA buffers are not being synchronized, so the PE could be reading stale data rather than the new data written by the ETR.

Regards

Mike

On Thu, 10 Apr 2025 at 02:33, Jie Gan jie.gan@oss.qualcomm.com wrote:

The byte-cntr function provided by the CTCU device is used to transfer data from the ETR buffer to the userspace. An interrupt is tiggered if the data size exceeds the threshold set in the BYTECNTRVAL register. The interrupt handler counts the number of triggered interruptions and the read function will read the data from the ETR buffer if the IRQ count is greater than 0. The read work will be conducted ASAP after the byte-cntr is started. Each successful read process will decrement the IRQ count by 1.

The byte cntr function will start when the device node is opened for reading, and the IRQ count will reset when the byte cntr function has stopped. When the file node is opened, the w_offset of the ETR buffer will be read and stored in byte_cntr_data, serving as the original r_offset (indicating where reading starts) for the byte counter function.

The work queue for the read operation will wake up once when ETR is stopped, ensuring that the remaining data in the ETR buffer has been flushed based on the w_offset read at the time of stopping.

The byte-cntr read work has integrated with the file node tmc_etr, e.g. /dev/tmc_etr0 /dev/tmc_etr1

There are two scenarios for the ETR file nodes with byte-cntr function:

1. BYTECNTRVAL register has configured -> byte-cntr read
2. BYTECNTRVAL register is disabled -> original behavior, flush the etr_buf

We still can flush the etr buffer once after the byte-cntr function has triggered.

1. Enable byte-cntr
2. Byte-cntr read
3. Disable byte-cntr
4. Flush etr buffer

Since the ETR operates in circular buffer mode, we cannot fully guarantee that no overwrites occur when the byte-cntr read function reads the data. The read function will read the data ASAP when the interrupt is triggered and we should not configure a threshold greater than the buffer size of the ETR buffer.

The following shell commands write threshold to BYTECNTRVAL registers.

Only enable byte-cntr for ETR0: echo 0x10000 > /sys/devices/platform/soc@0/4001000.ctcu/ctcu0/byte_cntr_val

Enable byte-cntr for both ETR0 and ETR1(support both hex and decimal values): echo 0x10000 4096 > /sys/devices/platform/soc@0/4001000.ctcu/ctcu0/byte_cntr_val

Setting the BYTECNTRVAL registers to 0 disables the byte-cntr function. Disable byte-cntr for ETR0: echo 0 > /sys/devices/platform/soc@0/4001000.ctcu/ctcu0/byte_cntr_val

Disable byte-cntr for both ETR0 and ETR1: echo 0 0 > /sys/devices/platform/soc@0/4001000.ctcu/ctcu0/byte_cntr_val

There is a minimum threshold to prevent generating too many interrupts. The minimum threshold is 4096 bytes. The write process will fail if user try to set the BYTECNTRVAL registers to a value less than 4096 bytes(except for 0).

Way to enable and start byte-cntr for ETR0: echo 0x10000 > /sys/devices/platform/soc@0/4001000.ctcu/ctcu0/byte_cntr_val echo 1 > /sys/bus/coresight/devices/tmc_etr0/enable_sink echo 1 > /sys/bus/coresight/devices/etm0/enable_source cat /dev/tmc_etr0

Testing case has conducted for the byte-cntr read work:

1. Setting the buffer_size of the ETR as large as possile, here is for ETR0 echo 0x1000000 > /sys/bus/coresight/devices/tmc_etr0/buffer_size
2. Setting the threshold for the ETR0 to 0x10000 echo 0x10000 > /sys/bus/coresight/devices/ctcu0/byte_cntr_val
3. Enable ETR0 echo 1 > /sys/bus/coresight/devices/tmc_etr0/enable_sink
4. Enable ETM0 as source and enable byte-cntr to read data echo 1 > /sys/bus/coresight/devices/etm0/enable_source; cat /dev/tmc_etr0 > /tmp/file_byte_cntr.bin &
5. Disable ETM0 echo 0 > /sys/bus/coresight/devices/etm0/enable_source
6. Disable byte-cntr and flush the etr buffer echo 0 > /sys/bus/coresight/devices/ctcu0/byte_cntr_val; cat /dev/tmc_etr0 > /tmp/file_etr0.bin ls -l /tmp

-rw-r--r-- 1 root root 12628960 Apr 28 17:44 file_byte_cntr.bin -rw-r--r-- 1 root root 12669296 Apr 28 17:45 file_etr0.bin

7. Deal with the file_etr0.bin with following command: dd if=/tmp/file_etr0.bin of=/tmp/file_etr0_aligned.bin bs=1

count=12628960 skip=40336 ls -l /tmp

-rw-r--r-- 1 root root 12628960 Apr 28 17:44 file_byte_cntr.bin -rw-r--r-- 1 root root 12669296 Apr 28 17:45 file_etr0.bin -rw-r--r-- 1 root root 12628960 Apr 28 17:49 file_etr0_aligned.bin

8. Compared file_byte_cntr.bin with file_etr0_aligned.bin and identified

they are competely same. diff file_byte_cntr.bin file_etr0_aligned.bin

======================= Changes in V2:

1. Removed the independent file node /dev/byte_cntr.
2. Integrated the byte-cntr's file operations with current ETR file node.
3. Optimized the driver code of the CTCU that associated with byte-cntr.
4. Add kernel document for the export API tmc_etr_get_rwp_offset.
5. Optimized the way to read the rwp_offset according to Mike's suggestion.
6. Removed the dependency of the dts patch.

Link to V1 - https://lore.kernel.org/all/20250310090407.2069489-1-quic_jiegan@quicinc.com...

Jie Gan (5): coresight: tmc: Introduce new APIs to get the RWP offset of ETR buffer dt-bindings: arm: Add an interrupt property for Coresight CTCU coresight: ctcu: Enable byte-cntr for TMC ETR devices coresight: tmc: add functions for byte-cntr operation arm64: dts: qcom: sa8775p: Add interrupts to CTCU device

.../bindings/arm/qcom,coresight-ctcu.yaml | 17 ++ arch/arm64/boot/dts/qcom/sa8775p.dtsi | 5 + drivers/hwtracing/coresight/Makefile | 2 +- .../coresight/coresight-ctcu-byte-cntr.c | 119 ++++++++++++ .../hwtracing/coresight/coresight-ctcu-core.c | 88 ++++++++- drivers/hwtracing/coresight/coresight-ctcu.h | 49 ++++- .../hwtracing/coresight/coresight-tmc-core.c | 29 ++- .../hwtracing/coresight/coresight-tmc-etr.c | 175 ++++++++++++++++++ drivers/hwtracing/coresight/coresight-tmc.h | 10 +- 9 files changed, 483 insertions(+), 11 deletions(-) create mode 100644 drivers/hwtracing/coresight/coresight-ctcu-byte-cntr.c

-- 2.34.1