Hi Mike.

Thank you for your response. I am sorry I was probably not very clear about my problem.

Let me give you a better example with this assembly code extracted from my user application:

0x16258 <__syscall>: ... 0x16270: ldm ip, {r3, r4, r5, r6} 0x16274: svc 0x00000000 0x16278: pop {r4, r5, r6, r7}

The execution of the instructions above are traced, but when the SVC instruction is executed, the user application gives way to kernel land function related to the system call. The kernel is not traced, but during the system call is executed, a computation result from the monitor is nested before returning to user land. In order to inform the FPGA monitor that the user application entered into a system call I am using PTM traces. When I tried to implement that behaviour, the monitor was not aware that the user application gives way to kernel functions because the monitor didn’t receive the waypoint update packet nor the exception branch address packet related to the SVC instruction at 0x16274. (http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ihi0035b/BHBIIEI... http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ihi0035b/BHBIIEIA.html)

The “execution flow” is: 1) Jump to __syscall function (at 0x16258) 2) SVC Instruction in the user application is executed (at 0x16274) 3) Entering into the Kernel land … (somewhere in the kernel land) 4) Waiting a response from the FPGA monitor (somewhere in the kernel land)

The traces in the FPGA monitor are: { … ; 0x16258 } (notice that the last trace is 0x16258, the address of __syscall function) As the traces don't include any waypoint update packet nor the exception branch address packet related to the SVC instruction at 0x16274 the FPGA monitor is not aware that the application is entered into a system call.

I tried to read all documentations available, but I didn’t find when exactly the waypoint update packet and/or the exception branch address packet are packed and emitted. I thought that by flushing the traces, at least the waypoint update packet (0x16274) will be flushed, but that's not the case. By doing some experimentations, I understood that when an SVC instruction is traced, the corresponding waypoint update packet and the exception branch address packet are available only when returning from that exception. Is that correct ?

To reach this conclusion, I tested to return from the kernel land without waiting the FPGA Monitor response (not wanted behaviour) and I had the result below: The “execution flow” is: 1) Jump to __syscall function (at 0x16258) 2) SVC Instruction in the user application is executed (at 0x16274) 3) Entering into the Kernel (somewhere in the kernel land) 4) Returning from kernel land (at 0x16278)

The traces in the FPGA monitor are: { … ; 0x16258 ; 0x16274 ; 0xffff0008 ; 0x16278 }

So my question is : Is it possible to force the waypoint update packet (0x16274) or/and the exception branch address packet (0xffff0008 ; 0x16278) to be flushed before returning from the SVC (at 0x16278) ?

Best regards,

Mounir

On 22 Jun 2020, at 17:10, Mike Leach mike.leach@linaro.org wrote:

Hi,

On Sun, 21 Jun 2020 at 22:28, Mounir NASR ALLAH <mounir.nasrallah@centralesupelec.fr mailto:mounir.nasrallah@centralesupelec.fr> wrote:

...

Hi all,

I am currently facing a problem about manually flushing traces. The traced user application runs on a Linux distribution with an FPGA Board based on a Zynq-7000 SoC. I am setting up coresight components for tracing only the user application PID and his address space, and I exclude tracing kernel. All traces are exported to a monitor present in the FPGA. The flow of the traces:

————————————— ——————— ————————— | | | | | | | ARM Cortex A9 (Core 0) | ==> | TPIU | ==> | FPGA Monitor | | / PTM | | | | | ————————————— ——————— —————————

When there is a syscall on the user application, I have the traces below:

… 16274 (a waypoint update packet) ffff0008 (the exception branch address) 16278 ..

The relative addresses/assembly instructions extracted from the ELF file are :

16270: e89c0078 ldm ip, {r3, r4, r5, r6} 16274: ef000000 svc 0x00000000 16278: e8bd00f0 pop {r4, r5, r6, r7}

During the execution of the syscall (on the Kernel side), the kernel needs to receive a response from the FPGA Monitor and wait actively for it. This response depends on calculations relative to all PTM traces before the syscall (SVC instruction at 16274). That’s why I need to flush all traces during the syscall, I tried to flush traces by setting the FOnMan_W of the Formatter and Flush Control Register (FFCR) into coresight-tpiu.c but that’s not working.

What is happening to make you think that the flush is not working? Based on the information above you are actually getting all available trace up to the SVC call,

Regards

Mike

...

Do you have an idea of how correctly flushing all traces to the FPGA Monitor on the kernel?

Best regards, Mounir

---

CoreSight mailing list CoreSight@lists.linaro.org mailto:CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight https://lists.linaro.org/mailman/listinfo/coresight

-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

Hi Zied,

Thank you for your quick response. You correctly understood the problem.

Here is the process that I use for flushing traces.

1) In coresight-etm3x.c, I set and clear the Programming bit (Setting the Programming bit or the OS Lock http://infocenter.arm.com/help/topic/com.arm.doc.ihi0035b/BHBHFBAC.html)

etm_set_prog(etmdrvdata[CPU_0]); etm_clr_prog(etmdrvdata[CPU_0]);

2) Then, I tried to flush the TPIU by setting the bit 6 (FOnMan) of the Formatter and Flush Control Register (Formatter and Flush Control Register, 0x304 http://infocenter.arm.com/help/topic/com.arm.doc.ddi0314h/Cbahcjai.html)

writel_relaxed( FFCR_FON_MAN, FFCR_addr);

Is it the right way to flush traces ?

I also tried to stop tracing in coresight-etm3x.c by calling etm_disable_hw :

etm_disable_hw(etmdrvdata[CPU_0]);

But I never receive the address of the SVC instruction. What do you mean by "the transport” ? If I understand I missed to flush an other component ?

Best regards,

On 22 Jun 2020, at 23:57, zied guermazi guermazi_zied@yahoo.com wrote:

hi Mounir If I understood you correctly you need a mean to catch the last executed instruction (SVC) in order to let the fpga monitor detect that it can start processing. usually, the sources and sinks of etm have a hardware fifos where the instruction is either still not emitted by the source of still not formatted by the sink. stopping the source tracing shall flush its pipelines, stoping the sink shall flush the formatter pipeline. the status of the pipelines can be checked through the coresight device registers (for example on ETB flush bit in their status Formatter and Flush Status Register) have you tried stopping and then restarting tracing when you enter the svc handler (in the kernel space) this should force coresight components to issue pending traces, be careful in the order of the stop action (start first by the source then the transport, then the sink)

I hope this helps

Kind regards Zied Guermazi

On Monday, June 22, 2020, 11:15:39 PM GMT+2, Mounir NASR ALLAH mounir.nasrallah@centralesupelec.fr wrote:

Hi Mike.

Thank you for your response. I am sorry I was probably not very clear about my problem.

Let me give you a better example with this assembly code extracted from my user application:

0x16258 <__syscall>: ... 0x16270: ldm ip, {r3, r4, r5, r6} 0x16274: svc 0x00000000 0x16278: pop {r4, r5, r6, r7}

The execution of the instructions above are traced, but when the SVC instruction is executed, the user application gives way to kernel land function related to the system call. The kernel is not traced, but during the system call is executed, a computation result from the monitor is nested before returning to user land. In order to inform the FPGA monitor that the user application entered into a system call I am using PTM traces. When I tried to implement that behaviour, the monitor was not aware that the user application gives way to kernel functions because the monitor didn’t receive the waypoint update packet nor the exception branch address packet related to the SVC instruction at 0x16274. (http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ihi0035b/BHBIIEI...)

The “execution flow” is:

1. Jump to __syscall function (at 0x16258)
2. SVC Instruction in the user application is executed (at 0x16274)
3. Entering into the Kernel land … (somewhere in the kernel land)
4. Waiting a response from the FPGA monitor (somewhere in the kernel land)

The traces in the FPGA monitor are: { … ; 0x16258 } (notice that the last trace is 0x16258, the address of __syscall function) As the traces don't include any waypoint update packet nor the exception branch address packet related to the SVC instruction at 0x16274 the FPGA monitor is not aware that the application is entered into a system call.

I tried to read all documentations available, but I didn’t find when exactly the waypoint update packet and/or the exception branch address packet are packed and emitted. I thought that by flushing the traces, at least the waypoint update packet (0x16274) will be flushed, but that's not the case. By doing some experimentations, I understood that when an SVC instruction is traced, the corresponding waypoint update packet and the exception branch address packet are available only when returning from that exception. Is that correct ?

To reach this conclusion, I tested to return from the kernel land without waiting the FPGA Monitor response (not wanted behaviour) and I had the result below: The “execution flow” is:

1. Jump to __syscall function (at 0x16258)
2. SVC Instruction in the user application is executed (at 0x16274)
3. Entering into the Kernel (somewhere in the kernel land)
4. Returning from kernel land (at 0x16278)

The traces in the FPGA monitor are: { … ; 0x16258 ; 0x16274 ; 0xffff0008 ; 0x16278 }

So my question is : Is it possible to force the waypoint update packet (0x16274) or/and the exception branch address packet (0xffff0008 ; 0x16278) to be flushed before returning from the SVC (at 0x16278) ?

Best regards,

Mounir

...

On 22 Jun 2020, at 17:10, Mike Leach mike.leach@linaro.org wrote:

Hi,

On Sun, 21 Jun 2020 at 22:28, Mounir NASR ALLAH mounir.nasrallah@centralesupelec.fr wrote:

...

Hi all,

I am currently facing a problem about manually flushing traces. The traced user application runs on a Linux distribution with an FPGA Board based on a Zynq-7000 SoC. I am setting up coresight components for tracing only the user application PID and his address space, and I exclude tracing kernel. All traces are exported to a monitor present in the FPGA. The flow of the traces:

————————————— ——————— ————————— | | | | | | | ARM Cortex A9 (Core 0) | ==> | TPIU | ==> | FPGA Monitor | | / PTM | | | | | ————————————— ——————— —————————

When there is a syscall on the user application, I have the traces below:

… 16274 (a waypoint update packet) ffff0008 (the exception branch address) 16278 ..

The relative addresses/assembly instructions extracted from the ELF file are :

16270: e89c0078 ldm ip, {r3, r4, r5, r6} 16274: ef000000 svc 0x00000000 16278: e8bd00f0 pop {r4, r5, r6, r7}

During the execution of the syscall (on the Kernel side), the kernel needs to receive a response from the FPGA Monitor and wait actively for it. This response depends on calculations relative to all PTM traces before the syscall (SVC instruction at 16274). That’s why I need to flush all traces during the syscall, I tried to flush traces by setting the FOnMan_W of the Formatter and Flush Control Register (FFCR) into coresight-tpiu.c but that’s not working.

What is happening to make you think that the flush is not working? Based on the information above you are actually getting all available trace up to the SVC call,

Regards

Mike

...

Do you have an idea of how correctly flushing all traces to the FPGA Monitor on the kernel?

Best regards, Mounir

---

CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight

-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

---

CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight

Hi Mounir,

Issuing a flush request from the sink (TPIU) should be sufficient to flush the ETM - but you do need to ensure that you wait for the flush to complete by monitoring the status flags. I also assume that you are unlocking the coresight components correctly before attempting to write to them.

However, issuing a flush does not flush the frame formatter in the TPIU, which may be keeping the last few bytes of data back until sufficient arrives to complete a frame. The only way to ensure this is flushed is to stop the fomatter / disable the TPIU.

Look @ coresight-tpiu.c - tpiu_disable_hw() for a sequence which flushes and stops the TPIU, and shows the correct coresight unlocking sequence.

Regards

Mike

On Tue, 23 Jun 2020 at 09:40, zied guermazi guermazi_zied@yahoo.com wrote:

hi Mounir please check if you have any funnels, or etrs... in the path between the source of traces and your sink (TPIU), those have to be stopped gracefully. Kind regards Zied

On Tuesday, June 23, 2020, 12:41:18 AM GMT+2, Mounir NASR ALLAH mounir.nasrallah@centralesupelec.fr wrote:

Hi Zied,

Thank you for your quick response. You correctly understood the problem.

Here is the process that I use for flushing traces.

1. In coresight-etm3x.c, I set and clear the Programming bit (Setting the Programming bit or the OS Lock)

etm_set_prog(etmdrvdata[CPU_0]); etm_clr_prog(etmdrvdata[CPU_0]);

2. Then, I tried to flush the TPIU by setting the bit 6 (FOnMan) of the Formatter and Flush Control Register (Formatter and Flush Control Register, 0x304)

writel_relaxed( FFCR_FON_MAN, FFCR_addr);

Is it the right way to flush traces ?

I also tried to stop tracing in coresight-etm3x.c by calling etm_disable_hw :

etm_disable_hw(etmdrvdata[CPU_0]);

But I never receive the address of the SVC instruction. What do you mean by "the transport” ? If I understand I missed to flush an other component ?

Best regards,

On 22 Jun 2020, at 23:57, zied guermazi guermazi_zied@yahoo.com wrote:

hi Mounir If I understood you correctly you need a mean to catch the last executed instruction (SVC) in order to let the fpga monitor detect that it can start processing. usually, the sources and sinks of etm have a hardware fifos where the instruction is either still not emitted by the source of still not formatted by the sink. stopping the source tracing shall flush its pipelines, stoping the sink shall flush the formatter pipeline. the status of the pipelines can be checked through the coresight device registers (for example on ETB flush bit in their status Formatter and Flush Status Register) have you tried stopping and then restarting tracing when you enter the svc handler (in the kernel space) this should force coresight components to issue pending traces, be careful in the order of the stop action (start first by the source then the transport, then the sink)

I hope this helps

Kind regards Zied Guermazi

On Monday, June 22, 2020, 11:15:39 PM GMT+2, Mounir NASR ALLAH mounir.nasrallah@centralesupelec.fr wrote:

Hi Mike.

Thank you for your response. I am sorry I was probably not very clear about my problem.

Let me give you a better example with this assembly code extracted from my user application:

0x16258 <__syscall>: ... 0x16270: ldm ip, {r3, r4, r5, r6} 0x16274: svc 0x00000000 0x16278: pop {r4, r5, r6, r7}

The execution of the instructions above are traced, but when the SVC instruction is executed, the user application gives way to kernel land function related to the system call. The kernel is not traced, but during the system call is executed, a computation result from the monitor is nested before returning to user land. In order to inform the FPGA monitor that the user application entered into a system call I am using PTM traces. When I tried to implement that behaviour, the monitor was not aware that the user application gives way to kernel functions because the monitor didn’t receive the waypoint update packet nor the exception branch address packet related to the SVC instruction at 0x16274. (http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ihi0035b/BHBIIEI...)

The “execution flow” is:

1. Jump to __syscall function (at 0x16258)
2. SVC Instruction in the user application is executed (at 0x16274)
3. Entering into the Kernel land … (somewhere in the kernel land)
4. Waiting a response from the FPGA monitor (somewhere in the kernel land)

The traces in the FPGA monitor are: { … ; 0x16258 } (notice that the last trace is 0x16258, the address of __syscall function) As the traces don't include any waypoint update packet nor the exception branch address packet related to the SVC instruction at 0x16274 the FPGA monitor is not aware that the application is entered into a system call.

I tried to read all documentations available, but I didn’t find when exactly the waypoint update packet and/or the exception branch address packet are packed and emitted. I thought that by flushing the traces, at least the waypoint update packet (0x16274) will be flushed, but that's not the case. By doing some experimentations, I understood that when an SVC instruction is traced, the corresponding waypoint update packet and the exception branch address packet are available only when returning from that exception. Is that correct ?

To reach this conclusion, I tested to return from the kernel land without waiting the FPGA Monitor response (not wanted behaviour) and I had the result below: The “execution flow” is:

1. Jump to __syscall function (at 0x16258)
2. SVC Instruction in the user application is executed (at 0x16274)
3. Entering into the Kernel (somewhere in the kernel land)
4. Returning from kernel land (at 0x16278)

The traces in the FPGA monitor are: { … ; 0x16258 ; 0x16274 ; 0xffff0008 ; 0x16278 }

So my question is : Is it possible to force the waypoint update packet (0x16274) or/and the exception branch address packet (0xffff0008 ; 0x16278) to be flushed before returning from the SVC (at 0x16278) ?

Best regards,

Mounir

On 22 Jun 2020, at 17:10, Mike Leach mike.leach@linaro.org wrote:

Hi,

On Sun, 21 Jun 2020 at 22:28, Mounir NASR ALLAH mounir.nasrallah@centralesupelec.fr wrote:

Hi all,

I am currently facing a problem about manually flushing traces. The traced user application runs on a Linux distribution with an FPGA Board based on a Zynq-7000 SoC. I am setting up coresight components for tracing only the user application PID and his address space, and I exclude tracing kernel. All traces are exported to a monitor present in the FPGA. The flow of the traces:

————————————— ——————— ————————— | | | | | | | ARM Cortex A9 (Core 0) | ==> | TPIU | ==> | FPGA Monitor | | / PTM | | | | | ————————————— ——————— —————————

When there is a syscall on the user application, I have the traces below:

… 16274 (a waypoint update packet) ffff0008 (the exception branch address) 16278 ..

The relative addresses/assembly instructions extracted from the ELF file are :

16270: e89c0078 ldm ip, {r3, r4, r5, r6} 16274: ef000000 svc 0x00000000 16278: e8bd00f0 pop {r4, r5, r6, r7}

During the execution of the syscall (on the Kernel side), the kernel needs to receive a response from the FPGA Monitor and wait actively for it. This response depends on calculations relative to all PTM traces before the syscall (SVC instruction at 16274). That’s why I need to flush all traces during the syscall, I tried to flush traces by setting the FOnMan_W of the Formatter and Flush Control Register (FFCR) into coresight-tpiu.c but that’s not working.

What is happening to make you think that the flush is not working? Based on the information above you are actually getting all available trace up to the SVC call,

Regards

Mike

Do you have an idea of how correctly flushing all traces to the FPGA Monitor on the kernel?

Best regards, Mounir

---

CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight

-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK

---

CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight

-- Mike Leach Principal Engineer, ARM Ltd. Manchester Design Centre. UK