---------- Forwarded message ---------- From: Mathieu Poirier mathieu.poirier@linaro.org Date: 3 November 2015 at 21:27 Subject: Collection of metadata in a multi-session environment To: private-opencsd private-opencsd@linaro.org
As reported in my other email (Trace bundles for ETMv3/v4) moving to a multi-session scheme his forcing me to rethink how metadata is collected as one can no longer assume that what gets read from sysFS actually pertains to the session being reported.
When looking at the required registers (once again, please refer to Mike's document, section 4.3.1 to 4.3.3) a lot of them are RO:
ETMv3/PTM: ETMIDR ETMv4: TRCIDR[0, ..., 13], TRCAUTHSTATUS
Some are RW but can easily be made RO when using the CS framework from Perf:
ETMv3/PTM: ETMTRACEID ETMv4: TRCTRACEIDR
In fact I remember having this conversation with Mike in San Francisco and nobody would cry if we were to let the framework decide the traceIDs configured on the tracers.
Anything that is RO can still be retrieved using the current sysFS driven method since their values don't change. That leaves us with the other RW registers:
ETMv3/PTM: ETMCR, ETMCCR ETMv4: TRCCONFIGR
The question is, what information is needed from these registers in order to do trace decoding? Is there anything in there that is coming from the HW that can't be deduced otherwise? From what I understand we are talking about user configurable options that could be communicated to the trace decoding library using other means than conveying the whole registers. Using a bitmap is the first option that comes to mind, i.e, if cycle accurate tracing is configured, bit X is set.
With the above in mind I need you guys to help me identify the mandatory information in those tracer registers that is needed for trace decoding. Once again we can discuss this further in our meeting tomorrow.
Thanks, Mathieu
When looking at the required registers (once again, please refer to Mike's document, section 4.3.1 to 4.3.3) a lot of them are RO:
ETMv3/PTM: ETMIDR ETMv4: TRCIDR[0, ..., 13], TRCAUTHSTATUS
Some are RW but can easily be made RO when using the CS framework from Perf:
ETMv3/PTM: ETMTRACEID ETMv4: TRCTRACEIDR
In fact I remember having this conversation with Mike in San Francisco and nobody would cry if we were to let the framework decide the traceIDs configured on the tracers.
Anything that is RO can still be retrieved using the current sysFS driven method since their values don't change. That leaves us with the other RW registers:
ETMv3/PTM: ETMCR, ETMCCR ETMv4: TRCCONFIGR
The question is, what information is needed from these registers in order to do trace decoding?
I believe it was all there in cs-decode.pdf (attached for those who haven't seen it). Any omissions, please let me know.
I don't believe ETMCCR is a RW register.
Is there anything in there that is coming from the HW that can't be deduced otherwise?
There are no changes coming from the HW that haven't been programmed in at some point by SW (or by extneral debug but that's a separate issue). So your problem is limited to
- one-time discovery of the configuration
- tracking software updates to the registers where these are not also tracked by sync packets.
So for perf you would want to emit some kind of "configuration changed" event if you notice a configuration change that isn't discoverable through sync packets. The good news is in ETMv4 much more of the changes are tracked in sync packets. You can probably emit that at the end of the trace stream since the trace configuration isn't going to change mid-stream. The key thing is to have the perf subsystem emit that event before it allows any more changes to the configuration for that core, not wait for the user tool to (ages later) query the configuration via sysfs.
Al
From what I understand we are talking about user configurable options that could be communicated to the trace decoding library using other means than conveying the whole registers. Using a bitmap is the first option that comes to mind, i.e, if cycle accurate tracing is configured, bit X is set.
With the above in mind I need you guys to help me identify the mandatory information in those tracer registers that is needed for trace decoding. Once again we can discuss this further in our meeting tomorrow.
Thanks, Mathieu _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
________________________________
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Mathieu,
I've been revisiting the requirements for the trace configuration information perf needs to pass through to the report/decode process as we discussed in the IRC chat on Wednesday.
Considering that the decode library is designed to be very generic, perf has more narrow requirements as it is focussed in A-profile instruction trace.
ETMv4. ----------
TRCCONFIGR => Strictly speaking there are no values in TRCCONFIGR that need passing forwards. If we assume that the trace is 100% valid (i.e. not corrupted in anyway) then the dynamic elements of TRCCONFIGR are supplied in the packets - as Al says, ETMv4 is largely self describing.
However at present the packet processor front end (which splits the stream into discrete packets) does use values in TRCCONFIGR to check that the packet stream is consistent - e.g. throwing errors if timestamp packets are seen when TS is not enabled, if ContexdID packets are seen when ContextID trace is disabled etc,etc. This is done as it is designed to be used in a wider variety of situations than just instruction trace decode in perf. Additionally, having the programmed value of the registers allows us to force sync the decoder to a given offset in the trace stream - this matches functionality present in Intel libipt.
Cortex-A profile will never emit data trace so the bits we really need are:- QE[14:13], TS[11], VMID[7], CID[6], CCI[4];
RS[12] is a nice to have as it saves a bit of work if we know if we are using a return stack - though we can assume there is one if IDR0 indicates it is implemented.
What I am considering doing is allowing a dynamic version of the packet processor to run, removing the consistency checks and removing the need for perf to pass on the CONFIGR in ETMv4 at all - though this will be a future version as it is not at all current. In this case the perf report could configure the decoder with CONFIGR as 0, set the decoder to run in dynamic mode and the relevant information will be updated automatically.
TRCTRACEIDR - as you say this can be effectively RO from perf perspective - the decoder simply needs the ID value to identify the correct data stream from the frame demuxer and allows the generic output elements to be tagged with this ID value.
TRCIDR0-2 - needed for etm features, etm arch version, size of elements TRCIDR8 - needed for max speculation depth. These are all RO so the full value should not be an issue.
TRCIDR9-13 - not needed - only required for data trace.
ETMv3/PTM ---------------- ETMCR => value directly affects the format of the incoming trace, so ideally full value is needed, though perf report could synthesize the value given the relevant bit values. This data cannot be determined from the trace so a value is needed.
Bits needed for instruction trace:- VMID[30], TS[28], CID[15:14], CycleAcc[12] If these features are passed through, perf report can synthesize the correct value to configure the decoder.
RO regs:- ETMIDR, ETMCCER, "ETMTRACEID" values as agreed.
Regards
Mike
---------------------------------------------------------------- Mike Leach +44 (0)1254 893911 (Direct) Principal Engineer +44 (0)1254 893900 (Main) Arm Blackburn Design Centre +44 (0)1254 893901 (Fax) Belthorn House Walker Rd mailto:mike.leach@arm.com Guide Blackburn BB1 2QE ----------------------------------------------------------------
-----Original Message----- From: CoreSight [mailto:coresight-bounces@lists.linaro.org] On Behalf Of Al Grant Sent: 04 November 2015 17:59 To: Mathieu Poirier; coresight@lists.linaro.org Subject: RE: Collection of metadata in a multi-session environment
When looking at the required registers (once again, please refer to Mike's document, section 4.3.1 to 4.3.3) a lot of them are RO:
ETMv3/PTM: ETMIDR ETMv4: TRCIDR[0, ..., 13], TRCAUTHSTATUS
Some are RW but can easily be made RO when using the CS framework from Perf:
ETMv3/PTM: ETMTRACEID ETMv4: TRCTRACEIDR
In fact I remember having this conversation with Mike in San Francisco and nobody would cry if we were to let the framework decide the traceIDs configured on the tracers.
Anything that is RO can still be retrieved using the current sysFS driven
method
since their values don't change. That leaves us with the other RW registers:
ETMv3/PTM: ETMCR, ETMCCR ETMv4: TRCCONFIGR
The question is, what information is needed from these registers in order to
do
trace decoding?
I believe it was all there in cs-decode.pdf (attached for those who haven't seen it). Any omissions, please let me know.
I don't believe ETMCCR is a RW register.
Is there anything in there that is coming from the HW that can't be deduced otherwise?
There are no changes coming from the HW that haven't been programmed in at some point by SW (or by extneral debug but that's a separate issue). So your problem is limited to
one-time discovery of the configuration
tracking software updates to the registers where these are not also tracked
by sync packets.
So for perf you would want to emit some kind of "configuration changed" event if you notice a configuration change that isn't discoverable through sync packets. The good news is in ETMv4 much more of the changes are tracked in sync packets. You can probably emit that at the end of the trace stream since the trace configuration isn't going to change mid-stream. The key thing is to have the perf subsystem emit that event before it allows any more changes to the configuration for that core, not wait for the user tool to (ages later) query the configuration via sysfs.
Al
From what I understand we are talking about user configurable options that could be communicated to the trace decoding
library
using other means than conveying the whole registers. Using a bitmap is the first option that comes to mind, i.e, if cycle accurate tracing is configured, bit
X
is set.
With the above in mind I need you guys to help me identify the mandatory information in those tracer registers that is needed for trace decoding. Once again we can discuss this further in our meeting tomorrow.
Thanks, Mathieu _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
________________________________
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Good afternoon,
I'm almost done putting all this in - the main difference between the new and old solution is that the configuration register is artificially constructed instead of being read from HW. At this time only cycle accurate and timestamp options are available. More can be added at will when we get traces decoded with the basic configuration. Options can be added fairly easily so get back to me if there is something that you absolutely need to be there to start integrating.
One question... On ETMv4, is TRCAUTHSTATUS needed? It was in your document but it's not listed below.
Many thanks for the detailed information, it was very useful. Mathieu
On 6 November 2015 at 10:19, Mike Leach Mike.Leach@arm.com wrote:
Hi Mathieu,
I've been revisiting the requirements for the trace configuration information perf needs to pass through to the report/decode process as we discussed in the IRC chat on Wednesday.
Considering that the decode library is designed to be very generic, perf has more narrow requirements as it is focussed in A-profile instruction trace.
ETMv4.
TRCCONFIGR => Strictly speaking there are no values in TRCCONFIGR that need passing forwards. If we assume that the trace is 100% valid (i.e. not corrupted in anyway) then the dynamic elements of TRCCONFIGR are supplied in the packets - as Al says, ETMv4 is largely self describing.
However at present the packet processor front end (which splits the stream into discrete packets) does use values in TRCCONFIGR to check that the packet stream is consistent - e.g. throwing errors if timestamp packets are seen when TS is not enabled, if ContexdID packets are seen when ContextID trace is disabled etc,etc. This is done as it is designed to be used in a wider variety of situations than just instruction trace decode in perf. Additionally, having the programmed value of the registers allows us to force sync the decoder to a given offset in the trace stream - this matches functionality present in Intel libipt.
Cortex-A profile will never emit data trace so the bits we really need are:- QE[14:13], TS[11], VMID[7], CID[6], CCI[4];
RS[12] is a nice to have as it saves a bit of work if we know if we are using a return stack - though we can assume there is one if IDR0 indicates it is implemented.
What I am considering doing is allowing a dynamic version of the packet processor to run, removing the consistency checks and removing the need for perf to pass on the CONFIGR in ETMv4 at all - though this will be a future version as it is not at all current. In this case the perf report could configure the decoder with CONFIGR as 0, set the decoder to run in dynamic mode and the relevant information will be updated automatically.
TRCTRACEIDR - as you say this can be effectively RO from perf perspective - the decoder simply needs the ID value to identify the correct data stream from the frame demuxer and allows the generic output elements to be tagged with this ID value.
TRCIDR0-2 - needed for etm features, etm arch version, size of elements TRCIDR8 - needed for max speculation depth. These are all RO so the full value should not be an issue.
TRCIDR9-13 - not needed - only required for data trace.
ETMv3/PTM
ETMCR => value directly affects the format of the incoming trace, so ideally full value is needed, though perf report could synthesize the value given the relevant bit values. This data cannot be determined from the trace so a value is needed.
Bits needed for instruction trace:- VMID[30], TS[28], CID[15:14], CycleAcc[12] If these features are passed through, perf report can synthesize the correct value to configure the decoder.
RO regs:- ETMIDR, ETMCCER, "ETMTRACEID" values as agreed.
Regards
Mike
Mike Leach +44 (0)1254 893911 (Direct) Principal Engineer +44 (0)1254 893900 (Main) Arm Blackburn Design Centre +44 (0)1254 893901 (Fax) Belthorn House Walker Rd mailto:mike.leach@arm.com Guide Blackburn BB1 2QE
-----Original Message----- From: CoreSight [mailto:coresight-bounces@lists.linaro.org] On Behalf Of Al Grant Sent: 04 November 2015 17:59 To: Mathieu Poirier; coresight@lists.linaro.org Subject: RE: Collection of metadata in a multi-session environment
When looking at the required registers (once again, please refer to Mike's document, section 4.3.1 to 4.3.3) a lot of them are RO:
ETMv3/PTM: ETMIDR ETMv4: TRCIDR[0, ..., 13], TRCAUTHSTATUS
Some are RW but can easily be made RO when using the CS framework from Perf:
ETMv3/PTM: ETMTRACEID ETMv4: TRCTRACEIDR
In fact I remember having this conversation with Mike in San Francisco and nobody would cry if we were to let the framework decide the traceIDs configured on the tracers.
Anything that is RO can still be retrieved using the current sysFS driven
method
since their values don't change. That leaves us with the other RW registers:
ETMv3/PTM: ETMCR, ETMCCR ETMv4: TRCCONFIGR
The question is, what information is needed from these registers in order to
do
trace decoding?
I believe it was all there in cs-decode.pdf (attached for those who haven't seen it). Any omissions, please let me know.
I don't believe ETMCCR is a RW register.
Is there anything in there that is coming from the HW that can't be deduced otherwise?
There are no changes coming from the HW that haven't been programmed in at some point by SW (or by extneral debug but that's a separate issue). So your problem is limited to
one-time discovery of the configuration
tracking software updates to the registers where these are not also tracked
by sync packets.
So for perf you would want to emit some kind of "configuration changed" event if you notice a configuration change that isn't discoverable through sync packets. The good news is in ETMv4 much more of the changes are tracked in sync packets. You can probably emit that at the end of the trace stream since the trace configuration isn't going to change mid-stream. The key thing is to have the perf subsystem emit that event before it allows any more changes to the configuration for that core, not wait for the user tool to (ages later) query the configuration via sysfs.
Al
From what I understand we are talking about user configurable options that could be communicated to the trace decoding
library
using other means than conveying the whole registers. Using a bitmap is the first option that comes to mind, i.e, if cycle accurate tracing is configured, bit
X
is set.
With the above in mind I need you guys to help me identify the mandatory information in those tracer registers that is needed for trace decoding. Once again we can discuss this further in our meeting tomorrow.
Thanks, Mathieu _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
One question... On ETMv4, is TRCAUTHSTATUS needed? It was in your document but it's not listed below.
The only reason to capture TRCAUTHSTATUS in trace metadata would be to indicate whether Secure trace was permitted and hence why the trace might or might not contain traces of Secure code. For decoding system-wide trace this might be an issue, for decoding trace captured by Linux it should never be. As Mike says, you might want to keep this anyway so the decoder can self-check itself - if it sees trace apparently in Secure state when TRCAUTHSTATUS says this isn't permitted then it knows it's gone wrong.
If Linux is running Non-Secure then any trace captured by Linux should not include Secure code - this would be a security exposure (you'd find it impossible to decode to opcode level anyway anyway without the image). There may be Secure code executing in response to interrupts etc. but you just can't know about it. (If you find you can do this on Juno that's a bug... I'm sure you can't do it on vendor SoCs.)
Al
Many thanks for the detailed information, it was very useful. Mathieu
On 6 November 2015 at 10:19, Mike Leach Mike.Leach@arm.com wrote:
Hi Mathieu,
I've been revisiting the requirements for the trace configuration information
perf needs to pass through to the report/decode process as we discussed in the IRC chat on Wednesday.
Considering that the decode library is designed to be very generic, perf has
more narrow requirements as it is focussed in A-profile instruction trace.
ETMv4.
TRCCONFIGR => Strictly speaking there are no values in TRCCONFIGR that
need passing forwards. If we assume that the trace is 100% valid (i.e. not corrupted in anyway) then the dynamic elements of TRCCONFIGR are supplied in the packets - as Al says, ETMv4 is largely self describing.
However at present the packet processor front end (which splits the stream
into discrete packets) does use values in TRCCONFIGR to check that the packet stream is consistent - e.g. throwing errors if timestamp packets are seen when TS is not enabled, if ContexdID packets are seen when ContextID trace is disabled etc,etc. This is done as it is designed to be used in a wider variety of situations than just instruction trace decode in perf. Additionally, having the programmed value of the registers allows us to force sync the decoder to a given offset in the trace stream - this matches functionality present in Intel libipt.
Cortex-A profile will never emit data trace so the bits we really need are:- QE[14:13], TS[11], VMID[7], CID[6], CCI[4];
RS[12] is a nice to have as it saves a bit of work if we know if we are using a
return stack - though we can assume there is one if IDR0 indicates it is implemented.
What I am considering doing is allowing a dynamic version of the packet
processor to run, removing the consistency checks and removing the need for perf to pass on the CONFIGR in ETMv4 at all - though this will be a future version as it is not at all current. In this case the perf report could configure the decoder with CONFIGR as 0, set the decoder to run in dynamic mode and the relevant information will be updated automatically.
TRCTRACEIDR - as you say this can be effectively RO from perf perspective -
the decoder simply needs the ID value to identify the correct data stream from the frame demuxer and allows the generic output elements to be tagged with this ID value.
TRCIDR0-2 - needed for etm features, etm arch version, size of elements TRCIDR8 - needed for max speculation depth. These are all RO so the full value should not be an issue.
TRCIDR9-13 - not needed - only required for data trace.
ETMv3/PTM
ETMCR => value directly affects the format of the incoming trace, so ideally
full value is needed, though perf report could synthesize the value given the relevant bit values. This data cannot be determined from the trace so a value is needed.
Bits needed for instruction trace:- VMID[30], TS[28], CID[15:14], CycleAcc[12] If these features are passed through, perf report can synthesize the correct value to configure the
decoder.
RO regs:- ETMIDR, ETMCCER, "ETMTRACEID" values as agreed.
Regards
Mike
Mike Leach +44 (0)1254 893911 (Direct) Principal Engineer +44 (0)1254 893900 (Main) Arm Blackburn Design Centre +44 (0)1254 893901 (Fax) Belthorn House Walker Rd mailto:mike.leach@arm.com Guide Blackburn BB1 2QE
-----Original Message----- From: CoreSight [mailto:coresight-bounces@lists.linaro.org] On Behalf Of Al Grant Sent: 04 November 2015 17:59 To: Mathieu Poirier; coresight@lists.linaro.org Subject: RE: Collection of metadata in a multi-session environment
When looking at the required registers (once again, please refer to Mike's document, section 4.3.1 to 4.3.3) a lot of them are RO:
ETMv3/PTM: ETMIDR ETMv4: TRCIDR[0, ..., 13], TRCAUTHSTATUS
Some are RW but can easily be made RO when using the CS framework from Perf:
ETMv3/PTM: ETMTRACEID ETMv4: TRCTRACEIDR
In fact I remember having this conversation with Mike in San Francisco and nobody would cry if we were to let the framework decide the traceIDs configured on the tracers.
Anything that is RO can still be retrieved using the current sysFS driven
method
since their values don't change. That leaves us with the other RW
registers:
ETMv3/PTM: ETMCR, ETMCCR ETMv4: TRCCONFIGR
The question is, what information is needed from these registers in order to
do
trace decoding?
I believe it was all there in cs-decode.pdf (attached for those who haven't seen it). Any omissions, please let me know.
I don't believe ETMCCR is a RW register.
Is there anything in there that is coming from the HW that can't be deduced otherwise?
There are no changes coming from the HW that haven't been programmed in at some point by SW (or by extneral debug but that's a separate issue). So your problem is limited to
one-time discovery of the configuration
tracking software updates to the registers where these are not
also tracked by sync packets.
So for perf you would want to emit some kind of "configuration changed" event if you notice a configuration change that isn't discoverable through sync packets. The good news is in ETMv4 much more of the changes are tracked in sync packets. You can probably emit that at the end of the trace stream since the trace configuration isn't going to change mid-stream. The key thing is to have the perf subsystem emit that event before it allows any more changes to the configuration for that core, not wait for the user tool to (ages later) query the configuration via sysfs.
Al
From what I understand we are talking about user configurable options that could be communicated to the trace decoding
library
using other means than conveying the whole registers. Using a bitmap is the first option that comes to mind, i.e, if cycle accurate tracing is configured, bit
X
is set.
With the above in mind I need you guys to help me identify the mandatory information in those tracer registers that is needed for trace decoding. Once again we can discuss this further in our meeting
tomorrow.
Thanks, Mathieu _______________________________________________ CoreSight mailing list CoreSight@lists.linaro.org https://lists.linaro.org/mailman/listinfo/coresight
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
________________________________
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-
Al Grant -
Mathieu Poirier -
Mike Leach