On Wed, Apr 24, 2019 at 06:42:18PM +0000, Al Grant wrote:
This is a frequent topology for CoreSight - the intervening ETF smooths out incoming trace which can be quite bursty depending on the processes being run on the cores.
Some ETF/ETBs have big enough buffers that they can capture significant amounts of trace in circular-buffer mode (with readout via the RRD register). This is useful in early stages of bringup when the ETR's path to main memory via the system interconnect may not have been established, but may also be useful to capture trace without any probe-effect on the system bus.
So it would be useful if perf test could try all possibilities on the path to the ETR - i.e. if there are ETBs on the path, put those into circular buffer mode and test those too.
One system I'm using right now has one CPU with CPU->ETB->ETB->ETR. Each ETB has an 8K buffer. So in fact it is 1:1 but with three options for where trace can be captured.
Off-list we have been discussing generic ways to specify the sink in which trace is collected, e.g. by a standard way of enumerating the possible sinks as seen from a CPU.
For example we could number the sinks seen by each CPU according to a walk of its outgoing trace path, with ETB numbered before ETR if they occur at the same level (or the other way round if you prefer).
This would cope with multiple topologies, in a generic way. It's quite similar to the way caches are represented in sysfs. If you want to test all possibilities you simply iterate through the sinks until perf tells you there are no more. For physically partitioned trace fabrics (including multi-socket) this is much simpler than specifying trace sinks individually, since an individual sink is only valid for a subset of cores.
Thanks a lot for suggestions, Al.
Combined Al's suggestions in this email and Mathieu's suggestions in another email, I'd like to summary two different testing methodologies:
- Source oriented testing (Al's suggestion)
Source oriented testing is based on every source (now only refers to CPU) and test its all possible sinks.
Pros: the testing is very sanity and can test all possibilities from sources to sinks.
Cons: the difficult thing is to find a general (and simple) method to traverse the pathes from sources to sinks.
Suzuki's patch '36/36: [RFC] coresight: Expose device connections via sysfs' will be helpful for analysis the path between one CPUs to its sinks.
Another option is to create sysfs nodes (as Al has mentioned) like '/sys/devices/system/cpu/cpuX/coresight/sinkX', so this can give out the CoreSight topology from the CPU's view. But I think we should give this low priority (we can try this method only if there have block issues with Suzuki's patch 36/36).
- Sink oriented testing (Mathieu's suggestion and this is also the method in my patch)
Sink oriented testing is to iterate every sink and generate trace data stream to it.
Pros: usually the testing will take less iterations than source oriented testing (at least this is true for the hardwares in my hand);
The sink cannot receive any trace data if the testing task doesn't run on the 'right' CPUs. So how we needs to know what's the connected CPUs to the specific sink, one possible solution is to create cpumask nodes for every sink, like below:
/sys/bus/coresight/devices/XXXXXXXX.[etr|etf|etb]/cpumask
Cons: the testing might be not very complete, it might not cover some corner cases.
So now I am bias to use source oriented testing as Al suggested, it's would be general enough and cover current & future topologies. If you have any furthermore comments, please let me know.
We do predict that future topologies will move towards a 1:1 relationship between ETM and ETR - with no STF between.
Meanwhile we need to support current silicon.
Totally agree.
Thanks! Leo Yan