Hi Mathieu,
On 16.08.2018 21:17, Mathieu Poirier wrote:
On Thu, 16 Aug 2018 at 08:04, Tomasz Nowicki tnowicki@caviumnetworks.com wrote:
Hello Mathieu,
On 14.08.2018 18:08, Mathieu Poirier wrote:
Good morning Tomasz,
On Tue, 14 Aug 2018 at 09:20, tn Tomasz.Nowicki@caviumnetworks.com wrote:
On 13.08.2018 18:47, Mathieu Poirier wrote:
On Wed, 8 Aug 2018 at 01:59, Tomasz Nowicki tnowicki@caviumnetworks.com wrote:
Hi Mathieu,
It's been a while but I am back to Coresight.
Let me remind my setup and the issue I am struggling with now.
Kernel baseline: https://github.com/Linaro/perf-opencsd (perf-opencsd-v4.16) OpenCSD: https://github.com/Linaro/OpenCSD.git (master)
The simplest Coresight components path I used as a start point: ETMv4.1 -> TDR -> FUNNEL -> ETF
As I mentioned TDR is built by Cavium and it was added to aggregate 128 inputs into one output rather than cascading funnels. TDR has its own driver just to keep path connected in Linux Coresight framework.
Here is how I catch some trace data: sudo perf record -C 0 -e cs_etm/@etf0/ --per-thread test_app
The above command line tells perf to trace everything that is happening on CPU0 for as long as "test_app" is executing. In this case the "--per-thread" option is ignored. This is called a CPU-wide trace scenario and is currently not supported for CS (I am currently working on it).
If you want to make sure "test_app" executes on CPU0 and that you trace just that you will need to use the "taskset" utility:
sudo perf record -e cs_etm/@etf0/ --per-thread taskset 0x1 test_app
My apologise, I used 'taskset' part in my command but forgot to mention. Anyway, thank you for clarifying that CPU-wide trace scenario is not supported for now.
An alternative to the above would be to CPU-hotplug out CPU128-255 while you are testing.
Yes, this helped to setup path between source and sink. Finally I am able to decode data with: # sudo perf record -e cs_etm/@etf0/ --per-thread test_app # sudo perf report --stdio
Perfect.
Above command traces everything related to test_app and all I can see in 'perf report' are userspace symbols. Is it possible to trace kernel side? I have tried to narrow down data using '-e cs_etm/@etf0/k' filter but got nothing in 'perf report'.
The ETF buffer is a pretty small one. As such it may be wrapping around and all you get is the user space part. Do you have an ETR on the platform? If so I suggest to download the latest coresight next branch [1] and try with a bigger buffer.
Yes, I have tried ETR with the same results. But it turns out ETMv4 driver enables EL1_NS for kernel tracing which is not true for VHE enabled system where kernel is running on EL2_NS. Bellow patch helps to fix this for me:
diff --git a/drivers/hwtracing/coresight/coresight-etm4x.c b/drivers/hwtracing/coresight/coresight-etm4x.c index f79b0ea85d76..dc22c1d6c7f3 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x.c +++ b/drivers/hwtracing/coresight/coresight-etm4x.c @@ -28,6 +28,7 @@ #include <linux/pm_runtime.h> #include <asm/sections.h> #include <asm/local.h> +#include <asm/virt.h>
#include "coresight-etm4x.h" #include "coresight-etm-perf.h" @@ -617,13 +618,11 @@ static u64 etm4_get_access_type(struct etmv4_config *config) * Bit[13] Exception level 1 - OS * Bit[14] Exception level 2 - Hypervisor * Bit[15] Never implemented - * - * Always stay away from hypervisor mode. */ - access_type = ETM_EXLEVEL_NS_HYP;
if (config->mode & ETM_MODE_EXCL_KERN) - access_type |= ETM_EXLEVEL_NS_OS; + access_type |= is_kernel_in_hyp_mode ? + ETM_EXLEVEL_NS_HYP : ETM_EXLEVEL_NS_OS;
if (config->mode & ETM_MODE_EXCL_USER) access_type |= ETM_EXLEVEL_NS_APP; @@ -881,7 +880,8 @@ void etm4_config_trace_mode(struct etmv4_config *config)
addr_acc = config->addr_acc[ETM_DEFAULT_ADDR_COMP]; /* clear default config */ - addr_acc &= ~(ETM_EXLEVEL_NS_APP | ETM_EXLEVEL_NS_OS); + addr_acc &= ~(ETM_EXLEVEL_NS_APP | ETM_EXLEVEL_NS_OS | + ETM_EXLEVEL_NS_HYP);
/* * EXLEVEL_NS, bits[15:12] @@ -892,7 +892,8 @@ void etm4_config_trace_mode(struct etmv4_config *config) * Bit[15] Never implemented */ if (mode & ETM_MODE_EXCL_KERN) - addr_acc |= ETM_EXLEVEL_NS_OS; + addr_acc |= is_kernel_in_hyp_mode ? + ETM_EXLEVEL_NS_HYP : ETM_EXLEVEL_NS_OS; else addr_acc |= ETM_EXLEVEL_NS_APP;
Thanks, Tomasz