Hi Kim,

I grabbed a Juno-r2 for an hour while the validation team where looking in the other direction :).....

On 15 March 2017 at 23:27, Kim Phillips <kim.phillips@arm.com> wrote:
On Wed, 15 Mar 2017 19:04:38 +0000
Mike Leach <mike.leach@linaro.org> wrote:

> I'm running r1 - the cs on r1 and r2 are identical - the only difference is
> the A57 A72 change. So I see cpu 0 and 1 as the A57, and 2-5 as the A53, I
> would expect the same on r2.

not so, according to this:

https://community.arm.com/dev-platforms/b/documents/posts/faq-how-do-i-change-which-cpus-are-released-out-of-reset-and-which-one-is-the-primary-for-baremetal-development

r1 boots off an A57, whereas r2 boots off an A53, leading to different
cpu assignments in Linux (cpu0 is the boot cpu).


​Well that's unnecessarily confusing - but confirmed on my R2 too...​

[    0.000000] Booting Linux on physical CPU 0x100
[    0.000000] Linux version 4.11.0-rc1-gc508373-dirty (mleach@ubu-16-04-vm) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11) ) #19 SMP PREEMPT Wed Mar 15 14:41:05 GMT 2017

root@linaro-developer:/home/mleach/csal/demos/juno_demo_setup# for i in 0 1 2 3 4 5; do cat /sys/devices/system/cpu/cpu$i/regs/identification/midr_el1; done
0x00000000410fd033
0x00000000410fd080
0x00000000410fd080
0x00000000410fd033
0x00000000410fd033
0x00000000410fd033

 
Either way, if I use taskset -c 0 - for the A53 - I still see the
failure, and on first try with gcc4.9-built binaries:

root@juno:~# dmesg | grep gcc
[    0.000000] Linux version 4.11.0-rc1-g0d15341 (kim@dupont) (gcc version 4.9.4 20151028 (prerelease) (Linaro GCC 4.9-2016.02) ) #3 SMP PREEMPT Tue Mar 14 22:41:34 CDT 2017
root@juno:~# taskset -c 0 ./perf record -e cs_etm/@20070000.etr/u --per-thread taskset -c 0 uname
[ 1727.185799] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.192885] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.199278] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.204460] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.210848] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.217714] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.224183] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.230666] coresight-tmc 20010000.etf: TMC disabled
[ 1727.235585] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.242049] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.249217] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.254992] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.262065] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.268456] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.273636] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.280022] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.286840] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.293308] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.299790] coresight-tmc 20010000.etf: TMC disabled
[ 1727.304708] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.311171] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.318339] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.324087] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.331160] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.337551] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.342730] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.349116] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.369918] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.376387] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.382871] coresight-tmc 20010000.etf: TMC disabled
[ 1727.387790] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.394254] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.401422] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.407207] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.414281] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.420672] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.425851] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.432237] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.439054] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.445522] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.452006] coresight-tmc 20010000.etf: TMC disabled
[ 1727.456924] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.463388] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.470556] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.476382] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.483456] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.489847] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.495027] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.501413] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.508244] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.514713] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.521196] coresight-tmc 20010000.etf: TMC disabled
[ 1727.526114] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.532577] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.539745] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.545526] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.552600] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.558990] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.564169] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.570556] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.577390] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.583858] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.590340] coresight-tmc 20010000.etf: TMC disabled
[ 1727.595259] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.601722] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.608890] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.614653] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.621727] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.628117] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.633297] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.639683] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.646495] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.652963] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.659446] coresight-tmc 20010000.etf: TMC disabled
[ 1727.664364] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.670828] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.677995] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.683740] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.690813] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.697203] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.702383] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.708769] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.715581] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.722049] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.728532] coresight-tmc 20010000.etf: TMC disabled
[ 1727.733450] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.739915] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.747083] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.752831] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.759904] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.766294] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.771473] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.777860] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.785106] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.791576] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.798059] coresight-tmc 20010000.etf: TMC disabled
[ 1727.802978] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.809442] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.816610] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.822396] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.829470] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.835860] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.841040] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.847427] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
[ 1727.854241] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.860709] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.867192] coresight-tmc 20010000.etf: TMC disabled
[ 1727.872111] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.878575] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.885742] coresight-tmc 20070000.etr: TMC-ETR disabled
[ 1727.891503] coresight-replicator-qcom 20120000.replicator: REPLICATOR enabled
[ 1727.898577] coresight-funnel 20150000.funnel: FUNNEL inport 0 enabled
[ 1727.904968] coresight-tmc 20010000.etf: TMC-ETF enabled
[ 1727.910147] coresight-funnel 20040000.funnel: FUNNEL inport 1 enabled
[ 1727.916534] coresight-funnel 230c0000.funnel: FUNNEL inport 0 enabled
Linux
[ 1727.936635] coresight-funnel 230c0000.funnel: FUNNEL inport 0 disabled
[ 1727.943106] coresight-funnel 20040000.funnel: FUNNEL inport 1 disabled
[ 1727.949589] coresight-tmc 20010000.etf: TMC disabled
[ 1727.954507] coresight-funnel 20150000.funnel: FUNNEL inport 0 disabled
[ 1727.960971] coresight-replicator-qcom 20120000.replicator: REPLICATOR disabled
[ 1727.968139] coresight-tmc 20070000.etr: TMC-ETR disabled
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote

...and the hard lock-up.


​So, running the stuff as usual, gcc 6.x built kernel and perf, as root and as my user login via sudo, various cores

​======================================================
root@linaro-developer:/home/mleach/perf-tools# taskset -c 2 ./perf record -e cs_etm/@20070000.etr/u --per-thread taskset -c 2 uname
Linux
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.077 MB perf.data ]
root@linaro-developer:/home/mleach/perf-tools# taskset -c 2 ./perf record -e cs_etm/@20070000.etr/u --per-thread taskset -c 2 uname
Linux
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.076 MB perf.data ]
root@linaro-developer:/home/mleach/perf-tools# taskset -c 0 ./perf record -e cs_etm/@20070000.etr/u --per-thread taskset -c 0 uname
Linux
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.076 MB perf.data ]
root@linaro-developer:/home/mleach/perf-tools#
root@linaro-developer:/home/mleach/perf-tools#
mleach@linaro-developer:~/perf-tools$ sudo taskset -c 0 ./perf record -e cs_etm/@20070000.etr/u --per-thread taskset -c 0 uname
Linux
[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.077 MB perf.data ]
​===========================================================


​All good - at least on this R2.

The only issues I have encountered with this board and kernel are that I get an error on the PCI bus during boot, and the rear network cable does not work - I have to use the front one. Could be a new kernel issue or a h/w fault on my board. Either way CoreSight is working normally

Regards

Mike




 
Thanks,

Kim

p.s. please inline & trim responses, thanks.

> On Wed, 15 Mar 2017 at 18:40, Kim Phillips <kim.phillips@arm.com> wrote:
>
> > On Wed, 15 Mar 2017 15:39:47 +0000
> > Mike Leach <mike.leach@linaro.org> wrote:
> >
> > > Checked gcc version and ran perf a few times
> > > ==============================================================
> > > ​root@linaro-developer:/home/mleach/perf-tools# strings -a perf | grep
> > > "GCC: ("
> > > GCC: (Linaro GCC 6.2-2016.11) 6.2.1 20161016
> > > root@linaro-developer:/home/mleach/perf-tools#
> > >
> > > root@linaro-developer:/home/mleach/perf-tools# taskset -c 2 ./perf
> > record
> > > -e cs_etm/@20070000.etr/u --per-thread taskset -c 2 uname
> > > Linux
> > > [ perf record: Woken up 2 times to write data ]
> > > [ perf record: Captured and wrote 0.077 MB perf.data ]
> > > root@linaro-developer:/home/mleach/perf-tools#
> > > root@linaro-developer:/home/mleach/perf-tools# taskset -c 2 ./perf
> > record
> > > -e cs_etm/@20070000.etr/u --per-thread taskset -c 2 uname
> > > Linux
> > > [ perf record: Woken up 1 times to write data ]
> > > [ perf record: Captured and wrote 0.077 MB perf.data ]
> > > root@linaro-developer:/home/mleach/perf-tools# taskset -c 2 ./perf
> > record
> > > -e cs_etm/@20070000.etr/u --per-thread taskset -c 2 uname
> > > Linux
> > > [ perf record: Woken up 2 times to write data ]
> > > [ perf record: Captured and wrote 0.077 MB perf.data ]
> > > root@linaro-developer:/home/mleach/perf-tools#
> > > ==============================================================
> > >
> > > ​Worked fine.​
> >
> > I like to include dmesg | grep gcc because sometimes I myself make
> > sloppy mistakes copying the kernel image around, but OK, I should have
> > said I gave it a lot of time before runs: several minutes.
> >
> > > ​so....I'm back on the funnel / trace amount and core differences.
> > > taskset -c 2 -> selects cpu 2 -> A53(0) on both juno-r1 and juno-r2. So
> > why
> > > on your system is the funnel for the A72 cluster (cpu 0-1) being
> > activated.
> > > if I use taskset -c 0 (or 1) then I get the A57 cluster funnel activated
> > as
> > > expected. (2-5 gets the A53 cluster funnel)
> > >
> > > This seems to be indicative of some .dts issue on your system / r2.
> > Nothing
> > > immediately obvious in the .dts that I can see.
> >
> > Using the device tree compiler to show all the differences, like so:
> >
> > $ dtc -I dtb -O dts -s juno.dtb > juno.dts
> > $ dtc -I dtb -O dts -s juno-r2.dtb > juno-r2.dts
> > $ diff -u juno.dts juno-r2.dts
> >
> > We see the cpus have been renumbered such that on r2, taskset -c 2
> > references an A-72 (it actually happens between r0 and r1, diff r1 r2
> > is a simple a57->a72 rename), and phandles that coresight components
> > reference also change, e.g.,  the 0x3d phandle that the etf@20140000
> > references used to point to the old sysctl@020000, but that node's
> > phandle gets changed to 0x3c.  Below is the full diff;  is a dts
> > patch for coresight on r1/r2 in order?
> >
> > I still think there's more to it since you were able to see the
> > inconsistent behaviour between two runs problem on r0.
> >
> > Thanks,
> >
> > Kim



--
Mike Leach
Principal Engineer, ARM Ltd.
Blackburn Design Centre. UK