On Tue, Jul 23, 2019 at 9:12 AM Leo Yan <leo.yan@linaro.org> wrote:

Hi Jeremy,

On Mon, Jul 22, 2019 at 11:42:57AM +0800, Jeremy Ng wrote:
> Hi Al,
>
> On Fri, Jul 19, 2019 at 5:16 PM Al Grant <Al.Grant@arm.com> wrote:
> >
> > Hi,
> >
> >
> >
> > CSAL can be tricky to use as a discovery tool. I’ve now added a Python
> tool
> >
> > which might be easier to use, especially if you have to work around bus
> >
> > hangs etc. Please see csscan.py here:
> >
> >
> >
> > https://github.com/ARM-software/CSAL/tree/master/coresight-tools
> >
> >
> >
> > You can do a scan using
> >
> >
> >
> > python csscan.py <rom-base-address>
> >
> >
> >
> > As well as listing configuration details for the devices, it can also
> discover
> >
> > ATB topology (not CTI yet, sorry) and report on what state the devices are
> >
> > currently in.
> >
> >
> >
> > There’s some explanation on the discovery process:
> >
> >
> >
> >
> https://github.com/ARM-software/CSAL/blob/master/coresight-tools/discovery.md
> >
> >
> >
> > Al
>
> Thank you for your email. My apologies for the late reply as I only have
> access to my work on weekdays.
>
> Anyway, I have tried the coresight-tools as updated on CSAL repo.
>
> Unfortunately, the device continues to hang at the same memory mapped
> address.
>
> Just to clarify the steps I have taken:
>
> 1. Since hikey960 is running on AOSP, Linux kernel 4.9, python is not
> installed
> 2. I installed python to the device by installing Termux APK, access
> internet
> and downloaded python through `pkg install python`
>
> 3. Next, i get access to root through adb and added a path to termux's bin
> files
> through `PATH=$PATH:/data/data/com.termux/files/usr/bin`
> 4. I ran `python /data/coresight-tools/csscan.py 0xEC000000`. Below shows
> the
> output for csscan and csls to show the similarites between the outputs.
>
> csscan.py
> ========
>
> hikey960:/ # python /data/coresight-tools/csscan.py 0xEC000000
>
>
>
> @0xec000000 0x327 0x000 r0.0 ROM table
> @0xec030000 0x000 0x000 r0.0 ROM table
> @0xec031000 0x23b 0x908 r2.0 funnel in-ports:7
> @0xec032000 0x23b 0x912 r4.0 port TPIU
> @0xec033000 0x23b 0x961 r1.0 buffer TMC:ETR
> memwidth:64 wb:8
> @0xec034000 0x23b 0x906 r4.0 CTI channels:4
> triggers:8
> @0xec035000 0x23b 0x963 r0.0 STM Arm STM rev1 ports:65536
> @0xec036000 0x23b 0x961 r1.0 fifo TMC:ETF
> size:4096 memwidth:128
> @0xec037000 0x23b 0x101 r1.0 CoreSight timestamp generator
> @0xec038000 0x23b 0x101 r1.0 CoreSight timestamp generator
> >--- device crash ---<
>
> csls
> ===
> hikey960:/ # csls
> ** CSLS: listing CoreSight config...
> ** CSLS: Using default ROM address 0xEC000000
> 00EC031000: 2.1 908 00000037 00/0F type= 4 - LINK [FUNNEL: 7 in ports]
> 00EC032000: 1.1 912 000000A0 00/0F type= 8 - SINK PORT [TPIU]
> 00EC033000: 1.2 961 00001B40 00/0F type= 7 - SINK BUFFER(ETR r/w size: 4K)
> [TMC: ETR configuration]
> 00EC034000: 4.1 906 00040800 00/0F type=10 - CTI
> 00EC035000: 3.6 963 00010000 00/0F type= 3 - SOURCE SWSTIM(65536) [STM ext
> ports only, 64-bit, 128 masters]
> 00EC036000: 2.3 961 00000480 00/0F type= 6 - LINK SINK BUFFER(4K) [TMC: ETF
> configuration]
> 00EC037000: type=13 O TIMESTAMP
> 00EC038000: type=13 O TIMESTAMP
> >--- device crash ---<

Sorry for some late ...

Except the CPUIdle should be disabled, we also need to enable
CoreSight clock; Could you try below patch for clock driver?
The patch sets the CoreSight clock as critical so the kernel will turn
on the clock by default:

diff --git a/drivers/clk/hisilicon/clk-hi3660.c b/drivers/clk/hisilicon/clk-hi3660.c
index 5b3ad26dcc77..e2243c761871 100644
--- a/drivers/clk/hisilicon/clk-hi3660.c
+++ b/drivers/clk/hisilicon/clk-hi3660.c
@@ -19,7 +19,7 @@ static const struct hisi_fixed_rate_clock hi3660_fixed_rate_clks[] = {
{ HI3660_CLK_PPLL2, "clk_ppll2", NULL, 0, 2880000000UL, },
{ HI3660_CLK_PPLL3, "clk_ppll3", NULL, 0, 1290000000, },
{ HI3660_CLK_SCPLL, "clk_scpll", NULL, 0, 245760000, },
- { HI3660_PCLK, "pclk", NULL, 0, 20000000, },
+ { HI3660_PCLK, "pclk", NULL, CLK_IS_CRITICAL, 20000000, },
{ HI3660_CLK_UART0_DBG, "clk_uart0_dbg", NULL, 0, 19200000, },
{ HI3660_CLK_UART6, "clk_uart6", NULL, 0, 19200000, },
{ HI3660_OSC32K, "osc32k", NULL, 0, 32764, },

I saw you said cannot download CoreSight DT binding patch for Hikey960,
so enclose the patch for reference.

Thanks,
Leo Yan

> I am also aware that CPUIdle must be switched off. I have a bash script that
> deactivates the CPUIdle settings, on top of the suggestions Leo Yan has made
> in the previous email.
>
> This is the script used:
> # For A53 Cluster
> for i in 0 1 2 3; do
> for j in 0 1 2 ; do
> adb shell "echo 1 > /sys/devices/system/cpu/cpu$i/cpuidle/state$j/disable";
> adb shell "echo 'cpu$i state$j disabled'";
> done;
> done
>
> # For A73 Cluster
> for i in 4 5 6 7; do
> for j in 0 1 2 ; do
> adb shell "echo 1 > /sys/devices/system/cpu/cpu$i/cpuidle/state$j/disable";
> adb shell "echo 'cpu$i state$j disabled'";
> done;
> done
>
> adb shell "exec 3<> /dev/cpu_dma_latency; echo 0 >&3"
>
> Hopefully I have given sufficient details and perhaps a solution
> can be derived.
>
> Once again, thank you very much for the tools and assistance!
>
> Yours Sincerely,
> Jeremy
>
> >
> >
> >
> >
> >
> > From: CoreSight <coresight-bounces@lists.linaro.org> On Behalf Of Jeremy
> Ng
> > Sent: 18 July 2019 04:54
> > To: leo.yan@linaro.org
> > Cc: coresight@lists.linaro.org
> > Subject: Re: Coresight Memory Mapped Addresses in Hikey960
> >
> >
> >
> > Hi Leo Yan,
> >
> > On Wed, Jul 17, 2019 at 6:04 PM Leo Yan <leo.yan@linaro.org> wrote:
> > >
> > > Hi Jeremy,
> > >
> > > On Wed, Jul 17, 2019 at 05:53:28PM +0800, Jeremy Ng wrote:
> > > > Dear Leo Yan,
> > > >
> > > > I will like to ask if there are any better documentations that detail
> the
> > > > memory mapped addresses for Coresight registers in Hikey960.
> > > >
> > > > I am currently referring to Chapter 2-9-1 in
> > > > http://mirror.lemaker.org/HiKey960_SoC_Reference_Manual.pdf. This is
> all
> > > > the information about coresight addresses that I could retrieve:
> > > >
> > > > 0xEC000000 0xED7FFFFF 24M CSSYS_APB
> > >
> > > As I know, the CoreSight registers address mapping is not contained
> > > in this pdf file.
> >
> > Are there any other pdf files that perhaps I could have access to with
> > all the addresses properly detailed?
> >
> >
> >
> > Just to improve understanding of what I am trying to accomplish, I will
> include
> >
> > a snippet of the code from CSAL.
> >
> >
> >
> > This is a snippet of the code that I am currently trying to write for
> hikey960,
> > in hope that I might be able to reproduce the results for the other
> example boards,
> > such as Juno, Snowball or TC2:
> >
> > static int do_registration_juno(struct cs_devices_t *devices)
> > {
> > enum { A53_0, A53_1, A53_2, A53_3, A57_0, A57_1 };
> > cs_device_t rep, etr, etf, fun_main, fun_a53, fun_a57, stm, tpiu,
> > sys_cti;
> > cs_device_t r1_fun_scp, r1_fun_common, r1_etf_scp;
> > #ifdef LIB_DEVICE_UNSUPPORTED
> > cs_device_t r1_cti_2, ela_a53, ela_a57;
> > #endif
> > int i;
> >
> > if (registration_verbose)
> > printf("CSDEMO: Registering CoreSight devices...\n");
> > cs_register_romtable(0x20000000);
> >
> > if (registration_verbose)
> > printf("CSDEMO: Registering CPU affinities...\n");
> >
> > /* CTI affinities */
> > cs_device_set_affinity(cs_device_register(0x22020000), A57_0);
> > cs_device_set_affinity(cs_device_register(0x22120000), A57_1);
> > cs_device_set_affinity(cs_device_register(0x23020000), A53_0);
> > cs_device_set_affinity(cs_device_register(0x23120000), A53_1);
> > cs_device_set_affinity(cs_device_register(0x23220000), A53_2);
> > cs_device_set_affinity(cs_device_register(0x23320000), A53_3);
> >
> > /* PMU affinities */
> > cs_device_set_affinity(cs_device_register(0x22030000), A57_0);
> > cs_device_set_affinity(cs_device_register(0x22130000), A57_1);
> > cs_device_set_affinity(cs_device_register(0x23030000), A53_0);
> > cs_device_set_affinity(cs_device_register(0x23130000), A53_1);
> > cs_device_set_affinity(cs_device_register(0x23230000), A53_2);
> > cs_device_set_affinity(cs_device_register(0x23330000), A53_3);
> >
> > /* ETMv4 affinities */
> > cs_device_set_affinity(cs_device_register(0x22040000), A57_0);
> > cs_device_set_affinity(cs_device_register(0x22140000), A57_1);
> > cs_device_set_affinity(cs_device_register(0x23040000), A53_0);
> > cs_device_set_affinity(cs_device_register(0x23140000), A53_1);
> > cs_device_set_affinity(cs_device_register(0x23240000), A53_2);
> > cs_device_set_affinity(cs_device_register(0x23340000), A53_3);
> >
> > if (registration_verbose)
> > printf("CSDEMO: Registering trace-bus connections...\n");
> >
> > /* funnels in clusters */
> > fun_a57 = cs_device_get(0x220C0000);
> > cs_atb_register(cs_cpu_get_device(A57_0, CS_DEVCLASS_SOURCE), 0,
> > fun_a57, 0);
> > cs_atb_register(cs_cpu_get_device(A57_1, CS_DEVCLASS_SOURCE), 0,
> > fun_a57, 1);
> >
> > fun_a53 = cs_device_get(0x230C0000);
> > cs_atb_register(cs_cpu_get_device(A53_0, CS_DEVCLASS_SOURCE), 0,
> > fun_a53, 0);
> > cs_atb_register(cs_cpu_get_device(A53_1, CS_DEVCLASS_SOURCE), 0,
> > fun_a53, 1);
> > cs_atb_register(cs_cpu_get_device(A53_2, CS_DEVCLASS_SOURCE), 0,
> > fun_a53, 2);
> > cs_atb_register(cs_cpu_get_device(A53_3, CS_DEVCLASS_SOURCE), 0,
> > fun_a53, 3);
> >
> >
> > /*common setup */
> > fun_main = cs_device_get(0x20040000);
> > stm = cs_device_get(0x20100000);
> > etf = cs_device_get(0x20010000);
> > rep = cs_device_get(0x20120000);
> > etr = cs_device_get(0x20070000);
> >
> > tpiu = cs_device_get(0x20030000);
> >
> > .
> >
> > .
> >
> >
> >
> > The addresses stated in the function code for Juno corresponds with the
> official
> >
> > Technical datasheet as found in
> >
> >
> https://www.arm.com/files/pdf/DDI0515D1a_juno_arm_development_platform_soc_trm.pdf
> ,
> >
> > page 3-21.
> >
> >
> >
> > Furthermore, I realized that there are missing programmable addresses in
> Hi3660 datasheet:
> >
> >
> >
> > 0xED800000 0xEFFFFFFF 40M Reserved
> > 0xEC000000 0xED7FFFFF 24M CSSYS_APB
> > 0xE9890000 0xE989FFFF 64K MMC0_NOC_Service_Target
> >
> >
> >
> > As seen in the data above, there is a gap from 0xE98A0000 to 0xEBFFFFFF
> >
> > I am particularly curious whether the coresight address' offset might be
> >
> > consistent across different boards.
> >
> >
> >
> > For example, CS_APB starts at 0x2100_0000 for Juno and CS_ROM is found at
> >
> > 0x2000_0000, 0x0100_0000 apart. I have tried to run `csls` on 0xEB00_0000
> for
> >
> > hikey960 but as with the previous attempts, the device crashes + there
> are less info
> >
> > output when probing from that address.
> >
> >
> >
> > i.e. hikey960:/ # csls
> > ** CSLS: listing CoreSight config...
> > ** CSLS: Using default ROM address 0xEB000000
> >
> > >--- crash ---<
> >
> >
> >
> >
> >
> > >
> > > On the other hand, could you check if below patch is sufficient for
> > > you? In this patch, it enabled CoreSight on Hikey960 and contains the
> > > addresses for CoreSight devices.
> > >
> https://archive.armlinux.org.uk/lurker/message/20190420.140035.74b936ef.en.html
> >
> >
> >
> > Unfortunately, I could not access this webpage from Singapore.
> >
> >
> >
> > On another note, my hikey960 is run on Linux kernel 4.9:
> >
> > hikey960:/ # uname -ar
> >
> > Linux localhost 4.9.176-12953-g7c09ed7b46a4-dirty #15 SMP PREEMPT Wed Jul
> 17 16:53:26 +08 2019 aarch64
> >
> >
> >
> > >
> > > > I will like to check if there are any datasheet for hikey960 that is
> as
> > > > detailed as that for Juno device (link here:
> > > >
> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0515b/CHDIFADJ.html
> > > > ).
> > > >
> > > > This is important to me as I am trying to acquire the device snapshot
> by
> > > > using CSAL API. I am not sure if there are other ways to do this
> since the
> > > > documentation hasn't been exactly straightforward.
> > > >
> > > > The reason for wanting the snapshot is to possibly run openCSD and
> decode
> > > > my trace binary files.
> > > >
> > > > I have followed the CSAL build documentations and ran `csls` on my
> > > > hikey960. The following output is seen but it crashed my device soon
> after:
> > > >
> > > > ```bash
> > > > hikey960:/ # csls
> > > > ** CSLS: listing CoreSight config...
> > > > ** CSLS: Using default ROM address 0xEC000000
> > > > 00EC031000: 2.1 908 00000037 00/0F type= 4 - LINK [FUNNEL: 7 in ports]
> > > > 00EC032000: 1.1 912 000000A0 00/0F type= 8 - SINK PORT [TPIU]
> > > > 00EC033000: 1.2 961 00001B40 00/0F type= 7 - SINK BUFFER(ETR r/w
> size: 4K)
> > > > [TMC: ETR configuration]
> > > > 00EC034000: 4.1 906 00040800 00/0F type=10 - CTI
> > > > 00EC035000: 3.6 963 00010000 00/0F type= 3 - SOURCE SWSTIM(65536)
> [STM ext
> > > > ports only, 64-bit, 128 masters]
> > > > 00EC036000: 2.3 961 00000480 00/0F type= 6 - LINK SINK BUFFER(4K)
> [TMC: ETF
> > > > configuration]
> > > > 00EC037000: type=13 O TIMESTAMP
> > > > 00EC038000: type=13 O TIMESTAMP
> > > > ```
> > > >
> > > > I will also like to understand why it might have crashed my device
> during
> > > > the execution of csls.
> > >
> > > Cool! Thanks for sharing the method.
> > >
> > > Could you try to disable CPUidles for Hikey960? E.g. you could add
> > > 'nohlt' in the kernel command line to prevent CPU idle states. Or you
> > > could use below command:
> > >
> > > # exec 3<> /dev/cpu_dma_latency; echo 0 >&3
> >
> >
> >
> > Previously, I disabled cpu_idle from linux kernel menuconfig and
> reflashed the kernel.
> >
> > I have also made a bash script to temporarily disable cpuidle. the script
> is:
> >
> >
> >
> > #!/bin/sh
> >
> >
> > # For A53 Cluster
> > for i in 0 1 2 3; do
> > for j in 0 1 2 ; do
> > echo 1 > /sys/devices/system/cpu/cpu$i/cpuidle/state$j/disable;
> > echo 'cpu$i state$j disabled';
> > done;
> > done
> >
> > # For A73 Cluster
> > for i in 4 5 6 7; do
> > for j in 0 1 2 ; do
> > echo 1 > /sys/devices/system/cpu/cpu$i/cpuidle/state$j/disable;
> > echo 'cpu$i state$j disabled';
> > done;
> > done
> >
> >
> >
> > I am not sure if these methods differ from your methods, but cpuidle
> >
> > in my device should have been switched off.
> >
> >
> >
> > Nonetheless, i tried your both methods.
> >
> >
> >
> > After the first method, my device still crash after csls command.
> >
> > How I added the kernel parameters is by adding the following line
> >
> > to the end of the script in
> ${AOSP_DIR}/device/linaro/hikey/hikey960/BoardConfig.mk:
> >
> >
> >
> > BOARD_KERNEL_CMDLINE += nohlt
> >
> >
> >
> > before rebuilding and reflashing the boot image into hikey960.
> >
> > Output of the kernel parameters added can be found:
> >
> > hikey960:/ # cat /proc/cmdline
>
>
> > androidboot.hardware=hikey960 firmware_class.path=/vendor/firmware
> efi=noruntime init=/init androidboot.boot_devices=soc/ff3b0000.ufs
> loglevel=15 overlay_mgr.overlay_dt_entry=hardware_cfg_enable_android_fstab
> nohlt buildvariant=eng console=ttyAMA6
> androidboot.serialno=2E6F5F3A0180CA83 initrd=0x7C00000,0xC2431
> >
> >
> >
> > Unfortunately, using both methods simultaneously did not work and csls
> >
> > still did not give me the full listing of the coresight components.
> >
> > The output from the csls command before it crashed
> >
> > remains the same as per our last email.
> >
> >
> >
> >
> >
> > >
> > > Thanks,
> > > Leo Yan
> >
> >
> >
> > Thank you for your reply and assistance!
> >
> >
> >
> > Jeremy
> >
> > IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended
> recipient, please notify the sender immediately and do not disclose the
> contents to any other person, use it for any purpose, or store or copy the
> information in any medium. Thank you.