-----Original Message----- From: CoreSight coresight-bounces@lists.linaro.org On Behalf Of Dominik Huber Sent: 28 April 2021 14:51 To: leo.yan@linaro.org; Mike Leach mike.leach@linaro.org Cc: coresight@lists.linaro.org Subject: Re: Help sought on which approach to take for tracing with CoreSight
On 23/04/2021 02:10, Leo Yan wrote:
Hi Dominik,
On Thu, Apr 22, 2021 at 04:24:26PM +0100, Mike Leach wrote:
[...]
For now, I am going to try implementing the sysfs tracing, since it
seemed easier at first glance. I found the documentation for the .ini files within the snapshots, and am now trying to create my own version of them. But I'm struggling to get the addresses for all the devices, e.g. within cpu_0.ini.
You could download the snapshot .ini files for Hikey from: https://people.linaro.org/~leo.yan/opencsd/opencsd_hikey/hikey_snapsho t.tgz
Just clarify:
- The hikey_snapshot.tgz contains the configuration files which were manually edited by myself and I tested it can work with OpenCSD's snapshot tool;
- I think you could continue to use CSAL tool to continue to retrieve the registers, so that can use the dumping registers to double check the settings in .ini.
Thank you, these snapshot files are a huge help! I've added my binary and its .so files to the "cpu_0.ini", but it doesn't work quite yet. When using the traced data from my own binary with the trc_pkt_lister program, it just says "END OF TRACE DATA" (see attachment "trc_pkt_lister- decode_output.txt"). Weirdly enough, when using the cstrace.bin from Leo's snapshot, I get lots of decoded trace. Of course, the streams aren't synchronized and it is not informative, but it has content. This made me wonder, whether I'm collecting the trace correctly, but I believe there's not much I can do wrong. I attached my simpletrace.sh, which enables trace, runs the binary on CPU-core 0, stops trace, and collects it. I also tried a variant, where I filter only the relevant address ranges, but with the same result. Did I understand correctly, that I need the code segment addresses for the binary and .so files? (the entries from "/proc/<pid>/maps" with the "x" executable flag)? I think my .ini files should be correct, but I've attached my "cpu_0.ini" and the output of "/proc/<pid>/maps" of my binary just in case. I also deactivated ASLR, to ensure the addresses stay constant.
I've read in the discovery.md from CSAL that they can be extracted by reading the ROM table of my Cortex-A53, but I'm doing/understanding something wrong. I cross-compiled the CSAL library and the csinfo-folder (which btw only compiled after inserting "MODULE_LICENSE("");" into the csinfo.c file). Then I copied the resulting csinfo.ko to my Hikey620 where I tried "sudo make" and it returned: Load CoreSight reporting module... expect "Resource temporarily
unavailable"
insmod csinfo.ko insmod: ERROR: could not insert module csinfo.ko: Resource temporarily unavailable Makefile:17: recipe for target 'load' failed make: *** [load] Error 1 If I understood correctly, that is intended, but should also give me the base address of the ROM-table afterward. But there was no other
output.
I do have CONFIG_DEVMEM enabled. I tried "sudo insmod csinfo.ko". directly as well, with the same result.
Looking at the code it appears that the output is printk(KERN_INFO ... ) Check the printk level is correct on your system.
It's good to use "dmesg" to check the kernel log, so it might give more hints for why fails to load kernel module. The compiling faiure with "MODULE_LICENSE" is weird, it's better to go back to check if have specified correctly for the kernel path?
Oh, of course, csinfo is a module, using printk, therefore the output also appears only on dmesg. There is indeed output. I've attached the dmesg output (load_csinfo_dmesg_output.txt), could you maybe take a quick glance at it, whether it looks as expected?
Sorry, I should have got to this thread earlier (I wrote csinfo and csscan). Yes, that is expected. You load the csinfo module so that it can read some system ID registers that aren't readable in userspace - it prints the values out as kernel log messages, and then unloads itself.
If there's a better way that involves the messages getting directly back to your shell when you load the module, or that avoids insmod complaining about the module 'failing to load', I'd be very happy to fix it to use that!
cssinfo exists only to discover where the main ROM table lives in the CPU's physical address space. It also prints some other ID registers. csinfo has no value if you already know the ROM table address... or if it turns out that the ROM table address register reads as zero, which is increasingly the case.
Using the address from the MPIDR register ROM table ("sudo ./csscan.py 0x00000000f6400000") causes the board to hang until I replug the power supply (see csscan_output.txt).
Unfortunately that is fairly typical. csscan has got as far as confirming that 0xf6400000 is a ROM table. It will then try and read each entry in the ROM table and interrogate the CoreSight devices pointed to. If those devices aren't accessible for any reason you may get a bus lockup. Sometimes it's due to them being powered off, which is why disabling CPUidle sometimes work - but other times, they are deliberately access-protected. At the top level, i.e. main ROM table, you may be encountering entries for the vendor's private system-level debug components.
Entries in the ROM table page itself may cause a bus lockup (i.e. you might be able to read the first entry at 0xf6400000 successfully, but 0xf6400010 might lock up).
You may be able to get further by excluding entries from the scan... possibly by hacking the csscan script. All csscan is doing at this stage is reading physical memory and following pointers. It doesn't actively start interacting with the devices unless you use the ATB topology discovery feature.
Al
Besides Mike's suggestion, I'd like to strongly suggest to disable CPUIdle when you debug CoreSight stuffs on Hikey620. So please add "nohlt" in the kernel command line.
I'm already using "nohlt", but thanks for the reminder.
Thanks, Leo
Thanks,
Dominik