On 10/18/23 10:34, Ard Biesheuvel wrote:

(cc Heinrich)

Hello Ben,

Thanks for the report.

On Wed, 18 Oct 2023 at 03:19, Ben Schneider ben@bens.haus wrote:

...

Hi Ard,

I have an ESPRESSObin Ultra (aarch64) that uses U-Boot as its bootloader. It shipped from the manufacturer with with v5.10, and I've been trying to upgrade. U-Boot supports booting Image directly via EFI (https://u-boot.readthedocs.io/en/latest/usage/cmd/bootefi.html), and I have been using it that way to successfully boot the system up to and including v6.0.19. However, v6.1 and v6.5 kernels fail to boot.

When booting successfully, the following messages are displayed:

EFI stub: Booting Linux Kernel...EFI stub: ERROR: FIRMWARE BUG: efi_loaded_image_t::image_base has bogus value EFI stub: ERROR: FIRMWARE BUG: kernel image not aligned on 64k boundary EFI stub: Using DTB from configuration table EFI stub: ERROR: Failed to install memreserve config table! EFI stub: Exiting boot services... [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]

I suspect many of the above error messages are simply attributable to using U-Boot to load an EFI stub and can be safely ignored given that the system boots and runs fine.

These messages are not typical for launching a kernel via the EFI stub from U-Boot. It should look like this:

=> load mmc 0:1 $fdt_addr_r boot/dtb 28846 bytes read in 6 ms (4.6 MiB/s) => load mmc 0:1 $kernel_addr_r boot/vmlinuz 53686664 bytes read in 2223 ms (23 MiB/s) => setenv bootargs root=/dev/mmcblk0p1 efi=debug earlyprintk initrd=boot/initrd.img => bootefi $kernel_addr_r $fdt_addr_r Card did not respond to voltage select! : -110 Failed to load EFI variables Booting /boot\vmlinuz EFI stub: Booting Linux Kernel... EFI stub: EFI_RNG_PROTOCOL unavailable EFI stub: Loaded initrd from command line option EFI stub: Using DTB from configuration table EFI stub: Exiting boot services... [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]

...

I suspect that these are not as harmless as you think. How old is the u-boot build on this platform?

...

When boot fails (v6.5), the following messages are displayed:

EFI stub: Booting Linux Kernel... EFI stub: ERROR: FIRMWARE BUG: efi_loaded_image_t::image_base has bogus value EFI stub: ERROR: FIRMWARE BUG: kernel image not aligned on 64k boundary EFI stub: ERROR: Failed to install memreserve config table! EFI stub: Using DTB from configuration table EFI stub: Exiting boot services... EFI stub: ERROR: Unable to construct new device tree. EFI stub: ERROR: Failed to update FDT and exit boot services

In case it's relevant, the device tree for this device is arch/arm64/boot/marvell/armada-3720-espressobin-ultra.dts

This is a uboot path, right? Not a linux path? Are you sure this DTS is compatible with the v6.5 kernel?

There is no arch/arm64/ in U-Boot. Maybe arch/arm64/boot/dts/marvell/armada-3720-espressobin-ultra.dts?

Best regards

Heinrich

...

Hopefully I've reported this in the correct place or that the information provided is sufficient to get it where it needs to be. Let me know if there is additional information I can provide. I am also able to use the device to test.

Please add some efi_warn() message inside the update_fdt() routine in drivers/firmware/efi/libstub/fdt.c to narrow down which call is causing it to return an error. Nothing in that code jumps out to me, but we regularly update libfdt in the kernel as well, so it might be a change in there that triggers this.

How old is the u-boot build on this platform?

U-Boot 2018.03-devel-18.12.3-g926d08c7ce (Apr 11 2022 - 15:48:13 +0800)

This appears to be the most recent version the manufacturer has released for this device. Source: https://github.com/globalscaletechnologies/u-boot-marvell.

...

arch/arm64/boot/marvell/armada-3720-espressobin-ultra.dts

This is a uboot path, right? Not a linux path? Are you sure this DTS is compatible with the v6.5 kernel?

Sorry for the confusion; that is the path in the linux source to the DTS used to compile the DTB that I am using to boot the device. I booted v5.15.135 using the DTB compiled from v6.5.7 source and that works fine. I also tried to boot v6.5.7 with the factory DTB and that failed.

Please add message inside the update_fdt() routine...

I added a bunch and here's what I got back:

EFI stub: Booting Linux Kernel... EFI stub: ERROR: FIRMWARE BUG: efi_loaded_image_t::image_base has bogus value EFI stub: ERROR: FIRMWARE BUG: kernel image not aligned on 64k boundary EFI stub: ERROR: Failed to install memreserve config table! EFI stub: Using DTB from configuration table EFI stub: Exiting boot services... EFI stub: Starting update_fdt()... EFI stub: fdt_num_mem_rsv() returned 1 EFI stub: fdt_subnode_offset() returned 8944 EFI stub: Setting bootargs=console=ttyMV0,115200 earlycon=ar3700_uart,0xd0012000 root=/dev/sda1 rw rootwait EFI stub: Adding FDT entries... EFI stub: fdt_setprop_var() for linux,uefi-system-table returned 0 EFI stub: fdt_setprop_var() for linux,uefi-mmap-start returned -11 EFI stub: update_fdt() failed with status -11 EFI stub: ERROR: Unable to construct new device tree. EFI stub: ERROR: Failed to update FDT and exit boot services

That's as far as I could get today but hopefully that starts to narrow it down. Appreciate the help!

Sincerely,

Ben

Oct 19, 2023, 07:21 by heinrich.schuchardt@canonical.com:

To which kernel and device-tree are the messages below related?

Sorry. I'm building/testing with v6.5.7 right now, but I have not been able to boot any kernel version >= 6.1. The device tree I used also came from linux v6.5.7. Its source is at arch/arm64/boot/dts/marvell/armada-3720-espressobin-ultra.dts.

Something in the structure of your device-tree is invalid.

The device does boot using kernel v5.15 and the device tree compiled from linux v6.5.7. The device tree could still very well be a problem. I just didn't start there because whatever problems the device tree may have don't prevent the device from booting for kernels prior to v6.1.

Please, check the load addresses in U-Boot. Is something overwriting the tail of the device-tree?

I have always loaded the device tree to 0x1000000 (16MiB) and the kernel to 0x2000000 (32MiB) for no particular reason except that's what the manufacturer did. armada-3720-espressobin-ultra.dtb is only 14K. I don't load an initramfs or anything else to memory with u-boot and the address used to load u-boot environment is 0x6000000.

Compiling upstream U-Boot's qemu_arm64_defconfig yields lib/efi_loader/dtbdump.efi. If you run this instead of the kernel, you can write the device-tree as it is passed in a configuration table to the ESP.

I compiled and ran this fine, but I was unable to save the device tree. I suspect this is because the program searches for an ESP, and there is none on the device. U-boot was compiled with support to load directly from an ext4 filesystem so I didn't bother setting one up. I will work on it. I can convert the .dtb on disk back to a human-readable .dts easily with dtc if that is helpful.

Thanks!

Ben

On 10/21/23 02:07, Ben Schneider wrote:

Oct 20, 2023, 01:25 by ben@bens.haus:

...

Oct 19, 2023, 07:21 by heinrich.schuchardt@canonical.com:

...

Compiling upstream U-Boot's qemu_arm64_defconfig yields lib/efi_loader/dtbdump.efi. If you run this instead of the kernel, you can write the device-tree as it is passed in a configuration table to the ESP.

I compiled and ran this fine, but I was unable to save the device tree. I suspect this is because the program searches for an ESP, and there is none on the device. U-boot was compiled with support to load directly from an ext4 filesystem so I didn't bother setting one up. I will work on it.

Hi Heinrich, I loaded dtbdump.efi from a FAT32 formatted partition with type EFI System, but attempts to run the save command return "Failed to open simple file system protocol". Sorry if there is something else I am missing.

On upstream U-Boot I cannot see this problem.

Best regards

Heinrich