On Mon, Nov 13, 2023 at 11:18:04AM -0600, Jon Humphreys wrote:

Hi all. I've organized some of my requests for clarification below. I appreciate you helping me understand the spec. Where the spec needs clarification itself, I'll create a ticket to track.

Basic (dumb) questions:

• why is EBBR dictating the location of firmwares? The firmware locations are a contract with the boot ROM and subsequent stages, leading up to the OSloader. The OSloader hand-off interface is the real EBBR contract with an OS provider, I would think.

Mostly it recommends rather that dictates...

My answer to this question had been that in order for the OS to update firmware, the firmware locations need to be standardized. But I've since learned that the firmware update mechanism should be abstracted via update capsules. So I am back to not understanding why the firmware locations aren't just a convention local to the board vendor.

I've never considered updates as applicable here (as you said, use capsules). However if a boot ROM is designed to allow the EBBR recommendations to be followed then:

1. It is easy to write OS installers that don't accidentally wipe out the system firmware.

2. It is possible to author boot/recovery media that are capable of booting on multiple systems.

Firmware locations:

• the EBBR states that a dedicated partition is preferred for storing firmware. This seems to imply that all firmwares will be in a single location. This isn't true in many cases (eg, some partitions aren't large enough and subsequent stages are loaded from a different partition). Is there guidance (or should be guidance) in EBBR on how to handle these scenarios?

Not sure what you mean here. In general if a partition containing a filesystem isn't big enough then we can just make it bigger...

If you mean what happens when the two more preferred approaches[1] are contradicted by the boot ROM (or rejected by its designer) then is it useful for EBBR to provide tertiary recommendations for such cases[2]? Weird stuff is not prohibited... but not recommended.

I'm not sure whether guidance here is useful. We are pretty far from the EBBR recommendations by this point[1] so doesn't more guidance simply encourage boot-ROM designers to ignore the recommendations?

[1]: *Not* having shared storage is most preferred. Storing firmware in a FAT filesystem is preferred of per-device hackery.

[2]: Strictly speaking this is already an implicit tertiary recommendation which is to prefer GPT partitioning over MBR...

• does the EBBR discourage placing firmware in the ESP, and if so, under what circumstances is it recommended (eg, trying to boot a stock OS image). See https://github.com/ARM-software/ebbr/issues/113

I'm not 100% sure about this. To be honest the *only* reason I can see to store firmware in the ESP is to make it easier to re-author stock boot images by enriching them with additional firmware.

However I don't think I got very involved with these discussions.

Device Tree:

• The EBBR requires a device tree file (or ACPI table). It makes complete theoretical sense that the board firmware should supply the DTB, since it is (in theory) a description of the hardware, which obviously doesn't change. But the unfortunate reality is that the device tree is always changing to stay in sync with the kernel. My experience is that OS images ship with a corresponding device tree. But the device tree location is one example where the EBBR does not specify a convention. How can the EBBR handle this reality so that we can standardize where an OS vendor can place its version of the DTB?

Does EBBR need to specify that? OS will normally provide an OS bootloader (grub, systemd-boot, etc) and since the OS "owns" the configuration for that bootloader is already has full control over where alternative DTs are loaded from.

Note also the discusison w.r.t. the EFI_DT_FIXUP_PROTOCOL which is a potentially useful EBBR feature to help support OS provided device trees.

ESP

• who is responsible for providing the ESP, the board firmware or the OS provider?

Generally speaking the ESP is provided by the OS. The whole point of avoiding shared storage and recommended that firmware be placed in a different partition is to allow this.

AFAIK the only circumstance where an EBBR platform is required to come pre-authored with an ESP is when the firmware is loaded from that ESP.

The goal of SystemReady is to separate the software packaging between board firmware and OS images, so that we eliminate the need for OS vendors to produce board specific images. The current practice in embedded is to deliver a monolithic, self-contained image, and in this scenario, standards don't matter as much as it is all self-contained.

Not sure I agree that standards don't matter in that space. When OS and firmware are a monolithic, self-contained image then it is possible to get away without standardization... but that doesn't mean standardization isn't useful in helping to reduce costs.

As I try to image what embedded looks like where boards ship with firmware installed and OS providers only provide the OS image, I'm running into these questions like who provides the ESP, where is the DTB located, etc.

I asked earlier whether there is any value in providing tertiary recommendations. One thing where EBBR is a little vague is the value of the secondary recommendation (firmware partition in shared storage).

For example one could imagine bootflow logic something like the following:

def boot_from(media): scan_gpt() p = find_partition_by_guid(DEVICE_SPECIFIC_GUID) ep = load_secondary_loader_from_raw_partition(p) jump_to(ep)

def boot(media) if (has_hardware_boot_partition(media)) boot_from(media.hardware_boot_partition() boot_from(media.regular_partition())

if not gpio(suppress_fixed_media): boot_from(SPINOR) boot(UFS) boot(eMMC) boot(SD)

This implements the most preferred approach (e.g. it tries to find secondary firmware without requiring the use of shared media) and simply doesn't bother messing about with implementing a filesystem.

However IMHO the above it still a fairly high quality boot ROM. It has good support for booting from separate hardware partitions and good enough de-brick capabilities (booting from SD without following all shared-storage recommendations).

If I am right about "fairly high quality" here then I guess I'd acknowledge that it isn't easy to come up with the above pseudo code by reading the EBBR standard!

Daniel.

On 11/14/23 00:07, Jon Humphreys wrote:

Daniel Thompson daniel.thompson@linaro.org writes:

...

On Mon, Nov 13, 2023 at 11:18:04AM -0600, Jon Humphreys wrote:

...

Basic (dumb) questions:

• why is EBBR dictating the location of firmwares? The firmware locations are a contract with the boot ROM and subsequent stages, leading up to the OSloader. The OSloader hand-off interface is the real EBBR contract with an OS provider, I would think.

Mostly it recommends rather that dictates...

ok, thanks for the reminder on should vs shall. Re-looking at the spec, there are no SHALLS regarding firmware locations.

...

...

Firmware locations:

• the EBBR states that a dedicated partition is preferred for storing firmware. This seems to imply that all firmwares will be in a single location. This isn't true in many cases (eg, some partitions aren't large enough and subsequent stages are loaded from a different partition). Is there guidance (or should be guidance) in EBBR on how to handle these scenarios?

Not sure what you mean here. In general if a partition containing a filesystem isn't big enough then we can just make it bigger...

Not always. Consider a eMMC hardware partition that is limited in size. For example, BeaglePlay will put the first stage SPL in the eMMC hardware partition and other stages (eg, u-boot) in the UDA partition, so that all stages will fit.

From what you have said, the main point is to reduce the possibility of the OS messing with the firmware, so the spec recommends that the firmware be in dedicated partitions. This is clear at the beginning of chapter 4:

     In general, EBBR compliant platforms should use dedicated
     storage for boot firmware images and data ...

What confused me is that 4.2 goes a bit further is specifying firmware to be stored in "a dedicated firmware partition". (But no SHALLs here!) Maybe replace "a dedicated firmware partition" with "dedicated firmware partitions".

...

...

Device Tree:

• The EBBR requires a device tree file (or ACPI table). It makes complete theoretical sense that the board firmware should supply the DTB, since it is (in theory) a description of the hardware, which obviously doesn't change. But the unfortunate reality is that the device tree is always changing to stay in sync with the kernel. My experience is that OS images ship with a corresponding device tree. But the device tree location is one example where the EBBR does not specify a convention. How can the EBBR handle this reality so that we can standardize where an OS vendor can place its version of the DTB?

Does EBBR need to specify that? OS will normally provide an OS bootloader (grub, systemd-boot, etc) and since the OS "owns" the configuration for that bootloader is already has full control over where alternative DTs are loaded from.

Note also the discusison w.r.t. the EFI_DT_FIXUP_PROTOCOL which is a potentially useful EBBR feature to help support OS provided device trees.

Your point is that if the OS cares enough about using a particular DTB, then it should specify it in the OS loader (eg, "devicetree" grub2 command), right? True, but I wouldn't say it is normal. I don't see Debian or Fedora using it. Instead I see them relying on conventions

GRUB's devicetree command is only available if secure boot is disabled. Ubuntu uses it on RISC-V platforms.

that don't work for all vendors. So there needs to be some influence to have these guys start using OS loader commands to explicitly specify the DTB.

And yes, DTB fix-up would be an issue.

U-Boot implements the EFI_DT_FIXUP_PROTOCOL for this purpose and Ubuntu's GRUB invokes it.

Alternately, we could add some standard to the EBBR that specifies where an OS should put a DTB if it wants the firmware to pick it up. Then, that DTB goes through the normal firmware handling, including fix-ups. (We would need to factor in security, though, but this is true regardless of the approaches taken.)

The problem here is that sometimes device-trees are not forward and backward compatible and EDK II and U-Boot don't know which kernel will be booted if GRUB is used.

Another gap is that there is no convention on signing device-trees. So any device-tree coming from the OS is a gap in secure boot. Signed unified kernel images (UKI) comprising kernel, initrd, device-trees could be a solution.

Best regards

Heinrich

On Mon, Nov 13, 2023 at 05:07:43PM -0600, Jon Humphreys wrote:

Daniel Thompson daniel.thompson@linaro.org writes:

...

On Mon, Nov 13, 2023 at 11:18:04AM -0600, Jon Humphreys wrote:

...

• why is EBBR dictating the location of firmwares? The firmware locations are a contract with the boot ROM and subsequent stages, leading up to the OSloader. The OSloader hand-off interface is the real EBBR contract with an OS provider, I would think.

Mostly it recommends rather that dictates...

ok, thanks for the reminder on should vs shall. Re-looking at the spec, there are no SHALLS regarding firmware locations.

I'm not sure whether it is imminent but I think it possible that at some point the current EBBR will become "EBBR level N" and the recommendations will become requirements for level N+1.

...

...

Firmware locations:

• the EBBR states that a dedicated partition is preferred for storing firmware. This seems to imply that all firmwares will be in a single location. This isn't true in many cases (eg, some partitions aren't large enough and subsequent stages are loaded from a different partition). Is there guidance (or should be guidance) in EBBR on how to handle these scenarios?

Not sure what you mean here. In general if a partition containing a filesystem isn't big enough then we can just make it bigger...

Not always. Consider a eMMC hardware partition that is limited in size. For example, BeaglePlay will put the first stage SPL in the eMMC hardware partition and other stages (eg, u-boot) in the UDA partition, so that all stages will fit.

There is lots of scope for confusion here since the plural form of firmware is firmware ;-) .

IMHO the size of a hardware partition (or SPINOR) isn't in scope for this particular EBBR recommendation! The recommendation you mention is that when firmware is located in shared storage that it be placed in a single partition.

Interestingly it sounds like BeaglePlay is an example where the EBBR recommendations w.r.t. shared storage can be applied without requiring changes to the boot ROM (e.g. if GPT/FAT support can be fitted into the SPL then adopting this EBBR recommendation is "just software" ;-) ).

From what you have said, the main point is to reduce the possibility of the OS messing with the firmware, so the spec recommends that the firmware be in dedicated partitions. This is clear at the beginning of chapter 4:

    In general, EBBR compliant platforms should use dedicated
    storage for boot firmware images and data ...

What confused me is that 4.2 goes a bit further is specifying firmware to be stored in "a dedicated firmware partition". (But no SHALLs here!) Maybe replace "a dedicated firmware partition" with "dedicated firmware partitions".

I've worked on platforms where the firmware mandates multiple partitions in the shared storage. The problem is these partitions proliferate because "let's have one partition for every microcontroller/co-processor on the SoC". On such systems even *before* we try to author media that can boot on multiple devices we already have 16+ partitions for firmware.

AFAIR remember the logic behind recommending a single FAT-formatted firmware partition is that read-only FAT drivers are fairly trivial and much preferable to using the GPT as a pseudo-filesystem. Thus I believe recommending a single firmware partition in shared storage is intentional.

...

...

Device Tree:

• The EBBR requires a device tree file (or ACPI table). It makes complete theoretical sense that the board firmware should supply the DTB, since it is (in theory) a description of the hardware, which obviously doesn't change. But the unfortunate reality is that the device tree is always changing to stay in sync with the kernel. My experience is that OS images ship with a corresponding device tree. But the device tree location is one example where the EBBR does not specify a convention. How can the EBBR handle this reality so that we can standardize where an OS vendor can place its version of the DTB?

Does EBBR need to specify that? OS will normally provide an OS bootloader (grub, systemd-boot, etc) and since the OS "owns" the configuration for that bootloader is already has full control over where alternative DTs are loaded from.

Note also the discusison w.r.t. the EFI_DT_FIXUP_PROTOCOL which is a potentially useful EBBR feature to help support OS provided device trees.

Your point is that if the OS cares enough about using a particular DTB, then it should specify it in the OS loader (eg, "devicetree" grub2 command), right? True, but I wouldn't say it is normal. I don't see Debian or Fedora using it. Instead I see them relying on conventions that don't work for all vendors. So there needs to be some influence to have these guys start using OS loader commands to explicitly specify the DTB.

Through initiatives like UKI the distros are going to increase convergence but why do generic distros need to provide identical ways to hack at the devicetree? They already have a common way to obtain the devicetree using UEFI *and* a common way to update obsolete firmware (e.g. old devicetrees) with fwupd and capsule update.

Once a distro has to "just know"[1] what devicetree is has to offer the firmware on order to boot then on it's way to not being a generic distro any more! That's fine and is entirely legitimate from an EBBR perspective. However expecting generic distros to standardize infrastructure that is only useful for custom distros is unrealistic.

[1] There is a difference between the distro "just knowing" and the administrator knowing. For example distros already offer infrastructure that gives the admin the freedom to customize the devicetree. For example, in Debian: https://gist.github.com/daniel-thompson/dbc23bd3f4560b0d363187de212e7510

And yes, DTB fix-up would be an issue.

Alternately, we could add some standard to the EBBR that specifies where an OS should put a DTB if it wants the firmware to pick it up. Then, that DTB goes through the normal firmware handling, including fix-ups. (We would need to factor in security, though, but this is true regardless of the approaches taken.)

When alternative devicetrees are required I think UKI is a pretty good standard to adopt. This is noteworthy here because UKI is mostly implemented by the OS...

Daniel.