Hi all. I need clarification on where firmware should be located when booting from removable shared storage. Statements in the intro of 4.2 seem to contradict later statements in 4.2.2 of the EBBR [1].
Section 4.2.2 states that to make it easier to use stock OS images on removable storage, firmware should be located on the ESP partition (so you don't have to create a dedicated firmware partition).
On removable media, firmware should be stored in the ESP under the /FIRMWARE directory structure as described in Firmware Partition Filesystem.
However, prior in the 4.2 section, it states:
The sections below discuss the requirements when using both fixed and removable storage. However, it should be noted that the recommended behaviour of firmware should be identical regardless of storage type. In both cases, the recommended boot sequence is to first search for firmware in a dedicated firmware partition, and second search for firmware in the ESP.
Why would the boot sequence for shared storage look in two places if it is always in the ESP? Or does the statement in 4.2.2 only apply in the case of using a stock OS image?
As a side question, I'm not sure of the practicality of recommending a boot sequence that looks in two places, at least for the very first firmware stages, as ROM usually is very simple and only looks in a single location.
thanks Jon
On 10/19/23 23:05, Jon Humphreys wrote:
Hi all. I need clarification on where firmware should be located when booting from removable shared storage. Statements in the intro of 4.2 seem to contradict later statements in 4.2.2 of the EBBR [1].
Section 4.2.2 states that to make it easier to use stock OS images on removable storage, firmware should be located on the ESP partition (so you don't have to create a dedicated firmware partition).
On removable media, firmware should be stored in the ESP under the /FIRMWARE directory structure as described in Firmware Partition Filesystem.
However, prior in the 4.2 section, it states:
The sections below discuss the requirements when using both fixed and removable storage. However, it should be noted that the recommended behaviour of firmware should be identical regardless of storage type. In both cases, the recommended boot sequence is to first search for firmware in a dedicated firmware partition, and second search for firmware in the ESP.
Why would the boot sequence for shared storage look in two places if it is always in the ESP? Or does the statement in 4.2.2 only apply in the case of using a stock OS image?
High Jon,
thanks for pointing out where the specification is unclear.
I guess a half-sentence is missing saying "if both a dedicated firmware partition and the EFI system partition are both supported by the firmware".
The EBBR is still evolving. If you have a suggestion for improvement, please, create an incident or merge request on Github. Consider joining one of our meetings. See https://github.com/ARM-software/ebbr/wiki/EBBR-Meetings#datetime
As a side question, I'm not sure of the practicality of recommending a boot sequence that looks in two places, at least for the very first firmware stages, as ROM usually is very simple and only looks in a single location.
The documentation is not only about what a boot ROM does. E.g. I have seen a boot ROM loading U-Boot SPL from flash and U-Boot SPL loading EDK II from /EFI/vendor/board/firmware.itb. But U-Boot SPL in this case would only scan the ESP.
Best regards
Heinrich
thanks Jon
Heinrich Schuchardt heinrich.schuchardt@canonical.com writes:
High Jon,
thanks for pointing out where the specification is unclear.
I guess a half-sentence is missing saying "if both a dedicated firmware partition and the EFI system partition are both supported by the firmware".
If I follow you, more precisely it is if a *separate* dedicated firmware partition and ESP are supported as firmware locations.
For example, with TI ROM, it always loads the firmware from the boot partition. So designating the dedicated firmware partition is done by making it the boot partition. It also supports firmware in the ESP, but only if you make the ESP the boot partition. Which then they are really the same partition.
But I question the purpose of the sentence at the end of 4.2:
In both cases, the recommended boot sequence is to first search for firmware in a dedicated firmware partition, and second search for firmware in the ESP.
because it is stated at the beginning of 4:
In general, EBBR compliant platforms should use dedicated storage for boot firmware images and data, independent of the storage used for OS partitions and the EFI System Partition (ESP).
Was the point of the sentence in 4.2 really that the EBBR prefers the choice of firmware placement in a dedicated partition, but if you have to, it can be in the ESP, *and* that this choice should be the same for fixed and removable storage?
The EBBR is still evolving. If you have a suggestion for improvement, please, create an incident or merge request on Github.
Created https://github.com/ARM-software/ebbr/issues/113 for clarification of 4.2.2.
Consider joining one of our meetings. See https://github.com/ARM-software/ebbr/wiki/EBBR-Meetings#datetime
I had wanted to bring this up in person, but I am on vacation for the next 2 weeks. I hope I can make the following meeting.
As a side question, I'm not sure of the practicality of recommending a boot sequence that looks in two places, at least for the very first firmware stages, as ROM usually is very simple and only looks in a single location.
The documentation is not only about what a boot ROM does. E.g. I have seen a boot ROM loading U-Boot SPL from flash and U-Boot SPL loading EDK II from /EFI/vendor/board/firmware.itb. But U-Boot SPL in this case would only scan the ESP.
So that raises another issue. Your statement above implies, I think, that different stages of firmware can live in different places, which does happen.
But I had read the spec as implying that firmware must be at a single location.
Can you clarify?
thanks Jon
On 10/20/23 08:01, Jon Humphreys wrote:
Heinrich Schuchardt heinrich.schuchardt@canonical.com writes:
High Jon,
thanks for pointing out where the specification is unclear.
I guess a half-sentence is missing saying "if both a dedicated firmware partition and the EFI system partition are both supported by the firmware".
If I follow you, more precisely it is if a *separate* dedicated firmware partition and ESP are supported as firmware locations.
For example, with TI ROM, it always loads the firmware from the boot partition. So designating the dedicated firmware partition is done by making it the boot partition. It also supports firmware in the ESP, but only if you make the ESP the boot partition. Which then they are really the same partition.
How does the TI ROM identify the boot partition? Is that bit 2 "Legacy BIOS Bootable" in the GPT partition attributes field?
But I question the purpose of the sentence at the end of 4.2:
In both cases, the recommended boot sequence is to first search for firmware in a dedicated firmware partition, and second search for firmware in the ESP.
because it is stated at the beginning of 4:
In general, EBBR compliant platforms should use dedicated storage for boot firmware images and data, independent of the storage used for OS partitions and the EFI System Partition (ESP).
Was the point of the sentence in 4.2 really that the EBBR prefers the choice of firmware placement in a dedicated partition, but if you have to, it can be in the ESP, *and* that this choice should be the same for fixed and removable storage?
The EBBR is still evolving. If you have a suggestion for improvement, please, create an incident or merge request on Github.
Created https://github.com/ARM-software/ebbr/issues/113 for clarification of 4.2.2.
Great.
Consider joining one of our meetings. See https://github.com/ARM-software/ebbr/wiki/EBBR-Meetings#datetime
I had wanted to bring this up in person, but I am on vacation for the next 2 weeks. I hope I can make the following meeting.
As a side question, I'm not sure of the practicality of recommending a boot sequence that looks in two places, at least for the very first firmware stages, as ROM usually is very simple and only looks in a single location.
The documentation is not only about what a boot ROM does. E.g. I have seen a boot ROM loading U-Boot SPL from flash and U-Boot SPL loading EDK II from /EFI/vendor/board/firmware.itb. But U-Boot SPL in this case would only scan the ESP.
So that raises another issue. Your statement above implies, I think, that different stages of firmware can live in different places, which does happen.
But I had read the spec as implying that firmware must be at a single location.
Can you clarify?
I agree with you that we should put some more thought into this chapter of the EBBR.
Firmware always lives in multiple locations, if you count the boot ROM as firmware.
The central target of the EBBR specification is that operating systems can boot without special adaptation to a specific firmware.
From a distro perspective the best solution is a compliant firmware that the distro does not have to care about because it is not on any medium controlled by the operating system. This is for instance given if a compliant EDK II or U-Boot is on SPI flash or in an eMMC boot partition.
Unfortunately not all hardware lives up to this. E.g. SPI flash may be missing or too small. In this case it is still preferable if the impact of having the firmware on a medium governed by the operating system remains small.
I personally would prefer firmware using the firmware folder in the ESP over using a firmware partition because the partitioning scheme would not be firmware specific.
Best regards
Heinrich
Heinrich Schuchardt heinrich.schuchardt@canonical.com writes:
On 10/20/23 08:01, Jon Humphreys wrote:
How does the TI ROM identify the boot partition? Is that bit 2 "Legacy BIOS Bootable" in the GPT partition attributes field?
Current devices are MBR only. Is is whatever gets set if you use the parted "set <p#> boot on" command.
So that raises another issue. Your statement above implies, I think, that different stages of firmware can live in different places, which does happen.
But I had read the spec as implying that firmware must be at a single location.
Can you clarify?
I agree with you that we should put some more thought into this chapter of the EBBR.
Firmware always lives in multiple locations, if you count the boot ROM as firmware.
The central target of the EBBR specification is that operating systems can boot without special adaptation to a specific firmware.
From a distro perspective the best solution is a compliant firmware that the distro does not have to care about because it is not on any medium controlled by the operating system. This is for instance given if a compliant EDK II or U-Boot is on SPI flash or in an eMMC boot partition.
Unfortunately not all hardware lives up to this. E.g. SPI flash may be missing or too small. In this case it is still preferable if the impact of having the firmware on a medium governed by the operating system remains small.
I am new to this discussion of course, but I had guessed that the EBBR was pushing for firmware to be in a filesystem (vs raw storage) exactly because it wanted the OS to have a conventional access to it so it can be updated.
I personally would prefer firmware using the firmware folder in the ESP over using a firmware partition because the partitioning scheme would not be firmware specific.
I argue the opposite (again I'm new to this so open to learning!). My view: we want all the firmware bits to be owned and specific to the board and the board/firmware presents a standard API to the OS. The OS should own the ESP so it can modify it, set boot order, etc. For these reasons, you want the firmware location to be different than the ESP. For updateability, of course the firmware's location can't be unknown to the OS.
But my point in this is about where it comes from. The firmware must come from the board provider, and the ESP from the OS provider, so they must be in different locations.
As I type this, I realize I have 2 assumptions that may not be correct:
1) the ESP should come from the OS provider. True? At least this seems to be the common practice.
2) the EBBR wants the OS to have direct access to the firmware for updates. True? Or is the thinking that the OS should not have direct access but should only update via other mechanisms like capsule update?
Best regards
Heinrich
thanks! Jon
On Fri, Oct 20, 2023 at 09:10:45AM -0500, Jon Humphreys wrote:
Heinrich Schuchardt heinrich.schuchardt@canonical.com writes:
I personally would prefer firmware using the firmware folder in the ESP over using a firmware partition because the partitioning scheme would not be firmware specific.
I argue the opposite (again I'm new to this so open to learning!). My view: we want all the firmware bits to be owned and specific to the board and the board/firmware presents a standard API to the OS. The OS should own the ESP so it can modify it, set boot order, etc. For these reasons, you want the firmware location to be different than the ESP. For updateability, of course the firmware's location can't be unknown to the OS.
But my point in this is about where it comes from. The firmware must come from the board provider, and the ESP from the OS provider, so they must be in different locations.
As I type this, I realize I have 2 assumptions that may not be correct:
- the ESP should come from the OS provider. True? At least this seems
to be the common practice.
Pretty much.
On a system with multiple OSes they are something of a shared space between multiple OS providers. That means the OS providers try not to stand on each others toes! That means, in general, OS installers are designed to leave the ESP mostly as they found it (apart from installing their bootloaders).
However an installer typically won't do much to stop the user choosing to reformat or resizing the ESP since, on systems without firmware in the ESP, this is an entirely legitimate thing to do when entirely replacing one OS with another.
- the EBBR wants the OS to have direct access to the firmware for
updates. True? Or is the thinking that the OS should not have direct access but should only update via other mechanisms like capsule update?
Not as far as I remember. Capsules are the preferred update mechanism.
Daniel.
On 23.10.23 13:30, Daniel Thompson wrote:
On Fri, Oct 20, 2023 at 09:10:45AM -0500, Jon Humphreys wrote:
Heinrich Schuchardt heinrich.schuchardt@canonical.com writes:
I personally would prefer firmware using the firmware folder in the ESP over using a firmware partition because the partitioning scheme would not be firmware specific.
I argue the opposite (again I'm new to this so open to learning!). My view: we want all the firmware bits to be owned and specific to the board and the board/firmware presents a standard API to the OS. The OS should own the ESP so it can modify it, set boot order, etc. For these reasons, you want the firmware location to be different than the ESP. For updateability, of course the firmware's location can't be unknown to the OS.
But my point in this is about where it comes from. The firmware must come from the board provider, and the ESP from the OS provider, so they must be in different locations.
As I type this, I realize I have 2 assumptions that may not be correct:
- the ESP should come from the OS provider. True? At least this seems
to be the common practice.
Pretty much.
On a system with multiple OSes they are something of a shared space between multiple OS providers. That means the OS providers try not to stand on each others toes! That means, in general, OS installers are designed to leave the ESP mostly as they found it (apart from installing their bootloaders).
However an installer typically won't do much to stop the user choosing to reformat or resizing the ESP since, on systems without firmware in the ESP, this is an entirely legitimate thing to do when entirely replacing one OS with another.
- the EBBR wants the OS to have direct access to the firmware for
updates. True? Or is the thinking that the OS should not have direct access but should only update via other mechanisms like capsule update?
Not as far as I remember. Capsules are the preferred update mechanism.
Daniel.
I prefer if the firmware is provided with the board and is not installed on any medium that the OS manages, e.g. SPI flash or an eMMC boot partitions.
Unfortunately many boards don't follow this approach. Their boot ROM loads the firmware from SD card or another mountable device.
The whole discussion is about these cases only.
Often it is the Linux distribution that supplies the firmware. See the u-boot source package in Debian and derivatives.
If the firmware lives on a block device also used by the operating system, we must find a solution minimizing conflicts.
Files in a firmware directory on the ESP have little impact on operating systems. I would prefer this to having device specific partitioning requirements.
Usage of capsule updates does not imply the storage location.
Best regards
Heinrich
On Mon, Oct 23, 2023 at 02:44:45PM +0200, Heinrich Schuchardt wrote:
On 23.10.23 13:30, Daniel Thompson wrote:
On Fri, Oct 20, 2023 at 09:10:45AM -0500, Jon Humphreys wrote:
Heinrich Schuchardt heinrich.schuchardt@canonical.com writes:
I personally would prefer firmware using the firmware folder in the ESP over using a firmware partition because the partitioning scheme would not be firmware specific.
I argue the opposite (again I'm new to this so open to learning!). My view: we want all the firmware bits to be owned and specific to the board and the board/firmware presents a standard API to the OS. The OS should own the ESP so it can modify it, set boot order, etc. For these reasons, you want the firmware location to be different than the ESP. For updateability, of course the firmware's location can't be unknown to the OS.
But my point in this is about where it comes from. The firmware must come from the board provider, and the ESP from the OS provider, so they must be in different locations.
As I type this, I realize I have 2 assumptions that may not be correct:
- the ESP should come from the OS provider. True? At least this seems
to be the common practice.
Pretty much.
On a system with multiple OSes they are something of a shared space between multiple OS providers. That means the OS providers try not to stand on each others toes! That means, in general, OS installers are designed to leave the ESP mostly as they found it (apart from installing their bootloaders).
However an installer typically won't do much to stop the user choosing to reformat or resizing the ESP since, on systems without firmware in the ESP, this is an entirely legitimate thing to do when entirely replacing one OS with another.
- the EBBR wants the OS to have direct access to the firmware for
updates. True? Or is the thinking that the OS should not have direct access but should only update via other mechanisms like capsule update?
Not as far as I remember. Capsules are the preferred update mechanism.
I prefer if the firmware is provided with the board and is not installed on any medium that the OS manages, e.g. SPI flash or an eMMC boot partitions.
Unfortunately many boards don't follow this approach. Their boot ROM loads the firmware from SD card or another mountable device.
The whole discussion is about these cases only.
Often it is the Linux distribution that supplies the firmware. See the u-boot source package in Debian and derivatives.
If the firmware lives on a block device also used by the operating system, we must find a solution minimizing conflicts.
No objection so far... we are definitely talking about least-bad solutions rather than the best ones!
Files in a firmware directory on the ESP have little impact on operating systems. I would prefer this to having device specific partitioning requirements.
I agree the impact on OS is modest but for fixed shared storage I still favour what is proposed by EBBR today: a separate partition that can be marked as a Required Partition (sure an ESP can also be marked as a Required Partition but I think the meaning of that would be ambiguous to an installer).
It's all very "last percent" though. As we already agreeds, most major fixed storage media supports something akin to an eMMC boot partition, UFS boot LU or NVMe namespace.
A "new" boot ROM is better of supporting loading firmware from these LUs. Thus loading firmware from firmware partition or ESP ends up being exclusively used for provisioning/debrick/recovery boots.
Daniel.
boot-architecture@lists.linaro.org
-
Daniel Thompson -
Heinrich Schuchardt -
Jon Humphreys