[explicitly cc'ing boot-arch to see if this solves the TI auth problems]
On 05/04/2018 11:45 AM, Alexander Graf wrote:
On 05/04/2018 04:56 PM, Grant Likely wrote:
On 04/05/2018 15:46, Grant Likely wrote:
On 03/05/2018 09:43, Daniel Thompson wrote:
On Wed, May 02, 2018 at 06:09:15PM -0400, Tom Rini wrote:
On Wed, May 02, 2018 at 10:48:11PM +0100, Grant Likely wrote:
On 02/05/2018 22:24, Tom Rini wrote: > On Wed, May 02, 2018 at 08:40:42PM +0100, Daniel Thompson wrote: >> >> In terms of the restrictions that come from EBBR mandating GPT: > > Can we step back, why is EBBR mandating GPT?
I think EBBR should recommend GPT, but allow MBR if the SoC boot masked rom conflicts with GPT.
In the early days of GPT there was also a hybrid GPT+MBR scheme that could list the boot partition in the MBR, but still have a full GPT. It isn't pretty, but there is precidence.
How about recommends GPT but allows MBR, no qualifiers. As you note there's a lot of ways to fiddle around and make it work, probably, on all of the existing SoCs that do magic offsets. But it's a lot easier to just allow MBR (what the SoCs were designed to have to live with) and guide line that in this case nothing before the first 2MiB be used by the OS. With a few more spec words around all of that so it's nice and formal :)
I'm OK to allow MBR.
I'd be inclined to require that protective partitions be used to defend the first 2MB though. They are more flexible and are a useful hint to anyone trying to manually partition.
In a different mail Tom wrote:
And I agree with the high level idea of needing to do something to protect systems with a magic in-use spot.
There are a couple of extra details.
- We can't allocate a GUID to discourage an automatic partitioner
from harming a protected partition. We only have the 8-bit partition type. Some potential candidates could be 0xA2 (which Altera appear use for a similar purpose), 0xE7 (wikipedia does not report existing uses) or 0xF0 (PA-RISC Linux boot loader).
- Boot ROMs that have built-in support for FAT could be hard coded to
require a specific partition type. > I think we don't need to worry about #2 to much: systems with built-in MBR/FAT parsing can use hybrid MBR/GPT to communicate to distros which partitions are protected because (presumably) they allow flexibility in placing the first sector of the FAT partition.
Can we start a list of SoCs that have special requirements on the boot eMMC/SD/USB? It would be useful to see the cross-section of requirements.
I've created a wiki page to start capturing data. From their we can decide what scenarios the spec needs to cover.
https://github.com/glikely/ebbr-for-discussion/wiki/Boot-Media-Behaviour
One more thought, are there many new SoCs still using hard coded offsets instead of an MBR or GPT? It may be that there aren't enough left for us to care about for EBBR level 0. In which case EBBR Level 0 should support both MBR and GPT (non-hybrid), and then narrow to GPT only in a future revision.
I don't think we'll be able to move away from MBR anytime soon. For a quick glimpse on what SoCs need to have blobs installed where, take a look at our image creation script:
https://build.opensuse.org/package/view_file/openSUSE:Factory:ARM:Live/JeOS/...
There you can see that we force to MBR (for various reasons) on:
* RPi (reads MBR) * i.MX 5x/6 (SPL at sector 2) * Exynos 4/5 (SPL is in sector 1, not sure about newer ones) * AM905 (SPL in sector 1) * Kirkwood (SPL in sector 1) * Zynq(MP) (convenient to store boot.bin on FAT partition in MBR)
I think we should simply dictate that EBBR compliant OSs do not start any partitions before a sane boundary (1MB? 2MB?). IIRC most partitioning tools these days won't start partitions before 1MB anyway, so if we declare 1MB a safe bound we essentially have no work to do.
I did skim (not study) the script. Looks like a good resource of current board info. I see you are using raw mode for many boards.
I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
On 05/04/2018 06:20 PM, William Mills wrote:
[explicitly cc'ing boot-arch to see if this solves the TI auth problems]
On 05/04/2018 11:45 AM, Alexander Graf wrote:
On 05/04/2018 04:56 PM, Grant Likely wrote:
On 04/05/2018 15:46, Grant Likely wrote:
On 03/05/2018 09:43, Daniel Thompson wrote:
On Wed, May 02, 2018 at 06:09:15PM -0400, Tom Rini wrote:
On Wed, May 02, 2018 at 10:48:11PM +0100, Grant Likely wrote: > On 02/05/2018 22:24, Tom Rini wrote: >> On Wed, May 02, 2018 at 08:40:42PM +0100, Daniel Thompson wrote: >>> In terms of the restrictions that come from EBBR mandating GPT: >> Can we step back, why is EBBR mandating GPT? > I think EBBR should recommend GPT, but allow MBR if the SoC boot > masked > rom conflicts with GPT. > > In the early days of GPT there was also a hybrid GPT+MBR scheme that > could list the boot partition in the MBR, but still have a full > GPT. It > isn't pretty, but there is precidence. How about recommends GPT but allows MBR, no qualifiers. As you note there's a lot of ways to fiddle around and make it work, probably, on all of the existing SoCs that do magic offsets. But it's a lot easier to just allow MBR (what the SoCs were designed to have to live with) and guide line that in this case nothing before the first 2MiB be used by the OS. With a few more spec words around all of that so it's nice and formal :)
I'm OK to allow MBR.
I'd be inclined to require that protective partitions be used to defend the first 2MB though. They are more flexible and are a useful hint to anyone trying to manually partition.
In a different mail Tom wrote:
And I agree with the high level idea of needing to do something to protect systems with a magic in-use spot.
There are a couple of extra details.
- We can't allocate a GUID to discourage an automatic partitioner
from harming a protected partition. We only have the 8-bit partition type. Some potential candidates could be 0xA2 (which Altera appear use for a similar purpose), 0xE7 (wikipedia does not report existing uses) or 0xF0 (PA-RISC Linux boot loader).
- Boot ROMs that have built-in support for FAT could be hard coded to
require a specific partition type. > I think we don't need to worry about #2 to much: systems with built-in MBR/FAT parsing can use hybrid MBR/GPT to communicate to distros which partitions are protected because (presumably) they allow flexibility in placing the first sector of the FAT partition.
Can we start a list of SoCs that have special requirements on the boot eMMC/SD/USB? It would be useful to see the cross-section of requirements.
I've created a wiki page to start capturing data. From their we can decide what scenarios the spec needs to cover.
https://github.com/glikely/ebbr-for-discussion/wiki/Boot-Media-Behaviour
One more thought, are there many new SoCs still using hard coded offsets instead of an MBR or GPT? It may be that there aren't enough left for us to care about for EBBR level 0. In which case EBBR Level 0 should support both MBR and GPT (non-hybrid), and then narrow to GPT only in a future revision.
I don't think we'll be able to move away from MBR anytime soon. For a quick glimpse on what SoCs need to have blobs installed where, take a look at our image creation script:
https://build.opensuse.org/package/view_file/openSUSE:Factory:ARM:Live/JeOS/...
There you can see that we force to MBR (for various reasons) on:
* RPi (reads MBR) * i.MX 5x/6 (SPL at sector 2) * Exynos 4/5 (SPL is in sector 1, not sure about newer ones) * AM905 (SPL in sector 1) * Kirkwood (SPL in sector 1) * Zynq(MP) (convenient to store boot.bin on FAT partition in MBR)
I think we should simply dictate that EBBR compliant OSs do not start any partitions before a sane boundary (1MB? 2MB?). IIRC most partitioning tools these days won't start partitions before 1MB anyway, so if we declare 1MB a safe bound we essentially have no work to do.
I did skim (not study) the script. Looks like a good resource of current board info. I see you are using raw mode for many boards.
Yes, we usually prefer raw mode over anything else, as it's then more consistent across the different platforms. Special "firmware partitions" just become maintenance headaches because then suddenly our partitioner needs adoption and awareness.
I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
Alex
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote:
On 05/04/2018 11:45 AM, Alexander Graf wrote: I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Regards, Andreas
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote:
On 05/04/2018 11:45 AM, Alexander Graf wrote: I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need. Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR If using GPT, firmware should place needed files in either: * Vendor registered dir in EFI partition * GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Bill
On Fri, May 4, 2018 at 1:41 PM, William Mills wmills@ti.com wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote:
On 05/04/2018 11:45 AM, Alexander Graf wrote: I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need. Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR If using GPT, firmware should place needed files in either: * Vendor registered dir in EFI partition * GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
This is for all the other stuff vendors stick in custom partitions if we're lucky or some fixed offset if we're not, right?
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Firmware includes the bootrom and the loaded firmware (UEFI or ATF) itself. This would mean the bootrom has to read a FAT filesystem. I'm assuming you didn't mean that, so we'd still have to allow for partitions. Perhaps we tighten it up with specific GUIDs for specific firmware. That would make firmware updates easier and generically implementable.
That issue aside, case 3 could be handled by just marking what is the recommended option in case 2. And recommended can be defined as going to be required in the future.
Rob
On 05/07/2018 11:49 AM, Rob Herring wrote:
On Fri, May 4, 2018 at 1:41 PM, William Mills wmills@ti.com wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote:
On 05/04/2018 11:45 AM, Alexander Graf wrote: I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need. Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR If using GPT, firmware should place needed files in either: * Vendor registered dir in EFI partition * GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
This is for all the other stuff vendors stick in custom partitions if we're lucky or some fixed offset if we're not, right?
Yes. Today people seem to be using a GPT partition for each of these low level objects and seem to be using the partition name to identify them. They are using very generic names like "boot".
This means that the disk image will only work with one SOC or perhaps even one board.
If people are going to use raw GPT partitions I think they should at least use a GUID GPT partition type to give certainty. Falling back to a name is OK and will help the casual user.
Bit 0 of the GPT attributes table is listed as "Platform required". Do current installers leave these partitions alone?
However, i would prefer N files in an EFI dir rather than tons of special partitions. See below.
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Firmware includes the bootrom and the loaded firmware (UEFI or ATF) itself. This would mean the bootrom has to read a FAT filesystem. I'm assuming you didn't mean that, so we'd still have to allow for partitions. Perhaps we tighten it up with specific GUIDs for specific firmware. That would make firmware updates easier and generically implementable.
I did actually mean that. Is your doubt based on code size / complexity or legal reasons?
TI has put FAT in its ROMs for years (granted w/o long name support.) Sure early versions of that code had some quicks but the complexity is manageable.
My understanding is that Microsoft has a special grant of its IP for EFI partitions so I would think people would be comfortable at least for the EFI FAT partition. ( IANAL ... )
So if you still disagree I can understand. The ROM could use a GPT partition but after that it should use EFI files.
That issue aside, case 3 could be handled by just marking what is the recommended option in case 2. And recommended can be defined as going to be required in the future.
Rob
Yes and Thanks, Bill
On Mon, May 7, 2018 at 12:17 PM, William Mills wmills@ti.com wrote:
On 05/07/2018 11:49 AM, Rob Herring wrote:
On Fri, May 4, 2018 at 1:41 PM, William Mills wmills@ti.com wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote:
On 05/04/2018 11:45 AM, Alexander Graf wrote: I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need. Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR If using GPT, firmware should place needed files in either: * Vendor registered dir in EFI partition * GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
This is for all the other stuff vendors stick in custom partitions if we're lucky or some fixed offset if we're not, right?
Yes. Today people seem to be using a GPT partition for each of these low level objects and seem to be using the partition name to identify them. They are using very generic names like "boot".
This means that the disk image will only work with one SOC or perhaps even one board.
If people are going to use raw GPT partitions I think they should at least use a GUID GPT partition type to give certainty. Falling back to a name is OK and will help the casual user.
Bit 0 of the GPT attributes table is listed as "Platform required". Do current installers leave these partitions alone?
I'll leave that to one of the OS folks.
However, i would prefer N files in an EFI dir rather than tons of special partitions. See below.
Yes, I certainly agree. I'd extend that to bootloader specific files that are just containers of files and config: uImage, FIT image, Android boot image, Android dtbo image
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Firmware includes the bootrom and the loaded firmware (UEFI or ATF) itself. This would mean the bootrom has to read a FAT filesystem. I'm assuming you didn't mean that, so we'd still have to allow for partitions. Perhaps we tighten it up with specific GUIDs for specific firmware. That would make firmware updates easier and generically implementable.
I did actually mean that. Is your doubt based on code size / complexity or legal reasons?
Both.
It's certainly not that hard or much code size to do FAT read-only filesystem support, but it's more that I think vendors would be reluctant to add support. I get push back just on using filesystems in 2nd stage bootloaders on Android. I would love to see bootrom's more standardized, but I'm doubtful.
TI has put FAT in its ROMs for years (granted w/o long name support.) Sure early versions of that code had some quicks but the complexity is manageable.
That's good to know. Is that opensource?
My understanding is that Microsoft has a special grant of its IP for EFI partitions so I would think people would be comfortable at least for the EFI FAT partition. ( IANAL ... )
After some further reading on the details, I agree. In any case, I think all the patents on long filenames have expired now anyway.
Rob
On 05/07/2018 02:46 PM, Rob Herring wrote:
On Mon, May 7, 2018 at 12:17 PM, William Mills wmills@ti.com wrote:
TI has put FAT in its ROMs for years (granted w/o long name support.) Sure early versions of that code had some quicks but the complexity is manageable.
That's good to know. Is that opensource?
In the ROM? No. But I think some permissively licensed reference code vendors could use in their ROMs would be the way to solve the complexity/correctness/consistency issue.
On 05/07/2018 08:46 PM, Rob Herring wrote:
On Mon, May 7, 2018 at 12:17 PM, William Mills wmills@ti.com wrote:
On 05/07/2018 11:49 AM, Rob Herring wrote:
On Fri, May 4, 2018 at 1:41 PM, William Mills wmills@ti.com wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote: > On 05/04/2018 11:45 AM, Alexander Graf wrote: > I am missing something. If there is only MBR and no GPT then there is > no GUID type for EFI Partition. So how does the firmware find the "EFI > Partition" and the default /efi/boot/boot*.efi file? Does it just use > the partition with boot flag? There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need. Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR If using GPT, firmware should place needed files in either: * Vendor registered dir in EFI partition * GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
This is for all the other stuff vendors stick in custom partitions if we're lucky or some fixed offset if we're not, right?
Yes. Today people seem to be using a GPT partition for each of these low level objects and seem to be using the partition name to identify them. They are using very generic names like "boot".
This means that the disk image will only work with one SOC or perhaps even one board.
If people are going to use raw GPT partitions I think they should at least use a GUID GPT partition type to give certainty. Falling back to a name is OK and will help the casual user.
Bit 0 of the GPT attributes table is listed as "Platform required". Do current installers leave these partitions alone?
I'll leave that to one of the OS folks.
I double checked with our YaST people. The "Platform required" bit in GPT gets translated to the "hidden" bit in parted. That bit is then happily ignored by our installer.
So the installer may consider the partition ok to remove.
Alex
On Fri, May 04, 2018 at 02:41:56PM -0400, William Mills wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote:
On 05/04/2018 11:45 AM, Alexander Graf wrote: I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need.
As in you want to require a pure protective MBR and prohibit any form of hybriding?
I'm OK with this... much simpler than requiring distros never to touch the MBR of a GPT system!
Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR
I'd still prefer that EBBR compliant firmware (where firmware is loaded from shared media) have a protective partition describing the first 2MB at the point the firmware is installed to the shared media.
If using GPT, firmware should place needed files in either:
- Vendor registered dir in EFI partition
- GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Not sure about this.
For eMMC we'd prefer the boot ROM just load the firmware from the boot partition, then the OS can do whatever it wants with the main media (which could ship completely unpartitioned and leave it to the OS).
For UFS, the case is similar, although with multi-layer partitioning the terminology is hard. Either way we should rely on flash partitioning so that the firmware can be entirely separated by hardware partition. Again the flash partition to OS will be installed has not particular need to be partitioned in any way at all.
When booting from media without any hardware partitioning the case #3 scheme looks good although even there I'm not sure about the need to have an image work on unrelated boards though. To be useful it implies that EBBR implementations from different vendors can be combined in one disk image (i.e. that vendors will make EBBR implementations available with a license to redistribute them). We probably don't want to go there... even by implication!
Daniel.
On 08.05.18 16:38, Daniel Thompson wrote:
On Fri, May 04, 2018 at 02:41:56PM -0400, William Mills wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote:
On 05/04/2018 11:45 AM, Alexander Graf wrote: I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
GPT or MBR. The UEFI spec allows for either of them and so should we.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need.
As in you want to require a pure protective MBR and prohibit any form of hybriding?
I'm OK with this... much simpler than requiring distros never to touch the MBR of a GPT system!
Yes :). Case 1 and 2 are bascially identical.
Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR
I'd still prefer that EBBR compliant firmware (where firmware is loaded from shared media) have a protective partition describing the first 2MB at the point the firmware is installed to the shared media.
The problem with a protective partition is that getting changes into partitioning tools is really hard - and there are many of those out there :).
Also, I think we should only declare 1MB as reserved. If someone needs more space, they should really go the dedicated / separate partition route and use something like SPL to read from one of the options below.
The reason 1MB is so convenient is that basically all partitioning tools today already give you a natural 1MB alignment. So there are no OS changes needed.
If using GPT, firmware should place needed files in either:
- Vendor registered dir in EFI partition
- GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
No need to couple these to GPT. You can always have a vendor registered directory in the ESP with MBR as well.
For GUID partitions a GPT is obviously necessary.
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Not sure about this.
For eMMC we'd prefer the boot ROM just load the firmware from the boot partition, then the OS can do whatever it wants with the main media (which could ship completely unpartitioned and leave it to the OS).
For UFS, the case is similar, although with multi-layer partitioning the terminology is hard. Either way we should rely on flash partitioning so that the firmware can be entirely separated by hardware partition. Again the flash partition to OS will be installed has not particular need to be partitioned in any way at all.
When booting from media without any hardware partitioning the case #3 scheme looks good although even there I'm not sure about the need to have an image work on unrelated boards though. To be useful it implies that EBBR implementations from different vendors can be combined in one disk image (i.e. that vendors will make EBBR implementations available with a license to redistribute them). We probably don't want to go there... even by implication!
Well, with a fully U-Boot stack for example this should be easily achievable. I don't think it's a bad idea to suggest. Imagine a world where you could download a single image that just happens to run on all 96boards!
Alex
On Tue, May 08, 2018 at 05:43:51PM +0200, Alexander Graf wrote:
On 08.05.18 16:38, Daniel Thompson wrote:
On Fri, May 04, 2018 at 02:41:56PM -0400, William Mills wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote:
On 05/04/2018 11:45 AM, Alexander Graf wrote: I am missing something. If there is only MBR and no GPT then there is no GUID type for EFI Partition. So how does the firmware find the "EFI Partition" and the default /efi/boot/boot*.efi file? Does it just use the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
GPT or MBR. The UEFI spec allows for either of them and so should we.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need.
As in you want to require a pure protective MBR and prohibit any form of hybriding?
I'm OK with this... much simpler than requiring distros never to touch the MBR of a GPT system!
Yes :). Case 1 and 2 are bascially identical.
Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR
I'd still prefer that EBBR compliant firmware (where firmware is loaded from shared media) have a protective partition describing the first 2MB at the point the firmware is installed to the shared media.
The problem with a protective partition is that getting changes into partitioning tools is really hard - and there are many of those out there :).
Sorry, don't follow.
Why does requiring an EBBR compliant firmware to provide a protective partition describing the first N megabytes imply updating all partitioning tools? It is not the OS that has to create it; it is the firmware author and they can choose a working tool for this (possibly in expert mode to create the non-aligned partition).
Also, I think we should only declare 1MB as reserved. If someone needs more space, they should really go the dedicated / separate partition route and use something like SPL to read from one of the options below.
The reason 1MB is so convenient is that basically all partitioning tools today already give you a natural 1MB alignment. So there are no OS changes needed.
Again, don't entirely follow.
"they should really go the dedicated / separate partition route" implies that implies that an installer should try to recognise it as a protective partition and not nuke it (unless the user really, really insists).
That also implies it can leave a pre-existing protective partition covering the first N megabytes alone doesn't it?
If using GPT, firmware should place needed files in either:
- Vendor registered dir in EFI partition
- GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
No need to couple these to GPT. You can always have a vendor registered directory in the ESP with MBR as well.
For GUID partitions a GPT is obviously necessary.
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Not sure about this.
For eMMC we'd prefer the boot ROM just load the firmware from the boot partition, then the OS can do whatever it wants with the main media (which could ship completely unpartitioned and leave it to the OS).
For UFS, the case is similar, although with multi-layer partitioning the terminology is hard. Either way we should rely on flash partitioning so that the firmware can be entirely separated by hardware partition. Again the flash partition to OS will be installed has not particular need to be partitioned in any way at all.
When booting from media without any hardware partitioning the case #3 scheme looks good although even there I'm not sure about the need to have an image work on unrelated boards though. To be useful it implies that EBBR implementations from different vendors can be combined in one disk image (i.e. that vendors will make EBBR implementations available with a license to redistribute them). We probably don't want to go there... even by implication!
Well, with a fully U-Boot stack for example this should be easily achievable. I don't think it's a bad idea to suggest. Imagine a world where you could download a single image that just happens to run on all 96boards!
I don't object to it. I just think its somewhat irrelevant. I'm not sure the distros should be shipping all these vendor firmwares (since they can and IMHO should assume the firmware is already installed on the board) but if distros don't do it then who is going to make the single image and why?
I care *so* much more about the case where we expect good quality firmware located on the eMMC/UFS that is able to boot that single image from *removable* media and then run an installer that can install the OS to the eMMC/UFS without destroying the firmware by mistake.
Daniel.
On 08.05.18 18:09, Daniel Thompson wrote:
On Tue, May 08, 2018 at 05:43:51PM +0200, Alexander Graf wrote:
On 08.05.18 16:38, Daniel Thompson wrote:
On Fri, May 04, 2018 at 02:41:56PM -0400, William Mills wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf:
On 05/04/2018 06:20 PM, William Mills wrote: > On 05/04/2018 11:45 AM, Alexander Graf wrote: > I am missing something. If there is only MBR and no GPT then there is > no GUID type for EFI Partition. So how does the firmware find the "EFI > Partition" and the default /efi/boot/boot*.efi file? Does it just use > the partition with boot flag?
There is a special partition ID for the ESP (0xEF), but U-Boot currently just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
GPT or MBR. The UEFI spec allows for either of them and so should we.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need.
As in you want to require a pure protective MBR and prohibit any form of hybriding?
I'm OK with this... much simpler than requiring distros never to touch the MBR of a GPT system!
Yes :). Case 1 and 2 are bascially identical.
Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR
I'd still prefer that EBBR compliant firmware (where firmware is loaded from shared media) have a protective partition describing the first 2MB at the point the firmware is installed to the shared media.
The problem with a protective partition is that getting changes into partitioning tools is really hard - and there are many of those out there :).
Sorry, don't follow.
Why does requiring an EBBR compliant firmware to provide a protective partition describing the first N megabytes imply updating all partitioning tools? It is not the OS that has to create it; it is the firmware author and they can choose a working tool for this (possibly in expert mode to create the non-aligned partition).
A lot of OSs have installers which end up repartitioning existing disk targets. I wouldn't vouch for them to not nuke non-ESP partitions :).
Also, I think we should only declare 1MB as reserved. If someone needs more space, they should really go the dedicated / separate partition route and use something like SPL to read from one of the options below.
The reason 1MB is so convenient is that basically all partitioning tools today already give you a natural 1MB alignment. So there are no OS changes needed.
Again, don't entirely follow.
"they should really go the dedicated / separate partition route" implies that implies that an installer should try to recognise it as a protective partition and not nuke it (unless the user really, really insists).
That also implies it can leave a pre-existing protective partition covering the first N megabytes alone doesn't it?
If we conclude that reusing the ESP is good enough then we don't need to manually cover anything. We can simply declare 0.2MB (end of GPT) - 1MB (start of first partition) of the target disk as reserved for firmware purposes. If we make it as easy as shoving all additional resources into the ESP, everything will "just work".
If using GPT, firmware should place needed files in either:
- Vendor registered dir in EFI partition
- GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
No need to couple these to GPT. You can always have a vendor registered directory in the ESP with MBR as well.
For GUID partitions a GPT is obviously necessary.
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Not sure about this.
For eMMC we'd prefer the boot ROM just load the firmware from the boot partition, then the OS can do whatever it wants with the main media (which could ship completely unpartitioned and leave it to the OS).
For UFS, the case is similar, although with multi-layer partitioning the terminology is hard. Either way we should rely on flash partitioning so that the firmware can be entirely separated by hardware partition. Again the flash partition to OS will be installed has not particular need to be partitioned in any way at all.
When booting from media without any hardware partitioning the case #3 scheme looks good although even there I'm not sure about the need to have an image work on unrelated boards though. To be useful it implies that EBBR implementations from different vendors can be combined in one disk image (i.e. that vendors will make EBBR implementations available with a license to redistribute them). We probably don't want to go there... even by implication!
Well, with a fully U-Boot stack for example this should be easily achievable. I don't think it's a bad idea to suggest. Imagine a world where you could download a single image that just happens to run on all 96boards!
I don't object to it. I just think its somewhat irrelevant. I'm not sure the distros should be shipping all these vendor firmwares (since they can and IMHO should assume the firmware is already installed on the board) but if distros don't do it then who is going to make the single image and why?
I care *so* much more about the case where we expect good quality firmware located on the eMMC/UFS that is able to boot that single image from *removable* media and then run an installer that can install the OS to the eMMC/UFS without destroying the firmware by mistake.
I agree.
Alex
On Tue, May 08, 2018 at 06:25:11PM +0200, Alexander Graf wrote:
On 08.05.18 16:38, Daniel Thompson wrote:
On Fri, May 04, 2018 at 02:41:56PM -0400, William Mills wrote:
Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR
I'd still prefer that EBBR compliant firmware (where firmware is loaded from shared media) have a protective partition describing the first 2MB at the point the firmware is installed to the shared media.
The problem with a protective partition is that getting changes into partitioning tools is really hard - and there are many of those out there :).
Sorry, don't follow.
Why does requiring an EBBR compliant firmware to provide a protective partition describing the first N megabytes imply updating all partitioning tools? It is not the OS that has to create it; it is the firmware author and they can choose a working tool for this (possibly in expert mode to create the non-aligned partition).
A lot of OSs have installers which end up repartitioning existing disk targets. I wouldn't vouch for them to not nuke non-ESP partitions :).
I wouldn't wish to vouch for this either... so even if EBBR does allocate a partition ID I agree it could take a long time to gain support from installers. Its also clearly the case that there may exist boot ROMs that force a different ID anyway.
However putting a requirement on firmware to put a partition over the first megabyte doesn't contradict anything either and it does permits the OS installer to offer a better user experience (i.e. not nuking partitions) over time.
Daniel.
Also, I think we should only declare 1MB as reserved. If someone needs more space, they should really go the dedicated / separate partition route and use something like SPL to read from one of the options below.
The reason 1MB is so convenient is that basically all partitioning tools today already give you a natural 1MB alignment. So there are no OS changes needed.
Again, don't entirely follow.
"they should really go the dedicated / separate partition route" implies that implies that an installer should try to recognise it as a protective partition and not nuke it (unless the user really, really insists).
That also implies it can leave a pre-existing protective partition covering the first N megabytes alone doesn't it?
If we conclude that reusing the ESP is good enough then we don't need to manually cover anything. We can simply declare 0.2MB (end of GPT) - 1MB (start of first partition) of the target disk as reserved for firmware purposes. If we make it as easy as shoving all additional resources into the ESP, everything will "just work".
If using GPT, firmware should place needed files in either:
- Vendor registered dir in EFI partition
- GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
No need to couple these to GPT. You can always have a vendor registered directory in the ESP with MBR as well.
For GUID partitions a GPT is obviously necessary.
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Not sure about this.
For eMMC we'd prefer the boot ROM just load the firmware from the boot partition, then the OS can do whatever it wants with the main media (which could ship completely unpartitioned and leave it to the OS).
For UFS, the case is similar, although with multi-layer partitioning the terminology is hard. Either way we should rely on flash partitioning so that the firmware can be entirely separated by hardware partition. Again the flash partition to OS will be installed has not particular need to be partitioned in any way at all.
When booting from media without any hardware partitioning the case #3 scheme looks good although even there I'm not sure about the need to have an image work on unrelated boards though. To be useful it implies that EBBR implementations from different vendors can be combined in one disk image (i.e. that vendors will make EBBR implementations available with a license to redistribute them). We probably don't want to go there... even by implication!
Well, with a fully U-Boot stack for example this should be easily achievable. I don't think it's a bad idea to suggest. Imagine a world where you could download a single image that just happens to run on all 96boards!
I don't object to it. I just think its somewhat irrelevant. I'm not sure the distros should be shipping all these vendor firmwares (since they can and IMHO should assume the firmware is already installed on the board) but if distros don't do it then who is going to make the single image and why?
I care *so* much more about the case where we expect good quality firmware located on the eMMC/UFS that is able to boot that single image from *removable* media and then run an installer that can install the OS to the eMMC/UFS without destroying the firmware by mistake.
I agree.
Alex
On 05/08/2018 12:25 PM, Alexander Graf wrote:
On 08.05.18 18:09, Daniel Thompson wrote:
On Tue, May 08, 2018 at 05:43:51PM +0200, Alexander Graf wrote:
On 08.05.18 16:38, Daniel Thompson wrote:
On Fri, May 04, 2018 at 02:41:56PM -0400, William Mills wrote:
On 05/04/2018 01:03 PM, Andreas Färber wrote:
Am 04.05.2018 um 18:50 schrieb Alexander Graf: > On 05/04/2018 06:20 PM, William Mills wrote: >> On 05/04/2018 11:45 AM, Alexander Graf wrote: >> I am missing something. If there is only MBR and no GPT then there is >> no GUID type for EFI Partition. So how does the firmware find the "EFI >> Partition" and the default /efi/boot/boot*.efi file? Does it just use >> the partition with boot flag? > > There is a special partition ID for the ESP (0xEF), but U-Boot currently > just searches on every partition that's marked bootable.
... and falls back to the first partition if none on the device is marked bootable.
Thanks guys, that makes sense.
So the running summary in my head looks like this:
Case #1: Platforms that have separate storage for firmware and OS
OS storage is simple std GPT with no funny MBR stuff. One image may work on many boards.
GPT or MBR. The UEFI spec allows for either of them and so should we.
Case #2: Platforms where firmware and OS share single storage
Storage uses either GPT or MBR (but only one) based on board need.
As in you want to require a pure protective MBR and prohibit any form of hybriding?
I'm OK with this... much simpler than requiring distros never to touch the MBR of a GPT system!
Yes :). Case 1 and 2 are bascially identical.
Perhaps in layout but not from a use case POV. Your not suggesting that we merge the two use cases correct? Case 1 works on all EBBR platforms of the given architecture. Case 2 is dedicated to a given board, SOC, SOC family etc.
Image is dedicated for one machine type (or closely related set). If raw mode is needed use MBR with 2MB reserved space OS should not touch reserved space in MBR
I'd still prefer that EBBR compliant firmware (where firmware is loaded from shared media) have a protective partition describing the first 2MB at the point the firmware is installed to the shared media.
The problem with a protective partition is that getting changes into partitioning tools is really hard - and there are many of those out there :).
Sorry, don't follow.
Why does requiring an EBBR compliant firmware to provide a protective partition describing the first N megabytes imply updating all partitioning tools? It is not the OS that has to create it; it is the firmware author and they can choose a working tool for this (possibly in expert mode to create the non-aligned partition).
A lot of OSs have installers which end up repartitioning existing disk targets. I wouldn't vouch for them to not nuke non-ESP partitions :).
Also, I think we should only declare 1MB as reserved. If someone needs more space, they should really go the dedicated / separate partition route and use something like SPL to read from one of the options below.
The reason 1MB is so convenient is that basically all partitioning tools today already give you a natural 1MB alignment. So there are no OS changes needed.
Again, don't entirely follow.
"they should really go the dedicated / separate partition route" implies that implies that an installer should try to recognise it as a protective partition and not nuke it (unless the user really, really insists).
That also implies it can leave a pre-existing protective partition covering the first N megabytes alone doesn't it?
If we conclude that reusing the ESP is good enough then we don't need to manually cover anything. We can simply declare 0.2MB (end of GPT) - 1MB (start of first partition) of the target disk as reserved for firmware purposes. If we make it as easy as shoving all additional resources into the ESP, everything will "just work".
If using GPT, firmware should place needed files in either:
- Vendor registered dir in EFI partition
- GUID identified partitions w/ attribute bit 0 set (can fallback to using names if GUID not found)
No need to couple these to GPT. You can always have a vendor registered directory in the ESP with MBR as well.
For GUID partitions a GPT is obviously necessary.
Should we consider a future case #3? Case #3: New Platforms where firmware and OS share storage
Storage uses std GPT with no exceptions. Image may work with N preknown boards that are unrelated. Firmware files are all in a vendor specified dir in the EFI partition
Not sure about this.
For eMMC we'd prefer the boot ROM just load the firmware from the boot partition, then the OS can do whatever it wants with the main media (which could ship completely unpartitioned and leave it to the OS).
For UFS, the case is similar, although with multi-layer partitioning the terminology is hard. Either way we should rely on flash partitioning so that the firmware can be entirely separated by hardware partition. Again the flash partition to OS will be installed has not particular need to be partitioned in any way at all.
When booting from media without any hardware partitioning the case #3 scheme looks good although even there I'm not sure about the need to have an image work on unrelated boards though. To be useful it implies that EBBR implementations from different vendors can be combined in one disk image (i.e. that vendors will make EBBR implementations available with a license to redistribute them). We probably don't want to go there... even by implication!
Well, with a fully U-Boot stack for example this should be easily achievable. I don't think it's a bad idea to suggest. Imagine a world where you could download a single image that just happens to run on all 96boards!
Lots of people distribute card images for RPi and BeagleBone. Yes, some vendors may not license their binaries for redistribution but should we punish those that do allow it? This should not be a requirement on the firmware for EBBR but we should make provision for those that allow it.
I don't object to it. I just think its somewhat irrelevant. I'm not sure the distros should be shipping all these vendor firmwares (since they can and IMHO should assume the firmware is already installed on the board) but if distros don't do it then who is going to make the single image and why?
I care *so* much more about the case where we expect good quality firmware located on the eMMC/UFS that is able to boot that single image from *removable* media and then run an installer that can install the OS to the eMMC/UFS without destroying the firmware by mistake.
I agree.
Yes, this is a very real case.
Alex
boot-architecture@lists.linaro.org
-
Alexander Graf -
Andreas Färber -
Daniel Thompson -
Rob Herring -
William Mills