Recently my angry post on Google+ [1] got so many comments that it was clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC vendor companies I thought that this will be best one.
1. https://plus.google.com/u/0/+MarcinJuszkiewicz/posts/J79qhndV6FY
All started when I got Pine64 board (based on Allwinner A64 SoC) and had same issue as on several boards in past - boot loader written in some random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu boot loader knew how to read MBR partition table and was able to load 1st stage boot loader (called MLO) from it as long it was FAT filesystem.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT partitioning to find where from load kernel (or another boot loader).
And then we have all those boards where vendors decided that SPI flash for boot loader is too expensive so it will be read from SD card instead. From any random place of course...
Then we have distributions. And instead of generating bunch of images per board they want to make one clean image which will be able to handle as much as possible.
If there are UEFI machines on a list of supported ones then GPT partitioning will be used, boot loader will be stored in "EFI system area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot) ones. They are usually handled by taking image from previous stage and adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information about 128 partitions. Sure, no one is using so many on such devices but still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR - from 1KB - from 8KB - any other random place
So scripts start to be sets of magic written to handle all those SoCs...
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
Hi,
On 5 May 2016 at 05:45, Marcin Juszkiewicz marcin.juszkiewicz@linaro.org wrote:
Recently my angry post on Google+ [1] got so many comments that it was clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC vendor companies I thought that this will be best one.
All started when I got Pine64 board (based on Allwinner A64 SoC) and had same issue as on several boards in past - boot loader written in some random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu boot loader knew how to read MBR partition table and was able to load 1st stage boot loader (called MLO) from it as long it was FAT filesystem.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT partitioning to find where from load kernel (or another boot loader).
And then we have all those boards where vendors decided that SPI flash for boot loader is too expensive so it will be read from SD card instead. From any random place of course...
Then we have distributions. And instead of generating bunch of images per board they want to make one clean image which will be able to handle as much as possible.
If there are UEFI machines on a list of supported ones then GPT partitioning will be used, boot loader will be stored in "EFI system area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot) ones. They are usually handled by taking image from previous stage and adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information about 128 partitions. Sure, no one is using so many on such devices but still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR
- from 1KB
- from 8KB
- any other random place
So scripts start to be sets of magic written to handle all those SoCs...
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
This has bugged me for a long time. I see that Rockchip stores its boot information 64 blocks into the device, allowing room for the GPT. This seems like the lowest common denominator for the case you raise. Perhaps ARM could create a set of guidelines here?
I don't think the media type is the problem. Obviously different manufacturers are going to choose media depending on their preferences, cost, etc. But it shouldn't be too hard to describe the boot architecture of a machine in some sort of simple human-readable file, and provide a software library to deal with it.
Regards, Simon
W dniu 05.05.2016 o 15:26, Simon Glass pisze:
This has bugged me for a long time. I see that Rockchip stores its boot information 64 blocks into the device, allowing room for the GPT. This seems like the lowest common denominator for the case you raise. Perhaps ARM could create a set of guidelines here?
I don't think the media type is the problem.
When it comes to media all was about split between "firmware device" which can be 1-2MB of SPI flash with just boot loaders + "system device" which can be SD/SATA/whatever and contain system and user data.
On Thu, May 5, 2016 at 12:45 PM, Marcin Juszkiewicz marcin.juszkiewicz@linaro.org wrote:
Recently my angry post on Google+ [1] got so many comments that it was clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC vendor companies I thought that this will be best one.
All started when I got Pine64 board (based on Allwinner A64 SoC) and had same issue as on several boards in past - boot loader written in some random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu boot loader knew how to read MBR partition table and was able to load 1st stage boot loader (called MLO) from it as long it was FAT filesystem.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT partitioning to find where from load kernel (or another boot loader).
And then we have all those boards where vendors decided that SPI flash for boot loader is too expensive so it will be read from SD card instead. From any random place of course...
Then we have distributions. And instead of generating bunch of images per board they want to make one clean image which will be able to handle as much as possible.
If there are UEFI machines on a list of supported ones then GPT partitioning will be used, boot loader will be stored in "EFI system area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot) ones. They are usually handled by taking image from previous stage and adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information about 128 partitions. Sure, no one is using so many on such devices but still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR
- from 1KB
- from 8KB
- any other random place
So scripts start to be sets of magic written to handle all those SoCs...
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
To try and summarize, what you're asking for is to define the usage model for eMMC/SD when both the firmware* and OS are stored on the same media. Some argue that these things should always be on separate devices, but while the debate is interesting, it doesn't match the reality of how hardware is being built. In which case, the derived requirements are:
1) Co-exist with MBR partitioning 2) Co-exist with GPT partitioning 3) Be detectable --- partitioning tools must respect it 4) Be speced. Write it down so that tool and SoC developers can see it as a requirement 5) Be usable regardless of firmware type (UEFI, U-Boot, Little Kernel, etc) 6) Support some form of firmware non-volatile storage (variable storage)
It would be really nice if we could also have: 7) Support SoCs that hardcode boot code to specific locations (after-MBR, 1K, 8K, random) - May not be able to support all variants, but it is a worthy design goal.
Agreed?
* I'm ignoring eMMC's separate boot area because that solution has firmware and OS logically separated. Strong recommendation is for SoCs to boot from boot area. Then normal GPT/MBR partitioning works just fine. The rest of this discussion only applies If the SoC cannot do that
(For the following discussion, I refer to the UEFI spec because that is where GPT is defined, but the expectation is that anything described here can equally be used by non-UEFI platforms)
I've just read through the UEFI GPT spec, and here are the constraints: - MBR must be at the start of LBA0 (0 - 0.5k) - Primary GPT must be at the start of LBA1 (0.5k to 4k, but may collide with fw), - It /seems/ like the GPT Header and GPT table can be separated by some blocks. The GPT header has a PartitionEntryLBA field which describes where the actual table of partitions starts. - GPTHeader is only 92 bytes. - It should be possible to have: GPTHeader @ start of LBA1 and GPTPartitionTable @ an LBA that doesn't conflict with firmware.
I think we have everything we need to work around the location of the FW boot image without breaking the UEFI spec. The biggest problem is making sure partitioning tools don't go stomping over required firmware data and rendering systems unbootable. I *think* we can solve that problem by extending the MBR definition to block out a required region and then work around that. Tools can generically see the special region in the MBR and work around it accordingly.
So, let me try to itemize the use cases:
a. SoC boots from address immediately after MBR. - We're hooped. - Completely incompatible with current GPT spec. - Options: - Use MBR partitioning - move GPT to LBA > 1 - this creates a new question of how does the OS find the GPT? - I would prefer to treat this hardware as legacy and keep GPT at LBA1
b. SoC boots from fixed address > 1k and < 4k(?) - Place protective MBR at LBA0 - describe firmware block in MBR + protective GPT region - Modify spec to define this new format - Place GPT Header at LBA1 - Place GPT Partition Table > end of firmware LBA - Teach tools to understand & respect protective MBA when formatting disk
c. SoC boots from special MBR partition - place protective MBA at LBA0 (same as b.) - Describe FW partition in MBR - Perhaps mark as movable? - Vendor tools must place FW partition > 4k - Modify spec to define this new format (same format as b.) - Place GPT Header and partition table normally. - Tools should notice FW partition isn't in the way - Generate the GPT in the normal way - Either: - Add FW partition to the GPT table, or - set FirstUsableLBA to be immediately after the FW partition (safest?)
d. SoC boots from GPT partition. - Add new partition type that tools can recognize and refuse to delete unless the user forces it. - Define the new partition type in the UEFI spec, along with the required behaviour
I think that covers it, but I haven't yet talked through what to do with firmware non-volatile storage. It could be handled as another special firmware volume, or it could be treated as appended to the fw image. Need a bit more thought here.
If this looks good, then I can write up a proposed change to the spec and everyone can go look at the impact on firmware, the kernel, and tools.
g.
On Thursday, May 5, 2016 1:45:57 PM CDT Marcin Juszkiewicz wrote:
Recently my angry post on Google+ [1] got so many comments that it was clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC vendor companies I thought that this will be best one.
All started when I got Pine64 board (based on Allwinner A64 SoC) and had same issue as on several boards in past - boot loader written in some random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu boot loader knew how to read MBR partition table and was able to load 1st stage boot loader (called MLO) from it as long it was FAT filesystem.
There is terribleness here as well. It was very fragile. dding the binaries into the correct offsets is much more reliable.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT partitioning to find where from load kernel (or another boot loader).
vboot brings about many other issues here as well. its not a great setup if you want to run anything other than chromeos
And then we have all those boards where vendors decided that SPI flash for boot loader is too expensive so it will be read from SD card instead. From any random place of course...
The locations are not really random, it is what is baked into the bootrom on the silicon and is SOC specific.
Then we have distributions. And instead of generating bunch of images per board they want to make one clean image which will be able to handle as much as possible.
If there are UEFI machines on a list of supported ones then GPT partitioning will be used, boot loader will be stored in "EFI system area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot) ones. They are usually handled by taking image from previous stage and adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information about 128 partitions. Sure, no one is using so many on such devices but still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR
- from 1KB
- from 8KB
- any other random place
So scripts start to be sets of magic written to handle all those SoCs...
This is why I did all the work to support extlinux.conf on arm, take away the magic, make it simple for users. but that does require that you have u-boot somewhere first.
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
96 boards failed misserably in my eye when they did not mandate flash storage for the bootloader
Is there a way to get it improved? At least for new designs?
combine together as a community and refuse to support hardware that does not meet minimum requirements, battery backed RTC and flash for initial board startup at the least.
Take the tegra jetson TK1, update the bootloader to upstream u-boot, you get PXE install and boot support, if you dd a Fedora image to a sd card and put it in it just boots and gives a good user experience. same is true if you update the u-boot on the trimslice. Solid Run is doing a terrible job here all of there devices require that you put u-boot somewhere, sdcard, start of ssd etc.
Dennis
On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz marcin.juszkiewicz@linaro.org wrote:
<snip>
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
Yes! I've added this suggestion to a list of suggestions for evolution of the 96boards spec.
On Thu, May 05, 2016 at 09:01:05PM +0530, Amit Kucheria wrote:
On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
Yes! I've added this suggestion to a list of suggestions for evolution of the 96boards spec.
We already went round the houses repeatedly on that one :(
On Thu, May 5, 2016 at 4:59 PM, Mark Brown broonie@kernel.org wrote:
On Thu, May 05, 2016 at 09:01:05PM +0530, Amit Kucheria wrote:
On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
Yes! I've added this suggestion to a list of suggestions for evolution of the 96boards spec.
We already went round the houses repeatedly on that one :(
It's not so dismal as all that! With the board size and price point the CE spec aims at, it is very unlikely that vendors will build boards with a separate storage device for FW.* However, by and large these platforms will be built with eMMC and eMMC has a separate boot area that is logically separate from the main pool. We can add a requirement for future versions to boot from the boot area when booting from eMMC.
While on the topic of the spec, we need to handle "optional" items in a better way. Rather than having toothless "optional" requirements. How about we have 2 levels of compliance. Something like "Silver" compliance which covers all the absolutely required items, and "Gold" which also implements the optional items. I bring this up because there are several SoCs that vendors want to build board for, but they many not be able to make the business case for respining the chip to meet an item of the spec. Such as device the SoC boot from.
* although anyone who really wants to treat it as separate storage can happily insert an SD card and dedicate the eMMC to firmware -- really, how is that different?.
On Thu, May 05, 2016 at 04:21:59PM +0100, Grant Likely wrote:
I think we have everything we need to work around the location of the FW boot image without breaking the UEFI spec. The biggest problem is making sure partitioning tools don't go stomping over required firmware data and rendering systems unbootable. I *think* we can solve that problem by extending the MBR definition to block out a required region and then work around that. Tools can generically see the special region in the MBR and work around it accordingly.
So, let me try to itemize the use cases:
I think there's one other slightly different angle on this which we should address at the same time, creating fresh boot media for a device ("I just unpacked my board and want to write a system image to a SD card"). If we can come up with a standard way of describing the requirements of boards then we could provide a shared database of this information that tools could use. This might also be useful for the less helpful requirements where it's hard to figure out what's going on from the media itself.
On Thu, May 5, 2016 at 5:53 PM, Mark Brown broonie@kernel.org wrote:
On Thu, May 05, 2016 at 04:21:59PM +0100, Grant Likely wrote:
I think we have everything we need to work around the location of the FW boot image without breaking the UEFI spec. The biggest problem is making sure partitioning tools don't go stomping over required firmware data and rendering systems unbootable. I *think* we can solve that problem by extending the MBR definition to block out a required region and then work around that. Tools can generically see the special region in the MBR and work around it accordingly.
So, let me try to itemize the use cases:
I think there's one other slightly different angle on this which we should address at the same time, creating fresh boot media for a device ("I just unpacked my board and want to write a system image to a SD card"). If we can come up with a standard way of describing the requirements of boards then we could provide a shared database of this information that tools could use. This might also be useful for the less helpful requirements where it's hard to figure out what's going on from the media itself.
Are you talking about for boards that don't have on-board media? For the boards with eMMC, I think the normal use-cases should always be to boot from eMMC (regardless of where the OS is installed). Booting from an SD shouldn't require anything special. The only time to boot firmware from SD is when upgrading the platform firmware, or doing firmware development.
For systems without on-board storage, yes I agree it would be good to have a standard way of describing the layout because the images will always need to be tailored for the board.
g.
On Thu, May 05, 2016 at 06:03:40PM +0100, Grant Likely wrote:
On Thu, May 5, 2016 at 5:53 PM, Mark Brown broonie@kernel.org wrote:
I think there's one other slightly different angle on this which we should address at the same time, creating fresh boot media for a device ("I just unpacked my board and want to write a system image to a SD card"). If we can come up with a standard way of describing the requirements of boards then we could provide a shared database of this information that tools could use. This might also be useful for the less helpful requirements where it's hard to figure out what's going on from the media itself.
Are you talking about for boards that don't have on-board media? For the boards with eMMC, I think the normal use-cases should always be to boot from eMMC (regardless of where the OS is installed). Booting from an SD shouldn't require anything special. The only time to boot firmware from SD is when upgrading the platform firmware, or doing firmware development.
For systems without on-board storage, yes I agree it would be good to have a standard way of describing the layout because the images will always need to be tailored for the board.
Yes, boards that don't have on board media are the big issue here - they're typically the lowest cost and hence quite widely used. The same infrastructure might also help deal with some of the oddballs that have unhelpful on board storage but that's much less important if it's even useful at all.
On Thu, May 5, 2016 at 9:29 PM, Mark Brown broonie@kernel.org wrote:
On Thu, May 05, 2016 at 09:01:05PM +0530, Amit Kucheria wrote:
On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
Yes! I've added this suggestion to a list of suggestions for evolution of the 96boards spec.
We already went round the houses repeatedly on that one :(
Yes, but it wasn't in the list. So if we want to reject something (or make it optional), I'd like us to document why we reached that decision for posterity.
As Grant mentioned elsewhere, the single-storage device configuration seems to be where a lot of the vendors seem to be focusing on.
I would add, that you need to draw a line in the sand between the 'consumer' (don't flame me, I am just struggling to find a better term) boards and those that are positioned for production/enterprise. We can't state that there is only one way that is the right way, that's not fair to anyone, and counter-culture to the opensource view. We can recommend a best practice, and I think this thread is a really good start at that, but it will always be a view that one can offer to the community. Adoption is a completely other thing.
Specifically for the 96boards, the spec is a recommended view, but its not meant to be constraining, however it does allow one to then show a best practice, that others can adopt. That's where the RPB comes in to play, again to demonstrate and not restrict.
On 5 May 2016 at 19:34, Amit Kucheria amit.kucheria@linaro.org wrote:
On Thu, May 5, 2016 at 9:29 PM, Mark Brown broonie@kernel.org wrote:
On Thu, May 05, 2016 at 09:01:05PM +0530, Amit Kucheria wrote:
On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz
Solution for existing SoCs is usually adding 1MB of SPI flash during
design
phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it
could be a
way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
Yes! I've added this suggestion to a list of suggestions for evolution of the 96boards spec.
We already went round the houses repeatedly on that one :(
Yes, but it wasn't in the list. So if we want to reject something (or make it optional), I'd like us to document why we reached that decision for posterity.
As Grant mentioned elsewhere, the single-storage device configuration seems to be where a lot of the vendors seem to be focusing on. _______________________________________________ cross-distro mailing list cross-distro@lists.linaro.org https://lists.linaro.org/mailman/listinfo/cross-distro
On Thu, May 05, 2016 at 07:47:41PM +0100, Martin Stadtler wrote:
Specifically for the 96boards, the spec is a recommended view, but its not meant to be constraining, however it does allow one to then show a best practice, that others can adopt. That's where the RPB comes in to play, again to demonstrate and not restrict.
Note that this is also going to the cross-distro list, we're probably getting a bit off topic with Linaro/96boards internal discussions like this.
On Thu, May 5, 2016 at 11:50 AM Martin Stadtler martin.stadtler@linaro.org wrote:
Specifically for the 96boards, the spec is a recommended view, but its not meant to be constraining, however it does allow one to then show a best practice, that others can adopt. That's where the RPB comes in to play, again to demonstrate and not restrict.
Sorry to jump in on this, since my horse in this race is pretty small...
That whole "best practice" point is REALLY important. But it's more nuanced than just "do it this way":
A. Vendors will do what they do, and they'll have their reasons. If the community offers them a "best practices" guideline, especially one that's easy to adopt (in full or in part), then hopefully they'll be less likely to stray.
B. If that best-practices document offers more than a one-size-fits-all recommendation, then so much the better. Again, it keeps necessary variants closer to the community than they might have been; a partial victory isn't a total loss.
C. (This is my main point.) When I see a document that says "best practices", then I understand that there's no binding requirement per se, and that it's likely that there will be deviations.
D. When that's the case, I'm more likely to produce code that's easily adapted to the various permutations of best-practices that I encounter, often with the blessing of my customer. Doubly so if those variations are predictable.
Otherwise, a customer will tell me to just "code to the requirement"---and that's all they'll pay for. Then my solution isn't likely to be as widely deployable, which cuts off opportunities to recycle that solution back to my point (A), even if the code becomes public.
I can't always code for every possibility. But, with a best-practices guideline that says "if you can do it X way, then do so; if you can't, but you can do Y, that's almost as good; else, for gods' sake don't do Z", I can better plan for where the changes might arise later.
Maybe I won't code the solution for everyone, but I'm likely to get a lot closer than I would have.
Just my $0.02.
b.g.
On Thu, May 5, 2016 at 11:50 AM Martin Stadtler martin.stadtler@linaro.org wrote:
I would add, that you need to draw a line in the sand between the 'consumer' (don't flame me, I am just struggling to find a better term) boards and those that are positioned for production/enterprise.
One way to draw the line is whether the board is delivered to be used as-is, vs. with the expectation of further development activities that will change the work product that shipped with the board.
In the first case, all you care is that it works when it leaves the dock. In the latter case, you care more that it can't be easily broken later.
That's a pretty big difference in my experience.
b.g.
+1
On 5 May 2016 at 20:08, Bill Gatliff bgat@billgatliff.com wrote:
On Thu, May 5, 2016 at 11:50 AM Martin Stadtler < martin.stadtler@linaro.org> wrote:
I would add, that you need to draw a line in the sand between the 'consumer' (don't flame me, I am just struggling to find a better term) boards and those that are positioned for production/enterprise.
One way to draw the line is whether the board is delivered to be used as-is, vs. with the expectation of further development activities that will change the work product that shipped with the board.
In the first case, all you care is that it works when it leaves the dock. In the latter case, you care more that it can't be easily broken later.
That's a pretty big difference in my experience.
b.g.
Bill Gatliff Embedded Linux training and consulting bgat@billgatliff.com (309) 453-3421
On 05/05/2016 12:05 PM, Bill Gatliff wrote:
On Thu, May 5, 2016 at 11:50 AM Martin Stadtler <martin.stadtler@linaro.org mailto:martin.stadtler@linaro.org> wrote:
Specifically for the 96boards, the spec is a recommended view, but its not meant to be constraining, however it does allow one to then show a best practice, that others can adopt. That's where the RPB comes in to play, again to demonstrate and not restrict.Sorry to jump in on this, since my horse in this race is pretty small...
That whole "best practice" point is REALLY important. But it's more nuanced than just "do it this way":
A. Vendors will do what they do, and they'll have their reasons. If the community offers them a "best practices" guideline, especially one that's easy to adopt (in full or in part), then hopefully they'll be less likely to stray.
B. If that best-practices document offers more than a one-size-fits-all recommendation, then so much the better. Again, it keeps necessary variants closer to the community than they might have been; a partial victory isn't a total loss.
C. (This is my main point.) When I see a document that says "best practices", then I understand that there's no binding requirement per se, and that it's likely that there will be deviations.
D. When that's the case, I'm more likely to produce code that's easily adapted to the various permutations of best-practices that I encounter, often with the blessing of my customer. Doubly so if those variations are predictable.
Otherwise, a customer will tell me to just "code to the requirement"---and that's all they'll pay for. Then my solution isn't likely to be as widely deployable, which cuts off opportunities to recycle that solution back to my point (A), even if the code becomes public.
I can't always code for every possibility. But, with a best-practices guideline that says "if you can do it X way, then do so; if you can't, but you can do Y, that's almost as good; else, for gods' sake don't do Z", I can better plan for where the changes might arise later.
Maybe I won't code the solution for everyone, but I'm likely to get a lot closer than I would have.
One thing I like to see in Best Practices guide is what is the benefit of the following the best practice. From that it is much easier to make the assessment of whether the promised result is worth following the guidance. All of the best practices people here are talking about appear to be geared toward a frictionless connection to the ARM Linux ecosystem. That's something many software focused Linaro participants care about, but is that something manufacturers care about? Usually I only hear about saving pennies leading to profits at scale being a priority. So if we can talk up gaining scale by following the practices, there's a better chance members will listen.
On 05.05.16 17:21, Grant Likely wrote:
On Thu, May 5, 2016 at 12:45 PM, Marcin Juszkiewicz marcin.juszkiewicz@linaro.org wrote:
Recently my angry post on Google+ [1] got so many comments that it was clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC vendor companies I thought that this will be best one.
All started when I got Pine64 board (based on Allwinner A64 SoC) and had same issue as on several boards in past - boot loader written in some random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu boot loader knew how to read MBR partition table and was able to load 1st stage boot loader (called MLO) from it as long it was FAT filesystem.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT partitioning to find where from load kernel (or another boot loader).
And then we have all those boards where vendors decided that SPI flash for boot loader is too expensive so it will be read from SD card instead. From any random place of course...
Then we have distributions. And instead of generating bunch of images per board they want to make one clean image which will be able to handle as much as possible.
If there are UEFI machines on a list of supported ones then GPT partitioning will be used, boot loader will be stored in "EFI system area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot) ones. They are usually handled by taking image from previous stage and adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information about 128 partitions. Sure, no one is using so many on such devices but still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR
- from 1KB
- from 8KB
- any other random place
So scripts start to be sets of magic written to handle all those SoCs...
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
To try and summarize, what you're asking for is to define the usage model for eMMC/SD when both the firmware* and OS are stored on the same media. Some argue that these things should always be on separate devices, but while the debate is interesting, it doesn't match the reality of how hardware is being built. In which case, the derived requirements are:
- Co-exist with MBR partitioning
- Co-exist with GPT partitioning
- Be detectable --- partitioning tools must respect it
- Be speced. Write it down so that tool and SoC developers can see it
as a requirement 5) Be usable regardless of firmware type (UEFI, U-Boot, Little Kernel, etc) 6) Support some form of firmware non-volatile storage (variable storage)
It would be really nice if we could also have: 7) Support SoCs that hardcode boot code to specific locations (after-MBR, 1K, 8K, random)
- May not be able to support all variants, but it is a worthy design goal.
Agreed?
- I'm ignoring eMMC's separate boot area because that solution has
firmware and OS logically separated. Strong recommendation is for SoCs to boot from boot area. Then normal GPT/MBR partitioning works just fine. The rest of this discussion only applies If the SoC cannot do that
(For the following discussion, I refer to the UEFI spec because that is where GPT is defined, but the expectation is that anything described here can equally be used by non-UEFI platforms)
I've just read through the UEFI GPT spec, and here are the constraints:
- MBR must be at the start of LBA0 (0 - 0.5k)
- Primary GPT must be at the start of LBA1 (0.5k to 4k, but may
collide with fw),
- It /seems/ like the GPT Header and GPT table can be separated by
some blocks. The GPT header has a PartitionEntryLBA field which describes where the actual table of partitions starts.
- GPTHeader is only 92 bytes.
- It should be possible to have: GPTHeader @ start of LBA1 and
GPTPartitionTable @ an LBA that doesn't conflict with firmware.
I think we have everything we need to work around the location of the FW boot image without breaking the UEFI spec. The biggest problem is making sure partitioning tools don't go stomping over required firmware data and rendering systems unbootable. I *think* we can solve that problem by extending the MBR definition to block out a required region and then work around that. Tools can generically see the special region in the MBR and work around it accordingly.
So what's the goal here? Are we trying to force GPT on systems whose vendors never intended them to run with GPT?
It really shouldn't matter at the end of the day whether we use GPT or MBR. All uEFI firmware implementations I'm aware of support both. So if you have a device whose bootloader collides with the GPT, just use MBR.
As for the "protection" mechanism, I don't think it's a problem either. IIRC parted starts to create partitions with a sensible alignment (1 or 2MB). Most boot loaders fit perfectly fine within that gap.
So this really boils down to
- use GPT for systems that were designed for it or - use MBR with alignment gap to first partition
end of story. There shouldn't be any middle ground. At least I haven't seen any so far :).
Alex
On Thu, May 05, 2016 at 12:44:57PM -0700, Brendan Conoboy wrote:
All of the best practices people here are talking about appear to be geared toward a frictionless connection to the ARM Linux ecosystem. That's something many software focused Linaro participants care about, but is that something manufacturers care about? Usually I only hear about saving pennies leading to profits at scale being a priority. So if we can talk up gaining scale by following the practices, there's a better chance members will listen.
The obvious angle on that is that if it's easier to pick up and just use work from the ecosystem then that's less special integration work that's needed per system reducing costs at the system vendor level.
On Thu, May 05, 2016 at 10:21:25PM +0200, Alexander Graf wrote:
On 05.05.16 17:21, Grant Likely wrote:
On Thu, May 5, 2016 at 12:45 PM, Marcin Juszkiewicz marcin.juszkiewicz@linaro.org wrote:
Recently my angry post on Google+ [1] got so many comments that it was clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC vendor companies I thought that this will be best one.
All started when I got Pine64 board (based on Allwinner A64 SoC) and had same issue as on several boards in past - boot loader written in some random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu boot loader knew how to read MBR partition table and was able to load 1st stage boot loader (called MLO) from it as long it was FAT filesystem.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT partitioning to find where from load kernel (or another boot loader).
And then we have all those boards where vendors decided that SPI flash for boot loader is too expensive so it will be read from SD card instead. From any random place of course...
Then we have distributions. And instead of generating bunch of images per board they want to make one clean image which will be able to handle as much as possible.
If there are UEFI machines on a list of supported ones then GPT partitioning will be used, boot loader will be stored in "EFI system area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot) ones. They are usually handled by taking image from previous stage and adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information about 128 partitions. Sure, no one is using so many on such devices but still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR
- from 1KB
- from 8KB
- any other random place
So scripts start to be sets of magic written to handle all those SoCs...
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
To try and summarize, what you're asking for is to define the usage model for eMMC/SD when both the firmware* and OS are stored on the same media. Some argue that these things should always be on separate devices, but while the debate is interesting, it doesn't match the reality of how hardware is being built. In which case, the derived requirements are:
- Co-exist with MBR partitioning
- Co-exist with GPT partitioning
- Be detectable --- partitioning tools must respect it
- Be speced. Write it down so that tool and SoC developers can see it
as a requirement 5) Be usable regardless of firmware type (UEFI, U-Boot, Little Kernel, etc) 6) Support some form of firmware non-volatile storage (variable storage)
It would be really nice if we could also have: 7) Support SoCs that hardcode boot code to specific locations (after-MBR, 1K, 8K, random)
- May not be able to support all variants, but it is a worthy design goal.
Agreed?
- I'm ignoring eMMC's separate boot area because that solution has
firmware and OS logically separated. Strong recommendation is for SoCs to boot from boot area. Then normal GPT/MBR partitioning works just fine. The rest of this discussion only applies If the SoC cannot do that
(For the following discussion, I refer to the UEFI spec because that is where GPT is defined, but the expectation is that anything described here can equally be used by non-UEFI platforms)
I've just read through the UEFI GPT spec, and here are the constraints:
- MBR must be at the start of LBA0 (0 - 0.5k)
- Primary GPT must be at the start of LBA1 (0.5k to 4k, but may
collide with fw),
- It /seems/ like the GPT Header and GPT table can be separated by
some blocks. The GPT header has a PartitionEntryLBA field which describes where the actual table of partitions starts.
- GPTHeader is only 92 bytes.
- It should be possible to have: GPTHeader @ start of LBA1 and
GPTPartitionTable @ an LBA that doesn't conflict with firmware.
I think we have everything we need to work around the location of the FW boot image without breaking the UEFI spec. The biggest problem is making sure partitioning tools don't go stomping over required firmware data and rendering systems unbootable. I *think* we can solve that problem by extending the MBR definition to block out a required region and then work around that. Tools can generically see the special region in the MBR and work around it accordingly.
So what's the goal here? Are we trying to force GPT on systems whose vendors never intended them to run with GPT?
It really shouldn't matter at the end of the day whether we use GPT or MBR. All uEFI firmware implementations I'm aware of support both. So if you have a device whose bootloader collides with the GPT, just use MBR.
As for the "protection" mechanism, I don't think it's a problem either. IIRC parted starts to create partitions with a sensible alignment (1 or 2MB). Most boot loaders fit perfectly fine within that gap.
So this really boils down to
- use GPT for systems that were designed for it or
- use MBR with alignment gap to first partition
end of story. There shouldn't be any middle ground. At least I haven't seen any so far :).
This, for many use cases is also true. A reason that various SoCs pick a magic location that is MBR compatible and not strictly GPT compatible is that they don't see a use case for GPT-not-MBR being used. By-and-large saying your SD card shall have an MBR and shall leave a gap that is also well aligned for the card anyhow is enough for (enough) firmware to reside in the magic locations.
On Thu, May 05, 2016 at 05:30:55PM +0100, Grant Likely wrote:
On Thu, May 5, 2016 at 4:59 PM, Mark Brown broonie@kernel.org wrote:
On Thu, May 05, 2016 at 09:01:05PM +0530, Amit Kucheria wrote:
On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
Yes! I've added this suggestion to a list of suggestions for evolution of the 96boards spec.
We already went round the houses repeatedly on that one :(
It's not so dismal as all that! With the board size and price point the CE spec aims at, it is very unlikely that vendors will build boards with a separate storage device for FW.* However, by and large these platforms will be built with eMMC and eMMC has a separate boot area that is logically separate from the main pool. We can add a requirement for future versions to boot from the boot area when booting from eMMC.
Saying that supporting the eMMC boot partitions shall be bootable AND documenting how the various relevant EXT_CSD flags need to be set be public is important. It won't do everyone good if you can't program the board to boot from the boot partition.
On Fri, May 6, 2016 at 2:10 AM, Tom Rini trini@konsulko.com wrote:
On Thu, May 05, 2016 at 05:30:55PM +0100, Grant Likely wrote:
On Thu, May 5, 2016 at 4:59 PM, Mark Brown broonie@kernel.org wrote:
On Thu, May 05, 2016 at 09:01:05PM +0530, Amit Kucheria wrote:
On Thu, May 5, 2016 at 5:15 PM, Marcin Juszkiewicz
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could be a way of showing how to make popular board. Instead it became yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
Yes! I've added this suggestion to a list of suggestions for evolution of the 96boards spec.
We already went round the houses repeatedly on that one :(
It's not so dismal as all that! With the board size and price point the CE spec aims at, it is very unlikely that vendors will build boards with a separate storage device for FW.* However, by and large these platforms will be built with eMMC and eMMC has a separate boot area that is logically separate from the main pool. We can add a requirement for future versions to boot from the boot area when booting from eMMC.
Saying that supporting the eMMC boot partitions shall be bootable AND documenting how the various relevant EXT_CSD flags need to be set be public is important. It won't do everyone good if you can't program the board to boot from the boot partition.
Yes, of course. This is definitely an aspirational item. It doesn't do any good for current SoCs that already have the boot scheme fixed, but it is a place where we can put pressure on vendors for what to do in future SoCs.
g.
On Fri, May 6, 2016 at 2:10 AM, Tom Rini trini@konsulko.com wrote:
On Thu, May 05, 2016 at 10:21:25PM +0200, Alexander Graf wrote:
On 05.05.16 17:21, Grant Likely wrote:
On Thu, May 5, 2016 at 12:45 PM, Marcin Juszkiewicz marcin.juszkiewicz@linaro.org wrote:
Recently my angry post on Google+ [1] got so many comments that it was clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC vendor companies I thought that this will be best one.
All started when I got Pine64 board (based on Allwinner A64 SoC) and had same issue as on several boards in past - boot loader written in some random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu boot loader knew how to read MBR partition table and was able to load 1st stage boot loader (called MLO) from it as long it was FAT filesystem.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT partitioning to find where from load kernel (or another boot loader).
And then we have all those boards where vendors decided that SPI flash for boot loader is too expensive so it will be read from SD card instead. From any random place of course...
Then we have distributions. And instead of generating bunch of images per board they want to make one clean image which will be able to handle as much as possible.
If there are UEFI machines on a list of supported ones then GPT partitioning will be used, boot loader will be stored in "EFI system area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot) ones. They are usually handled by taking image from previous stage and adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information about 128 partitions. Sure, no one is using so many on such devices but still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR
- from 1KB
- from 8KB
- any other random place
So scripts start to be sets of magic written to handle all those SoCs...
Solution for existing SoCs is usually adding 1MB of SPI flash during design phase of device and store boot loader(s) there. But it is so expensive someone would say when it is in 10-30 cents range...
To try and summarize, what you're asking for is to define the usage model for eMMC/SD when both the firmware* and OS are stored on the same media. Some argue that these things should always be on separate devices, but while the debate is interesting, it doesn't match the reality of how hardware is being built. In which case, the derived requirements are:
- Co-exist with MBR partitioning
- Co-exist with GPT partitioning
- Be detectable --- partitioning tools must respect it
- Be speced. Write it down so that tool and SoC developers can see it
as a requirement 5) Be usable regardless of firmware type (UEFI, U-Boot, Little Kernel, etc) 6) Support some form of firmware non-volatile storage (variable storage)
It would be really nice if we could also have: 7) Support SoCs that hardcode boot code to specific locations (after-MBR, 1K, 8K, random)
- May not be able to support all variants, but it is a worthy design goal.
Agreed?
- I'm ignoring eMMC's separate boot area because that solution has
firmware and OS logically separated. Strong recommendation is for SoCs to boot from boot area. Then normal GPT/MBR partitioning works just fine. The rest of this discussion only applies If the SoC cannot do that
(For the following discussion, I refer to the UEFI spec because that is where GPT is defined, but the expectation is that anything described here can equally be used by non-UEFI platforms)
I've just read through the UEFI GPT spec, and here are the constraints:
- MBR must be at the start of LBA0 (0 - 0.5k)
- Primary GPT must be at the start of LBA1 (0.5k to 4k, but may
collide with fw),
- It /seems/ like the GPT Header and GPT table can be separated by
some blocks. The GPT header has a PartitionEntryLBA field which describes where the actual table of partitions starts.
- GPTHeader is only 92 bytes.
- It should be possible to have: GPTHeader @ start of LBA1 and
GPTPartitionTable @ an LBA that doesn't conflict with firmware.
I think we have everything we need to work around the location of the FW boot image without breaking the UEFI spec. The biggest problem is making sure partitioning tools don't go stomping over required firmware data and rendering systems unbootable. I *think* we can solve that problem by extending the MBR definition to block out a required region and then work around that. Tools can generically see the special region in the MBR and work around it accordingly.
So what's the goal here? Are we trying to force GPT on systems whose vendors never intended them to run with GPT?
It really shouldn't matter at the end of the day whether we use GPT or MBR. All uEFI firmware implementations I'm aware of support both. So if you have a device whose bootloader collides with the GPT, just use MBR.
As for the "protection" mechanism, I don't think it's a problem either. IIRC parted starts to create partitions with a sensible alignment (1 or 2MB). Most boot loaders fit perfectly fine within that gap.
So this really boils down to
- use GPT for systems that were designed for it or
- use MBR with alignment gap to first partition
end of story. There shouldn't be any middle ground. At least I haven't seen any so far :).
That is a good point. MBR is a pain to deal with, but I don't think there is anything that it is absolutely required for in UEFI land.
This, for many use cases is also true. A reason that various SoCs pick a magic location that is MBR compatible and not strictly GPT compatible is that they don't see a use case for GPT-not-MBR being used. By-and-large saying your SD card shall have an MBR and shall leave a gap that is also well aligned for the card anyhow is enough for (enough) firmware to reside in the magic locations.
I think the key issue is how do the tools know what they are allowed to do. If (for example) the system is booted into a recovery/install image and needs to repartition and install onto the eMMC, can we get away from the tools requiring board specific knowledge? A generic partitioning/install tool needs to know: - Is an MBR required (ie. SoC reads MBR to find firmware) - Is FW location at a fixed offset? - Is GPT supported?
If the tools can get that information, then it can make good decisions when reimaging a device, and it will make Marcin happier I'm sure. :-)
I /don't/ think the general tools should need to know how to install the firmware itself. That is still a nasty board-specific problem.... (although even here we could make things better if we had a spec for managing the SoC's FW partition. I would like to see separate steps for FW provisioning, and OS install. ie. A board-specific tool to prep an SD card with the bare minimum for FW, and then generic tools to complete partitioning and install.
For on-board devices (eMMC), FW provisioning should only be needed once For removable devices (SD), FW provisioning is only needed when FW must be on SD. (ie. no-onboard eMMC, or for FW recovery/upgrade)
g.
On 06.05.16 13:03, Grant Likely wrote:
On Fri, May 6, 2016 at 2:10 AM, Tom Rini trini@konsulko.com wrote:
On Thu, May 05, 2016 at 10:21:25PM +0200, Alexander Graf wrote:
So what's the goal here? Are we trying to force GPT on systems whose vendors never intended them to run with GPT?
It really shouldn't matter at the end of the day whether we use GPT or MBR. All uEFI firmware implementations I'm aware of support both. So if you have a device whose bootloader collides with the GPT, just use MBR.
As for the "protection" mechanism, I don't think it's a problem either. IIRC parted starts to create partitions with a sensible alignment (1 or 2MB). Most boot loaders fit perfectly fine within that gap.
So this really boils down to
- use GPT for systems that were designed for it or
- use MBR with alignment gap to first partition
end of story. There shouldn't be any middle ground. At least I haven't seen any so far :).
That is a good point. MBR is a pain to deal with, but I don't think there is anything that it is absolutely required for in UEFI land.
This, for many use cases is also true. A reason that various SoCs pick a magic location that is MBR compatible and not strictly GPT compatible is that they don't see a use case for GPT-not-MBR being used. By-and-large saying your SD card shall have an MBR and shall leave a gap that is also well aligned for the card anyhow is enough for (enough) firmware to reside in the magic locations.
I think the key issue is how do the tools know what they are allowed to do. If (for example) the system is booted into a recovery/install image and needs to repartition and install onto the eMMC, can we get
We really only have 2 cases: eMMC and SD. In both cases, I would expect that there is some workable partition label on the target installation medium.
away from the tools requiring board specific knowledge? A generic partitioning/install tool needs to know:
- Is an MBR required (ie. SoC reads MBR to find firmware)
- Is FW location at a fixed offset?
- Is GPT supported?
So none of these matter. If we can guarantee that
* labels don't get recreated, only partitions change and * partitions start earliest at 2MB
then we have it all covered, because ...
If the tools can get that information, then it can make good decisions when reimaging a device, and it will make Marcin happier I'm sure. :-)
I /don't/ think the general tools should need to know how to install the firmware itself. That is still a nasty board-specific problem.... (although even here we could make things better if we had a spec for managing the SoC's FW partition. I would like to see separate steps for FW provisioning, and OS install. ie. A board-specific tool to prep an SD card with the bare minimum for FW, and then generic tools to complete partitioning and install.
... as you mention here, the installation path on such a system would involve a specialized image as your "installer image" which includes FW at whereever it has to be plus the installer binaries. That installer would usually load itself into RAM, use a server on the internet as installation source and simply repartition the SD card / eMMC.
So while it's repartitioning, we need to make sure that we don't remove the RPi FAT partition for example. But that's easy - mark it as MBR EFI partition (Label type 0xef) and fix any issues in your installer if it thinks it wants to remove it. I think today the RPi firmware doesn't boot from 0xef labels, but I'm sure we can talk to them to fix that.
For all hard coded offset firmware, just make sure that your repartitioning doesn't touch the first 2MB and your firmware will stay alive from the days when your SD card contained the installer.
For on-board devices (eMMC), FW provisioning should only be needed once For removable devices (SD), FW provisioning is only needed when FW must be on SD. (ie. no-onboard eMMC, or for FW recovery/upgrade)
So that only leaves the "install from SD to eMMC" case. And that is simply an installation issue, there's nothing we can do generically to fix that. If you want to support that case, put the respective logic to dump firmware to the eMMC into your installer. Or into some pre-stage on the SD card. Or create a separate SD image that "upgrades firmware" on the system.
Alex
On Fri, May 6, 2016 at 12:15 PM, Alexander Graf agraf@suse.de wrote:
On 06.05.16 13:03, Grant Likely wrote:
On Fri, May 6, 2016 at 2:10 AM, Tom Rini trini@konsulko.com wrote:
On Thu, May 05, 2016 at 10:21:25PM +0200, Alexander Graf wrote:
So what's the goal here? Are we trying to force GPT on systems whose vendors never intended them to run with GPT?
It really shouldn't matter at the end of the day whether we use GPT or MBR. All uEFI firmware implementations I'm aware of support both. So if you have a device whose bootloader collides with the GPT, just use MBR.
As for the "protection" mechanism, I don't think it's a problem either. IIRC parted starts to create partitions with a sensible alignment (1 or 2MB). Most boot loaders fit perfectly fine within that gap.
So this really boils down to
- use GPT for systems that were designed for it or
- use MBR with alignment gap to first partition
end of story. There shouldn't be any middle ground. At least I haven't seen any so far :).
That is a good point. MBR is a pain to deal with, but I don't think there is anything that it is absolutely required for in UEFI land.
This, for many use cases is also true. A reason that various SoCs pick a magic location that is MBR compatible and not strictly GPT compatible is that they don't see a use case for GPT-not-MBR being used. By-and-large saying your SD card shall have an MBR and shall leave a gap that is also well aligned for the card anyhow is enough for (enough) firmware to reside in the magic locations.
I think the key issue is how do the tools know what they are allowed to do. If (for example) the system is booted into a recovery/install image and needs to repartition and install onto the eMMC, can we get
We really only have 2 cases: eMMC and SD. In both cases, I would expect that there is some workable partition label on the target installation medium.
away from the tools requiring board specific knowledge? A generic partitioning/install tool needs to know:
- Is an MBR required (ie. SoC reads MBR to find firmware)
- Is FW location at a fixed offset?
- Is GPT supported?
So none of these matter. If we can guarantee that
- labels don't get recreated, only partitions change and
- partitions start earliest at 2MB
then we have it all covered, because ...
If the tools can get that information, then it can make good decisions when reimaging a device, and it will make Marcin happier I'm sure. :-)
I /don't/ think the general tools should need to know how to install the firmware itself. That is still a nasty board-specific problem.... (although even here we could make things better if we had a spec for managing the SoC's FW partition. I would like to see separate steps for FW provisioning, and OS install. ie. A board-specific tool to prep an SD card with the bare minimum for FW, and then generic tools to complete partitioning and install.
... as you mention here, the installation path on such a system would involve a specialized image as your "installer image" which includes FW at whereever it has to be plus the installer binaries. That installer would usually load itself into RAM, use a server on the internet as installation source and simply repartition the SD card / eMMC.
So while it's repartitioning, we need to make sure that we don't remove the RPi FAT partition for example. But that's easy - mark it as MBR EFI partition (Label type 0xef) and fix any issues in your installer if it thinks it wants to remove it. I think today the RPi firmware doesn't boot from 0xef labels, but I'm sure we can talk to them to fix that.
For all hard coded offset firmware, just make sure that your repartitioning doesn't touch the first 2MB and your firmware will stay alive from the days when your SD card contained the installer.
Yes, that makes sense. One tweak I'd make, is I'd still create a partition table entry to cover the fixed-address region. Then the information is explicitly there.
For on-board devices (eMMC), FW provisioning should only be needed once For removable devices (SD), FW provisioning is only needed when FW must be on SD. (ie. no-onboard eMMC, or for FW recovery/upgrade)
So that only leaves the "install from SD to eMMC" case. And that is simply an installation issue, there's nothing we can do generically to fix that. If you want to support that case, put the respective logic to dump firmware to the eMMC into your installer. Or into some pre-stage on the SD card. Or create a separate SD image that "upgrades firmware" on the system.
Agreed.
Okay, that simplifies things quite a bit. It still needs to be documented, and I'd like to get some of this discussion into the UEFI spec. I'll draft something.
BTW, I've looked into the limitations on MBR vs. GPT. From the UEFI spec:
The following list outlines the advantages of using the GPT disk layout over the legacy Master Boot Record (MBR) disk layout: • Logical Block Addresses (LBAs) are 64 bits (rather than 32 bits). • Supports many partitions (rather than just four primary partitions). • Provides both a primary and backup partition table for redundancy. • Uses version number and size fields for future expansion. • Uses CRC32 fields for improved data integrity. • Defines a GUID for uniquely identifying each partition. • Uses a GUID and attributes to define partition content type. • Each partition contains a 36 character human readable name.
GPT does have better data integrity and redundancy, and a larger maximum size.... but I haven't seen many >2TB eMMC or SD devices around. :-) None of these are show stoppers. As long as the provisioning tool does initial setup correctly, including choosing MBR vs GPT, then the OS install step doesn't need to do anything special.
g.
On 05/05/2016 09:26 AM, Simon Glass wrote:
Perhaps ARM could create a set of guidelines here?
Hahaha! Funniest thing I've heard all week :)
Of course, that would be the ideal, as would an industry that acted rationally and listened to such requirements (frankly, I personally would make basic platform standards a requirement of licensing the architecture or any of its pieces). On server, we do it correctly by forcefully separating platform flash from boot media and using "big pointy stick" (aka OS vendor clout) to enforce compliance. While people on non-server parts continue to misuse SD cards and other media to contain bootloaders, they're going to get what they are asking for: a mess. They can't really want real general purpose OS support until they understand that to do this properly requires that you don't force distros to ship firmware (sure, others will point out that distros do this today and it just about works for some boards, etc.).
But is it really worth trying after so long of the right thing not happening? If anyone really cared about making general purpose distros boot on embedded boards, efforts to compel standards would have happened years ago. To do it right, we would need to have a couple of vendors involved who could compel vendors to comply.
Jon.
On Fri, May 06, 2016 at 12:20:40PM -0400, Jon Masters wrote:
But is it really worth trying after so long of the right thing not happening? If anyone really cared about making general purpose distros boot on embedded boards, efforts to compel standards would have happened years ago. To do it right, we would need to have a couple of vendors involved who could compel vendors to comply.
Distros care and currently do ship on such systems - Debian stable lists a bunch of boards (something like 20 IIRC) as actively tested for example. The board and SoC vendors are to an extent irrelevant here, when people do this they often don't use any of the software the board vendors provide and just use things like upstream u-boot. Coming up with something that covers all the cases with minimal code for the distros will make life easier for them, if board vendors want to pick it up as well in what they're shipping (and ideally in how they spec their media usage) that's great but we're already winning even if all the board vendors totally ignore it.
Hi Mark,
On 05/06/2016 01:10 PM, Mark Brown wrote:
On Fri, May 06, 2016 at 12:20:40PM -0400, Jon Masters wrote:
But is it really worth trying after so long of the right thing not happening? If anyone really cared about making general purpose distros boot on embedded boards, efforts to compel standards would have happened years ago. To do it right, we would need to have a couple of vendors involved who could compel vendors to comply.
Note: by standards above, I specifically mean "separate platform flash" in addition to all of the other associated things. Actually, to do it right you need two separate platform flash modules - one with EFI, the other with parameters (such as MAC addresses, and the like). Today, we're very lucky if there's even one flash chip on the board to house firmware (where it belongs) rather than on an SD card, etc.
What will happen next ;) is that someone will say "in your opinion, Jon". We can't generally agree that it's "wrong" to build systems without flash on the board to house the firmware. So getting others to agree to do it is going to be probably never achievable.
Distros care and currently do ship on such systems - Debian stable lists a bunch of boards (something like 20 IIRC) as actively tested for example. The board and SoC vendors are to an extent irrelevant here,
I get your point, but for separate flash and getting vendors to ship firmware, they very much need to be involved. Today, we can't agree as an industry who is on the hook for this. With an enterprise hat on, I get to compel vendors to do the only sane thing (in my opinion) which is "thou shalt ship EFI, on flash that we don't touch". And those who screwed up and put EFI parameters on hidden disk partitions, or thought EFI variables were a place to store MAC and platform parameters are slowly found and forced to comply with the way the industry works. But on embedded, spending a few cents to do the "right" thing is something that isn't going to happen unless everyone mandates and pushes for it.
Jon.
On Fri, May 06, 2016 at 02:03:42PM -0400, Jon Masters wrote:
On 05/06/2016 01:10 PM, Mark Brown wrote:
On Fri, May 06, 2016 at 12:20:40PM -0400, Jon Masters wrote:
But is it really worth trying after so long of the right thing not happening? If anyone really cared about making general purpose distros boot on embedded boards, efforts to compel standards would have happened years ago. To do it right, we would need to have a couple of vendors involved who could compel vendors to comply.
Note: by standards above, I specifically mean "separate platform flash" in addition to all of the other associated things. Actually, to do it right you
Oh, right. In that case this is all irrelevant anyway, there's no need to worry about magic areas of the disk and distros can just do whatever.
Distros care and currently do ship on such systems - Debian stable lists a bunch of boards (something like 20 IIRC) as actively tested for example. The board and SoC vendors are to an extent irrelevant here,
I get your point, but for separate flash and getting vendors to ship firmware, they very much need to be involved. Today, we can't agree as an industry who is on the hook for this. With an enterprise hat on, I get to
Personally I'm not convinced it's particularly worth worrying about - there's enough sensible ways to build systems where the separate storage for bootloaders just isn't solving problems people have that such things are going to be around for a while and inevitably people will end up wanting to run distros on them so at least the community distros need to work with them.
compel vendors to do the only sane thing (in my opinion) which is "thou shalt ship EFI, on flash that we don't touch". And those who screwed up and put EFI parameters on hidden disk partitions, or thought EFI variables were a place to store MAC and platform parameters are slowly found and forced to comply with the way the industry works. But on embedded, spending a few cents to do the "right" thing is something that isn't going to happen unless everyone mandates and pushes for it.
It's not just cents, it's also things like board area, manufacturing process, usage models and for some use cases customers who want full control over the software stack. For some kinds of system like the enterprise market what you're describing is absolutely the right way to go but there's other segments where it either isn't solving problems people have or is currently actively worse so there's no compelling reason to adopt. But this is a bit off topic...
-
Alexander Graf -
Amit Kucheria -
Bill Gatliff -
Brendan Conoboy -
Dennis Gilmore -
Grant Likely -
Jon Masters -
Marcin Juszkiewicz -
Mark Brown -
Martin Stadtler -
Simon Glass -
Tom Rini