Loading a variable number of "modules"

Grant Likely grant.likely at secretlab.ca
Thu Jun 30 04:40:47 UTC 2011 

Okay, I think we have our topic for tomorrow's meeting.  comments below...

On Wed, Jun 29, 2011 at 7:37 AM, Olivier Martin <Olivier.Martin at arm.com> wrote:
> I also had a thought about a 'general purpose ARM bootloader' approach after our phone call last week.
>
> Linaro could provide a reference grub-style module/library/application which would be Boot Firmware agnostic but using wrapper functions to call firmware specific functions. These wrappers functions would allow to take advantages of the platform firmware features.
> There would be an implementation of this wrapper for each supported boot firmware (eg: UEFI, u-boot, Barebox, etc). The wrapper API could export generic functions to load files from the network/filesystem/etc, load and store non volatile environment variables, etc
> This OS loader could also be responsible for the DT edition if required. And it will be built in a suitable manner for the targeted boot firmware (static library, EFI application, baremetal application, etc).
>
> The main benefit would be a common (user) interface between the different Linaro's platforms. The HW vendors would also not be tied to a specific boot firmware implementation. The wrapper API would define the requirements for the boot firmware.
>
> Even if it is a reference implementation, HW/OS vendors could implement their own version as soon as their implementation ensure the same features (eg: boot from PXE, DT edition, etc).

So from both Loïc and Olivier's posts, I discern two proposals on the table:

1) a bootloader runtime ABI usable by a generic 'osloader' executable.
2) a rich boot descriptor file format that instructs firmware how and
where to load binary images before booting the client
(kernel/osloader/memtest/grub/whatever).

These two options aren't even mutually exclusive.  It seems to me that
option 1) is also Loïc's first concluding option, and it is what
Olivier is talking about above.  Option 2) is Loïc's second concluding
option.  Jump in and correct me if I'm wrong here.

Just to be backwards, I'll start with my thoughts on option 2.  I
think this is exactly what we need.  It would be incredibly powerful
to have a standardized data file or even simple script language for
loading images into RAM, updating the DT to point to the images, and
executing one of the binaries, and it could be designed in such a way
as to fetch images from any of: the same place the boot script was
loaded from, an http/ftp/tftp url, a local storage url, or firmware's
default configuration storage.

Downsides: each firmware needs to implement the scripting language or
data parser, plus a scripting language needs to be created.  Neither
are great scenarios.  However, the first could be mitigated if we
create a BSD licensed library that does the parsing (and hooks into
the firmware-specific apis).  For the second, I'd rather investigate
and adopt an existing data/script format than create something new.
Surely there is something suitable that is straight forward, simple,
and doesn't require a huge amount of resources to implement.  (/me
wonder's if Lua would be suitable for the job).  U-Boot does have a
scripting language, but it's pretty ad-hoc and centric to the U-Boot
way of doing things.  I doubt it is the thing to standardize on.

Option 1 both intrigues me and frightens me.  Intrigues me because it
would be /really cool/ to have a reliable runtime ABI for client
programs regardless of firmware implementation.  Frightens me since I
greatly fear that this will be difficult to get fimrware to implement
correctly, far more so than pulling in a data/script parser.  I worry
that it will take a lot longer to both define and get all the major
Firmware projects to adopt.  Also, I suspect that option 2 actually
provides all of the features that are actually needed.  I'd like to
hear thoughts and use cases of what a runtime ABI can provide that
option 2 wouldn't be able to handle.

Having said that, I think that from a development standpoint a runtime
ABI actually fits in well with what needs to be done to implement
option 2.  In some ways the only difference between the two is that
option 1 defines an ABI at the machine code level, while option 2
defines it at the data/script/interpreter level.  Either way, the
provided behaviours are going to need to be the same.  If we started
with standardizing a data/script format, it could also be used to lay
out and define what is required for a binary ABI, and it also takes
the pressure away from needing to get a binary ABI defined and
implemented /right/freaking/now/.  :-)

One final point on a runtime ABI; just like with a data/script format,
I don't think inventing something new is the right thing to do.  I'd
much rather adopt something existing than delve into new territory.

Let's discuss at the meeting.

g.

>
> Olivier
>
>
>
> -----Original Message-----
> From: boot-architecture-bounces at lists.linaro.org [mailto:boot-architecture-bounces at lists.linaro.org] On Behalf Of Loïc Minier
> Sent: 23 June 2011 17:33
> To: boot-architecture at lists.linaro.org
> Subject: Loading a variable number of "modules"
>
>        Hey
>
>  (followup to today's call0
>
>  This is an attempt to write down an use case which expresses the
>  concept of "modules" that the boot architecture would cover.  I don't
>  really like the name modules, if you have better names please propose!
>
>  So here's the writeup; this is completely fictional, and is just meant
>  to illustrate modules.
>
>  Here are the steps that could be taken to load a plain linux kernel:
>  a) your SoC starts up in a SoC-specific way
>  b) the boot architecture mandates how vmlinuz is loaded and started
>  c) control is passed to linux
>  d) profit!1!
>
>  in a very dumbed down version of our boot architecture, we could say
>  for b):
>    a file named "bootme" is loaded from the first FAT partition of the
>    SD card; its type is detected and if it's a linux kernel, it's
>    started with a device tree blob provided by the firmware to the
>    kernel
>
> Good:
>  * easy to install or update a kernel by replacing the vmlinuz file
>
> Bad:
>  * miss way to pass kernel cmdline
>  * miss way to pass (optional) initrd
>  * miss way to load/replace the DT
>
>  all of the above bad points could be alleviated by pushing this data in
>  the vmlinuz file, but it's not as practical as the modern linux systems
>  we know.
>   Another approach would be to put U-Boot or Barebox in the "bootme"
>  file, but then U-Boot or Barebox would have to decide how to load the
>  kernel and implement drivers for the SD card and this wouldn't be a
>  SoC-agnostic boot image anymore.
>   So the only possible approach I see in this case is to put a general
>  purpose ARM bootloader in the "bootme" file which will talk to firmware
>  to load additional files from the SD card (thanks to non-resident
>  support from the firmware); this would then go away once linux runs.
>
>
>  Now to "modules": instead of defining b) as just loading a single
>  "bootme" file, we'd define a config file which lists what to load (e.g.
>  something like grub.cfg/menu.lst) so that we could pass combinations
>  such as:
>  - vmlinuz + initrd + DT
>  - vmlinuz + DT
>  - Xen + DT + the OS modules it should load (linux and initrd)
>  it would also allow setting things like kernel cmdline or other OS
>  specific things.
>
>  Maybe I misunderstood Grant's idea, but it felt like he was mandating a
>  single payload from being loaded and just deferring to that payload.  I
>  feel that if we go that route, we've specified too little to be able to
>  construct useful images and have deffered to an implementation specific
>  architecture (even if it could support multiple SoCs).
>
>
>
>  So my conclusion is that we should specify:
>  * either a single payload to start which would be something like a
>   generic ARM bootloader talking to the firmware to load the other bits
>   (e.g. via UEFI), but then also how it loads the next things
>  * or a config file which points at "modules" (by lack of a better name)
>   to load; these will get loaded by SoC specific code (likely an UEFI
>   implementation)
>  the main difference is whether we specify how a generic ARM bootloader
>  is loaded and then defer to it to decide how to load the OS, or whether
>  we specify OS-ish bits to load, and leave the loading to the
>  implementation.
>
>
>  Does that make any sense?
>
>    Cheers,
> --
> Loïc Minier
>
> _______________________________________________
> boot-architecture mailing list
> boot-architecture at lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/boot-architecture
>
>
> -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.
>
>
> _______________________________________________
> boot-architecture mailing list
> boot-architecture at lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/boot-architecture
>



-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

More information about the boot-architecture mailing list