Hi all,
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their systems configured as it works fairly well, recently i sent in a patch implementing most of what I would like to see for the wandboard[1]
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
pxe boot suport, two reasons, one is to be able to easily network boot the distro installer, the other is to use sysboot support post install with a extlinux.conf file to specify the kernel, initrd and bootargs
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
fdt will be automatically loaded and provided fedora ships dtbs in /boot/dtb-<kernelver>/ I am not sure where other distros provide them, however u-boot should automatically load dtb and provide access to a fdt in a ${fdt_addr} variable that can be used by pxe_cmd but still allows the user to specify their own in extlinux.conf if desired. ideally the devicetree files need to be decoupled from the kernel, along the lines of what is being discussed in the kernel now[2]. Distros should agree on a single location for the dtbs to be placed. /boot/dtb/ or /boot/dtbs/ are probably good locations. u-boot can then look in the path with and without /boot
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
longer term being able to edit the boot arguments should also be an option with the menu functionality.
""" # extlinux.conf generated by anaconda
ui menu.c32
menu autoboot Welcome to Fedora. Automatic boot in # second{,s}. Press a key for options. menu title Fedora Boot Options. menu hidden
timeout 60 #totaltimeout 9000
label Fedora (3.9.9-302.fc19.armv7hl) 19 (Schrödinger’s Cat) kernel /vmlinuz-3.9.9-302.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.9-302.fc19.armv7hl.img
label Fedora (3.9.4-301.fc19.armv7hl) 19 (Schrödinger’s Cat) kernel /vmlinuz-3.9.4-301.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.4-301.fc19.armv7hl.img """ the above is an example of a extlinux.conf file on a fedora 19 system that works with the u-boot as shipped by calxeda on thier highbank energy cards. some of the valid extlinux options written out by anaconda cause noise on ARM as they are not implemented longer term it would be good to deal with them correctly. """ Ignoring unknown command: ui Ignoring malformed menu command: autoboot Ignoring malformed menu command: hidden Fedora Boot Options. 1: Fedora (3.9.9-302.fc19.armv7hl) 19 (Schrödinger’s Cat) 2: Fedora (3.9.4-301.fc19.armv7hl) 19 (Schrödinger’s Cat) Enter choice: """ is the output on boot, the user can then enter 1 or 2 to select a kernel to boot. if nothing is selected the first option is booted after the timeout expires.
What this gets us in the end is that with a unified kernel, device tree and u-boot implementing the system specific knowledge, supporting new ARM systems is along the same lines as supporting new x86 systems. Get the drivers and platform support in the mainline kernel and as long as the Distro enables it it will just work in the next OS release. The one part then that we as Distro's need to do is to ensure we install u-boot into the disk correctly for the board in question if it is a system that does not ship with some kind of storage for u-boot. If it is a system with NOR, Nand, SPI storage for u-boot it is then entirely in the vendors hand, and assuming that they implement everything, all distros will be able to simply support them. Giving users and vendors more choice in what to run on their hardware.
I may not be entirely clear here and there may be things I am missing, I feel it is a good place to start the discussion.
Regards
Dennis
[1] http://lists.denx.de/pipermail/u-boot/2013-July/159522.html [2] https://lwn.net/Articles/560523/
On 3 August 2013 08:11, Dennis Gilmore dennis@ausil.us wrote:
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
I looked into doing this for QEMU's boot loader once. I wasn't able to come up with a solution because there's no way given a zImage to determine how big it will be uncompressed, so all you can do is make a best-guess about where to put other things. Maybe I missed a way to do this cleanly though?
thanks -- PMM
On Sat, Aug 03, 2013 at 11:08:57AM +0100, Peter Maydell wrote:
On 3 August 2013 08:11, Dennis Gilmore dennis@ausil.us wrote:
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
I looked into doing this for QEMU's boot loader once. I wasn't able to come up with a solution because there's no way given a zImage to determine how big it will be uncompressed, so all you can do is make a best-guess about where to put other things. Maybe I missed a way to do this cleanly though?
Well, that depends on how repellent using a not-raw-zImage format is, even if you don't have to know the load address at creation time. One can create a FIT image with kernel (or N kernels), N device tree blobs and ramdisk (or N ramdisks), and have U-Boot handle the rest of the work. All of that it is there today, aside from having boards enable FIT images by default. Having the right device tree be picked and used is one of those "almost there" things.
Dear Dennis Gilmore,
In message 20130803021104.1fecaf5a@adria.ausil.us you wrote:
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their
When talking about simplifying and unifying things, you should not start with the mistake of an architecture specific view. ARM may be what you are interested in, and where most of the current development is happening, but it is not the only architecture around.
Also, it may make sense to start with leaning back a bit and looking how other architectures solved some of the issues ARM is having. My feeling is that some of them are actually home-made.
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
Actually it is a horror thought to me that U-Boot could degrade into such a horrible envrionment as the boot loaders (BIOS, UEFI etc.) provide, both to the developer and to the end user.
No, this is definitely NOT where we waht to head for.
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
This may be interesting for you, but faci it: the majority of boards are not using ext4 based root file systems, nor xfs. Most of them are strictly embedded devices, and the use of JFFS2 (yes, still!) and UBI/UBIFS on NOR and NAND flash is far more common.
pxe boot suport, two reasons, one is to be able to easily network boot the distro installer, the other is to use sysboot support post install with a extlinux.conf file to specify the kernel, initrd and bootargs
PXE may be interesting for a very small percentage of applications, but it is nothing to unify on. Existing DHCP / TFTP / NFS support is much more commonly used.
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a
Actually this is strictly an ARM issue. No other architecture has such problems. U-Boot images (both the old, legacy uImage format, and much more so the FIT imagesformat) have a large number of advantages, especially for software management on embedded systems.
Also, a pretty common requirement in U-Boot land is boot time optimization. The common approach of standard distros with a multi-stage intialization sequence including booting from an intial ramdisk before mounting the real root file system is usually prohibitive in such szenarios.
user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
Again, this is a problem home-made by ARM Linux. I am not ware of any such issues with other architectures.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
This is easy to say, but difficult to impossible to implement in the general case. The problem of dealing with compressed images where you usually don't know in advance how big they will become when uncompressed) has already been mentioned. The property especially of the ARM Linux kernel (in default setup) to copy itself around if it deems fit and some other such issues make things really complicated.
When you talk of unifying, this must be done on both sides - the boot loader cannot solve or fix up all the issues caused elsewhere.
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
Such a requirement can never be fulfilled. For example, there are many embedded systems which have all kind of devices attached which would go haywire if you start sending console output to them.
Fedora Boot Options. 1: Fedora (3.9.9-302.fc19.armv7hl) 19 (Schrödinger’s Cat) 2: Fedora (3.9.4-301.fc19.armv7hl) 19 (Schrödinger’s Cat) Enter choice: """ is the output on boot, the user can then enter 1 or 2 to select a kernel to boot. if nothing is selected the first option is booted after the timeout expires.
You always assume that there is a user, who provides manual input, and such. This is simply not true. Actually this is not even true on x86 systems, and if you ever try running any standard x86 board in a remote-operated way, you will quickly realize what I mean. Yes, there are versions of boot loaders available that support some kind of serial console interface. But then - there are all kinds of PCI cards which require things like pressing "ALT + 3" to enter specific modes. Ever tried that on a serial console?
What this gets us in the end is that with a unified kernel, device tree and u-boot implementing the system specific knowledge, supporting new ARM systems is along the same lines as supporting new x86 systems. Get
As mentioned before, this requires not only work on the U-Boot side. And you have to keep in mind that your vision is actually for a small sector of the market, for one specific application case, while embedded devices have totally different requirements. And last but not least, let's keep the head up so we can see over the rim of our plate: not all the world's an ARM box, and any such unification should be done in an architecture-independent way.
Best regards,
Wolfgang Denk
Hi Wolfgang,
On Sun, 04 Aug 2013 21:48:00 +0200 Wolfgang Denk wd@denx.de wrote:
Dear Dennis Gilmore,
In message 20130803021104.1fecaf5a@adria.ausil.us you wrote:
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their
When talking about simplifying and unifying things, you should not start with the mistake of an architecture specific view. ARM may be what you are interested in, and where most of the current development is happening, but it is not the only architecture around.
sure, the use cases of u-boot in generic linux that I know of today are all ARM based but it doesn't mean it needs to be or always will be.
Also, it may make sense to start with leaning back a bit and looking how other architectures solved some of the issues ARM is having. My feeling is that some of them are actually home-made.
Home-made in what way? at least as I see it and I may be completely wrong. The biggest hurdles for us a distro maker is, supporting a new board means putting logic into the installer and the tools used in kernel updating to understand how to detect the system, and deal with the peculiarities of the system. I have had the anaconda guys reject patches because they do not feel that the installer should know that. when systems using u-boot are the only ones
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
Actually it is a horror thought to me that U-Boot could degrade into such a horrible envrionment as the boot loaders (BIOS, UEFI etc.) provide, both to the developer and to the end user.
No, this is definitely NOT where we waht to head for.
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
This may be interesting for you, but faci it: the majority of boards are not using ext4 based root file systems, nor xfs. Most of them are strictly embedded devices, and the use of JFFS2 (yes, still!) and UBI/UBIFS on NOR and NAND flash is far more common.
in embedded use yes. I realise the the use case im talking about is not the most typical for u-boot and its not right for all targets, but its what id like to see supported on hardware that is targeted at being supported by generic distros. I
pxe boot suport, two reasons, one is to be able to easily network boot the distro installer, the other is to use sysboot support post install with a extlinux.conf file to specify the kernel, initrd and bootargs
PXE may be interesting for a very small percentage of applications, but it is nothing to unify on. Existing DHCP / TFTP / NFS support is much more commonly used.
its interesting in the use cases im trying to target
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a
Actually this is strictly an ARM issue. No other architecture has such problems. U-Boot images (both the old, legacy uImage format, and much more so the FIT imagesformat) have a large number of advantages, especially for software management on embedded systems.
Also, a pretty common requirement in U-Boot land is boot time optimization. The common approach of standard distros with a multi-stage intialization sequence including booting from an intial ramdisk before mounting the real root file system is usually prohibitive in such szenarios.
Its not an ARM issue, the same thing would exist on PPC if we were trying to use a generic Distro on it. its a difference between a embedded where you control the whole stack and build the software optimised for it, and the generic distro where you build it to the lowest common denominator, and want it to run in many different ways on different systems.
user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
Again, this is a problem home-made by ARM Linux. I am not ware of any such issues with other architectures.
again its a use case issue.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
This is easy to say, but difficult to impossible to implement in the general case. The problem of dealing with compressed images where you usually don't know in advance how big they will become when uncompressed) has already been mentioned. The property especially of the ARM Linux kernel (in default setup) to copy itself around if it deems fit and some other such issues make things really complicated.
When you talk of unifying, this must be done on both sides - the boot loader cannot solve or fix up all the issues caused elsewhere.
it may well be. i suggested it because sadly things get bigger all the time. if we hardcode in locations what gives plenty of room today may not be enough tomorrow.
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
Such a requirement can never be fulfilled. For example, there are many embedded systems which have all kind of devices attached which would go haywire if you start sending console output to them.
again different use cases. not all use case have, need or want this.
Fedora Boot Options. 1: Fedora (3.9.9-302.fc19.armv7hl) 19 (Schrödinger’s Cat) 2: Fedora (3.9.4-301.fc19.armv7hl) 19 (Schrödinger’s Cat) Enter choice: """ is the output on boot, the user can then enter 1 or 2 to select a kernel to boot. if nothing is selected the first option is booted after the timeout expires.
You always assume that there is a user, who provides manual input, and such. This is simply not true. Actually this is not even true on x86 systems, and if you ever try running any standard x86 board in a remote-operated way, you will quickly realize what I mean. Yes, there are versions of boot loaders available that support some kind of serial console interface. But then - there are all kinds of PCI cards which require things like pressing "ALT + 3" to enter specific modes. Ever tried that on a serial console?
yes I have, its not what I would call fun. I don't assume that there is always a user there to provide input, I assume that when a user needs to provide input there is a way for them to do it. We can't assume that users have serial console access.
What this gets us in the end is that with a unified kernel, device tree and u-boot implementing the system specific knowledge, supporting new ARM systems is along the same lines as supporting new x86 systems. Get
As mentioned before, this requires not only work on the U-Boot side. And you have to keep in mind that your vision is actually for a small sector of the market, for one specific application case, while embedded devices have totally different requirements. And last but not least, let's keep the head up so we can see over the rim of our plate: not all the world's an ARM box, and any such unification should be done in an architecture-independent way.
Thank you for your valuable feedback. We have two very different use-cases and ideally need to have a sane way to support both. I do not know if the hardware targeted for use with generic linux distros is also used in embedded use cases. all of what I've proposed would work just as well on systems using a generic distro regardless of the architecture. I certainly don't want to make things harder for the embedded use cases.
Dennis
On 08/03/2013 01:11 AM, Dennis Gilmore wrote:
Hi all,
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their systems configured as it works fairly well, recently i sent in a patch implementing most of what I would like to see for the wandboard[1]
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
I agree. I'd certainly like to see various distros have regular "PC-style" installers for Tegra boards.
I have attempted to write the "bootcmd" in upstream U-Boot for Tegra devices in a way that helps this. More on that below.
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
That's only partially relevant for U-Boot. Two things U-Boot needs to support:
* Filesystem read for whatever filesystem /boot is part of (it could very well be the / filesystem too, but U-Boot doesn't care, except for / vs. /boot in path names). This is a matter of adding the right flags to each board's U-Boot config file.
* Some way of determining which partition it should load things from.
For this point, I'd suggest MBR's bootable flag, or GPT's similar flags. I always intended to implement a sub-command of U-Boot's "part" command that read the partition flags and picked the correct partition to use based on those flags...
The rest of the stuff (swap, LVM, ...) seems entirely related to the distro itself and/or whatever gets put into the initrd.
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
+1 from me.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
The way I've approached this for Tegra is to have the default environment define certain standard environment variables that U-Boot scripts can use, without a care. For example, the kernel should be loaded at ${kernel_addr_r}, initrd at ${ramdisk_addr_r}, DTB at ${fdt_addr_r}, etc.
I chose the values of those variables in the Tegra environment to support reasonable max sizes. The kernel itself has a max size based on practicality and the ARM ISA and jump range. There's no reason the compressed kernel should be larger than the uncompressed kernel, etc.
fdt will be automatically loaded and provided fedora ships dtbs in /boot/dtb-<kernelver>/ I am not sure where other distros provide them, however u-boot should automatically load dtb and provide access to a fdt in a ${fdt_addr} variable that can be used by pxe_cmd but still allows the user to specify their own in extlinux.conf if desired. ideally the devicetree files need to be decoupled from the kernel, along the lines of what is being discussed in the kernel now[2]. Distros should agree on a single location for the dtbs to be placed. /boot/dtb/ or /boot/dtbs/ are probably good locations. u-boot can then look in the path with and without /boot
The model I chose on Tegra is slightly different.
For credit, I was heavily inspired by the default bootcmd from downstream TrimSlice U-Boot and I think also the upstream Calxeda U-Boot.
Tegra's U-Boot searches all known boot devices for /boot/boot.scr (and some other filenames), sets up some variables to record where it was found, loads the file into RAM, and treats that as a U-Boot script. The boot script then determines everything else that happens at boot; loading the kernel/initrd/DTB, and booting it.
This lets boot.scr control e.g.:
* Whether the distro wraps zImage into a uImage or not, since the distro-supplied script determines whether bootz or bootm is executed.
* Whether the distro has an initrd or not (I currently don't use one typically).
* The filename of the initrd and DTB, since the location/name/... of these is probably somewhat distro-specific.
* Whatever extra kernel command-line parameters the distro cares to pass to the kernel, without having to add them to the U-Boot environment, but rather just to boot.scr, which is a much more accessible file in a regular file-system.
In other words, I've tried to define a "boot interface" between U-Boot and the distro (which is to simply load/execute boot.scr, and provide some data to boot.scr in environment variables), rather than making U-Boot have complete knowledge of how to boot the distro.
Also, I have set up all boards so that they define standard U-Boot environment variables ${soc}, ${board} that define which HW the code is running on. This can be used to automatically construct the DTB filename, and hence makes boot.scr pretty much HW-agnostic.
An example boot.scr that I use is shown below, with some extra comments for the purposes of this email:
========== # I anticipate replacing this hard-coded assignment with e.g.: # part find-bootable ${devtype} ${devnum} rootpart setenv rootpart 1
# Find partition UUID (not fs UUID) at run-time, so script adapts to # whatever is there without regenerating it part uuid ${devtype} ${devnum}:${rootpart} uuid
# Here is the use of ${uuid} from aboe # bootargs_extra allows the user to interrupt boot and add to the # kernel command-line setenv bootargs ... root=PARTUUID=${uuid} ${bootargs_extra}
# You could load a "uEnv.txt" here too, which could set some variables # that e.g. get added to bootargs etc., if you want plain-text disk- # based configuration of this boot script.
# Complete generic... load ${devtype} ${devnum}:${rootpart} ${kernel_addr_r} /boot/zImage
load ${devtype} ${devnum}:${rootpart} ${ramdisk_addr_r} /boot/initrd.gz # So that we don't need to regenerate script when initrd size changes ramdisk=${ramdisk_addr_r}:0x${filesize} # :-( setenv initrd_high 0xffffffff
load ${devtype} ${devnum}:${rootpart} ${fdt_addr_r} \ /boot/${soc}-${board}.dtb
bootz ${kernel_addr_r} ${ramdisk} ${fdt_addr_r} ==========
For specifics in the code, see the definition of CONFIG_EXTRA_ENV_SETTINGS in U-Boot's include/configs/tegra-common-post.h, and work backwards from there.
Note that I'm also in the process of pushing a project to github that creates a few boot.scr that fit into this model. I've written the code, and hope to have IP approval to upload it very soon. Aside from the example above, it also supports netboot, initrd being optional, hard-coding various extra stuff into bootargs, etc.
You can also see some hard-coded boot scripts that I hope to get added to the Ubuntu installer image generator at:
https://code.launchpad.net/~srwarren/debian-installer/tegra/+merge/175967
(I'd love to have similar support added to other distros; I just had to pick one to work on first)
What this gets us in the end is that with a unified kernel, device tree and u-boot implementing the system specific knowledge, supporting new ARM systems is along the same lines as supporting new x86 systems. Get the drivers and platform support in the mainline kernel and as long as the Distro enables it it will just work in the next OS release. The one part then that we as Distro's need to do is to ensure we install u-boot into the disk correctly for the board in question if it is a system that does not ship with some kind of storage for u-boot. If it is a system with NOR, Nand, SPI storage for u-boot it is then entirely in the vendors hand, and assuming that they implement everything, all distros will be able to simply support them. Giving users and vendors more choice in what to run on their hardware.
At least for Tegra, the U-Boot binary won't (ever?) be part of the file-system, but is rather always somewhere dedicated (eMMC boot sectors rather than the main user area, SPI, or NAND for example). As such, the model I'd like to push for Tegra is that you use HW-specific tools to install U-Boot, then use a distro installer SD card, USB device, or even CD-/DVD-ROM image to install the distro, and that the distro doesn't have to know anything at all about the bootloader, except to install a hopefully entirely HW-agnostic boot.scr that I detailed above.
I know that some other SoCs store U-Boot etc. on storage-partitions/filesystems instead. Perhaps distros should always assume that the "U-Boot partition" is sacred, and just large enough for the firmware, and put all their files on other partitions. The only difference between Tegra and OMAP then would be an extra partition, and hence the distro's / or /boot would be either partition 1 or 2:
Tegra: Boot flash: U-Boot storage partition 1: / (boot.scr, DTB, zImage, etc.)
(p1 marked bootable)
OMAP: storage partition 1: U-Boot storage partition 2: / (boot.scr, DTB, zImage, etc.)
(p2 marked bootable)
I may not be entirely clear here and there may be things I am missing, I feel it is a good place to start the discussion.
I like where I think you're going!
On 08/05/2013 12:39 PM, Stephen Warren wrote: ...
Note that I'm also in the process of pushing a project to github that creates a few boot.scr that fit into this model. I've written the code, and hope to have IP approval to upload it very soon. Aside from the example above, it also supports netboot, initrd being optional, hard-coding various extra stuff into bootargs, etc.
Oh, that was quick - I got IP approval, and it's pushed to: https://github.com/NVIDIA/tegra-uboot-scripts
It's not directly relevant to this thread, but the scripts to flash U-Boot onto a Tegra device are also at:
https://github.com/NVIDIA/tegra-uboot-flasher-scripts/blob/master/README-use...
... which should give you an idea of the usage-model I'm pushing for flashing U-Boot on Tegra devices.
On Mon, 05 Aug 2013 12:48:25 -0600 Stephen Warren swarren@wwwdotorg.org wrote:
On 08/05/2013 12:39 PM, Stephen Warren wrote: ...
Note that I'm also in the process of pushing a project to github that creates a few boot.scr that fit into this model. I've written the code, and hope to have IP approval to upload it very soon. Aside from the example above, it also supports netboot, initrd being optional, hard-coding various extra stuff into bootargs, etc.
Oh, that was quick - I got IP approval, and it's pushed to: https://github.com/NVIDIA/tegra-uboot-scripts
very interesting to see.
It's not directly relevant to this thread, but the scripts to flash U-Boot onto a Tegra device are also at:
https://github.com/NVIDIA/tegra-uboot-flasher-scripts/blob/master/README-use...
nice, i will look at getting these packaged up and included in fedora.
... which should give you an idea of the usage-model I'm pushing for flashing U-Boot on Tegra devices.
On 08/05/2013 02:26 PM, Dennis Gilmore wrote:
On Mon, 05 Aug 2013 12:48:25 -0600 Stephen Warren swarren@wwwdotorg.org wrote:
On 08/05/2013 12:39 PM, Stephen Warren wrote: ...
Note that I'm also in the process of pushing a project to github that creates a few boot.scr that fit into this model. I've written the code, and hope to have IP approval to upload it very soon. Aside from the example above, it also supports netboot, initrd being optional, hard-coding various extra stuff into bootargs, etc.
Oh, that was quick - I got IP approval, and it's pushed to: https://github.com/NVIDIA/tegra-uboot-scripts
very interesting to see.
It's not directly relevant to this thread, but the scripts to flash U-Boot onto a Tegra device are also at:
https://github.com/NVIDIA/tegra-uboot-flasher-scripts/blob/master/README-use...
nice, i will look at getting these packaged up and included in fedora.
Note that I haven't put a huge amount of thought into distro packaging for the tool set. The usage model of the tools is:
1) Sync the source 2) Build the U-Boot/DTB/BCT/flash-image binaries 3) Run another command to flash them
I assume that a distro package would run (1) and (2) to generate the package, and the user would then install the package and run (3). Doing anything else would mean an odd model w.r.t. the use of "repo" to pull in multiple git repos into the source tree, or the package would end up being nothing more than a copy of the source tree that a developer would sync. Hence, the package would include some specific version of the U-Boot binary/binaries.
So, I'm not 100% sure if it's a good model to package it up? Perhaps it'd make sense...
If do you package it up, please make sure to package U-Boot v2013.07 and nothing later at the moment (you'll need to manually adjust the version of u-boot.git that gets sync'd during the package build process), since the very latest version of u-boot/master doesn't boot on Tegra, due to a bug that will hopefully be fixed in the next couple of days.
On Mon, 05 Aug 2013 12:39:03 -0600 Stephen Warren swarren@wwwdotorg.org wrote:
On 08/03/2013 01:11 AM, Dennis Gilmore wrote:
Hi all,
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their systems configured as it works fairly well, recently i sent in a patch implementing most of what I would like to see for the wandboard[1]
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
I agree. I'd certainly like to see various distros have regular "PC-style" installers for Tegra boards.
I have attempted to write the "bootcmd" in upstream U-Boot for Tegra devices in a way that helps this. More on that below.
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
That's only partially relevant for U-Boot. Two things U-Boot needs to support:
- Filesystem read for whatever filesystem /boot is part of (it could
very well be the / filesystem too, but U-Boot doesn't care, except for / vs. /boot in path names). This is a matter of adding the right flags to each board's U-Boot config file.
right, but at the least it needs to be ext4 not all boards today read ext4, btrfs may be something down the road also. u-boot doesnt need to care too much. it just needs to look in / and /boot
- Some way of determining which partition it should load things from.
For this point, I'd suggest MBR's bootable flag, or GPT's similar flags. I always intended to implement a sub-command of U-Boot's "part" command that read the partition flags and picked the correct partition to use based on those flags...
The rest of the stuff (swap, LVM, ...) seems entirely related to the distro itself and/or whatever gets put into the initrd.
Mostly i was trying to show that where the other bits live doesn't matter.
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
+1 from me.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
The way I've approached this for Tegra is to have the default environment define certain standard environment variables that U-Boot scripts can use, without a care. For example, the kernel should be loaded at ${kernel_addr_r}, initrd at ${ramdisk_addr_r}, DTB at ${fdt_addr_r}, etc.
I really don't want to need to deal with variable names either. I really want to just say load a kernel initramfs and here is some arguments and have the rest just work.
I chose the values of those variables in the Tegra environment to support reasonable max sizes. The kernel itself has a max size based on practicality and the ARM ISA and jump range. There's no reason the compressed kernel should be larger than the uncompressed kernel, etc.
another perfectly valid way to do it. a initramfs should be able to be non existant or 200mb big and just work. I suggested dynamically allocated solely because something we consider outrageous today may be necessary tomorrow.
fdt will be automatically loaded and provided fedora ships dtbs in /boot/dtb-<kernelver>/ I am not sure where other distros provide them, however u-boot should automatically load dtb and provide access to a fdt in a ${fdt_addr} variable that can be used by pxe_cmd but still allows the user to specify their own in extlinux.conf if desired. ideally the devicetree files need to be decoupled from the kernel, along the lines of what is being discussed in the kernel now[2]. Distros should agree on a single location for the dtbs to be placed. /boot/dtb/ or /boot/dtbs/ are probably good locations. u-boot can then look in the path with and without /boot
The model I chose on Tegra is slightly different.
For credit, I was heavily inspired by the default bootcmd from downstream TrimSlice U-Boot and I think also the upstream Calxeda U-Boot.
Tegra's U-Boot searches all known boot devices for /boot/boot.scr (and some other filenames), sets up some variables to record where it was found, loads the file into RAM, and treats that as a U-Boot script. The boot script then determines everything else that happens at boot; loading the kernel/initrd/DTB, and booting it.
This lets boot.scr control e.g.:
While i think its a noble goal I really don't think that boot.scr is really very suitable for distros. I do not know of a great way to setup menus its why i want to use the sysboot command provided by pxe boot and load a extlinux style config file. uboot handles all the details of the devices and memory locations behind the scenes. users get confused anytime they need to run mkimage. I really want to take it out of the equation.
- Whether the distro wraps zImage into a uImage or not, since the
distro-supplied script determines whether bootz or bootm is executed.
- Whether the distro has an initrd or not (I currently don't use one
typically).
- The filename of the initrd and DTB, since the location/name/... of
these is probably somewhat distro-specific.
I think that distros should be able to agree on a common location. names are defined upstream. omap has a quite nice way to work out the filename of the dtb you just need to know/workout where to load it.
- Whatever extra kernel command-line parameters the distro cares to
pass to the kernel, without having to add them to the U-Boot environment, but rather just to boot.scr, which is a much more accessible file in a regular file-system.
In other words, I've tried to define a "boot interface" between U-Boot and the distro (which is to simply load/execute boot.scr, and provide some data to boot.scr in environment variables), rather than making U-Boot have complete knowledge of how to boot the distro.
Also, I have set up all boards so that they define standard U-Boot environment variables ${soc}, ${board} that define which HW the code is running on. This can be used to automatically construct the DTB filename, and hence makes boot.scr pretty much HW-agnostic.
An example boot.scr that I use is shown below, with some extra comments for the purposes of this email:
========== # I anticipate replacing this hard-coded assignment with e.g.: # part find-bootable ${devtype} ${devnum} rootpart setenv rootpart 1
# Find partition UUID (not fs UUID) at run-time, so script adapts to # whatever is there without regenerating it part uuid ${devtype} ${devnum}:${rootpart} uuid
# Here is the use of ${uuid} from aboe # bootargs_extra allows the user to interrupt boot and add to the # kernel command-line setenv bootargs ... root=PARTUUID=${uuid} ${bootargs_extra}
# You could load a "uEnv.txt" here too, which could set some variables # that e.g. get added to bootargs etc., if you want plain-text disk- # based configuration of this boot script.
# Complete generic... load ${devtype} ${devnum}:${rootpart} ${kernel_addr_r} /boot/zImage
load ${devtype} ${devnum}:${rootpart} ${ramdisk_addr_r} /boot/initrd.gz # So that we don't need to regenerate script when initrd size changes ramdisk=${ramdisk_addr_r}:0x${filesize} # :-( setenv initrd_high 0xffffffff
load ${devtype} ${devnum}:${rootpart} ${fdt_addr_r} \ /boot/${soc}-${board}.dtb
bootz ${kernel_addr_r} ${ramdisk} ${fdt_addr_r}
While this is a step forward, its still much more than we need to do if we enable pxe support and use sysboot to load a config file the says load this kernel and initramfs and pass these bootargs. My initial post had a fully working example.
For specifics in the code, see the definition of CONFIG_EXTRA_ENV_SETTINGS in U-Boot's include/configs/tegra-common-post.h, and work backwards from there.
Note that I'm also in the process of pushing a project to github that creates a few boot.scr that fit into this model. I've written the code, and hope to have IP approval to upload it very soon. Aside from the example above, it also supports netboot, initrd being optional, hard-coding various extra stuff into bootargs, etc.
You can also see some hard-coded boot scripts that I hope to get added to the Ubuntu installer image generator at:
https://code.launchpad.net/~srwarren/debian-installer/tegra/+merge/175967
(I'd love to have similar support added to other distros; I just had to pick one to work on first)
It is not likely to get into fedora's installer. we currently have extlinux support which is really simple and arm-boot-config at: https://git.fedorahosted.org/cgit/arm-boot-config.git its a mostrosity of hush scripting to enable menus and configuration options. but it does require board specific knowledge to enable support for new systems. but i definetly want to enable people to use boot.scr if they choose to
What this gets us in the end is that with a unified kernel, device tree and u-boot implementing the system specific knowledge, supporting new ARM systems is along the same lines as supporting new x86 systems. Get the drivers and platform support in the mainline kernel and as long as the Distro enables it it will just work in the next OS release. The one part then that we as Distro's need to do is to ensure we install u-boot into the disk correctly for the board in question if it is a system that does not ship with some kind of storage for u-boot. If it is a system with NOR, Nand, SPI storage for u-boot it is then entirely in the vendors hand, and assuming that they implement everything, all distros will be able to simply support them. Giving users and vendors more choice in what to run on their hardware.
At least for Tegra, the U-Boot binary won't (ever?) be part of the file-system, but is rather always somewhere dedicated (eMMC boot sectors rather than the main user area, SPI, or NAND for example). As such, the model I'd like to push for Tegra is that you use HW-specific tools to install U-Boot, then use a distro installer SD card, USB device, or even CD-/DVD-ROM image to install the distro, and that the distro doesn't have to know anything at all about the bootloader, except to install a hopefully entirely HW-agnostic boot.scr that I detailed above.
need to add pxe support in there. there is no reason we couldnt make installer isos, we dont because we do not know of any hardware supporting it. but a installer sdcard or usb stick are things we want to support as well as network installing using pxe booting.
I know that some other SoCs store U-Boot etc. on storage-partitions/filesystems instead. Perhaps distros should always assume that the "U-Boot partition" is sacred, and just large enough for the firmware, and put all their files on other partitions. The only difference between Tegra and OMAP then would be an extra partition, and hence the distro's / or /boot would be either partition 1 or 2:
Tegra: Boot flash: U-Boot storage partition 1: / (boot.scr, DTB, zImage, etc.)
(p1 marked bootable)
OMAP: storage partition 1: U-Boot storage partition 2: / (boot.scr, DTB, zImage, etc.)
(p2 marked bootable)
for fedora 20 i plan to use raw MLO support for OMAP and have it load the u-boot.img from the first bootable partition. SuSE are doing this today but the patch they have enabling it needs some work to support all the different ways things are setup today. for fedora 19 we used a 20mb vfat partiton that contains MLO u-boot.img and a uEnv.txt loading a boot.scr from /boot/ on the 3rd partition partition 1: 20mb for u-boot mounted at /boot/uboot partition 2: 512mb swap partition 3: /
but overall i like the idea,
I may not be entirely clear here and there may be things I am missing, I feel it is a good place to start the discussion.
I like where I think you're going!
Thank you, I am open to feedback and options to achieve our goals that will help all distros and not break things for use cases that are not our primary interest.
Dennis
On Mon, Aug 05, 2013 at 02:50:59PM -0500, Dennis Gilmore wrote:
On Mon, 05 Aug 2013 12:39:03 -0600 Stephen Warren swarren@wwwdotorg.org wrote:
On 08/03/2013 01:11 AM, Dennis Gilmore wrote:
[snip]
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
The way I've approached this for Tegra is to have the default environment define certain standard environment variables that U-Boot scripts can use, without a care. For example, the kernel should be loaded at ${kernel_addr_r}, initrd at ${ramdisk_addr_r}, DTB at ${fdt_addr_r}, etc.
I really don't want to need to deal with variable names either. I really want to just say load a kernel initramfs and here is some arguments and have the rest just work.
Note that the pxe menu parsing code in turn relies on ${kernel_addr_r} and ${ramdisk_addr_r} (and ${bootargs}) to know where to place what's requested.
On 08/05/2013 01:50 PM, Dennis Gilmore wrote:
On Mon, 05 Aug 2013 12:39:03 -0600 Stephen Warren swarren@wwwdotorg.org wrote:
On 08/03/2013 01:11 AM, Dennis Gilmore wrote:
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot.
...
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
That's only partially relevant for U-Boot. Two things U-Boot needs to support:
- Filesystem read for whatever filesystem /boot is part of (it could
very well be the / filesystem too, but U-Boot doesn't care, except for / vs. /boot in path names). This is a matter of adding the right flags to each board's U-Boot config file.
right, but at the least it needs to be ext4 not all boards today read ext4, btrfs may be something down the road also. u-boot doesnt need to care too much. it just needs to look in / and /boot
Well, ext4 is nice, but presumably everything could work just as well with ext3 or ext2. It's not really an issue other than arguing semantics though; U-Boot has ext4 support so it's presumably just a matter of persuading the relevant board maintainers to enable it, or sending them patches to do so?
...
[ Tegra boot.scr discussion ]
While this is a step forward, its still much more than we need to do if we enable pxe support and use sysboot to load a config file the says load this kernel and initramfs and pass these bootargs. My initial post had a fully working example.
Oh, I must have not read your email correctly. I thought the config file you posted was as example of what you wanted to achieve, rather than something which already worked.
Is it just a matter of turning on some config options in upstream U-Boot to enable this on Tegra? If so, I'd be very happy to create a patch to do so, test it with Fedora's images (got a link/README/...?) and send the patch to be applied.
Dear Dennis,
In message 20130805145059.14c35ebf@adria.ausil.us you wrote:
right, but at the least it needs to be ext4 not all boards today read ext4, btrfs may be something down the road also. u-boot doesnt need to care too much. it just needs to look in / and /boot
Where exactly do you raw the line here? Do we have to support RAID / DM devices, too? What about LVM? If you look for "regular system usage", using such technologies is more or less standard today. Will we need that?
The rest of the stuff (swap, LVM, ...) seems entirely related to the distro itself and/or whatever gets put into the initrd.
Mostly i was trying to show that where the other bits live doesn't matter.
Well, what about the case where /boot resides - say - on a multi-drive RAID1 array?
While this is a step forward, its still much more than we need to do if we enable pxe support and use sysboot to load a config file the says load this kernel and initramfs and pass these bootargs. My initial post had a fully working example.
Is my understanding correct that boot times are only a secondary concern to you, if any?
for fedora 20 i plan to use raw MLO support for OMAP and have it load the u-boot.img from the first bootable partition. SuSE are doing this
Please look into Tom's proposal to got the SPL / Falcon mode way. I fully agre with him there.
Best regards,
Wolfgang Denk
On 08/05/2013 02:43 PM, Wolfgang Denk wrote: ...
Please look into Tom's proposal to got the SPL / Falcon mode way. I fully agre with him there.
From my reading of doc/README.falcon, in order to use it, you still must
set everything up in order to do a full non-falcon boot, and then simply save a serialized version of the zImage/DTB/bootargs/... using a special U-Boot command.
As such, in order to use falcon mode, we still have to solve all the same problems that Dennis mentioned here so that the distro can set up falcon mode, and falcon mode is just an optimization that you can use once you've done that.
If we don't solve the problems Dennis mentioned, how would a distro be able to automatically script the initial setup of the serialized data that falcon mode boots?
And also, if a distro installs an updated kernel, how does it tell U-Boot to invalidate the previously serialized data that falcon mode uses to boot, and re-create it using the new kernel image?
So to me, falcon mode seems like a somewhat unrelated topic, and purely an optimization.
On Mon, Aug 05, 2013 at 02:49:58PM -0600, Stephen Warren wrote:
On 08/05/2013 02:43 PM, Wolfgang Denk wrote: ...
Please look into Tom's proposal to got the SPL / Falcon mode way. I fully agre with him there.
From my reading of doc/README.falcon, in order to use it, you still must set everything up in order to do a full non-falcon boot, and then simply save a serialized version of the zImage/DTB/bootargs/... using a special U-Boot command.
You must setup a certain amount of things, yes. And from what you've documented in the i2c thread about the Tegra orer, it depends a bit on the SoC.
[snip]
And also, if a distro installs an updated kernel, how does it tell U-Boot to invalidate the previously serialized data that falcon mode uses to boot, and re-create it using the new kernel image?
Dirty "secret" time, with device trees, it's just the device tree we loaded, with whatever run-time fixups are done. Doing this from the Linux side is a solvable problem.
So to me, falcon mode seems like a somewhat unrelated topic, and purely an optimization.
Yes, it's an optimization that I would like to see taken advantage of. I like the speed at which my laptop boots into Linux, and I would like to see that just as much if I was using an ARM board in a generic laptop enclosure.
On 08/05/2013 03:00 PM, Tom Rini wrote:
On Mon, Aug 05, 2013 at 02:49:58PM -0600, Stephen Warren wrote:
On 08/05/2013 02:43 PM, Wolfgang Denk wrote: ...
Please look into Tom's proposal to got the SPL / Falcon mode way. I fully agre with him there.
From my reading of doc/README.falcon, in order to use it, you still must set everything up in order to do a full non-falcon boot, and then simply save a serialized version of the zImage/DTB/bootargs/... using a special U-Boot command.
You must setup a certain amount of things, yes. And from what you've documented in the i2c thread about the Tegra orer, it depends a bit on the SoC.
[snip]
And also, if a distro installs an updated kernel, how does it tell U-Boot to invalidate the previously serialized data that falcon mode uses to boot, and re-create it using the new kernel image?
Dirty "secret" time, with device trees, it's just the device tree we loaded, with whatever run-time fixups are done. Doing this from the Linux side is a solvable problem.
Oh! Where does the zImage binary come from then? The documentation explicitly says it's written to flash by "spl export", and I assume that's so that the SPL (that's executing falcon mode) doesn't have to initialize SD card drivers, filesystems, etc. in order to load the zImage from the card, but rather simply a single flash driver to read the exported image. If falcon mode has to do all that, why is it any faster than the main U-Boot doing exactly the same?
On Mon, Aug 05, 2013 at 03:09:55PM -0600, Stephen Warren wrote:
On 08/05/2013 03:00 PM, Tom Rini wrote:
On Mon, Aug 05, 2013 at 02:49:58PM -0600, Stephen Warren wrote:
On 08/05/2013 02:43 PM, Wolfgang Denk wrote: ...
Please look into Tom's proposal to got the SPL / Falcon mode way. I fully agre with him there.
From my reading of doc/README.falcon, in order to use it, you still must set everything up in order to do a full non-falcon boot, and then simply save a serialized version of the zImage/DTB/bootargs/... using a special U-Boot command.
You must setup a certain amount of things, yes. And from what you've documented in the i2c thread about the Tegra orer, it depends a bit on the SoC.
[snip]
And also, if a distro installs an updated kernel, how does it tell U-Boot to invalidate the previously serialized data that falcon mode uses to boot, and re-create it using the new kernel image?
Dirty "secret" time, with device trees, it's just the device tree we loaded, with whatever run-time fixups are done. Doing this from the Linux side is a solvable problem.
Oh! Where does the zImage binary come from then? The documentation explicitly says it's written to flash by "spl export", and I assume that's so that the SPL (that's executing falcon mode) doesn't have to initialize SD card drivers, filesystems, etc. in order to load the zImage from the card, but rather simply a single flash driver to read the exported image. If falcon mode has to do all that, why is it any faster than the main U-Boot doing exactly the same?
'spl export' sets up the args blob that would be used, for ATAGs or the ready to go device tree. The time savings depends on how long (or, how quickly) your platform goes from power on to U-Boot asking you to interrupt auto-boot. When we were talking about this last year at the developers meeting, it didn't seem like a big deal to some platforms that Simon Glass had as they come up quickly. On others (like say beaglebone) skipping out on probing various things in U-Boot is a time win.
Hi Wolfgang,
On Mon, 05 Aug 2013 22:43:39 +0200 Wolfgang Denk wd@denx.de wrote:
Dear Dennis,
In message 20130805145059.14c35ebf@adria.ausil.us you wrote:
right, but at the least it needs to be ext4 not all boards today read ext4, btrfs may be something down the road also. u-boot doesnt need to care too much. it just needs to look in / and /boot
Where exactly do you raw the line here? Do we have to support RAID / DM devices, too? What about LVM? If you look for "regular system usage", using such technologies is more or less standard today. Will we need that?
I think not supporting dm or lvm is fine. raid1 mdraid for /boot would be nice down the road. I say this from the view of an enterprise arm based server hardware with a pair of sas drives for the filesystem. I would think that the vendor that produces said hardware would be on the hook for writing the support. what we need to ensure is that the installer know what valid options it can support are. if we can't do /boot on raid we cant do it.
The rest of the stuff (swap, LVM, ...) seems entirely related to the distro itself and/or whatever gets put into the initrd.
Mostly i was trying to show that where the other bits live doesn't matter.
Well, what about the case where /boot resides - say - on a multi-drive RAID1 array?
usually you can read from one drive from an array without setting up the raid array first.
While this is a step forward, its still much more than we need to do if we enable pxe support and use sysboot to load a config file the says load this kernel and initramfs and pass these bootargs. My initial post had a fully working example.
Is my understanding correct that boot times are only a secondary concern to you, if any?
boot times are not a primary concern, but that doesn't mean we need to make it needlessly long. Interrupting a boot to provide input is perfectly a valid option. Until a few minutes ago I did not know it was an option. making it short and sweet for most cases and if you need to do something different the user needs to do something. I can get behind supporting.
for fedora 20 i plan to use raw MLO support for OMAP and have it load the u-boot.img from the first bootable partition. SuSE are doing this
Please look into Tom's proposal to got the SPL / Falcon mode way. I fully agre with him there.
It is going to be my afternoon reading.
Thank you
Dennis
On 08/05/2013 03:08 PM, Dennis Gilmore wrote:
Hi Wolfgang,
On Mon, 05 Aug 2013 22:43:39 +0200 Wolfgang Denk wd@denx.de wrote:
Dear Dennis,
In message 20130805145059.14c35ebf@adria.ausil.us you wrote:
right, but at the least it needs to be ext4 not all boards today read ext4, btrfs may be something down the road also. u-boot doesnt need to care too much. it just needs to look in / and /boot
Where exactly do you raw the line here? Do we have to support RAID / DM devices, too? What about LVM? If you look for "regular system usage", using such technologies is more or less standard today. Will we need that?
I think not supporting dm or lvm is fine. raid1 mdraid for /boot would be nice down the road. I say this from the view of an enterprise arm based server hardware with a pair of sas drives for the filesystem. I would think that the vendor that produces said hardware would be on the hook for writing the support. what we need to ensure is that the installer know what valid options it can support are. if we can't do /boot on raid we cant do it.
Do we even know this on x86?
Perhaps so, since the distro installs grub, and knows which version of grub is installed, and hence knows if it supports RAID-1?
Pluggable protocol modules a la UEFI would solve that;-)
Dear Stephen Warren,
In message 520021E4.4070500@wwwdotorg.org you wrote:
Pluggable protocol modules a la UEFI would solve that;-)
Load UEFI as payload from U-Boot instead of Linux, and let UEFI do the rest of the work?
<me> runs and hides ;-)
Best regards,
Wolfgang Denk
On Sat, Aug 03, 2013 at 02:11:04AM -0500, Dennis Gilmore wrote:
Hi all,
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their systems configured as it works fairly well, recently i sent in a patch implementing most of what I would like to see for the wandboard[1]
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
OK. But lets remember up front that this is because in x86 land we've got a quite long standing fixed memory map.
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
That's all distro-details, yes. Support for reading files shoved inside filesystems inside containers is "just" a matter of developing / leveraging the code from elsewhere, if there's sufficient interest and desire.
pxe boot suport, two reasons, one is to be able to easily network boot the distro installer, the other is to use sysboot support post install with a extlinux.conf file to specify the kernel, initrd and bootargs
So, CONFIG_CMD_PXE enabled, for part of it. And someone to develop code to parse extlinux.conf which is the same config file PXELINUX uses, right, and then shoot up a menu based on it.
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
bootz is fine, raw initrd is fine. I would say that "updating the initramfs" is done by the distro scripts anyhow and not by hand from memory.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
Well, how does this happen on x86, specifically for dealing with the kernel?
fdt will be automatically loaded and provided fedora ships dtbs in /boot/dtb-<kernelver>/ I am not sure where other distros provide them, however u-boot should automatically load dtb and provide access to a fdt in a ${fdt_addr} variable that can be used by pxe_cmd but still allows the user to specify their own in extlinux.conf if desired.
I know Ubuntu shoves them under /lib, so this is an area where the distro-policy side of things will have to do some of the work. U-Boot can say what the base-name is, but we need to be given the directory path. I would also suggest at first that we don't want the user providing us memory locations to write this/that/something else to.
ideally the devicetree files need to be decoupled from the kernel, along the lines of what is being discussed in the kernel now[2].
Ideally, yes. And for everyones sake I hope that when this does happen, some thought is given about how vendors should store things on the device.
Distros should agree on a single location for the dtbs to be placed. /boot/dtb/ or /boot/dtbs/ are probably good locations. u-boot can then look in the path with and without /boot
Yes, this is something the distros need to have a chat about and coordinate on.
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
Sure, I believe this works today, for some values of boards with supported displays and USB keyboards.
longer term being able to edit the boot arguments should also be an option with the menu functionality.
Sure.
""" # extlinux.conf generated by anaconda
ui menu.c32
menu autoboot Welcome to Fedora. Automatic boot in # second{,s}. Press a key for options. menu title Fedora Boot Options. menu hidden
timeout 60 #totaltimeout 9000
label Fedora (3.9.9-302.fc19.armv7hl) 19 (Schr??dinger???s Cat) kernel /vmlinuz-3.9.9-302.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.9-302.fc19.armv7hl.img
label Fedora (3.9.4-301.fc19.armv7hl) 19 (Schr??dinger???s Cat) kernel /vmlinuz-3.9.4-301.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.4-301.fc19.armv7hl.img """ the above is an example of a extlinux.conf file on a fedora 19 system that works with the u-boot as shipped by calxeda on thier highbank energy cards. some of the valid extlinux options written out by anaconda cause noise on ARM as they are not implemented longer term it would be good to deal with them correctly. """ Ignoring unknown command: ui Ignoring malformed menu command: autoboot Ignoring malformed menu command: hidden Fedora Boot Options. 1: Fedora (3.9.9-302.fc19.armv7hl) 19 (Schr??dinger???s Cat) 2: Fedora (3.9.4-301.fc19.armv7hl) 19 (Schr??dinger???s Cat) Enter choice: """ is the output on boot, the user can then enter 1 or 2 to select a kernel to boot. if nothing is selected the first option is booted after the timeout expires.
Sure.
Now, I'd like to make some, or at least one suggestion. Distros should make use of falcon mode (SPL boots Linux directly), or at least offer it to the user. Similar to how I can configure, in the distro, a grub menu to show up, timeout and boot a default, or just boot the default unless I'm spamming escape during boot-up.
Now, since most of what you ask for exists today, can you write up what config options you'd want enabled and what a default environment looks like, for highbank or wandboard so that it works for generic distros, and we can start talking about what fallbacks would be needed so they can still be used in other contexts just as easily?
And when I say "Sure" above, I mean patches will be reviewed, if submitted.
Thanks!
On Mon, 5 Aug 2013 15:01:32 -0400 Tom Rini trini@ti.com wrote:
On Sat, Aug 03, 2013 at 02:11:04AM -0500, Dennis Gilmore wrote:
Hi all,
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their systems configured as it works fairly well, recently i sent in a patch implementing most of what I would like to see for the wandboard[1]
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
OK. But lets remember up front that this is because in x86 land we've got a quite long standing fixed memory map.
yes, but omap, tegra, imx etc all know that they need to start at some address 0x0000800, 0x10008000, 0x4000000 etc so the starting point is known per soc.
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
That's all distro-details, yes. Support for reading files shoved inside filesystems inside containers is "just" a matter of developing / leveraging the code from elsewhere, if there's sufficient interest and desire.
absolutely, just in for completeness since some systems today don't have ext4 support for instance.
pxe boot suport, two reasons, one is to be able to easily network boot the distro installer, the other is to use sysboot support post install with a extlinux.conf file to specify the kernel, initrd and bootargs
So, CONFIG_CMD_PXE enabled, for part of it. And someone to develop code to parse extlinux.conf which is the same config file PXELINUX uses, right, and then shoot up a menu based on it.
yes, the sysboot command that comes with CONFIG_CMD_PXE already does it.
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
bootz is fine, raw initrd is fine. I would say that "updating the initramfs" is done by the distro scripts anyhow and not by hand from memory.
for the most part yes, its built at rpm install time. sometimes a user decides to rebuild for different reasons.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
Well, how does this happen on x86, specifically for dealing with the kernel?
I'm not entirely sure dynamically assigned may not be feasible. just trying to avoid unexpected boot failure in the future because we didnt make sure there was enough room for something.
fdt will be automatically loaded and provided fedora ships dtbs in /boot/dtb-<kernelver>/ I am not sure where other distros provide them, however u-boot should automatically load dtb and provide access to a fdt in a ${fdt_addr} variable that can be used by pxe_cmd but still allows the user to specify their own in extlinux.conf if desired.
I know Ubuntu shoves them under /lib, so this is an area where the distro-policy side of things will have to do some of the work. U-Boot can say what the base-name is, but we need to be given the directory path. I would also suggest at first that we don't want the user providing us memory locations to write this/that/something else to.
I totally agree distros need to come together here and agree on something like we did with the hardware floating point linker location. and memory locations need to be not provided.
ideally the devicetree files need to be decoupled from the kernel, along the lines of what is being discussed in the kernel now[2].
Ideally, yes. And for everyones sake I hope that when this does happen, some thought is given about how vendors should store things on the device.
Distros should agree on a single location for the dtbs to be placed. /boot/dtb/ or /boot/dtbs/ are probably good locations. u-boot can then look in the path with and without /boot
Yes, this is something the distros need to have a chat about and coordinate on.
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
Sure, I believe this works today, for some values of boards with supported displays and USB keyboards.
longer term being able to edit the boot arguments should also be an option with the menu functionality.
Sure.
""" # extlinux.conf generated by anaconda
ui menu.c32
menu autoboot Welcome to Fedora. Automatic boot in # second{,s}. Press a key for options. menu title Fedora Boot Options. menu hidden
timeout 60 #totaltimeout 9000
label Fedora (3.9.9-302.fc19.armv7hl) 19 (Schr??dinger???s Cat) kernel /vmlinuz-3.9.9-302.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.9-302.fc19.armv7hl.img
label Fedora (3.9.4-301.fc19.armv7hl) 19 (Schr??dinger???s Cat) kernel /vmlinuz-3.9.4-301.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.4-301.fc19.armv7hl.img """ the above is an example of a extlinux.conf file on a fedora 19 system that works with the u-boot as shipped by calxeda on thier highbank energy cards. some of the valid extlinux options written out by anaconda cause noise on ARM as they are not implemented longer term it would be good to deal with them correctly. """ Ignoring unknown command: ui Ignoring malformed menu command: autoboot Ignoring malformed menu command: hidden Fedora Boot Options. 1: Fedora (3.9.9-302.fc19.armv7hl) 19 (Schr??dinger???s Cat) 2: Fedora (3.9.4-301.fc19.armv7hl) 19 (Schr??dinger???s Cat) Enter choice: """ is the output on boot, the user can then enter 1 or 2 to select a kernel to boot. if nothing is selected the first option is booted after the timeout expires.
Sure.
Now, I'd like to make some, or at least one suggestion. Distros should make use of falcon mode (SPL boots Linux directly), or at least offer it to the user. Similar to how I can configure, in the distro, a grub menu to show up, timeout and boot a default, or just boot the default unless I'm spamming escape during boot-up.
If that's an option absolutely. I didn't mention it because I did not think it was an option and I do not see it as necessary to unify things. but it is a great usability option to have for users. how is it configured?
Now, since most of what you ask for exists today, can you write up what config options you'd want enabled and what a default environment looks like, for highbank or wandboard so that it works for generic distros, and we can start talking about what fallbacks would be needed so they can still be used in other contexts just as easily?
And when I say "Sure" above, I mean patches will be reviewed, if submitted.
Thanks!
Thanks for the feedback. Most of what I want is there today and just needs to be enabled.
On Mon, Aug 05, 2013 at 03:11:20PM -0500, Dennis Gilmore wrote:
On Mon, 5 Aug 2013 15:01:32 -0400 Tom Rini trini@ti.com wrote:
On Sat, Aug 03, 2013 at 02:11:04AM -0500, Dennis Gilmore wrote:
[snip]
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
bootz is fine, raw initrd is fine. I would say that "updating the initramfs" is done by the distro scripts anyhow and not by hand from memory.
for the most part yes, its built at rpm install time. sometimes a user decides to rebuild for different reasons.
Right. So, lets me ask. In .deb-based land, I build my own kernels, and yelling cursing and screaming at the pains of doing things by hand, I use the 'deb-pkg' target in the kernel as that hooks into all the right things and I stop having to ^R/^O my history to not break my system (or look at my own script or whatever). What's it like in Fedora land, with initramfses? Do users do make bzImage/modules/install in the kernel or expect to use rpm-pkg to get something that hooks in just right?
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
Well, how does this happen on x86, specifically for dealing with the kernel?
I'm not entirely sure dynamically assigned may not be feasible. just trying to avoid unexpected boot failure in the future because we didnt make sure there was enough room for something.
Well, this is an important thing. How does this work, or what are the limits on x86? If someone can go dig up the maximum uncompressed kernel size before grub/syslinux/whathaveyou start stomping on ramdisks, we can document that in whatever comes out of this saying that one must (using spec language) make sure there's $X between the kernel and ramdisk load addrs (and perhaps even X+Y so that we can place the device tree in not-going-to-end-up-in-CONFIG_HIGHMEM-area).
fdt will be automatically loaded and provided fedora ships dtbs in /boot/dtb-<kernelver>/ I am not sure where other distros provide them, however u-boot should automatically load dtb and provide access to a fdt in a ${fdt_addr} variable that can be used by pxe_cmd but still allows the user to specify their own in extlinux.conf if desired.
I know Ubuntu shoves them under /lib, so this is an area where the distro-policy side of things will have to do some of the work. U-Boot can say what the base-name is, but we need to be given the directory path. I would also suggest at first that we don't want the user providing us memory locations to write this/that/something else to.
I totally agree distros need to come together here and agree on something like we did with the hardware floating point linker location. and memory locations need to be not provided.
ideally the devicetree files need to be decoupled from the kernel, along the lines of what is being discussed in the kernel now[2].
Ideally, yes. And for everyones sake I hope that when this does happen, some thought is given about how vendors should store things on the device.
Distros should agree on a single location for the dtbs to be placed. /boot/dtb/ or /boot/dtbs/ are probably good locations. u-boot can then look in the path with and without /boot
Yes, this is something the distros need to have a chat about and coordinate on.
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
Sure, I believe this works today, for some values of boards with supported displays and USB keyboards.
longer term being able to edit the boot arguments should also be an option with the menu functionality.
Sure.
""" # extlinux.conf generated by anaconda
ui menu.c32
menu autoboot Welcome to Fedora. Automatic boot in # second{,s}. Press a key for options. menu title Fedora Boot Options. menu hidden
timeout 60 #totaltimeout 9000
label Fedora (3.9.9-302.fc19.armv7hl) 19 (Schr??dinger???s Cat) kernel /vmlinuz-3.9.9-302.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.9-302.fc19.armv7hl.img
label Fedora (3.9.4-301.fc19.armv7hl) 19 (Schr??dinger???s Cat) kernel /vmlinuz-3.9.4-301.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.4-301.fc19.armv7hl.img """ the above is an example of a extlinux.conf file on a fedora 19 system that works with the u-boot as shipped by calxeda on thier highbank energy cards. some of the valid extlinux options written out by anaconda cause noise on ARM as they are not implemented longer term it would be good to deal with them correctly. """ Ignoring unknown command: ui Ignoring malformed menu command: autoboot Ignoring malformed menu command: hidden Fedora Boot Options. 1: Fedora (3.9.9-302.fc19.armv7hl) 19 (Schr??dinger???s Cat) 2: Fedora (3.9.4-301.fc19.armv7hl) 19 (Schr??dinger???s Cat) Enter choice: """ is the output on boot, the user can then enter 1 or 2 to select a kernel to boot. if nothing is selected the first option is booted after the timeout expires.
Sure.
Now, I'd like to make some, or at least one suggestion. Distros should make use of falcon mode (SPL boots Linux directly), or at least offer it to the user. Similar to how I can configure, in the distro, a grub menu to show up, timeout and boot a default, or just boot the default unless I'm spamming escape during boot-up.
If that's an option absolutely. I didn't mention it because I did not think it was an option and I do not see it as necessary to unify things. but it is a great usability option to have for users. how is it configured?
See doc/README.falcon and then yes, we would need to say here are how some parts of that (such as the don't try kernel, do u-boot key) are set.
Now, since most of what you ask for exists today, can you write up what config options you'd want enabled and what a default environment looks like, for highbank or wandboard so that it works for generic distros, and we can start talking about what fallbacks would be needed so they can still be used in other contexts just as easily?
And when I say "Sure" above, I mean patches will be reviewed, if submitted.
Thanks!
Thanks for the feedback. Most of what I want is there today and just needs to be enabled.
Good to hear!
On Mon, 5 Aug 2013 16:25:45 -0400 Tom Rini trini@ti.com wrote:
On Mon, Aug 05, 2013 at 03:11:20PM -0500, Dennis Gilmore wrote:
On Mon, 5 Aug 2013 15:01:32 -0400 Tom Rini trini@ti.com wrote:
On Sat, Aug 03, 2013 at 02:11:04AM -0500, Dennis Gilmore wrote:
[snip]
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
bootz is fine, raw initrd is fine. I would say that "updating the initramfs" is done by the distro scripts anyhow and not by hand from memory.
for the most part yes, its built at rpm install time. sometimes a user decides to rebuild for different reasons.
Right. So, lets me ask. In .deb-based land, I build my own kernels, and yelling cursing and screaming at the pains of doing things by hand, I use the 'deb-pkg' target in the kernel as that hooks into all the right things and I stop having to ^R/^O my history to not break my system (or look at my own script or whatever). What's it like in Fedora land, with initramfses? Do users do make bzImage/modules/install in the kernel or expect to use rpm-pkg to get something that hooks in just right?
generally we expect users to just do a yum update and the new kernel is automatically installed and a new initramfs is made for the kernel. they would run dracut to make a new initramfs if for instance they hit a systemd bug and needed to get the newer systemd binaries into the initramfs or in the case of when we dir the usr move feature and moved /lib /bin and /sbin into /usr the user needed to rebuild the initramfs including the module to do the conversion process.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
Well, how does this happen on x86, specifically for dealing with the kernel?
I'm not entirely sure dynamically assigned may not be feasible. just trying to avoid unexpected boot failure in the future because we didnt make sure there was enough room for something.
Well, this is an important thing. How does this work, or what are the limits on x86? If someone can go dig up the maximum uncompressed kernel size before grub/syslinux/whathaveyou start stomping on ramdisks, we can document that in whatever comes out of this saying that one must (using spec language) make sure there's $X between the kernel and ramdisk load addrs (and perhaps even X+Y so that we can place the device tree in not-going-to-end-up-in-CONFIG_HIGHMEM-area).
Sure, ill see what I can find out.
fdt will be automatically loaded and provided fedora ships dtbs in /boot/dtb-<kernelver>/ I am not sure where other distros provide them, however u-boot should automatically load dtb and provide access to a fdt in a ${fdt_addr} variable that can be used by pxe_cmd but still allows the user to specify their own in extlinux.conf if desired.
I know Ubuntu shoves them under /lib, so this is an area where the distro-policy side of things will have to do some of the work. U-Boot can say what the base-name is, but we need to be given the directory path. I would also suggest at first that we don't want the user providing us memory locations to write this/that/something else to.
I totally agree distros need to come together here and agree on something like we did with the hardware floating point linker location. and memory locations need to be not provided.
ideally the devicetree files need to be decoupled from the kernel, along the lines of what is being discussed in the kernel now[2].
Ideally, yes. And for everyones sake I hope that when this does happen, some thought is given about how vendors should store things on the device.
Distros should agree on a single location for the dtbs to be placed. /boot/dtb/ or /boot/dtbs/ are probably good locations. u-boot can then look in the path with and without /boot
Yes, this is something the distros need to have a chat about and coordinate on.
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
Sure, I believe this works today, for some values of boards with supported displays and USB keyboards.
longer term being able to edit the boot arguments should also be an option with the menu functionality.
Sure.
""" # extlinux.conf generated by anaconda
ui menu.c32
menu autoboot Welcome to Fedora. Automatic boot in # second{,s}. Press a key for options. menu title Fedora Boot Options. menu hidden
timeout 60 #totaltimeout 9000
label Fedora (3.9.9-302.fc19.armv7hl) 19 (Schr??dinger???s Cat) kernel /vmlinuz-3.9.9-302.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.9-302.fc19.armv7hl.img
label Fedora (3.9.4-301.fc19.armv7hl) 19 (Schr??dinger???s Cat) kernel /vmlinuz-3.9.4-301.fc19.armv7hl append console=ttyAMA0,115200 root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet LANG=en_US.UTF-8 initrd /initramfs-3.9.4-301.fc19.armv7hl.img """ the above is an example of a extlinux.conf file on a fedora 19 system that works with the u-boot as shipped by calxeda on thier highbank energy cards. some of the valid extlinux options written out by anaconda cause noise on ARM as they are not implemented longer term it would be good to deal with them correctly. """ Ignoring unknown command: ui Ignoring malformed menu command: autoboot Ignoring malformed menu command: hidden Fedora Boot Options. 1: Fedora (3.9.9-302.fc19.armv7hl) 19 (Schr??dinger???s Cat) 2: Fedora (3.9.4-301.fc19.armv7hl) 19 (Schr??dinger???s Cat) Enter choice: """ is the output on boot, the user can then enter 1 or 2 to select a kernel to boot. if nothing is selected the first option is booted after the timeout expires.
Sure.
Now, I'd like to make some, or at least one suggestion. Distros should make use of falcon mode (SPL boots Linux directly), or at least offer it to the user. Similar to how I can configure, in the distro, a grub menu to show up, timeout and boot a default, or just boot the default unless I'm spamming escape during boot-up.
If that's an option absolutely. I didn't mention it because I did not think it was an option and I do not see it as necessary to unify things. but it is a great usability option to have for users. how is it configured?
See doc/README.falcon and then yes, we would need to say here are how some parts of that (such as the don't try kernel, do u-boot key) are set.
Great thank you
Now, since most of what you ask for exists today, can you write up what config options you'd want enabled and what a default environment looks like, for highbank or wandboard so that it works for generic distros, and we can start talking about what fallbacks would be needed so they can still be used in other contexts just as easily?
And when I say "Sure" above, I mean patches will be reviewed, if submitted.
Thanks!
Thanks for the feedback. Most of what I want is there today and just needs to be enabled.
Good to hear!
On Mon, Aug 05, 2013 at 03:54:03PM -0500, Dennis Gilmore wrote:
On Mon, 5 Aug 2013 16:25:45 -0400 Tom Rini trini@ti.com wrote:
On Mon, Aug 05, 2013 at 03:11:20PM -0500, Dennis Gilmore wrote:
On Mon, 5 Aug 2013 15:01:32 -0400 Tom Rini trini@ti.com wrote:
On Sat, Aug 03, 2013 at 02:11:04AM -0500, Dennis Gilmore wrote:
[snip]
bootz and raw initrd support. not having to wrap kernels and initrds really is a must. raw initrd support means that its much simpler for a user to rebuild an initramfs if needed and bootz means we do not need to worry about making sure that we specify the correct addresses to load the kernel to when calling mkimage.
bootz is fine, raw initrd is fine. I would say that "updating the initramfs" is done by the distro scripts anyhow and not by hand from memory.
for the most part yes, its built at rpm install time. sometimes a user decides to rebuild for different reasons.
Right. So, lets me ask. In .deb-based land, I build my own kernels, and yelling cursing and screaming at the pains of doing things by hand, I use the 'deb-pkg' target in the kernel as that hooks into all the right things and I stop having to ^R/^O my history to not break my system (or look at my own script or whatever). What's it like in Fedora land, with initramfses? Do users do make bzImage/modules/install in the kernel or expect to use rpm-pkg to get something that hooks in just right?
generally we expect users to just do a yum update and the new kernel is automatically installed and a new initramfs is made for the kernel. they would run dracut to make a new initramfs if for instance they hit a systemd bug and needed to get the newer systemd binaries into the initramfs or in the case of when we dir the usr move feature and moved /lib /bin and /sbin into /usr the user needed to rebuild the initramfs including the module to do the conversion process.
Still in this quite hypothetical situation, dracut would take care of invoking mkimage, in a way that doesn't need SoC-specific magic whacked into it. Hypothetically, if we wanted to go down this path.
On 08/05/2013 02:11 PM, Dennis Gilmore wrote:
On Mon, 5 Aug 2013 15:01:32 -0400 Tom Rini trini@ti.com wrote:
On Sat, Aug 03, 2013 at 02:11:04AM -0500, Dennis Gilmore wrote:
Hi all,
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their systems configured as it works fairly well, recently i sent in a patch implementing most of what I would like to see for the wandboard[1]
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
OK. But lets remember up front that this is because in x86 land we've got a quite long standing fixed memory map.
yes, but omap, tegra, imx etc all know that they need to start at some address 0x0000800, 0x10008000, 0x4000000 etc so the starting point is known per soc.
If you can require a recent U-Boot, and place requirements on any U-Boot that you will support, then you can simply say "${kernel_addr_r}" instead of writing code to detect which SoC is in use, and manually coming up with the correct value to use. This is *the* reason I switched Tegra's default environment over to using the standard kernel_addr_r variable, so that Tegra boards worked the same as other standardized boards, and boot scripts could make assumptions re: the existence of the kernel_addr_r variable, and "just work" the same everywhere.
Similar arguments apply to many other things, such as board name, calculating DTB filename, etc.
I took a quick look at Fedora's arm-boot-config package. It seems like it's approach is to detect which U-Boot is present, then create a boot.scr that will work with it. This script will be different on different boards due to the differences in U-Boot features etc. The approach I was assuming was that for distros to support a board, you'd need a recent upstream U-Boot with certain features enabled (this is going to be true even if we all go enable PXE support...), hence the same boot.scr could be used anywhere, hence most of a-b-c could be radically simplified down to a single code-path. Then, I think the only thing missing would be some interactive menu stuff to choose which kernel to boot.
But, if the PXE support provides the interactive menu and solves the other board-specific issues, then perhaps that's a reasonable approach too.
On 08/05/2013 01:45 PM, Stephen Warren wrote:
I took a quick look at Fedora's arm-boot-config package. It seems like it's approach is to detect which U-Boot is present, then create a boot.scr that will work with it. This script will be different on different boards due to the differences in U-Boot features etc. The approach I was assuming was that for distros to support a board, you'd need a recent upstream U-Boot with certain features enabled (this is going to be true even if we all go enable PXE support...), hence the same boot.scr could be used anywhere, hence most of a-b-c could be radically simplified down to a single code-path. Then, I think the only thing missing would be some interactive menu stuff to choose which kernel to boot.
Not exactly. Arm-boot-config generates a single boot.scr that will run on any supported board. The first stage of the boot.scr detects the kind of board it's running on and sets things up accordingly (Do I have bootz? What's a good kernel load address? Do I need to load a dtb? What IO devices should I scan for loading the kernel? etc). After it's figured that out it looks through its list of kernels and boots the most recent one with a matching dtb. It also generates crude menus ("run menu" etc). It's not pretty, but it works well enough to allow us to distribute a single disk image with multiple that boots on a variety of platforms. As the author, I fully supported Dennis's initiative to make it obsolete as soon as humanly possible: It is vastly preferable to get consensus amongst the distributions. How to turn that into working with all the hardware makers is is to my mind the more daunting prospect. Chain loading uboots comes to mind. ew.
But, if the PXE support provides the interactive menu and solves the other board-specific issues, then perhaps that's a reasonable approach too.
Yes, it's a step in a useful direction.
[Just now checking the cross-distro folder, oops]
On 08/05/2013 05:03 PM, Brendan Conoboy wrote:
On 08/05/2013 01:45 PM, Stephen Warren wrote:
I took a quick look at Fedora's arm-boot-config package. It seems like it's approach is to detect which U-Boot is present, then create a boot.scr that will work with it. This script will be different on different boards due to the differences in U-Boot features etc. The approach I was assuming was that for distros to support a board, you'd need a recent upstream U-Boot with certain features enabled (this is going to be true even if we all go enable PXE support...), hence the same boot.scr could be used anywhere, hence most of a-b-c could be radically simplified down to a single code-path. Then, I think the only thing missing would be some interactive menu stuff to choose which kernel to boot.
Not exactly. Arm-boot-config generates a single boot.scr that will run on any supported board. The first stage of the boot.scr detects the kind of board it's running on and sets things up accordingly (Do I have bootz? What's a good kernel load address? Do I need to load a dtb? What IO devices should I scan for loading the kernel? etc). After it's figured that out it looks through its list of kernels and boots the most recent one with a matching dtb. It also generates crude menus ("run menu" etc). It's not pretty, but it works well enough to allow us to distribute a single disk image with multiple that boots on a variety of platforms. As the author, I fully supported Dennis's initiative to make it obsolete as soon as humanly possible: It is vastly preferable to get consensus amongst the distributions. How to turn that into working with all the hardware makers is is to my mind the more daunting prospect. Chain loading uboots comes to mind. ew.
Well, I think this is a solvable problem. Generally, vendors that want commodity support for their hardware work with the community to some degree. Providing both documentation and examples for "If you want most distros to work out of the box, you must.." should get traction.
Hi Dennis,
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
I personally don't see a reason for xfs on /boot (or the journalling nature of ext4, but thats besides the point). I agree though that a filesystem that supports symlinks is a must for being able to reasonably well install a regular kernel image (or multiple parallel) on /boot.
Therefore, the openSUSE on ARM team has a locally patched version of u-boot that handles booting from extX directly, because we did not like to use FAT or anything similar for /boot, and didn't see the need for adding a special /load (or similar) partition that is only there to make u-boot happy.
We've not always received love for this patch, given that all the people who "know how to do things" where confused because our way of building an image was so different than what is documented on the net everywhere, but it makes the layout feel more natural for somebody coming from non-ARM architectures.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
on openSUSE for ARM, we're converging to using only one boot.scr template, that is only templatized on the various addresses that are needed to be passed down and the flavor name (mostly for being able to print the right information during boot).
If it is reasonably possible to avoid the templatisation of the addresses, I'm all for it. I don't know if it is feasible, and for us it is not really an issue (unless it is a board supported by the zimage flavor).
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
True. not easy to detect though which input/outputs are connected. We have issues due to that as well.
Greetings, Dirk
On 08/08/2013 12:48 PM, Dirk Müller wrote: ...
Therefore, the openSUSE on ARM team has a locally patched version of u-boot that handles booting from extX directly, because we did not like to use FAT or anything similar for /boot, and didn't see the need for adding a special /load (or similar) partition that is only there to make u-boot happy.
Could you expand upon what "handles booting from extX directly" means? Upstream U-Boot has supported ext2/3 for as long as I've been involved with it (which admittedly isn't that long), and ext4 support was added recently. This allows U-Boot commands "extload" or "load" to access ext* just like any other file-system. Is there something more involved when you say "booting from extX directly" beyond just the extload/load commands?
...
on openSUSE for ARM, we're converging to using only one boot.scr template, that is only templatized on the various addresses that are needed to be passed down and the flavor name (mostly for being able to print the right information during boot).
If it is reasonably possible to avoid the templatisation of the addresses, I'm all for it. I don't know if it is feasible, and for us it is not really an issue (unless it is a board supported by the zimage flavor).
I think it's reasonable to require that boards supported by generic distros have upgraded/recent U-Boots that export a standard set of environment variables that define the various addresses, so that boot.scr authors don't have to care about platform differences, but rather simply use those variables. For example, kernel_addr_r, ramdisk_addr_r, fdt_addr_r, etc.
On Thu, Aug 08, 2013 at 02:01:12PM -0600, Stephen Warren wrote:
On 08/08/2013 12:48 PM, Dirk M?ller wrote: ...
Therefore, the openSUSE on ARM team has a locally patched version of u-boot that handles booting from extX directly, because we did not like to use FAT or anything similar for /boot, and didn't see the need for adding a special /load (or similar) partition that is only there to make u-boot happy.
Could you expand upon what "handles booting from extX directly" means? Upstream U-Boot has supported ext2/3 for as long as I've been involved with it (which admittedly isn't that long), and ext4 support was added recently. This allows U-Boot commands "extload" or "load" to access ext* just like any other file-system. Is there something more involved when you say "booting from extX directly" beyond just the extload/load commands?
I think (and hope!) this is SPL loading u-boot from ext* rather than having to be from FAT. This should be doable in such a way that we can handle both FAT and ext* in the same binary, now that we have generic load commands now.
Hi Stephen,
Could you expand upon what "handles booting from extX directly" means? Upstream U-Boot has supported ext2/3 for as long as I've been involved with it (which admittedly isn't that long), and ext4 support was added recently. This allows U-Boot commands "extload" or "load" to access ext* just like any other file-system. Is there something more involved when you say "booting from extX directly" beyond just the extload/load commands?
Sorry, I meant loading the SPL, which in some of the boot scripts are loaded from FAT still.
I think it's reasonable to require that boards supported by generic distros have upgraded/recent U-Boots that export a standard set of environment variables that define the various addresses, so that boot.scr authors don't have to care about platform differences, but rather simply use those variables. For example, kernel_addr_r, ramdisk_addr_r, fdt_addr_r, etc.
Sure, no problems with that.
Greetings, Dirk
On Thu, 8 Aug 2013 20:48:24 +0200 Dirk Müller dirk@dmllr.de wrote:
Hi Dennis,
right or wrong we want things to be simple for the user and to largely look like a linux system on x86 would. The user and distro should never need to worry about memory locations
so this would mean similar partitioning. i.e. /boot on ext4 root and swap on lvm or as raw partitions. people should be able to have just / on ext4 also. Down the road xfs /boot support would be a nice to have.
I personally don't see a reason for xfs on /boot (or the journalling nature of ext4, but thats besides the point). I agree though that a filesystem that supports symlinks is a must for being able to reasonably well install a regular kernel image (or multiple parallel) on /boot.
you are free to make that choice. I don't want to close doors or force people to do things only one way.
Therefore, the openSUSE on ARM team has a locally patched version of u-boot that handles booting from extX directly, because we did not like to use FAT or anything similar for /boot, and didn't see the need for adding a special /load (or similar) partition that is only there to make u-boot happy.
I've seen the patch and it is a start but it needs a bit of work to support all the options people could want from it.
We've not always received love for this patch, given that all the people who "know how to do things" where confused because our way of building an image was so different than what is documented on the net everywhere, but it makes the layout feel more natural for somebody coming from non-ARM architectures.
Right that is the ultimate goal.
when it comes to memory addressing a distro and user shouldn't need to know anything. Ideally u-boot will auto allocate addresses based on the size of loaded objects. starting with a base address internal to u-boot you load a kernel, when loading an initramfs u-boot automatically calculates an address that ensures it does not overlap with the kernel. same for a fdt if loaded. I say auto calculated because what we think today will be enough room may not be tomorrow, dynamically calculating gives the flexibility for whatever may come.
on openSUSE for ARM, we're converging to using only one boot.scr template, that is only templatized on the various addresses that are needed to be passed down and the flavor name (mostly for being able to print the right information during boot).
my proposal should enable you to use one boot.scr with variables. however my goal is to use a extlinux.conf file that is exactly the same format as those used for years in syslinux on install cd/dvds. I would love all distros to move this way as it gives you a nice menu and a simple interface with the option to select any installed kernel.
If it is reasonably possible to avoid the templatisation of the addresses, I'm all for it. I don't know if it is feasible, and for us it is not really an issue (unless it is a board supported by the zimage flavor).
output and input should happen on all possible devices and not just the serial console. If a user has a trimslice with only a HDMI monitor and usb keyboard they should be able to see the menu listing their possible kernels and be able to choose which one they want to boot.
True. not easy to detect though which input/outputs are connected. We have issues due to that as well.
sure, some of it is definitely things that are down the road nice to haves.
Dennis
On 08/03/2013 01:11 AM, Dennis Gilmore wrote:
Hi all,
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their systems configured as it works fairly well, recently i sent in a patch implementing most of what I would like to see for the wandboard[1]
There's also the possibility of chain-loading e.g. Grub from U-Boot, which I think would satisfy at least some of your desires, although there would still be a need for U-Boot's bootcmd to know to modified to be able to load Grub, and also enable e.g. CONFIG_API, so still needs some U-Boot work.
https://wiki.linaro.org/LEG/Engineering/Kernel/GRUBonUBOOT
I got it working pretty easily on one Tegra board...
An advantage would be familiar config file format and UI.
Dear Stephen Warren,
In message 52056B16.7050102@wwwdotorg.org you wrote:
There's also the possibility of chain-loading e.g. Grub from U-Boot, which I think would satisfy at least some of your desires, although there would still be a need for U-Boot's bootcmd to know to modified to be able to load Grub, and also enable e.g. CONFIG_API, so still needs some U-Boot work.
It appears boot time and increased complexity are no concern to you?
Best regards,
Wolfgang Denk
On 08/09/2013 04:49 PM, Wolfgang Denk wrote:
Dear Stephen Warren,
In message 52056B16.7050102@wwwdotorg.org you wrote:
There's also the possibility of chain-loading e.g. Grub from U-Boot, which I think would satisfy at least some of your desires, although there would still be a need for U-Boot's bootcmd to know to modified to be able to load Grub, and also enable e.g. CONFIG_API, so still needs some U-Boot work.
It appears boot time and increased complexity are no concern to you?
I don't think this is any different than a BIOS booting grub on x86.
On 08/09/2013 05:00 PM, Stephen Warren wrote:
On 08/09/2013 04:49 PM, Wolfgang Denk wrote:
Dear Stephen Warren,
In message 52056B16.7050102@wwwdotorg.org you wrote:
There's also the possibility of chain-loading e.g. Grub from U-Boot, which I think would satisfy at least some of your desires, although there would still be a need for U-Boot's bootcmd to know to modified to be able to load Grub, and also enable e.g. CONFIG_API, so still needs some U-Boot work.
It appears boot time and increased complexity are no concern to you?
I don't think this is any different than a BIOS booting grub on x86.
I suppose one way of addressing this might be for U-Boot to support grub.cfg; either by integrating some code from the Grub project or a re-implementation.
This ends up pretty similar to Dennis' original suggestion, but with the advantage that that config file format is something already widely generated by distros.
Dear Stephen Warren,
In message 5205748D.7060308@wwwdotorg.org you wrote:
It appears boot time and increased complexity are no concern to you?
I don't think this is any different than a BIOS booting grub on x86.
Probably not. But do we really have to chose the worst possible example as guide?
Why should the boot process take a minute or more when it could be done in a few seconds instead?
I think we should focus on what users want and need, not what makes life easy for developers.
Best regards,
Wolfgang Denk
On Sat, Aug 10, 2013 at 12:11:38PM +0200, Wolfgang Denk wrote:
Why should the boot process take a minute or more when it could be done in a few seconds instead?
It takes your x86 kit a minute to load grub? Hyperbole doesn't help advance technical discussions. Chaining another bootloader into the mix, assuming you don't add arbitrary interruptable config delays into each one, is probably a loss of tenths of a second at most.
I think we should focus on what users want and need, not what makes life easy for developers.
Not all users want or need the sane thing. Users who want the fastest possibly boot path are welcome to use u-boot (or find something even faster) directly, and no one is suggesting taking away u-boot's ability to boot a kernel and replacing that with grub.
What people are suggesting is that distributions might want to chain u-boot into grub to provide a consistent view to user-space which, given the number of times I get new ARM users of my distros asking questions like "how do I configure grub on this board?" seems like it might not be a bad plan for a generic distro solution.
There's also the whole hyperscale story here, where server admins tend to like things to be as similar as possible across platforms. It's one of the massive selling features of distributions that port to multiple architectures: once you're up and running, you don't care that you installed to ARM, POWER, x86, etc. But the one thing that appears to confuse a lot of sysadmins is that every arch uses a different boot method, and there's no reason we can't make this easier.
(FWIW, I intend to finally switch Ubuntu/PPC to using grub2 as well, for similar reasons. yaboot isn't exactly rocket science to use, but it's *different*, and for some users, a little different is all it takes to get frustrated and drop a platform on the floor).
... Adam
On Sat, Aug 03, 2013 at 02:11:04AM -0500, Dennis Gilmore wrote:
Hi all,
I wanted to start a discussion on defining a unified feature set that makes it simpler for the different distros to support ARM systems using u-boot. I have based a lot of my thoughts on how calxeda ship their systems configured as it works fairly well, recently i sent in a patch implementing most of what I would like to see for the wandboard[1]
So, lets try and bring this back around. It seems like the general concensous was on Yes for an opt-in and documented set of requirements. Can you please patch one of the boards you have with the set of required variable names and commands (and spelled out, rather than relying on config_cmd_default.h) ? I imagine we still need to chat a bit more about if a boot script should come from the distro or be built-in, but I also believe that if you write it into your patch as a U-Boot built-in, that'll further the discussion along nicely :) Thanks!
-
Adam Conrad -
Brendan Conoboy -
Dennis Gilmore -
Dirk Müller -
Peter Maydell -
Stephen Warren -
Tom Rini -
Wolfgang Denk