Hey
There are a bunch of places where we generate uImages and which duplicate information.
a kernel build most commonly outputs a zImage; this is then converted to an uImage for u-boot consumption with some rune like: mkimage -A arm -O linux -T kernel -C none \ -a 0x80008000 -e 0x80008000 -n Linux \ -d vmlinuz-2.6.35-1001-omap \ uImage-linaro
Linaro's platform wants to output media for various boards, probably as a single rootfs + separate boot bits for each SoC, but probably as multiple [ rootfs + boot bits ] for now.
I'd also like to setup daily kernel builds where people can directly grab uImages for testing.
Linaro's platform provides .debs with zImage which get converted to an updated uImage at upgrade time (and get flashed when that makes sense).
So there are already three places where this data is relevant: a) daily kernel builds b) live-helper image builds c) flash-kernel helper for kernel upgrade
Finally, there are two other places in the .deb ecosystem where this data is also copied: in debian-installer (both in Debian and Ubuntu), and in debian-cd (mostly in Ubuntu, but applies to Debian). d) debian-installer is a package in charge of collecting the bootloader and kernel pieces and generating installation media such as netboot files which contain the installer e) debian-cd is standalone software to create installation media such as ISOs or SD card images These two pieces are not used by Linaro right now, but it makes sense to try to come up with a solution which is usable by these pieces too.
So there is way too much duplication of this data; there are some duplications we could possibly avoid, for instance we could try calling flash-kernel's logic from live-helper, but it's not really a good approach for the debian-installer build, debian-cd, or the daily kernel builds. flash-kernel currently relies on being run on the target because it pokes /proc/cpuinfo, /proc/mtd and /dev/mtdblock*, or in Ubuntu the boot SD card; it's a "host install", not an installation towards a target.
We want a solution which is usable from a x86 build host to create an ARM image (daily kernel builds, debian-cd).
I see two ways to approach this problem: - data driven: we have data files for each board which document the addresses to pass to mkimage; all interested software parses the data and uses it to generate images - scripts: we have scripts for each board to do any board specific stuff; all interested software calls into the scripts to generate images
I think what would likely work is a combination of the two. We should have generic "task" scripts ("I want to generate a bootloader kernel image from a zImage"), and board-specific scripts for board-specific logic -- I think we don't really need board-specific scripts for Linaro because we try to unify all platforms to the same boot interface, but in practice it will be needed for other architectures and we can only get wide acceptance of such a replacement if we support more than just u-boot. This should all be backed by plain simple data files as much as possible so that extending support to one more board can often be done by dropping a data file in a directory.
So what kind of operations do we want to be able to do?
* generate an u-boot kernel image from a zImage - input: zImage, kernel load address - output: uImage * generate an u-boot initrd image from an initrd.gz - input: initrd.gz, initrd load address - output: uInitrd * generate a mostly boilerplate u-boot boot script with specific cmdline opts - input: cmdline opts, script load address - output: boot.cmd
I wonder whether we want to wrap other operations such as: * writing the kernel/initrd/script to flash or to a specific partition according to config * creation of boot media (partition table, vfat etc.) * other bootloaders
Comments welcome! Other use cases, existing software, whatever! :-)
Cheers
On 10 Aug 24, Loïc Minier wrote:
So what kind of operations do we want to be able to do?
- generate an u-boot kernel image from a zImage
- input: zImage, kernel load address
- output: uImage
- generate an u-boot initrd image from an initrd.gz
- input: initrd.gz, initrd load address
- output: uInitrd
- generate a mostly boilerplate u-boot boot script with specific cmdline opts
- input: cmdline opts, script load address
- output: boot.cmd
I wonder whether we want to wrap other operations such as:
- writing the kernel/initrd/script to flash or to a specific partition according to config
- creation of boot media (partition table, vfat etc.)
- other bootloaders
Comments welcome! Other use cases, existing software, whatever! :-)
Here are a few usecases that are important to me as a kernel developer:
* A way to install new kernel .debs from x86 dev machine - Currently the only way to do it is from a fully booted system
* Ability to have a 'backup' kernel/initramfs in case of broken kernels - A script running on x86 should allow me to revert to the backup kernel incase the new kernel does not boot
* Easy way to modify kernel cmdline - Script to dump existing kernel cmdline and to modify it easily
/Amit
On Tue, Aug 24, 2010, Amit Kucheria wrote:
- Ability to have a 'backup' kernel/initramfs in case of broken kernels
incase the new kernel does not boot
- A script running on x86 should allow me to revert to the backup kernel
Keeping a backup is a good idea, but installing it from your x86 wont work in the general case. It only makes sense when the boot uImage is on the SD card, not when it's in flash.
Thanks for the other ideas
On 10 Aug 24, Loïc Minier wrote:
On Tue, Aug 24, 2010, Amit Kucheria wrote:
- Ability to have a 'backup' kernel/initramfs in case of broken kernels
incase the new kernel does not boot
- A script running on x86 should allow me to revert to the backup kernel
Keeping a backup is a good idea, but installing it from your x86 wont work in the general case. It only makes sense when the boot uImage is on the SD card, not when it's in flash.
Thanks for the other ideas
Right, I was only talking about the SD card case. I thought we were moving away from kernel-in-flash since some future boards don't even have onboard flash.
/Amit
On Tue, Aug 24, 2010 at 12:09 PM, Amit Kucheria amit.kucheria@linaro.orgwrote:
On 10 Aug 24, Loïc Minier wrote:
On Tue, Aug 24, 2010, Amit Kucheria wrote:
- Ability to have a 'backup' kernel/initramfs in case of broken kernels
- A script running on x86 should allow me to revert to the backup
kernel
incase the new kernel does not boot
Keeping a backup is a good idea, but installing it from your x86 wont work in the general case. It only makes sense when the boot uImage is on the SD card, not when it's in flash.
Thanks for the other ideas
Right, I was only talking about the SD card case. I thought we were moving away from kernel-in-flash since some future boards don't even have onboard flash.
assuming we have support to cross install debs on your sdcard, wouldn''t it be good enough for your case to keep the old deb and install that rather than keeping the backup on the sdcard?
On Tue, Aug 24, 2010, Alexander Sack wrote:
assuming we have support to cross install debs on your sdcard, wouldn''t it be good enough for your case to keep the old deb and install that rather than keeping the backup on the sdcard?
It's a valid point; also, the kernel changes ABI so frequently that you'd probably have a couple of ABIs as backup kernels.
BTW: installing a linux-image works fine in a qemu-backed chroot; I do that all the time. The one catch is that if one has the system configured to run flash-kernel, that will break. But for images I create myself, I usually qemu-debootstrap + install a linux-image.deb and that works fine. It would work equally well on an installed system, but one has to remember to disable the flash-kernel step before installing a new kernel .deb.
On 10 Aug 24, Loïc Minier wrote:
On Tue, Aug 24, 2010, Alexander Sack wrote:
assuming we have support to cross install debs on your sdcard, wouldn''t it be good enough for your case to keep the old deb and install that rather than keeping the backup on the sdcard?
It's a valid point; also, the kernel changes ABI so frequently that you'd probably have a couple of ABIs as backup kernels.
Yes, makes sense. If we can install to SD from an x86 machine..
BTW: installing a linux-image works fine in a qemu-backed chroot; I do that all the time. The one catch is that if one has the system configured to run flash-kernel, that will break. But for images I create myself, I usually qemu-debootstrap + install a linux-image.deb and that works fine. It would work equally well on an installed system, but one has to remember to disable the flash-kernel step before installing a new kernel .deb.
I don't want to debootstrap everytime I want to try out a change. We need something that works for an installed system if we are to convince people to try out Linaro kernels.
/Amit
Hi,
On Mon, Aug 23, 2010 at 11:20 PM, Loïc Minier loic.minier@linaro.org wrote:
[...]
a kernel build most commonly outputs a zImage; this is then converted to an uImage for u-boot consumption with some rune like: mkimage -A arm -O linux -T kernel -C none \ -a 0x80008000 -e 0x80008000 -n Linux \ -d vmlinuz-2.6.35-1001-omap \ uImage-linaro
Couldn't we simply use the kernel tree "make uImage" rule, and put the uImage in the kernel binary packages, rather than reduplicating this elsewhere? Of course, which kernel tree targets to build may then become board-specific, which might be seen as a disadvantage.
I've tended to treat the kernel tree rule as the canonical way of generating a valid uImage, though maybe not everyone will agree with that.
Cheers ---Dave
On Tue, Aug 24, 2010 at 11:28 AM, Dave Martin dave.martin@linaro.org wrote:
Hi,
On Mon, Aug 23, 2010 at 11:20 PM, Loïc Minier loic.minier@linaro.org wrote:
[...]
a kernel build most commonly outputs a zImage; this is then converted to an uImage for u-boot consumption with some rune like: mkimage -A arm -O linux -T kernel -C none \ -a 0x80008000 -e 0x80008000 -n Linux \ -d vmlinuz-2.6.35-1001-omap \ uImage-linaro
Couldn't we simply use the kernel tree "make uImage" rule, and put the uImage in the kernel binary packages, rather than reduplicating this elsewhere? Of course, which kernel tree targets to build may then become board-specific, which might be seen as a disadvantage.
As you rightly pointed out, every arch/bootloader combination could then have a different potentional target image format. So the kernel package sticks to the simplicity of generating a zImage. And flash-kernel(?) has the logic of converting the zImage into whatever format is acceptable to the bootloader.
I've tended to treat the kernel tree rule as the canonical way of generating a valid uImage, though maybe not everyone will agree with that.
I use it for my own kernels too.
/Amit
On Tue, Aug 24, 2010, Dave Martin wrote:
Couldn't we simply use the kernel tree "make uImage" rule, and put the uImage in the kernel binary packages, rather than reduplicating this elsewhere? Of course, which kernel tree targets to build may then become board-specific, which might be seen as a disadvantage.
ISTR someone commented upstream a long time ago that the uImage target was clutter and that u-boot should learn to cope with zImage as other bootloaders do.
Problem with the uImage target is that we want different uImages built from the same kernel flavours, and it's not really worth having one binary kernel package per board, it's nicer to have a single binary kernel package for e.g. all of omap3 and convert the zImage to what's needed for a specific board at package installation time.
What might make sense is keeping the data in the kernel though: certainly the uImage and uInitrd load addresses are best kept within the kernel?
I've tended to treat the kernel tree rule as the canonical way of generating a valid uImage, though maybe not everyone will agree with that.
It would probably be valid for daily kernel builds; but it's only one of the use cases; any boot media generation (e.g. linaro-media-create or live-helper, or debian-cd, or debian-installer) couldn't use that, and it doesn't make sense for the flash-kernel use case either. There even are some platforms which support both zImage and uImage (e.g. imx51/redboot and imx51/u-boot), albeit Linaro isn't too interested in these the plan needs to support them if we want it to go in distros.
On Tue, 24 Aug 2010, Loïc Minier wrote:
On Tue, Aug 24, 2010, Dave Martin wrote:
Couldn't we simply use the kernel tree "make uImage" rule, and put the uImage in the kernel binary packages, rather than reduplicating this elsewhere? Of course, which kernel tree targets to build may then become board-specific, which might be seen as a disadvantage.
ISTR someone commented upstream a long time ago that the uImage target was clutter and that u-boot should learn to cope with zImage as other bootloaders do.
This could have been me, or if it wasn't me then I would have strongly approved. I even posted some notes about the de facto header that is found at the front of the zImage binary (that was many years ago and therefore I can't find an archive link right now):
Offset into zImage Value Description 0x24 0x016F2818 Magic number used to identify this is an ARM Linux zImage 0x28 start address The address the zImage starts at 0x2C end address The address the zImage ends at
The start and end offsets can be used to validate the length of the provided zImage (size = end - start).
The zImage code is (in most cases) Position Independent Code (PIC) so may be loaded anywhere within the available address space. The start address in that case is zero.
If U-Boot really insists on its uImage format in flash, then I really think that the mkimage functionality could be made into U-Boot itself.
Problem with the uImage target is that we want different uImages built from the same kernel flavours, and it's not really worth having one binary kernel package per board, it's nicer to have a single binary kernel package for e.g. all of omap3 and convert the zImage to what's needed for a specific board at package installation time.
Like I said though, if all of OMAP3 can share the same kernel image, the corresponding uImage is likely to be identical too.
What might make sense is keeping the data in the kernel though: certainly the uImage and uInitrd load addresses are best kept within the kernel?
No. The initrd load address has to be passed to the kernel and that should remain that way so to preserve flexibility in its placement. The zImage can be loaded anywhere -- it is just mkimage that insists on a load address (and that's a problem with U-Boot not the kernel).
I've tended to treat the kernel tree rule as the canonical way of generating a valid uImage, though maybe not everyone will agree with that.
It would probably be valid for daily kernel builds; but it's only one of the use cases; any boot media generation (e.g. linaro-media-create or live-helper, or debian-cd, or debian-installer) couldn't use that, and it doesn't make sense for the flash-kernel use case either. There even are some platforms which support both zImage and uImage (e.g. imx51/redboot and imx51/u-boot), albeit Linaro isn't too interested in these the plan needs to support them if we want it to go in distros.
It would be great if we agreed that U-Boot should be able to cope with zImage directly. Adding the necessary support to U-Boot is trivial.
Nicolas
It would be great if we agreed that U-Boot should be able to cope with zImage directly. Adding the necessary support to U-Boot is trivial.
I did come across a ramdom version of U-Boot which could load zImages, but I don't have a pointer to that any more.
Also, does anyone know why U-Boot always prints "loading legacy image" when loading images created with mkimage? Kinda suggests the U-Boot folks think this image format has been superseded by _something_ but I never figured out why.
Cheers ---Dave
On Wed, Aug 25, 2010, Dave Martin wrote:
Also, does anyone know why U-Boot always prints "loading legacy image" when loading images created with mkimage? Kinda suggests the U-Boot folks think this image format has been superseded by _something_ but I never figured out why.
From a quick look at U-Boot, it defaults to IMAGE_FORMAT_LEGACY if there is the U-Boot image magic number, else if U-Boot is built with FDT (DeviceTree) support, it uses IMAGE_FORMAT_FIT. In any other case, the image is considered invalid (IMAGE_FORMAT_INVALID).
I found this old submission of bootz to boot zImages: http://lists.denx.de/pipermail/u-boot/2003-June/001589.html
The patches had many issues (I for one would have been unhappy about having a new bootz command, but perhaps I'm missing the reason why bootm can't be used).
On Tue, 24 Aug 2010, Dave Martin wrote:
Hi,
On Mon, Aug 23, 2010 at 11:20 PM, Loïc Minier loic.minier@linaro.org wrote:
[...]
a kernel build most commonly outputs a zImage; this is then converted to an uImage for u-boot consumption with some rune like: mkimage -A arm -O linux -T kernel -C none \ -a 0x80008000 -e 0x80008000 -n Linux \ -d vmlinuz-2.6.35-1001-omap \ uImage-linaro
Couldn't we simply use the kernel tree "make uImage" rule, and put the uImage in the kernel binary packages, rather than reduplicating this elsewhere? Of course, which kernel tree targets to build may then become board-specific, which might be seen as a disadvantage.
I think this is a flaw with U-Boot's image format. We've introduced in the last merge window the ability for zImage to be completely position independent. But the U-Boot's mkimage tool still insists on a load address to be part of the uImage itself instead of having the possibility to only decide at load time where to load the image.
But until now, the kernel zImage had to encode at least the address of the decompressed kernel which is related to the physical RAM offset. And physical RAM offset is also related to the address used by mkimage. So for the time being, there is little point not building the uImage at kernel build time because if you need a different address passed to mkimage then you still (at the moment) need a different kernel image anyway.
Right now we can have a single kernel binary that can boot on multiple boards if they're using the same SOC family. That also means that they are using the same load address with mkimage. Otherwise you need a different kernel binary already. So if a given kernel is always going to be booted by U-Boot, then the uImage format can as well be built with the kernel "make uImage" target right away.
Nicolas
-
Alexander Sack -
Amit Kucheria -
Dave Martin -
Loïc Minier -
Nicolas Pitre