There has been quite a lot of discussion of image creation, how many flavours there should be, where we store them and for how long, how they get generated and delivered etc. And there were many sessions at UDS as well as a couple of threads on this list covering this.
All this discussion left me feeling that people had got a little fixated on pre-generated images, and the alternative approach of using an installer has not received enough attention. Only the server sessions even mentioned this and even there we are still planning to start with an image and move to an installer later (IIRC).
So I just thought I'd bring this up as a point to bear in mind as we move along. I really think we should be taking the installer option more seriously in the future, and not trying to serve the whole world of possibilities with images.
There are clearly advantages to the 'just dd an image onto removable media and plug it in to get started' and having one of these available for that 'instant start' fix makes a lot of sense. I'm not suggesting we should stop doing that; this is the 'crack' to get new people addicted :-)
But the more you get into having different images for different machines and purposes, the more you suffer from the combinatorial explosion problem. And all those images are actually just combinations of packages for the hardware and the software wanted, which are trivially specified and installed by a general-purpose installer (an example of which already exists in Debian, working across all architectures, using the same software to run the install as is ultimately installed on the machines).
By using an installer (actually just a mini-distro image, which could be made exactly as the images we are currently making) running on the target hardware you gain the ability to make as many 'tasks' (currently images) as you like, and support as many boards as you want to support without the combinatorial explosion. And it can be skinned and pre-configured in different ways for different use-cases (fully automated/simple task selection/fully configurable, headless/gui, netboot/SD image and so on).
I realise that this involves some work, but so does all that image-generation, and I do believe that the flexibility of this approach is powerful and we shouldn't ignore it to the degree we currently are.
Pre-built images comes from an 'embedded systems' view of the world. Real computers can run installers to put their operating systems on and set themselves up, and we should be taking advantage of that. ARM computers _are_ real computers now and we should be treating them as such.
Yes it takes a lot longer on initial boot, which is why I agree that there should be at least one 'instant gratification' image, but for real work, having to install your OS the first time you boot a board is not a big deal, and provides enormous flexibility. We can make it slick and painless.
So, that's all. We don't necessarily need to have a big debate about this now, I'd really just like to make sure that this is properly on the agenda for the next cycle, and that we don't all get so stuck in the 'everything must be supplied as pre-generated image' mindset that we back ourselves into a very inefficient corner.
Wookey
On Wednesday 01 June 2011, Wookey wrote:
Pre-built images comes from an 'embedded systems' view of the world. Real computers can run installers to put their operating systems on and set themselves up, and we should be taking advantage of that. ARM computers are real computers now and we should be treating them as such.
I've been claiming for some time that the devices we are dealing with are actually a new class of devices that are neither "embedded" nor "real computer". You have the option to install anything you like in a system like Android or an Ubuntu based netbook, but installing one from scratch requires much more customization than installing Ubuntu on a PC. E.g. most devices today require their own custom kernel.
I absolutely agree that we should consequently think beyond image generation, but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer.
Yes it takes a lot longer on initial boot, which is why I agree that there should be at least one 'instant gratification' image, but for real work, having to install your OS the first time you boot a board is not a big deal, and provides enormous flexibility. We can make it slick and painless.
My main question to this is "install from where?". Optical drives are probably out of the question here, and if you use a single SD card as both the source for your packages and the root file system to install them on, it still sounds rather pointless to have an installer.
If the answer is that we always want to install from the network, then there might not be too much work required at all: As you say, we still need an image file for the installer and it can probably be quite similar to the one debootstrap output we are already using, except that it boots into whatever frontend to apt is being used this decade in order to complete the installation.
So, that's all. We don't necessarily need to have a big debate about this now, I'd really just like to make sure that this is properly on the agenda for the next cycle, and that we don't all get so stuck in the 'everything must be supplied as pre-generated image' mindset that we back ourselves into a very inefficient corner.
Thanks for bringing it up.
Arnd
+++ Arnd Bergmann [2011-06-01 16:11 +0200]:
On Wednesday 01 June 2011, Wookey wrote:
I absolutely agree that we should consequently think beyond image generation, but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer.
My main question to this is "install from where?".
I'd say the default case (at least for current hardware) is booting from SD or USB stick and installing from the network. (Which is how I install PCs these days too - it's a very long time since I got a CD out :-)).
All sorts of things are possible (and a well-designed installer will be flexible about sources and sinks, as the existing Debian one is), but if we only supported the above option I think that would cover most of what we want to support. (ARM servers might want different boot media?)
but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer
I'm not sure I follow you here. Are you suggesting that there is some third way between a locally-bootable installer image and pre-built images? (In which case what - I don't see this), or just that CDs are no longer the default media (agreed).
Wookey
On Wed, Jun 01, 2011 at 03:36:47PM +0100, Wookey wrote:
+++ Arnd Bergmann [2011-06-01 16:11 +0200]:
On Wednesday 01 June 2011, Wookey wrote:
I absolutely agree that we should consequently think beyond image generation, but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer.
My main question to this is "install from where?".
I'd say the default case (at least for current hardware) is booting from SD or USB stick and installing from the network. (Which is how I install PCs these days too - it's a very long time since I got a CD out :-)).
All sorts of things are possible (and a well-designed installer will be flexible about sources and sinks, as the existing Debian one is), but if we only supported the above option I think that would cover most of what we want to support. (ARM servers might want different boot media?)
but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer
I'm not sure I follow you here. Are you suggesting that there is some third way between a locally-bootable installer image and pre-built images? (In which case what - I don't see this), or just that CDs are no longer the default media (agreed).
A basic pre-built image with network and packager functionality is arguably almost an installer anyway.
Anrdoid aside, if any of our images is not just an "apt-get install" command away from nano, we should be asking ourselves why not.
Separate question how big is Debian-installer, in terms of filesystem and RAM footprint?
If we can move the entire installation system to a ramfs on boot, we can unmount and free up the boot device, allowing the system to be installed in-place.
This might also require Linux's idea of which devices are "removable" to be overridden though, so that they can be repartitioned without a reboot. I think the kernel hard-codes this for some of our boards currently; i.e., the boot SD slot may be considered non- removable. I don't know how easy it is do get around this.
Cheers ---Dave
On Wednesday 01 June 2011 16:56:07 Dave Martin wrote:
On Wed, Jun 01, 2011 at 03:36:47PM +0100, Wookey wrote:
+++ Arnd Bergmann [2011-06-01 16:11 +0200]:
On Wednesday 01 June 2011, Wookey wrote:
I absolutely agree that we should consequently think beyond image generation, but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer.
My main question to this is "install from where?".
I'd say the default case (at least for current hardware) is booting from SD or USB stick and installing from the network. (Which is how I install PCs these days too - it's a very long time since I got a CD out :-)).
Yes, me too.
but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer
I'm not sure I follow you here. Are you suggesting that there is some third way between a locally-bootable installer image and pre-built images? (In which case what - I don't see this), or just that CDs are no longer the default media (agreed).
One approach that seems to be getting more popular these days is to have a bootable system as a USB image, with a way to clone that installation to another drive. This is arguably a bit different from the classic installer where you boot a very small image (not made of regular packages but e.g. udeb instead of deb, or purely busybox based) that installs a system to the final destination from scratch.
If we can move the entire installation system to a ramfs on boot, we can unmount and free up the boot device, allowing the system to be installed in-place.
This is probably the main question: If we want an installer, should it be something that boots as an initramfs and is able to install in a very flexible way, or do we instead build a minimal image that basically includes everything needed to add more stuff through apt-get, possibly with a way to clone itself to another drive?
Arnd
On Wed, Jun 01, 2011 at 07:32:10PM +0200, Arnd Bergmann wrote:
On Wednesday 01 June 2011 16:56:07 Dave Martin wrote:
On Wed, Jun 01, 2011 at 03:36:47PM +0100, Wookey wrote:
+++ Arnd Bergmann [2011-06-01 16:11 +0200]:
On Wednesday 01 June 2011, Wookey wrote:
I absolutely agree that we should consequently think beyond image generation, but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer.
My main question to this is "install from where?".
I'd say the default case (at least for current hardware) is booting from SD or USB stick and installing from the network. (Which is how I install PCs these days too - it's a very long time since I got a CD out :-)).
Yes, me too.
but that doesn't necessarily mean that a CD image to perform an unattended installation is a better answer
I'm not sure I follow you here. Are you suggesting that there is some third way between a locally-bootable installer image and pre-built images? (In which case what - I don't see this), or just that CDs are no longer the default media (agreed).
One approach that seems to be getting more popular these days is to have a bootable system as a USB image, with a way to clone that installation to another drive. This is arguably a bit different
i.e., "live CD"? Ubuntu's installers for ARM have followed this approach.
The complication for ARM is that you can probably only boot from one device, which can complicate things like bootloader installation.
from the classic installer where you boot a very small image (not made of regular packages but e.g. udeb instead of deb, or purely busybox based) that installs a system to the final destination from scratch.
If we can move the entire installation system to a ramfs on boot, we can unmount and free up the boot device, allowing the system to be installed in-place.
This is probably the main question: If we want an installer, should it be something that boots as an initramfs and is able to install in a very flexible way, or do we instead build a minimal image that basically includes everything needed to add more stuff through apt-get, possibly with a way to clone itself to another drive?
Option two is certainly the thing we are closest to right now (i.e., nano). We don't have "clonability" yet, though when the installed system is removable storace (MMC/SD/USB) it's pertty easy to clone offline anyway.
A self-contained installer which runs from initramfs (or whatever) would be more effort, so the feasibility really depends on whether there's something available which can already nearly do this.
Cheers ---Dave
+++ Dave Martin [2011-06-01 15:56 +0100]:
Separate question how big is Debian-installer, in terms of filesystem and RAM footprint?
There are various flavours. Primarily:
1) a 'full' image which is 160MB and includes the base system that is installed (so you can get a system without network access),
2) a minimal image which is 33MB and needs a network to download the packages to be installed.
3) initrd-based headless installers for various arm boxes which vary from 4.5 to 18Mb (including kernel) e.g: http://ftp.debian.org/debian/dists/stable/main/installer-armel/current/image...
The D-I system is very flexible. But it does depend on the building of udebs (minimal versions of debs which are used to make installed bootable images). You could make a rather fatter installer out of normal debs, but that would preclude the initrd flavour.
I don't know how much ram is needed, but it works on the 32Mb NSLU2 so 'not much', at least in headless form. More for the GUI version.
If we can move the entire installation system to a ramfs on boot, we can unmount and free up the boot device, allowing the system to be installed in-place.
This is possible. On most of the currently-supported-by-debian arm devices a console and some uboot runes are required to get things installed and defaulting to booting off the desired device. Some (such as thecus n2100) support a web+ssh install: http://www.cyrius.com/debian/iop/n2100/install.html
Things are somewhat simplified if we are only worrying about devices which already boot from USB or SD/MMC.
This might also require Linux's idea of which devices are "removable" to be overridden though, so that they can be repartitioned without a reboot. I think the kernel hard-codes this for some of our boards currently; i.e., the boot SD slot may be considered non- removable. I don't know how easy it is do get around this.
This is only a problem if repartitioning is needed.
Wookey
On Wed, Jun 01, 2011 at 11:37:30PM +0100, Wookey wrote:
+++ Dave Martin [2011-06-01 15:56 +0100]:
Separate question how big is Debian-installer, in terms of filesystem and RAM footprint?
There are various flavours. Primarily:
a 'full' image which is 160MB and includes the base system that is installed (so you can get a system without network access),
a minimal image which is 33MB and needs a network to download the
packages to be installed.
- initrd-based headless installers for various arm boxes which vary
from 4.5 to 18Mb (including kernel) e.g: http://ftp.debian.org/debian/dists/stable/main/installer-armel/current/image...
The D-I system is very flexible. But it does depend on the building of udebs (minimal versions of debs which are used to make installed bootable images). You could make a rather fatter installer out of normal debs, but that would preclude the initrd flavour.
I don't know how much ram is needed, but it works on the 32Mb NSLU2 so 'not much', at least in headless form. More for the GUI version.
If we can move the entire installation system to a ramfs on boot, we can unmount and free up the boot device, allowing the system to be installed in-place.
This is possible. On most of the currently-supported-by-debian arm devices a console and some uboot runes are required to get things installed and defaulting to booting off the desired device. Some (such as thecus n2100) support a web+ssh install: http://www.cyrius.com/debian/iop/n2100/install.html
Things are somewhat simplified if we are only worrying about devices which already boot from USB or SD/MMC.
This might also require Linux's idea of which devices are "removable" to be overridden though, so that they can be repartitioned without a reboot. I think the kernel hard-codes this for some of our boards currently; i.e., the boot SD slot may be considered non- removable. I don't know how easy it is do get around this.
This is only a problem if repartitioning is needed.
Partitioning is normally at least desirable during install, no?
---Dave
-
Arnd Bergmann -
Dave Martin -
Wookey