Hi,
During LDS we agreed to provide prebuilt Linaro images at our milestones[1], for users who can't/won't run linaro-media-create. One thing we forgot to discuss, though, is if it's worth doing so for all image types or just for the LEB(s).
Our downloads page (http://www.linaro.org/downloads/) has 4 different image types, but the email for weekly testing, for example, lists only UbuntuDesktop as official and the others as community images. More importantly, though, I'm not sure the people who just want to try Linaro quickly will be interested in anything other than UbuntuDesktop at this point, so we might as well avoid the extra work to provide images that are unlikely to be of any use.
Can anybody think of other reasons why we should provide images of all types?
[1] https://wiki.linaro.org/Platform/Infrastructure/Specs/PrebuiltImages
On 20 May 2011 15:38, Guilherme Salgado guilherme.salgado@linaro.org wrote:
During LDS we agreed to provide prebuilt Linaro images at our milestones[1], for users who can't/won't run linaro-media-create. One thing we forgot to discuss, though, is if it's worth doing so for all image types or just for the LEB(s).
Our downloads page (http://www.linaro.org/downloads/) has 4 different image types, but the email for weekly testing, for example, lists only UbuntuDesktop as official and the others as community images. More importantly, though, I'm not sure the people who just want to try Linaro quickly will be interested in anything other than UbuntuDesktop at this point, so we might as well avoid the extra work to provide images that are unlikely to be of any use.
Can anybody think of other reasons why we should provide images of all types?
The short answer is "if we care enough to release it we should care enough to release it in a format that people can easily use". If there are too many image types then we should cut down on the number so we're producing fewer artefacts of all types.
At a bare minimum, prebuilt SD card images should come in a "general desktop" and the "nano" small size version. But those are just my two use cases (one for functionality and one for small download and disk size); it seems likely that some of the people who would be interested in the other image types will have all the constraints we discussed at UDS that make linaro-media-create not a usable solution for them.
We're only doing this once a month, we should just produce images for everything rather than trying to second guess which we can get away with not generating.
-- PMM
On Fri, 20 May 2011 16:03:16 +0100, Peter Maydell peter.maydell@linaro.org wrote:
We're only doing this once a month, we should just produce images for everything rather than trying to second guess which we can get away with not generating.
While that's a nice sentiment, we can't ignore the disk space usage that this implies.
If we take as an example 6 supported boards, and 6 images (3x2G, 1x1G, 1x.5G, 1x.1G), that's ~48G per month, so in one year ~0.5T of images. I haven't done the maths to factor in the bandwidth here.
We can only do this if we change our image retention policy. I think that the benefit of this change is worth doing that, but we can't naïvely start generating all of this without considering these questions.
Thanks,
James
P.S. if anyone replies saying that it doesn't matter because disk space is cheap, I shall be forced to nominate you as Linaro sysadmin for this cycle.
On 20 May 2011 17:12, James Westby james.westby@linaro.org wrote:
On Fri, 20 May 2011 16:03:16 +0100, Peter Maydell peter.maydell@linaro.org wrote:
We're only doing this once a month, we should just produce images for everything rather than trying to second guess which we can get away with not generating.
While that's a nice sentiment, we can't ignore the disk space usage that this implies.
If we take as an example 6 supported boards, and 6 images (3x2G, 1x1G, 1x.5G, 1x.1G), that's ~48G per month, so in one year ~0.5T of images. I haven't done the maths to factor in the bandwidth here.
We can only do this if we change our image retention policy. I think that the benefit of this change is worth doing that, but we can't naïvely start generating all of this without considering these questions.
If disk space is a problem then we can take the other approach that was suggested at UDS, which is that we autogenerate images on the fly as they are asked for and keep a cache of the most recently used ones. I thought the reason we opted to do the simple "just statically generate images for each monthly release" was exactly that for a monthly thing we thought the disk space usage was acceptable.
-- PMM
On 20 May 2011 17:12, James Westby james.westby@linaro.org wrote:
On Fri, 20 May 2011 16:03:16 +0100, Peter Maydell peter.maydell@linaro.org wrote:
We're only doing this once a month, we should just produce images for everything rather than trying to second guess which we can get away with not generating.
While that's a nice sentiment, we can't ignore the disk space usage that this implies.
If we take as an example 6 supported boards, and 6 images (3x2G, 1x1G, 1x.5G, 1x.1G), that's ~48G per month, so in one year ~0.5T of images. I haven't done the maths to factor in the bandwidth here.
I don't quite understand your maths there; if you look at the Ubuntu ARM images they are 540MB for netbook and 200MB for headless (compressed). So at 6 boards to support that's ~4.2GB/month or ~50GB/year which is a lot less scary.
Dave
On Fri, 20 May 2011 17:32:27 +0100, David Gilbert david.gilbert@linaro.org wrote:
On 20 May 2011 17:12, James Westby james.westby@linaro.org wrote:
On Fri, 20 May 2011 16:03:16 +0100, Peter Maydell peter.maydell@linaro.org wrote:
We're only doing this once a month, we should just produce images for everything rather than trying to second guess which we can get away with not generating.
While that's a nice sentiment, we can't ignore the disk space usage that this implies.
If we take as an example 6 supported boards, and 6 images (3x2G, 1x1G, 1x.5G, 1x.1G), that's ~48G per month, so in one year ~0.5T of images. I haven't done the maths to factor in the bandwidth here.
I don't quite understand your maths there; if you look at the Ubuntu ARM images they are 540MB for netbook and 200MB for headless (compressed). So at 6 boards to support that's ~4.2GB/month or ~50GB/year which is a lot less scary.
We decided that we would pregenerate one image size with some free space that can be directly written to an SD card.
Thanks,
James
On Fri, 2011-05-20 at 12:49 -0400, James Westby wrote:
On Fri, 20 May 2011 17:32:27 +0100, David Gilbert david.gilbert@linaro.org wrote:
On 20 May 2011 17:12, James Westby james.westby@linaro.org wrote:
On Fri, 20 May 2011 16:03:16 +0100, Peter Maydell peter.maydell@linaro.org wrote:
We're only doing this once a month, we should just produce images for everything rather than trying to second guess which we can get away with not generating.
While that's a nice sentiment, we can't ignore the disk space usage that this implies.
If we take as an example 6 supported boards, and 6 images (3x2G, 1x1G, 1x.5G, 1x.1G), that's ~48G per month, so in one year ~0.5T of images. I haven't done the maths to factor in the bandwidth here.
I don't quite understand your maths there; if you look at the Ubuntu ARM images they are 540MB for netbook and 200MB for headless (compressed). So at 6 boards to support that's ~4.2GB/month or ~50GB/year which is a lot less scary.
We decided that we would pregenerate one image size with some free space that can be directly written to an SD card.
But when compressed the free space doesn't make any difference, does it? I've gzipped a 2GB desktop image last week and it went down to around 500MB IIRC.
On Mon, 23 May 2011 13:11:37 -0300, Guilherme Salgado guilherme.salgado@linaro.org wrote:
But when compressed the free space doesn't make any difference, does it? I've gzipped a 2GB desktop image last week and it went down to around 500MB IIRC.
That's a good point, I forgot that we are compressing them.
Therefore I propose the following:
* Generate all image types for two releases (not 11.05) * After that time take a look at the download counts and re-evaluate the tradeoffs, and whether some things are worth having at all.
Thanks,
James
On May 24, 2011, at 9:12 AM, James Westby wrote:
On Mon, 23 May 2011 13:11:37 -0300, Guilherme Salgado guilherme.salgado@linaro.org wrote:
But when compressed the free space doesn't make any difference, does it? I've gzipped a 2GB desktop image last week and it went down to around 500MB IIRC.
That's a good point, I forgot that we are compressing them.
Therefore I propose the following:
- Generate all image types for two releases (not 11.05)
So that's 5 images { nano, server, alip, LEB, developer} x 9 board types { beagle (C & Xm), IGEP, imx51, iMX53, Overo, Panda, Orion, U8500, Versatile Express} x1 file system type {ext4?} = 45 prebuilt images
Look right?
- After that time take a look at the download counts and re-evaluate the tradeoffs, and whether some things are worth having at all.
Thanks,
James
Regards, Tom
"We want great men who, when fortune frowns will not be discouraged." - Colonel Henry Knox w) tom.gall att linaro.org w) tom_gall att vnet.ibm.com h) tom_gall att mac.com
On Tue, 24 May 2011 09:22:14 -0500, Tom Gall tom_gall@vnet.ibm.com wrote:
So that's 5 images { nano, server, alip, LEB, developer} x 9 board types { beagle (C & Xm), IGEP, imx51, iMX53, Overo, Panda, Orion, U8500, Versatile Express} x1 file system type {ext4?} = 45 prebuilt images
Look right?
Looks about right. There are more hwpacks than that currently, I'm not sure what criteria we should use for deciding which to provide prebuilt images for. Alexander?
Thanks,
James
P.S. if anyone replies saying that it doesn't matter because disk space is cheap, I shall be forced to nominate you as Linaro sysadmin for this
cycle.
Ha! :) Only for this cycle?
I don't quite understand your maths there; if you look at the Ubuntu ARM images they are 540MB for netbook and 200MB for headless (compressed). So at 6 boards to support that's ~4.2GB/month or ~50GB/year which is a lot less scary.
The Ubuntu Images have an extra bit that happens on first boot where it
expands itself to consume your entire SD card.
-Paul Larson
On 20 May 2011 17:50, Paul Larson paul.larson@linaro.org wrote:
I don't quite understand your maths there; if you look at the Ubuntu ARM images they are 540MB for netbook and 200MB for headless (compressed). So at 6 boards to support that's ~4.2GB/month or ~50GB/year which is a lot less scary.
The Ubuntu Images have an extra bit that happens on first boot where it expands itself to consume your entire SD card.
Sounds a good trick; can't we just use that as well?
Dave
On 20 May 2011 18:10, David Gilbert david.gilbert@linaro.org wrote:
On 20 May 2011 17:50, Paul Larson paul.larson@linaro.org wrote:
The Ubuntu Images have an extra bit that happens on first boot where it expands itself to consume your entire SD card.
Sounds a good trick; can't we just use that as well?
We discussed that at the UDS session; although it's good for an image which is intended to be installed once and then used a lot thereafter, it doesn't work quite so well for one-off testing, since it makes the initial startup a lot slower (and the extra work can be unhelpful if you were eg interested in timing or profiling system boot).
-- PMM
On 20 May 2011 18:27, Peter Maydell peter.maydell@linaro.org wrote:
On 20 May 2011 18:10, David Gilbert david.gilbert@linaro.org wrote:
On 20 May 2011 17:50, Paul Larson paul.larson@linaro.org wrote:
The Ubuntu Images have an extra bit that happens on first boot where it expands itself to consume your entire SD card.
Sounds a good trick; can't we just use that as well?
We discussed that at the UDS session; although it's good for an image which is intended to be installed once and then used a lot thereafter, it doesn't work quite so well for one-off testing, since it makes the initial startup a lot slower (and the extra work can be unhelpful if you were eg interested in timing or profiling system boot).
OK, so that just needs to have the resize disabled by default, and a command that is run that enables it and causes it to happen at next boot.
Dave
Hi,
On 20 May 2011 15:03, Peter Maydell peter.maydell@linaro.org wrote:
We're only doing this once a month, we should just produce images for everything rather than trying to second guess which we can get away with not generating.
Instead of trying to 2nd guess and preserve disk space, what about checking the fact? As we have the apache logs of ${releases/snapshots}.linaro.org since July 2010, we can extract which images are the most popular and provide these prebuilt.
Cheers,
Fathi
To be added to that list in the yet-to-be-named rediculously small image as well as linaro-server.
Linaro-server hopefully we'll see added to the daily snapshots fairly soon as I have the seed etc already put together for it. pico or whatever it will be called is probably a bit further off.
But getting back to your question, I don't think we should produce prebuilt images for all derivations. I would pick the most popular image and couple of the most popular board types as a start. If lots of people download and there's clammering for more, than hopefully would be easy enough to adjust.
That's my 2c.
Regards, Tom
On Fri, May 20, 2011 at 9:38 AM, Guilherme Salgado guilherme.salgado@linaro.org wrote:
Hi,
During LDS we agreed to provide prebuilt Linaro images at our milestones[1], for users who can't/won't run linaro-media-create. One thing we forgot to discuss, though, is if it's worth doing so for all image types or just for the LEB(s).
Our downloads page (http://www.linaro.org/downloads/) has 4 different image types, but the email for weekly testing, for example, lists only UbuntuDesktop as official and the others as community images. More importantly, though, I'm not sure the people who just want to try Linaro quickly will be interested in anything other than UbuntuDesktop at this point, so we might as well avoid the extra work to provide images that are unlikely to be of any use.
Can anybody think of other reasons why we should provide images of all types?
[1] https://wiki.linaro.org/Platform/Infrastructure/Specs/PrebuiltImages
-- Guilherme Salgado https://launchpad.net/~salgado
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
-
David Gilbert -
Fathi Boudra -
Guilherme Salgado -
James Westby -
Paul Larson -
Peter Maydell -
Tom Gall -
Tom Gall