Hey
As a followup to IRC conversations around backports, releases and QA today, I'd like to hear what others think of our Linaro PPAs. I'll start with some history and proposals:
We created a fairly ad hoc PPA layout for the 10.11 cycle, with ppa:linaro-maintainers/tools ppa:linaro-maintainers/override ppa:linaro-maintainers/kernel
and nowadays we also have this PPA for toolchain backports: ppa:linaro-maintainers/toolchain
and misc other PPAs which I don't really know much about: ppa:linaro-maintainers/user-platforms ppa:linaro-toolchain-dev/ppa ppa:linaro-infrastructure/launch-control ppa:linaro-infrastructure/launch-control-snapshots (and probably many more)
There are multiple problems with our current approach: * ~linaro-maintainers membership is poorly defined * it's not clear which software should go in which PPA * it's not clear when we can update which PPAs, e.g. can we update them after 6-monthly releases? how do we QA/validate updates?
In general, it's good to avoid PPA proliferation, both for sanity and for the confort of our end users, but I think a consistent set of PPAs is more important than trying to optimize the number of PPAs to the smallest subset possible.
Some ideas on addressing this:
* have software ownership split by team; ~toolchain owns gcc-linaro and qemu-linaro, ~infrastructure owns * have each team decide between either: - a single PPA for all their outputs (e.g. ~infrastructure/ppa for linaro-image-tools and gcc-log-analyzer) - or multiple PPAs, one per software (e.g. ~toolchain/gcc-linaro PPA for gcc-linaro, ~toolchain/qemu-linaro PPA for qemu-linaro) * optionally, additional PPAs for dailies * optionally, additional PPAs for RC releases
[ this is inspired from the set of PPAs for bzr: http://wiki.bazaar.canonical.com/DistroDownloads#Ubuntu%20and%20derivatives ]
In addition, we'd have a single overlay PPA which would be used for rootfs builds. We could keep the existing one below ~linaro-maintainers.
Open questions: - list of ~linaro-teams - do we upload latest release of e.g. gcc-linaro to the natty toolchain PPA if it already gets uploaded to natty proper?
Thanks,
On 20 January 2011 18:30, Loïc Minier loic.minier@linaro.org wrote:
Hey
As a followup to IRC conversations around backports, releases and QA today, I'd like to hear what others think of our Linaro PPAs. I'll start with some history and proposals:
To my mind the important constraint is that there should be some relatively easy thing that we can say to someone 'and this is how you get a stable Linaro'; and by that I mean the whole thing - a set of tools that build and run everything else including a kernel and preferably debug it.
By 'easy' I mean relatively simple, but critically pretty difficult to do wrong or miss a step.
There should be a 'stable' version of the entire set - i.e. we're not asking someone else to pick a particular kernel/tools/etc - so that it's easy for people to know what they should get and it 'should just work'. And it should stay working up until at least the next release.
Dave
On Fri, Jan 21, 2011 at 7:44 AM, David Gilbert david.gilbert@linaro.org wrote:
On 20 January 2011 18:30, Loïc Minier loic.minier@linaro.org wrote:
Hey
As a followup to IRC conversations around backports, releases and QA today, I'd like to hear what others think of our Linaro PPAs. I'll start with some history and proposals:
To my mind the important constraint is that there should be some relatively easy thing that we can say to someone 'and this is how you get a stable Linaro'; and by that I mean the whole thing - a set of tools that build and run everything else including a kernel and preferably debug it.
By 'easy' I mean relatively simple, but critically pretty difficult to do wrong or miss a step.
There should be a 'stable' version of the entire set - i.e. we're not asking someone else to pick a particular kernel/tools/etc - so that it's easy for people to know what they should get and it 'should just work'. And it should stay working up until at least the next release.
Agreed. I'd like an easy way of getting pre-built binaries of all the stable enough Linaro outputs. On the toolchain side this would include the latest monthly releases of Linaro GCC, GDB, and QEMU in native and cross versions as appropriate. A single PPA for the whole of Linaro would be nice.
-- Michael
On Fri, Jan 21, 2011 at 10:10:08AM +1300, Michael Hope wrote:
To my mind the important constraint is that there should be some relatively easy thing that we can say to someone 'and this is how you get a stable Linaro'; and by that I mean the whole thing - a set of tools that build and run everything else including a kernel and preferably debug it.
By 'easy' I mean relatively simple, but critically pretty difficult to do wrong or miss a step.
There should be a 'stable' version of the entire set - i.e. we're not asking someone else to pick a particular kernel/tools/etc - so that it's easy for people to know what they should get and it 'should just work'. And it should stay working up until at least the next release.
Agreed. I'd like an easy way of getting pre-built binaries of all the stable enough Linaro outputs. On the toolchain side this would include the latest monthly releases of Linaro GCC, GDB, and QEMU in native and cross versions as appropriate. A single PPA for the whole of Linaro would be nice.
If each WG publishes packages in their own PPA, it's simple enough to have these packages (source+binaries) copied on a regular basis to a common "mix master" PPA. As long as there's no confusion between this and the overlay PPA, that should be easy. (After all, as a shining example, the toolchain package we want shipped with the 6-month release is the one integrated in Ubuntu and used to build all the binaries... not the latest monthly gcc-linaro release.)
On Thu, Jan 20, 2011 at 5:35 PM, Steve Langasek steve.langasek@linaro.org wrote:
On Fri, Jan 21, 2011 at 10:10:08AM +1300, Michael Hope wrote:
Agreed. I'd like an easy way of getting pre-built binaries of all the stable enough Linaro outputs. On the toolchain side this would include the latest monthly releases of Linaro GCC, GDB, and QEMU in native and cross versions as appropriate. A single PPA for the whole of Linaro would be nice.
I very much like the idea of the linaro-maintainers overlay PPA being the central place for released (and supported) packages that will be included in the daily image builds.
If each WG publishes packages in their own PPA, it's simple enough to have these packages (source+binaries) copied on a regular basis to a common "mix master" PPA. As long as there's no confusion between this and the overlay PPA, that should be easy. (After all, as a shining example, the toolchain package we want shipped with the 6-month release is the one integrated in Ubuntu and used to build all the binaries... not the latest monthly gcc-linaro release.)
Further I also agree that on a team by team basis if they want their own PPA that their own team can define what and how packages are uploaded for easier access and use for WIP or candidate items for linaro-maintainers/overlay. This class of PPAs should never be used for the daily build. This makes great sense to me.
I equally agree that we should avoid a proliferation of PPAs as this will just cloud matters which makes it confusing for linaro devs as well as those that might use our software.
The meego/kwin/chrome stuff might potentially be an oddball exception in so much that the collective set of package is just that, a set of packages. Mixing collections of packages such as these with packages that are not part of the set could be confusing.
Regards, Tom
-- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer http://www.debian.org/ slangasek@ubuntu.com vorlon@debian.org
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux)
iQIVAwUBTTjGpFaNMPMhshM9AQjL5w//V+PfWWFZ+enuMXYM1U6UKJF+r3H1eT91 V4bUJrd34GQacRGGCfetcXvOrlRIQwhT+9qINMFz+OBRQlG+NJ4fMZM5bbNeLwmL qV+HGQ6E3SNvrkEpdZKcOGUikilCHzDbzWWtebrTb0YjxWMeMPHBGtfrETn9C7GE QFyUI12TNneSei/oeqvfDJf8fq46pkcOSLIuRz9FAyY6ltFpSVqebPbreWA6+cni Feu/ncBkE4WVyykSMU20mWtYGfK00XPiUuseH9AurykQRUznzNmDuwMv/iLY+88z tjHxmKyKap0VfjRaMHZXtrY4qUHOs7XtK2UaC5zO0OC9hHsNJEUU9BIwafRQjcMI xqdcJ0CyCWETiXkAZ4tQ3YTxyCDky/bVvubTVOXhEmB/mb7d8S8fxqnTuSf/qPX6 kE7bDdkpiMA0VlCmci+QVNVq6Kw1L30zMjukcyyHXsK1URNUQkXkmzj3NbfelglJ Vgofu0ybq8hCE5oavik8Sk6xz9oziHss0iG2JrGMQNHwL2WxQ1aPOISk7ve6BOFT zIh7HIfRR/1inzaNWcE+KFet1lXqRA6tyNGWyWITyXR4AuiSQ9d3nCP5qhdh4dHM lmdmUBLt7t2Wj9pigMSBuf8PIe0CNjXRN02uWZOTOqGxyLnnUtserpoA8wG7thb1 XKiNohhkEJQ= =Nek2 -----END PGP SIGNATURE-----
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
Agreed. I'd like an easy way of getting pre-built binaries of all the stable enough Linaro outputs. On the toolchain side this would include the latest monthly releases of Linaro GCC, GDB, and QEMU in native and cross versions as appropriate. A single PPA for the whole of Linaro would be nice.
I'm curious to hear what packaging methods would be used for things like the toolchain which have consumers beyond ubuntu. Are we saying this will only be Ubuntu packages or is there consensus around providing static tarballs ?
I ask this because I've seen a number of requests about getting hold of a binary Linaro toolchain release and not everyone uses Ubuntu as their host distribution.
Ramana
-- Michael
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On 21 January 2011 13:22, Ramana Radhakrishnan ramana.radhakrishnan@arm.com wrote:
I'm curious to hear what packaging methods would be used for things like the toolchain which have consumers beyond ubuntu. Are we saying this will only be Ubuntu packages or is there consensus around providing static tarballs ?
I ask this because I've seen a number of requests about getting hold of a binary Linaro toolchain release and not everyone uses Ubuntu as their host distribution.
My intention for the qemu-linaro releases is that the tarball will be the primary released object, with ubuntu packages provided only for convenience...
-- PMM
On Fri, Jan 21, 2011, Ramana Radhakrishnan wrote:
I'm curious to hear what packaging methods would be used for things like the toolchain which have consumers beyond ubuntu. Are we saying this will only be Ubuntu packages or is there consensus around providing static tarballs ?
For outputs of WGs like gcc-linaro, gdb-linaro, qemu-linaro, linux-linaro, u-boot-linaro, ... we expect the official release to be a source tarball which can be consumed/integrated anywhere, not limited to Ubuntu. But as our primary developer platform is Ubuntu, we also package these outputs in the current Ubuntu release and we should also provide PPA versions for: * people running older / stable versions of Ubuntu * times when the Ubuntu development repo is frozen (e.g. before an Ubuntu release)
I ask this because I've seen a number of requests about getting hold of a binary Linaro toolchain release and not everyone uses Ubuntu as their host distribution.
You mean a CS-style .tar.gz with a standalone toolchain? This came up frequenly; it's a question of whether we want to spend the extra effort of supporting this output, and whether we'll be able to manage / we'll be interested in the support of the users of such a release. I'm sure Michael Hope can expand on this, but I think the main argument is that we are interested in supporting packagers integrating our toolchain in their distros, but less interested in direct support of hundreds of people consuming the toolchain directly -- we'd rather crowdsource this to distros.
On Thu, Jan 20, 2011 at 07:30:35PM +0100, Loïc Minier wrote:
As a followup to IRC conversations around backports, releases and QA today, I'd like to hear what others think of our Linaro PPAs. I'll start with some history and proposals:
We created a fairly ad hoc PPA layout for the 10.11 cycle, with ppa:linaro-maintainers/tools ppa:linaro-maintainers/override ppa:linaro-maintainers/kernel
and nowadays we also have this PPA for toolchain backports: ppa:linaro-maintainers/toolchain
and misc other PPAs which I don't really know much about: ppa:linaro-maintainers/user-platforms ppa:linaro-toolchain-dev/ppa ppa:linaro-infrastructure/launch-control ppa:linaro-infrastructure/launch-control-snapshots (and probably many more)
There are multiple problems with our current approach:
- ~linaro-maintainers membership is poorly defined
- it's not clear which software should go in which PPA
- it's not clear when we can update which PPAs, e.g. can we update them after 6-monthly releases? how do we QA/validate updates?
Alexander and I discussed the linaro-maintainers team status last week at the sprint; as I mentioned to Loïc on IRC, the basic plan going forward is this:
- blueprint handling will be split off of linaro-maintainers to a separate linaro-drivers team, leaving -maintainers for ppas and bzr branches
- many of the existing members of linaro-maintainers will have their membership deactivated
- membership in -maintainers will be managed by the current admins accordingly; only people who need to upload to these ppas or commit directly to the branches will be in this team, and the admins will determine based on package sponsorship history when a prospective uploader is ready for direct commit/upload access
- the overlay ppa is the only one that should be enabled for Linaro release images, and anyone who needs additional packages not available there included in the images should work with the DevPlatform team to get them into either the Ubuntu archive or this ppa
I do not want us to be managing per-image PPAs for any 6-month release images, nor do I want us to be pulling multiple PPAs into the sources for our images. It is the responsibility of the DevPlatform team to assemble the output of the working groups into a cohesive, integrated whole that showcases the Linaro development, and this necessarily implies a certain amount of selectivity in terms of which versions of component releases we take in from the working groups and when. The "component release" plan put forward by Kiko involves a relatively lightweight QA process on the monthly releases and I don't want to impose further overhead on these monthly releases per se, which means instead that integrating component releases into the Linaro eval images / 6 month releases (by way of either the Ubuntu archive or the overlay ppa) involves a manual pull. This is something I intend to discuss with the WG TLs over the next week to ensure we're aligned with them in terms of what outputs we should be integrating when.
As for the 'tools' ppa, I had envisioned this as the single ppa that developers should enable on their desktop systems to get tools they need to work with, and develop for, current Linaro development images, particularly when those desktops are running older versions of Ubuntu. So backports of the current version of qemu, for instance, that are able to boot the latest images, and linaro-image-tools that work with the new 11.05 hwpack format and can pass the right boot args for newer kernels, are the kind of thing I would like to see here. I don't intend us to guarantee that the packages won't cause regressions vs. those versions shipped in that Ubuntu release - indeed, we know for certain that the new linaro-image-tools won't work with images released with 10.11 - but they will be the best tools we can give you for working with Linaro images while minimizing regressions where possible.
The 'kernel' ppa is something of a mixed bag. There are a variety of test kernels, bsp kernels and pre-release kernels available there, including those we used for our bsp hwpacks in 10.11. It lacks a clear policy, and I'm pretty sure we don't want Landing Teams to be bottlenecked on this single PPA for their ability to build kernel packages (and hwpacks). But I wouldn't do away with this altogether; it's clear to me that John has been getting good use out of it.
In general, it's good to avoid PPA proliferation, both for sanity and for the confort of our end users, but I think a consistent set of PPAs is more important than trying to optimize the number of PPAs to the smallest subset possible.
Some ideas on addressing this:
- have software ownership split by team; ~toolchain owns gcc-linaro and qemu-linaro, ~infrastructure owns
- have each team decide between either:
- a single PPA for all their outputs (e.g. ~infrastructure/ppa for linaro-image-tools and gcc-log-analyzer)
- or multiple PPAs, one per software (e.g. ~toolchain/gcc-linaro PPA for gcc-linaro, ~toolchain/qemu-linaro PPA for qemu-linaro)
- optionally, additional PPAs for dailies
- optionally, additional PPAs for RC releases
I think it's a good idea to have per-WG PPAs where the working groups can upload packages of their monthly component releases, of dailies, and any other WG output they want packages of - however they happen to be split up.
Open questions:
- do we upload latest release of e.g. gcc-linaro to the natty toolchain PPA if it already gets uploaded to natty proper?
Provided that the PPA uploads use proper version numbers (i.e., don't shadow any version number that would be used in the Ubuntu or Debian archives), I think this is reasonable. We can't generally guarantee any particular time frame for inclusion in natty after the WG component is released, so I think it's simpler to have a policy of "always PPA upload".
Thanks,
On Thu, Jan 20, 2011, Steve Langasek wrote:
- the overlay ppa is the only one that should be enabled for Linaro release images
[...]
I like the clear limits you're setting to the overlay PPA (rejecting any other concurrent overlays)
As for the 'tools' ppa, I had envisioned this as the single ppa that developers should enable on their desktop systems to get tools they need to work with, and develop for, current Linaro development images, particularly when those desktops are running older versions of Ubuntu
That's relatively close to the reason we created it for during the 10.05 cycle: we basically had to allow people to create/use Linaro images from a lucid install, and not everything was in lucid.
So I'm also happy with this, keeping in mind that it's on a best effort basis, in particular:
I don't intend us to guarantee that the packageswon't cause regressions vs. those versions shipped in that Ubuntu release - indeed, we know for certain that the new linaro-image-tools won't work with images released with 10.11 - but they will be the best tools we can give you for working with Linaro images while minimizing regressions where possible.
and that's in part what triggered this thread, because I wanted to eventually provide lucid/maverick folks with updated l-i-t, but I was suddenly challenged to QA these extensively. There should only be a decently lightweight QA for the tools PPA.
The 'kernel' ppa is something of a mixed bag. There are a variety of test kernels, bsp kernels and pre-release kernels available there, including those we used for our bsp hwpacks in 10.11. It lacks a clear policy, and I'm pretty sure we don't want Landing Teams to be bottlenecked on this single PPA for their ability to build kernel packages (and hwpacks). But I wouldn't do away with this altogether; it's clear to me that John has been getting good use out of it.
To me it sounds like this should be a ~linaro-kernel-wg/release PPA, with eventually some /beta or /daily PPA; or perhaps his own ~jcrigby PPA
Provided that the PPA uploads use proper version numbers (i.e., don't shadow any version number that would be used in the Ubuntu or Debian archives), I think this is reasonable. We can't generally guarantee any particular time frame for inclusion in natty after the WG component is released, so I think it's simpler to have a policy of "always PPA upload".
Ok; this is how I felt as well, simply because of freezes
Thanks!
-
David Gilbert -
Loïc Minier -
Michael Hope -
Peter Maydell -
Ramana Radhakrishnan -
Steve Langasek -
Tom Gall