Hey!
10 days ago, I met with John and Nicolas and with the help of Steve we discussed how an "upstreamish" release process for u-boot-linaro and linux-linaro would look like.
I'm sorry for sending the notes out to late, please find them below.
Thanks
== Attendees ==
Nicolas John Steve Loic
== Minutes ==
* Releases every month * Release announcement on linaro-announce@ * Would we release both -next and -stable? even if no change was done? * Probably an upstream stable release every week or so
* Between releases, merge upstream stable updates as they show up; before release, verify that we have the latest stable release
* John wonders whether we need the Ubuntu changes? * AppArmor went upstream * aufs isn't used in Linaro * but we might want to be the upstream for the Ubuntu ARM team, so it makes sense to have the Ubuntu changes in our Ubuntu packages
* We don't really need to test the Ubuntu changes in our dailies; simpler to just build daily zImages
* We should include Ubuntu patches in ubuntu, but not in daily builds
* What's the kernel PPA for? Is it for staging and testing, or an overlay? * The .debs are kind of specific to Ubuntu and distros in general, and we already integrate with Ubuntu
* Should we produce .debs in a PPA or something before doing a kernel release, or just before upload to Ubuntu?
* Kernel team is producing vanilla .debs, would be possible to do the same?
* How would we test a new kernel, .deb is simpler than modules.tgz, zImage with hardcoded modules is easier to test though * Nicolas would prefer testing zImage; easier this way * Configs would differ though, and might regress either way * Linaro KCWG and Linaro .debs would use different configs
* We're doing releases for users, and they mostly expect .debs, but also .rpms
* Conclusion: * Daily zImage with everything compiled in, no need to worry about modules * Daily .deb builds that have modules split out, but no Ubuntu patches * Monthly release of the above * Monthly integration of that tree into Linaro Platform if possible * linux-linaro-next + linaro-linaro-stable, same thing
Can add stuff progressively, starting without zImage
Align a monthly release with the 6-monthly release, e.g. kernel release at the same time as platform release for 10.11
* u-boot: same thing, except we don't have Ubuntu configs and we don't have the modules problem; could even do a single daily built
* Nicolas would ensure that we have builds for linux trees, and John for u-boot, and each of them would post to the tracker * Then send Call for testing to people with the hardware, report bugs, and if we have everything tested, release it
* How do we test? * u-boot? * just loading a kernel probably * perhaps recovery procedure: make sure we have a way to load a kernel from an external source * loading a kernel from SD (for platforms which have it) * loading a kernel from the network * could test network * linux? * boots * display output * network * SD R/W * USB: storage at least * NAND: shows up in /proc/mtd if supported * user platforms should push for other test cases like audio if needed on some boards * what rootfs to test on? * for now use our minimal image, or the installed system * future: a small initramfs with static busybox ACTION slangasek to talk to ppearse and tgall to see how we could do that
* John: The current configs are way too large, we need to have something smaller for Linaro * Needs at least to support everything on the board * Need some template for the common kernel config parts ACTION jcrigby to discuss with Frederik ideas about default/common configs ACTION nico to create a minimal config using the kernel system ACTION lool to send Ubuntu kernel checker over email
ACTION lool document release process on the wiki
Hey
Per the topic quoted below (kernel + u-boot release process), I've setup the awesome hudson CI server on my personal system until we can move it to the datacenter; it's accessible at http://hudson.dooz.org/
It currently builds various branches (repositories really) of u-boot with multiple config: ca9x4_ct_vxp, omap3_beagle, omap4_panda, mx51evk. (ca9x4_ct_vxp isn't upstream yet)
On Mon, Sep 06, 2010, Loïc Minier wrote:
Daily zImage with everything compiled in, no need to worry about modules
Daily .deb builds that have modules split out, but no Ubuntu patches
Monthly release of the above
Monthly integration of that tree into Linaro Platform if possible
linux-linaro-next + linaro-linaro-stable, same thing
u-boot: same thing, except we don't have Ubuntu configs and we don't have the modules problem; could even do a single daily built
Next step are daily kernel builds, but I need some minimal configs for that.
On Tue, Sep 7, 2010 at 2:41 PM, Loïc Minier loic.minier@linaro.org wrote:
HeyPer the topic quoted below (kernel + u-boot release process), I've setup the awesome hudson CI server on my personal system until we can move it to the datacenter; it's accessible at http://hudson.dooz.org/
This looks great! Just out of curiosity, did you look at anything else such as buildbot maybe? I'm curious what reasons you had for choosing hudson over other alternatives.
Thanks, Paul Larson
Show and tell time. Here's mine: http://ex.seabright.co.nz/helpers/buildlog
It's all Makefile based in the background. I've used buildbot before and been very happy with it but it was too hard to integrate with bzr.
-- Michael
On Wed, Sep 8, 2010 at 9:03 AM, Paul Larson paul.larson@linaro.org wrote:
On Tue, Sep 7, 2010 at 2:41 PM, Loïc Minier loic.minier@linaro.org wrote:
Hey
Per the topic quoted below (kernel + u-boot release process), I've setup the awesome hudson CI server on my personal system until we can move it to the datacenter; it's accessible at http://hudson.dooz.org/
This looks great! Just out of curiosity, did you look at anything else such as buildbot maybe? I'm curious what reasons you had for choosing hudson over other alternatives.
Thanks, Paul Larson
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On Tue, Sep 07, 2010, Paul Larson wrote:
This looks great! Just out of curiosity, did you look at anything else such as buildbot maybe? I'm curious what reasons you had for choosing hudson over other alternatives.
I did! Thanks for asking! :-)
In fact, before I started looking for some daily build infrastructure, I had heard two strong recommendations of hudson, but I still went the buildbot route because as nice as it sounded, Hudson is Java and I thought it would be pain to extend and integrate.
So I went with Buildbot, and sure enough it's Python. After some 4 hours, I had it decently setup to pull from a git repo and build one u-boot branch for one flavour. I had to tweak a couple of minor things to do it, nothing major, and I mostly time learning the slightly special semantics / workflow of buildbot. I also realized that I'd have to extend it to do some of things I needed, like publication of the results.
Then I read another strong recommendation for hudson in a thread on a Canonical mailing-list; the case study was rich and complex, so a significant amount of time had been put into it. I decided to try Hudson out, and I was immediately very successful at doing the same things as with buildbot and more. Once you've played with Hudson, Buildbot feels very basic. Hudson has many plugins and extension points, while Buildbot is more like a good structure for a custom infrastructure. Out of the box, Hudson comes with many plugins and a feature-rich web UI which gets you to the meat of your own problems faster, but the underlying structure is as rich.
Now that I've set it up, I certainly know it was the best choice, but I can also see how it's going to be hard to extend this setup for other tasks. For instance, if I wanted to have a different web UI to serve the results, or if I wanted to setup git watches on multiple trees to trigger a build. It's all doable, but it ends up being as hard as with buildbot if you go significantly away of what it's designed to do.
I suspect the large Buildbot instances out there are all pretty much forks of the upstream Buildbot which are unlikely to move to a newer buildbot; they used buildbot as a good, simple, transparent mini-structure for integration builds and extended it, while Hudson encourages the plugin mechanism and continuous upgrades of your Hudson deployment just a bit more, so that it built a larger base of features out of the box.
I have complete notes of the step I took and I even have my buildbot instance on my laptop (stopped); I'm happy to share these with you if you'd like to read more or would like to reproduce.
On 10 Sep 08, Loïc Minier wrote:
On Tue, Sep 07, 2010, Paul Larson wrote:
This looks great! Just out of curiosity, did you look at anything else such as buildbot maybe? I'm curious what reasons you had for choosing hudson over other alternatives.
[snip]
I have complete notes of the step I took and I even have my buildbot instance on my laptop (stopped); I'm happy to share these with you if you'd like to read more or would like to reproduce.
I'd be interested in config files, etc. for hudson. I'd like to set it up to compile kernels that I'm maintaining.
Also, when are you packaging it into Debian/Ubuntu? ;)
/Amit
On Wed, Sep 08, 2010, Amit Kucheria wrote:
I'd be interested in config files, etc. for hudson. I'd like to set it up to compile kernels that I'm maintaining.
So my main driver for Hudson right now are Linaro daily u-boot and kernel builds; we could work on setting up some useful Linaro builds for you, I just didn't find a way to propagate a separate repo of configs elegantly so that I can combine a kernel tree + a config to start a build. I need to be reasonnable as long as my main home server is doing the builds, but if I take the time to hook Hudson to EC2, I shall have infinite build power.
Also, when are you packaging it into Debian/Ubuntu? ;)
Hehe, I know when to keep away from packaging something ;-)
It's a combination of factors, but to sum up I'm unlikely to package it into Debian/Ubuntu: - it's moving fast, they release about one time per week - it's java, which is a pain to package properly - I know it's going to suck time to get the updates working properly or disabled
Overall, I'm moving ownership of this service to IS and the general Hudson topic should go to Infrastructure.
If you're desperate, there are some .debs available upstream; the APT repo is broken though (lags way behind). There's an APT repo with experimental packages too (I understand they are from their experimental branch).
-
Amit Kucheria -
Loïc Minier -
Michael Hope -
Paul Larson