Hey
I'd like to propose moving our Hudson stuff to Launchpad hosted branches. This would be useful to: * share our stuff, perhaps in common repos across hudson instances (see below) * have everybody follow best practices of keeping as much as possible under a VCS * help move/factor hudson instances (e.g. factor two instances in one, move from home server or cloud to Linaro/Canonical IS datacenter)
I see two ways to approach this problem: * each team owns a hudson instance, and stores its config and scripts in the team's namespace; this doesn't encourage factoring instances together though * we create a virtual team of people caring for hudson stuff, e.g. ~linaro-hudson-hackers, and we put branches below that
We will need to keep the hudson configs private because they might hold private data like hudson's secret key or the userdb, perhaps with some private bzr branches? or can we move to openid and avoid storing the other secret keys already?
The branch with build script would of course be public.
We could have some cron committing each config (changed from the web UI) hourly, and pushing it to LP, and a cron pulling the latest version of the scripts hourly (or a special hudson job which does either).
For scripts, we'd have to agree on some conventions such as namespaces for Android build scripts (e.g. android-list-changes), hudson job definitions (e.g. job-build-u-boot, or job-build-android).
Two things which might slow us down if we want to proceed: * Hudson might get renamed due to the situation with Oracle * getting private branches * we need some kind of Launchpad accounts for the hudson instances to commit stuff
Paul, James, Michael Hudson, (Alexander?), and anybody, what are your thoughts?
Cheers,
On Thu, 27 Jan 2011 15:35:44 +0100, Loïc Minier loic.minier@linaro.org wrote:
HeyI'd like to propose moving our Hudson stuff to Launchpad hosted branches. This would be useful to:
What do you mean by stuff? Configs? Generically useful scripts? Although I'm not really sure how much of the latter there will be.
- share our stuff, perhaps in common repos across hudson instances (see below)
- have everybody follow best practices of keeping as much as possible under a VCS
- help move/factor hudson instances (e.g. factor two instances in one, move from home server or cloud to Linaro/Canonical IS datacenter)
I don't know how much having the config in bzr is going to make merging two instances easier... I guess a bit.
I see two ways to approach this problem:
- each team owns a hudson instance, and stores its config and scripts in the team's namespace; this doesn't encourage factoring instances together though
- we create a virtual team of people caring for hudson stuff, e.g. ~linaro-hudson-hackers, and we put branches below that
TBH, one team per hudson is probably about right. My instincts about how much will be reasonably shareable between instances might be wrong though.
We will need to keep the hudson configs private because they might hold private data like hudson's secret key or the userdb, perhaps with some private bzr branches? or can we move to openid and avoid storing the other secret keys already?
The hudson-ec2 plugin requires having the ec2 credentials in the config, which is another reason to keep it private.
The branch with build script would of course be public.
We could have some cron committing each config (changed from the web UI) hourly, and pushing it to LP, and a cron pulling the latest version of the scripts hourly (or a special hudson job which does either).
For scripts, we'd have to agree on some conventions such as namespaces for Android build scripts (e.g. android-list-changes), hudson job definitions (e.g. job-build-u-boot, or job-build-android).
Two things which might slow us down if we want to proceed:
- Hudson might get renamed due to the situation with Oracle
I don't think this is a good thing to block on. We should deal with it when it happens, not make guesses about what and when things might change IMHO.
- getting private branches
This is easy these days; any admin or member of ~bazaar-experts can twiddle a branch to private.
- we need some kind of Launchpad accounts for the hudson instances to commit stuff
Paul, James, Michael Hudson, (Alexander?), and anybody, what are your thoughts?
I think broadly this makes sense.
I have a related question, which is what do you have in your .bzrignore for your branches that contain hudson configs? I just have this currently:
slave-*.log ./jobs/*/builds
which avoids checking in the output of the builds. Is there anything else that should be excluded?
Cheers, mwh
On Fri, Jan 28, 2011, Michael Hudson-Doyle wrote:
What do you mean by stuff? Configs? Generically useful scripts? Although I'm not really sure how much of the latter there will be.
Both
I see storing configs more as some kind of backup / safety net in the ideal case. Most of the logic should be in separate scripts; for instance, I moved my u-boot jobs from various trees to use a common build-u-boot shell wrapper which does the right thing. Similarly I have build-qemu, build-linux, build-cross-gdb etc.
I'm also sharing some utility services; for instance I have a script which runs commands in a schroot, then destroys it.
- share our stuff, perhaps in common repos across hudson instances (see below)
- have everybody follow best practices of keeping as much as possible under a VCS
- help move/factor hudson instances (e.g. factor two instances in one, move from home server or cloud to Linaro/Canonical IS datacenter)
I don't know how much having the config in bzr is going to make merging two instances easier... I guess a bit.
The main thing I'd like to share are hudson scripts, like the chroot thing above, or a script to download the latest linaro snapshot or hwpack.
I see two ways to approach this problem:
- each team owns a hudson instance, and stores its config and scripts in the team's namespace; this doesn't encourage factoring instances together though
- we create a virtual team of people caring for hudson stuff, e.g. ~linaro-hudson-hackers, and we put branches below that
TBH, one team per hudson is probably about right. My instincts about how much will be reasonably shareable between instances might be wrong though.
I realize I'm also trying to have people who care about hudson be in the same virtual team as to have our hudson instances relatively on par, and not maintained in completely different ways. Having the configs accessible to folks maintaining other hudson instances in Linaro allows them to see how other instances are handled and copy the good ideas or suggest improvements (albeit TBH the XML format is not nice to read).
Two things which might slow us down if we want to proceed:
- Hudson might get renamed due to the situation with Oracle
I don't think this is a good thing to block on. We should deal with it when it happens, not make guesses about what and when things might change IMHO.
agreed
I have a related question, which is what do you have in your .bzrignore for your branches that contain hudson configs? I just have this currently: slave-*.log ./jobs/*/builds which avoids checking in the output of the builds. Is there anything else that should be excluded?
I'm storing /srv/hudson.dooz.org in bzr, which contains the hudson home under /home, scripts under /bin and the bzr repo for itself under /bzr. I have a slightly longer list of things in my .bzrignore (attached)
On Fri, 28 Jan 2011 11:11:03 +0100, Loïc Minier loic.minier@linaro.org wrote:
On Fri, Jan 28, 2011, Michael Hudson-Doyle wrote:
What do you mean by stuff? Configs? Generically useful scripts? Although I'm not really sure how much of the latter there will be.
Both
I see storing configs more as some kind of backup / safety net in the ideal case. Most of the logic should be in separate scripts;
Ah ok, that makes sense. I've certainly quickly settled on a pattern where the scripts you give to hudson are basically "get a script from a bzr branch, run it".
for instance, I moved my u-boot jobs from various trees to use a common build-u-boot shell wrapper which does the right thing. Similarly I have build-qemu, build-linux, build-cross-gdb etc.
Funnily enough I have something called "build-android" too, that asac started. It's at lp:~mwhudson/linaro-android/ec2-tools, along with some other bits. It's a slightly grotty shell script, but I guess some of it would be reusable, indeed.
I'm also sharing some utility services; for instance I have a script which runs commands in a schroot, then destroys it.
Do you expect these things to mainly be shell? My instincts to rewrite things in Python are reappearing I guess.
- share our stuff, perhaps in common repos across hudson instances (see below)
- have everybody follow best practices of keeping as much as possible under a VCS
- help move/factor hudson instances (e.g. factor two instances in one, move from home server or cloud to Linaro/Canonical IS datacenter)
I don't know how much having the config in bzr is going to make merging two instances easier... I guess a bit.
The main thing I'd like to share are hudson scripts, like the chroot thing above, or a script to download the latest linaro snapshot or hwpack.
Right.
I see two ways to approach this problem:
- each team owns a hudson instance, and stores its config and scripts in the team's namespace; this doesn't encourage factoring instances together though
- we create a virtual team of people caring for hudson stuff, e.g. ~linaro-hudson-hackers, and we put branches below that
TBH, one team per hudson is probably about right. My instincts about how much will be reasonably shareable between instances might be wrong though.
I realize I'm also trying to have people who care about hudson be in the same virtual team as to have our hudson instances relatively on par, and not maintained in completely different ways. Having the configs accessible to folks maintaining other hudson instances in Linaro allows them to see how other instances are handled and copy the good ideas or suggest improvements (albeit TBH the XML format is not nice to read).
So it's at least as much about sharing and being able to find ideas and patterns as sharing actual code? That makes sense to me.
Two things which might slow us down if we want to proceed:
- Hudson might get renamed due to the situation with Oracle
I don't think this is a good thing to block on. We should deal with it when it happens, not make guesses about what and when things might change IMHO.
agreed
However... it's now happening. Hopefully it won't be too disruptive!
I have a related question, which is what do you have in your .bzrignore for your branches that contain hudson configs? I just have this currently: slave-*.log ./jobs/*/builds which avoids checking in the output of the builds. Is there anything else that should be excluded?
I'm storing /srv/hudson.dooz.org in bzr, which contains the hudson home under /home, scripts under /bin and the bzr repo for itself under /bzr. I have a slightly longer list of things in my .bzrignore (attached)
-- Loïc Minier /*.war /bzr/** /ccache/** /home/.owner /home/jobs/**/builds/ /home/jobs/**/lastSuccessful /home/jobs/**/lastStable /home/jobs/**/nextBuildNumber /home/jobs/*/scm-polling.log /home/jobs/*/workspace/ /home/plugins/ /home/updates/ /home/users/ /home/war/ /home/Connection Activity monitoring to slaves.log /home/Fingerprint cleanup.log /home/Workspace clean-up.log /home/queue.xml.bak /home/secret.key
OK, that's definitely more things ignored than I have. Probably all sensible too.
The hudson .deb that they provide has the config root also be the hudson users' home dir, which also results in some cruft appearing in the directory (.bzr.log etc). I should look in to how to change that...
Cheers, mwh
On Mon, Jan 31, 2011 at 12:56 AM, Michael Hudson-Doyle michael.hudson@canonical.com wrote:
On Fri, 28 Jan 2011 11:11:03 +0100, Loïc Minier loic.minier@linaro.org wrote:
On Fri, Jan 28, 2011, Michael Hudson-Doyle wrote:
What do you mean by stuff? Configs? Generically useful scripts? Although I'm not really sure how much of the latter there will be.
Both
I see storing configs more as some kind of backup / safety net in the ideal case. Most of the logic should be in separate scripts;
Ah ok, that makes sense. I've certainly quickly settled on a pattern where the scripts you give to hudson are basically "get a script from a bzr branch, run it".
yes, thats my take as well. And the actual script with the main build logic should be independent from hudson if possible.
for instance, I moved my u-boot jobs from various trees to use a common build-u-boot shell wrapper which does the right thing. Similarly I have build-qemu, build-linux, build-cross-gdb etc.
Funnily enough I have something called "build-android" too, that asac started. It's at lp:~mwhudson/linaro-android/ec2-tools, along with some other bits. It's a slightly grotty shell script, but I guess some of it would be reusable, indeed.
I would like to see the "hudson" build submission interface generic. e.g. make just one hudson job that delegates execution to a concrete backend like "build-android", "build-android-toolchain", build-chromeos, etc.
such hudson job would then just takes _one_ "xml" config as input parameter that is then used to select the build script/backend ...
e.g.
<linaro-cloud-buildd> <android>. ... </android> </linaro-cloud-buildd>
=> goes to build-android backend
and similarly <android-toolchain> would go to build-android-toolchain backend etc.
-
Alexander Sack -
Loïc Minier -
Michael Hudson-Doyle