Hello,
Some time passed since last update on Gerrit deployment, that's because work on complete AOSP mirroring to out tree took longer than expected. All in all, following was done:
Revamped branching in our toolchain/* components, freed room for upstream branches, mirrored them.
Mirrored AOSP kernel components. That was something I was putting off until latest, knowing that it would bring enough burden, like increasing storage space, increasing sync time, etc. Until last I wasn't sure if they should mirrored, but something which turned scale is recent talk about possibility to provide image for consumer phones from Google (for which we may want to hack kernels as provided by AOSP). Other point was just having complete AOSP mirror, and writing that question off forever, freeing space for other work. So, I proceeded with doing it, which soon led to OutOfMemory in Gerrit, so it's probably good that it got uncovered during deployment, than later. Thanks to IS, memory and Gerrit size were increased, and kernel imports finished successfully.
That means that we have complete mirror of upstream AOSP tree, and out tree is a proper superset of AOSP. The only workitem left is setting up automated upstream syncing via cron (so far I've been doing this manually), and we have nice tree set up with upstream at our fingertips (without having availability issues during builds, etc.), and at the same time, have all freedom to do any stuff on top of it (branching, tagging, etc.)
I also updated Linaro Gerrit howto: https://wiki.linaro.org/Platform/Android/Gerrit , which now should have all info to have one started quickly with Gerrit, and cover all aspects of Gerrit setup (like upstream mirroring and permission settings). I'd appreciate review of that and letting me know if something is missing there.
Finally few points we can continue with to elaborate our usage of Gerrit:
1. Set up branch policy (naming, etc.) and enforce it on Gerrit level. This may require revamping branching in other toolchain/* components (upstreamed not from AOSP), but in the end we'll get really robust development setup.
2. Turn off review bypass option which was available during transition process. I guess Android team if comfortable by now with new process (there're more than 80 patches passed thru review by now!), so once 11.08 release is out, it would be good time to do that.
On 23 August 2011 07:46, Paul Sokolovsky paul.sokolovsky@linaro.org wrote:
Hello,
Some time passed since last update on Gerrit deployment, that's because work on complete AOSP mirroring to out tree took longer than expected. All in all, following was done:
Revamped branching in our toolchain/* components, freed room for upstream branches, mirrored them.
Mirrored AOSP kernel components. That was something I was putting off until latest, knowing that it would bring enough burden, like increasing storage space, increasing sync time, etc. Until last I wasn't sure if they should mirrored, but something which turned scale is recent talk about possibility to provide image for consumer phones from Google (for which we may want to hack kernels as provided by AOSP). Other point was just having complete AOSP mirror, and writing that question off forever, freeing space for other work. So, I proceeded with doing it, which soon led to OutOfMemory in Gerrit, so it's probably good that it got uncovered during deployment, than later. Thanks to IS, memory and Gerrit size were increased, and kernel imports finished successfully.
That means that we have complete mirror of upstream AOSP tree, and out tree is a proper superset of AOSP. The only workitem left is setting up automated upstream syncing via cron (so far I've been doing this manually), and we have nice tree set up with upstream at our fingertips (without having availability issues during builds, etc.), and at the same time, have all freedom to do any stuff on top of it (branching, tagging, etc.)
I also updated Linaro Gerrit howto: https://wiki.linaro.org/Platform/Android/Gerrit , which now should have all info to have one started quickly with Gerrit, and cover all aspects of Gerrit setup (like upstream mirroring and permission settings). I'd appreciate review of that and letting me know if something is missing there.
This looks really good. Please share this with the repo mailing list. Your great work has really helped other people trying to set up their own Gerrit instances.
Finally few points we can continue with to elaborate our usage of Gerrit:
- Set up branch policy (naming, etc.) and enforce it on Gerrit level.
This may require revamping branching in other toolchain/* components (upstreamed not from AOSP), but in the end we'll get really robust development setup.
Agreed. One big thing that we need to work on is how we're going to handle kernel upgrades - I think some special branch-naming may be part of this solution.
- Turn off review bypass option which was available during transition
process. I guess Android team if comfortable by now with new process (there're more than 80 patches passed thru review by now!), so once 11.08 release is out, it would be good time to do that.
+1 for most. I think the only thing we have to watch out for is kernel maintainers needing to push big updates out-of-band.
-- Best Regards, Paul
Linaro.org | Open source software for ARM SoCs Follow Linaro: http://www.facebook.com/pages/Linaro http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
On 08/26/2011 12:12 AM, Somebody in the thread at some point said:
- Set up branch policy (naming, etc.) and enforce it on Gerrit level.
This may require revamping branching in other toolchain/* components (upstreamed not from AOSP), but in the end we'll get really robust development setup.
Agreed. One big thing that we need to work on is how we're going to handle kernel upgrades - I think some special branch-naming may be part of this solution.
- Turn off review bypass option which was available during transition
process. I guess Android team if comfortable by now with new process (there're more than 80 patches passed thru review by now!), so once 11.08 release is out, it would be good time to do that.
+1 for most. I think the only thing we have to watch out for is kernel maintainers needing to push big updates out-of-band.
Special case of that is going to be rebase branches as opposed to history ones, where 'disruptive' changes are going to be normal.
How well what are basically externally generated and managed kernels will fit into Gerrit flow is something I don't really understand. But I don't think it will be as simple as the case where the kernel is a history tree managed entirely through gerrit flow and everyone is happy.
-Andy
On 26 August 2011 00:04, Andy Green andy.green@linaro.org wrote:
On 08/26/2011 12:12 AM, Somebody in the thread at some point said:
- Set up branch policy (naming, etc.) and enforce it on Gerrit level.
This may require revamping branching in other toolchain/* components (upstreamed not from AOSP), but in the end we'll get really robust development setup.
Agreed. One big thing that we need to work on is how we're going to handle kernel upgrades - I think some special branch-naming may be part of this solution.
- Turn off review bypass option which was available during transition
process. I guess Android team if comfortable by now with new process (there're more than 80 patches passed thru review by now!), so once 11.08 release is out, it would be good time to do that.
+1 for most. I think the only thing we have to watch out for is kernel maintainers needing to push big updates out-of-band.
Special case of that is going to be rebase branches as opposed to history ones, where 'disruptive' changes are going to be normal.
How well what are basically externally generated and managed kernels will fit into Gerrit flow is something I don't really understand. But I don't think it will be as simple as the case where the kernel is a history tree managed entirely through gerrit flow and everyone is happy.
I think it can be simple as long as we lay it out. The general plan would be:
We track a kernel tree branch that's considered tip. We pull from that via repo and do builds etc.
That tree is manged by Gerrit with a maintainer backdoor.
A Gerrit change makes it into the tree after its been reviewed, approved and passes regression.
The maintainer approves the change.
The maintainer has the ability to reorder, rebase or do what ever they want outside of Gerrit. Their only extra duty is to ensure that patches that have been accepted are now in the tree. Anything that hasn't been accepted would just follow the normal flow since each change is merged against the current state of the tree.
My main issue with stgit is that its only useful for a single person to manage a patchset on top of a history tree. When many people then rely on that tree, something like stgit makes it hard for them to deal since the history is always changing. Now the really cool thing about Gerrit is that it doesn't care if the tree has been rebased. It will just try and merge the changeset on top of the current tree, whatever the history so in that case Gerrit is actually enabling developers to work with trees that are continually rebased (like the linux kernel).
-Andy
-- Andy Green | TI Landing Team Leader Linaro.org │ Open source software for ARM SoCs | Follow Linaro http://facebook.com/pages/Linaro/155974581091106 - http://twitter.com/#%21/linaroorg - http://linaro.org/linaro-blog
On 08/27/2011 02:15 AM, Somebody in the thread at some point said:
How well what are basically externally generated and managed kernels will fit into Gerrit flow is something I don't really understand. But I don't think it will be as simple as the case where the kernel is a history tree managed entirely through gerrit flow and everyone is happy.
My main issue with stgit is that its only useful for a single person
Stgit is a nice adjunct to git that lets you juggle and reorder patchsets easily; gerrit seems to be some kind of slashcode 0.01 one could knock up in a few days using LAMP where patches are the article and get voted on by folks. They are unrelated.
to manage a patchset on top of a history tree. When many people then rely on that tree, something like stgit makes it hard for them to deal since the history is always changing. Now the really cool thing about
That's nothing to do with stgit but "rebase flow". You can very well do rebases generally with pure git.
Android tree is just one of the things we're looking after, it is very convenient for us to have a rebase flow that let's us focuses on one intermediate vanilla enabled tree, so we do work once for that common basis, that is then rebased against the various bases that people want including Android. I use stgit-based scripts internally to manage that but none of our output exposes that, it's all pure normal git.
Gerrit is that it doesn't care if the tree has been rebased. It will just try and merge the changeset on top of the current tree, whatever the history so in that case Gerrit is actually enabling developers to work with trees that are continually rebased (like the linux kernel).
Linus' tree is a history tree, fwiw.
For kernel, gerrit users even generating the patches to propose requires git. So gerrit is just a sort of secondary 'social' option on top of git, which remains mandatory.
I want to respect people's contribution, but I don't see how external voting on patches is going to help me: either it fits at my tree's level, has a use and doesn't conflict in which case I'll take it, or it's not going to work out for me and it won't work out any better if one or more people clicked +1 on it.
I don't think we can dispense with one person attempting to understand what the patches are and what they buy us, what their future is and if they want them which trees they go on, etc, by a kind of isithotornot.linaro.org.
-Andy
On Sat, 2011-08-27 at 21:29 +0800, Andy Green wrote:
On 08/27/2011 02:15 AM, Somebody in the thread at some point said:
Gerrit is that it doesn't care if the tree has been rebased. It will just try and merge the changeset on top of the current tree, whatever the history so in that case Gerrit is actually enabling developers to work with trees that are continually rebased (like the linux kernel).
Linus' tree is a history tree, fwiw.
For kernel, gerrit users even generating the patches to propose requires git. So gerrit is just a sort of secondary 'social' option on top of git, which remains mandatory.
I want to respect people's contribution, but I don't see how external voting on patches is going to help me: either it fits at my tree's level, has a use and doesn't conflict in which case I'll take it, or it's not going to work out for me and it won't work out any better if one or more people clicked +1 on it.
I don't think we can dispense with one person attempting to understand what the patches are and what they buy us, what their future is and if they want them which trees they go on, etc, by a kind of isithotornot.linaro.org.
I think you're greatly mis-characterizing what gerrit does. *No one* maintains a code tree like a democracy.
I too suspect the voting (which is really just review logging, something most mature projects require and use, be it "Acked-by" lines or whatever) won't be of much use to us, as we're still fairly loose with our change management and we trust tree maintainers.
As a maintainer, I think the formal submission tracking (ie: here are changes submitted that have not yet been included) is really the part that is potentially valuable to us. It makes sure the work folks submit doesn't just get lost in churn and busy crunches.
Additionally, a more minor perk is that it allows series of patches to be merged without the manual plucking from your inbox, saving to files, copying them to my git tree server, and then git am'ing them in.
Now, again, I'm not a fan of gerrit. And outside of the Android work, I *really* don't see it providing any value to Linaro (personally I'd like to see a tool more like patchworks integrated with git that provides the same benefits as above).
But I've got an open mind and am willing to try it out and figure out a workflow. I suspect for me it really won't be anything other then a different location to "git push" to.
If we do actually start to see an inflow of kernel patches through gerrit, we can see how that workflow functions and have a concrete discussion on pros and cons (worse case, I git cherry pick the commits I like to my tree and push it out, or we can ask to see if gerrit can be configured to mail patches to us). But I really really doubt this will be a common thing.
Again, I've volunteered to lead the way on the kernel side in this trial run, and (while there have been some initial configuration hiccups) it really has been almost zero overhead/impact to my workflow so far.
(And forgive me for posting and running here, as I'm out of the office for the next two weeks - including Linux Plumbers, so I may not respond to further replies for some time. So feel free to have the last word. :)
thanks -john
On 08/28/2011 05:20 AM, Somebody in the thread at some point said:
On Sat, 2011-08-27 at 21:29 +0800, Andy Green wrote:
On 08/27/2011 02:15 AM, Somebody in the thread at some point said:
Gerrit is that it doesn't care if the tree has been rebased. It will just try and merge the changeset on top of the current tree, whatever the history so in that case Gerrit is actually enabling developers to work with trees that are continually rebased (like the linux kernel).
Linus' tree is a history tree, fwiw.
For kernel, gerrit users even generating the patches to propose requires git. So gerrit is just a sort of secondary 'social' option on top of git, which remains mandatory.
I want to respect people's contribution, but I don't see how external voting on patches is going to help me: either it fits at my tree's level, has a use and doesn't conflict in which case I'll take it, or it's not going to work out for me and it won't work out any better if one or more people clicked +1 on it.
I don't think we can dispense with one person attempting to understand what the patches are and what they buy us, what their future is and if they want them which trees they go on, etc, by a kind of isithotornot.linaro.org.
I think you're greatly mis-characterizing what gerrit does. *No one* maintains a code tree like a democracy.
I went and looked at it last week, clicked around a bit and report what I saw.
I too suspect the voting (which is really just review logging, something most mature projects require and use, be it "Acked-by" lines or whatever) won't be of much use to us, as we're still fairly loose with our change management and we trust tree maintainers.
Yeah.
As a maintainer, I think the formal submission tracking (ie: here are changes submitted that have not yet been included) is really the part that is potentially valuable to us. It makes sure the work folks submit doesn't just get lost in churn and busy crunches.
Additionally, a more minor perk is that it allows series of patches to be merged without the manual plucking from your inbox, saving to files, copying them to my git tree server, and then git am'ing them in.
Now, again, I'm not a fan of gerrit. And outside of the Android work, I *really* don't see it providing any value to Linaro (personally I'd like to see a tool more like patchworks integrated with git that provides the same benefits as above).
But I've got an open mind and am willing to try it out and figure out a workflow. I suspect for me it really won't be anything other then a different location to "git push" to.
I think we are having different takes on it (although I appreciate reading your somewhat nuanced view) because application of patches doesn't mean for me what it means to a history branch.
The patch does not just go on top of HEAD and then you're done, it needs to be placed at the right topic branch layer in tracking; may need to be adapted to apply to an intermediate branch (eg, tilt-3.0) that is on a different pinned release basis, then the intermediate branch rebased against the final basis like, eg, your android tree to provide the final output.
Can Gerrit understand or track that?
If we do actually start to see an inflow of kernel patches through gerrit, we can see how that workflow functions and have a concrete discussion on pros and cons (worse case, I git cherry pick the commits I like to my tree and push it out, or we can ask to see if gerrit can be configured to mail patches to us). But I really really doubt this will be a common thing.
Again, I've volunteered to lead the way on the kernel side in this trial run, and (while there have been some initial configuration hiccups) it really has been almost zero overhead/impact to my workflow so far.
(And forgive me for posting and running here, as I'm out of the office for the next two weeks - including Linux Plumbers, so I may not respond to further replies for some time. So feel free to have the last word. :)
Hey you're forgiven dude ^^ Have a nice time at Plumbers.
-Andy
On Sun, 2011-08-28 at 10:35 +0800, Andy Green wrote:
On 08/28/2011 05:20 AM, Somebody in the thread at some point said:
But I've got an open mind and am willing to try it out and figure out a workflow. I suspect for me it really won't be anything other then a different location to "git push" to.
I think we are having different takes on it (although I appreciate reading your somewhat nuanced view) because application of patches doesn't mean for me what it means to a history branch.
The patch does not just go on top of HEAD and then you're done, it needs to be placed at the right topic branch layer in tracking; may need to be adapted to apply to an intermediate branch (eg, tilt-3.0) that is on a different pinned release basis, then the intermediate branch rebased against the final basis like, eg, your android tree to provide the final output.
Right. So yes, the applying a series of patches directly to the tree isn't as big of a perk for your topic branches.
But... One way it can work is by applying them on top the gerrit head, you can then pluck them via git cherry-pick into your own tree and into the right topic branch. Then the next time you rebase/push, the patch should be included there. If gerrit provides a changeid on the merged patch (this I've not tried, so maybe it doesn't), it should then be able to track that the patch is also merged in your newly rebased HEAD.
But again, I really doubt there will be much in the way of kernel patches being submitted via gerrit (for most I suspect its more work to submit via gerrit then it is to mail patches). So I'd really only focus on gerrit as just a different public git repo to push your work to.
thanks -john
Hello Zach,
On Thu, 25 Aug 2011 18:12:01 -0500 Zach Pfeffer zach.pfeffer@linaro.org wrote:
[]
I also updated Linaro Gerrit howto: https://wiki.linaro.org/Platform/Android/Gerrit , which now should have all info to have one started quickly with Gerrit, and cover all aspects of Gerrit setup (like upstream mirroring and permission settings). I'd appreciate review of that and letting me know if something is missing there.
This looks really good. Please share this with the repo mailing list. Your great work has really helped other people trying to set up their own Gerrit instances.
I did so per your previous request, during LC.
Finally few points we can continue with to elaborate our usage of Gerrit:
- Set up branch policy (naming, etc.) and enforce it on Gerrit
level. This may require revamping branching in other toolchain/* components (upstreamed not from AOSP), but in the end we'll get really robust development setup.
Agreed. One big thing that we need to work on is how we're going to handle kernel upgrades - I think some special branch-naming may be part of this solution.
- Turn off review bypass option which was available during
transition process. I guess Android team if comfortable by now with new process (there're more than 80 patches passed thru review by now!), so once 11.08 release is out, it would be good time to do that.
+1 for most. I think the only thing we have to watch out for is kernel maintainers needing to push big updates out-of-band.
Whenever I talk about Gerrit workflow, I mean "Gerrit workflow for Android platform components, skipping kernels" ;-). There would be apparently 2 different workflows, and we need to settled main Android workflow and let kernel guys try and use Gerrit and see what's comfortable for them (starting with complete git access). We already have beginning of such setup, with John being in "Kernel Hackers" group, which allows rebases and upstream pulling for kernel/linux-android.
-
Andy Green -
John Stultz -
john stultz
-
Paul Sokolovsky -
Zach Pfeffer