Hi -
Recently at Linaro we've started a new tree linaro-androidization-tracking, which is pretty much "common-3.1"[1] at the moment on 3.1-rc8 basis.
http://git.linaro.org/gitweb?p=people/andygreen/kernel-tilt.git%3Ba=summary
We have already been running a kernel tree tracking Linus HEAD since 2.6.39, which has OMAP4 / Panda enablement stuff on top of Linus HEAD, so we have some experience with the rough and tumble.
What we missed having was an "all year round" Androidization tree that we could combine it with to casually create Android versions of the work we were doing on Vanilla Linus HEAD basis. We used common-3.0 for that for a while late in the kernel release cycle when it was tracking Linus HEAD itself and that was great. But common-3.0 like the name suggests is really a release tree, and it stops tracking at release, and a new one starts up only late in the next kernel release cycle.
Actually, it would be a big advantage for many folks to not be doing their Android kernel development on lagging releases, but by tracking Linus HEAD. They would have access to latest stuff already and they don't have to think about backport or later forwardport stuff to a new release, it's constantly tracking HEAD and being adapted.
One side-effect of using this tracking methodology is if they want release trees, they can just clone their tracking branch at the time Linus HEAD is at a release point and bam, a hopefully fully working release tree is there. Another very nice side-effect is they can do the bulk of the work once on a Vanilla Linus HEAD basis, and get and Android version of that work routinely by merging or rebasing in the Androidization tree - instead of doing and validating work twice on separate Vanilla and Android trees.
So linaro-androidization-tracking is our attempt at that Linus HEAD Androidization tree, moving it on regularly and fixing breakage as it happens throughout the cycle. It's generic same as common-, it should be usable for any arch or board to Androidize a vanilla kernel that is on the same Linus HEAD basis just by merge or rebase action.
It seemed this effort might be interesting for the guys that make the common- trees, since if there was a tracking Androidization tree, in fact common- releases for release trees like common-3.1 would just naturally fall out of it when Linus HEAD was at 3.1. So I'd be very happy to hear any opinions about that.
Another side-issue is we are also looking at refactoring the Androidization patchset into topic branches, at the moment they're kind of reflecting the history that they were applied on plus or minus some munging around. So, all the net core patches for example would be together in one series, then all the wireless ones in a series on top of that, etc. It seems they would be easier to maintain then, opinions on that approach are also welcome.
-Andy
[1] TI have a tree on omapzoom where we got the patches from
http://git.omapzoom.org/?p=kernel/omap.git%3Ba=shortlog%3Bh=refs/heads/p-and...
On Wed, 5 Oct 2011, Andy Green wrote:
Actually, it would be a big advantage for many folks to not be doing their Android kernel development on lagging releases, but by tracking Linus HEAD. They would have access to latest stuff already and they don't have to think about backport or later forwardport stuff to a new release, it's constantly tracking HEAD and being adapted.
Amen!
Nicolas
On 10/05/2011 08:37 AM, Andy Green wrote:
Hi -
Recently at Linaro we've started a new tree linaro-androidization-tracking, which is pretty much "common-3.1"[1] at the moment on 3.1-rc8 basis.
http://git.linaro.org/gitweb?p=people/andygreen/kernel-tilt.git%3Ba=summary
We have already been running a kernel tree tracking Linus HEAD since 2.6.39, which has OMAP4 / Panda enablement stuff on top of Linus HEAD, so we have some experience with the rough and tumble.
What we missed having was an "all year round" Androidization tree that we could combine it with to casually create Android versions of the work we were doing on Vanilla Linus HEAD basis. We used common-3.0 for that for a while late in the kernel release cycle when it was tracking Linus HEAD itself and that was great. But common-3.0 like the name suggests is really a release tree, and it stops tracking at release, and a new one starts up only late in the next kernel release cycle.
Actually, it would be a big advantage for many folks to not be doing their Android kernel development on lagging releases, but by tracking Linus HEAD. They would have access to latest stuff already and they don't have to think about backport or later forwardport stuff to a new release, it's constantly tracking HEAD and being adapted.
One side-effect of using this tracking methodology is if they want release trees, they can just clone their tracking branch at the time Linus HEAD is at a release point and bam, a hopefully fully working release tree is there. Another very nice side-effect is they can do the bulk of the work once on a Vanilla Linus HEAD basis, and get and Android version of that work routinely by merging or rebasing in the Androidization tree - instead of doing and validating work twice on separate Vanilla and Android trees.
So linaro-androidization-tracking is our attempt at that Linus HEAD Androidization tree, moving it on regularly and fixing breakage as it happens throughout the cycle. It's generic same as common-, it should be usable for any arch or board to Androidize a vanilla kernel that is on the same Linus HEAD basis just by merge or rebase action.
It seemed this effort might be interesting for the guys that make the common- trees, since if there was a tracking Androidization tree, in fact common- releases for release trees like common-3.1 would just naturally fall out of it when Linus HEAD was at 3.1. So I'd be very happy to hear any opinions about that.
Another side-issue is we are also looking at refactoring the Androidization patchset into topic branches, at the moment they're kind of reflecting the history that they were applied on plus or minus some munging around. So, all the net core patches for example would be together in one series, then all the wireless ones in a series on top of that, etc. It seems they would be easier to maintain then, opinions on that approach are also welcome.
I've been working on a set of broken-out Android patches, in quilt format, with exactly the same goal. It's much easier (at least for me) to maintain the differences as distinct patches using quilt, rather than as a series (or set of branches) of git commits.
But possibly I'm just missing how to do this easily with git. I would very much like to compare approaches, and possibly share the workload of maintaining such a thing. -- Tim
============================= Tim Bird Architecture Group Chair, CE Workgroup of the Linux Foundation Senior Staff Engineer, Sony Network Entertainment =============================
On 6 October 2011 02:36, Tim Bird tim.bird@am.sony.com wrote:
On 10/05/2011 08:37 AM, Andy Green wrote:
Another side-issue is we are also looking at refactoring the Androidization patchset into topic branches, at the moment they're kind of reflecting the history that they were applied on plus or minus some munging around. So, all the net core patches for example would be together in one series, then all the wireless ones in a series on top of that, etc. It seems they would be easier to maintain then, opinions on that approach are also welcome.
I've been working on a set of broken-out Android patches, in quilt format, with exactly the same goal. It's much easier (at least for me) to maintain the differences as distinct patches using quilt, rather than as a series (or set of branches) of git commits.
But possibly I'm just missing how to do this easily with git. I would very much like to compare approaches, and possibly share
Thanks a lot for the reply Tim. I can describe how I manage this flow in detail. Actually, since we would interact through pure git, it's not necessary to unify the patch management scheme to be able to cooperate. But since I spent so much time on exactly this subject I guess we both find it interesting to describe it.
There are a number of valid ways to come at managing the topic patchsets and quilt is certainly one of them. Another approach that would work is to use git cherry-pick to throw around series of patches from one branch to another. The way I have been using though is stgit.
stgit has the advantages of quilt in terms of managing itself the members of the series. If you try to do it with git cherry-pick, you have to have explicit secondary branches floating around to take the series from and keep tracking of what came from where and what basis it was on, etc, it doesn't seem it will scale well to dozens of topic and multiple branches. stgit has a further characteristic that it stores applied / pushed patches from the series in absolutely pure git commits, so at any time you can push what you have to remote git and that's exactly what people will pull with no stgit on their side. Stgit is like quilt in terms of patch stack operation, push, pop, float, "goto", support for rebase, re-ordering, apply conflict management, import, export, mail and so on. Stgit series, what is and is not applied, all state in fact, is bound to a specific git branch. So when you checkout a different branch in git, the stgit context for that branch is made active and nothing is lost about stgit in the previous branch. In that way, it's like some extra features integrated on top of git.
Here's a tree I maintain with many topic branches using that flow
http://git.linaro.org/gitweb?p=people/andygreen/kernel-tilt.git%3Ba=shortlog...
if you go back through the gitweb history, you can see it is formed from a set of other tracking-topic- branches each rebased on top of each other. I have a set of scripts to automate this rebasing. If I have something to fix or change at tracking-topic-hdmiaudio for example, I would move to that branch, change or add patches there, build and confirm the kernel works at that patchlevel, then use my scripts to rebase the topic branches on top of it, confirm we didn't break anything, and push the changed branches to public git.
The main downside is management by stgit is slow. I have a fast build machine but stgit just uses one thread and favours correctness over speed, it's constantly checking to see if any files were changed behind its back in the tree for example. Fully rebasing all my trees is a matter of 90 minutes, although that includes a bunch of special rebase trees that are different from what we're discussing.
The most important characteristic with managing the series at each topic level is that the topic patch series must strictly be sitting purely on top of the basis. git rebase, another candidate for handling the topic series, depending on the relationship of the two trees, will not always provide that feature in my experience.
the workload of maintaining such a thing.
That's great we have had the same thoughts looking at the same situation.
What I am expecting is we get an easy ride until Linus kernel 3.1 release, at which point a backlog of disruptive patches will be applied to Linus HEAD for 3.2-rc0 and there is some work to do. We should keep in contact and figure out how to react to it when the extent of it is known. If it's reasonable amount of work I can take care of it, if it's getting a bit much it will be really great to be able to figure out how to share the load.
Something else to consider about refactor scheme is when an official common-3.2 is coming say at 3.2-rc7. Unless we are able to interest common- maintainers in our scheme, that will not be structured according to our refactor. I gave this some thought, the worst that can happen is we have to discard the refactor and just take common-3.2 when the tracking basis is the same; consumers of the Androidization tree won't notice anything. More likely though, a large proportion of topic patches in, say, net core area are going to be unchanged or have minor tweaks. In that case, if we make sure that our tree and this first common-3.2 are on the same basis version, it should be possible to pick and adapt common-3.2 content on to the refactor and crucially, use git diff to confirm that the resulting tree is 100% identical at that point. So we can be certain that we have everything imported perfectly from the common- update even though the series is completely different. Thereafter, we can pick new patches going on common-3.2 one by one into the refactored tree. So, I think that case is probably manageable.
-Andy
On 6 October 2011 10:07, Andy Green andy.green@linaro.org wrote:
On 6 October 2011 02:36, Tim Bird tim.bird@am.sony.com wrote:
On 10/05/2011 08:37 AM, Andy Green wrote:
Another side-issue is we are also looking at refactoring the Androidization patchset into topic branches, at the moment they're kind of reflecting the history that they were applied on plus or minus some munging around. So, all the net core patches for example would be together in one series, then all the wireless ones in a series on top of that, etc. It seems they would be easier to maintain then, opinions on that approach are also welcome.
I've been working on a set of broken-out Android patches, in quilt format, with exactly the same goal. It's much easier (at least for me) to maintain the differences as distinct patches using quilt, rather than as a series (or set of branches) of git commits.
But possibly I'm just missing how to do this easily with git. I would very much like to compare approaches, and possibly share
Thanks a lot for the reply Tim. I can describe how I manage this flow in detail. Actually, since we would interact through pure git, it's not necessary to unify the patch management scheme to be able to cooperate. But since I spent so much time on exactly this subject I guess we both find it interesting to describe it.
Further to this I have now re-ordered the patchset into 20 topics in the following order, on top of 3.1-rc9+ from yesterday:
linaro-androidization-topic-base - stuff impacting core Linux linaro-androidization-topic-debug - logging, debug related features linaro-androidization-topic-wakelock - wakelock core stuff linaro-androidization-topic-earlysuspend - earlysuspend linaro-androidization-topic-mem - pmem, low memory killer linaro-androidization-topic-net-core - core network patches linaro-androidization-topic-netfilter - netfilter patches linaro-androidization-topic-net-wireless - wireless related linaro-androidization-topic-usb - usb, mostly gadget and composite linaro-androidization-topic-input - input subsystem related ./ HID linaro-androidization-topic-cpufreq - cpufreq linaro-androidization-topic-ion - ion graphical memory manager linaro-androidization-topic-mmc - mmc / sdio related linaro-androidization-topic-bluetooth - bluetooth linaro-androidization-topic-rtc - rtc / alarm linaro-androidization-topic-pm - suspend flow and other power management linaro-androidization-topic-binder - binder linaro-androidization-topic-fs - yaffs, vfat, filesystem, block linaro-androidization-topic-gpio - timed gpio linaro-androidization-topic-misc - just a compass driver right now
I confirmed that the diff between the original branch we issued and this new one is absolutely null, so we can be certain the reorder makes no difference to the resulting tree.
You can see what's going on easiest if you look at the last branch linaro-androidization-topic-misc, which is the "output" of this process
http://git.linaro.org/gitweb?p=people/andygreen/kernel-tilt.git%3Ba=shortlog...
and work backwards, you can see it's sitting on top of the other branches in reverse order.
In terms of patch counts, it's like this
linaro-androidization-topic-base 38 linaro-androidization-topic-debug 34 linaro-androidization-topic-wakelock 14 linaro-androidization-topic-earlysuspend 7 linaro-androidization-topic-mem 16 linaro-androidization-topic-net-core 19 linaro-androidization-topic-netfilter 23 linaro-androidization-topic-net-wireless 120 linaro-androidization-topic-usb 79 linaro-androidization-topic-input 27 linaro-androidization-topic-cpufreq 14 linaro-androidization-topic-ion 12 linaro-androidization-topic-mmc 22 linaro-androidization-topic-bluetooth 10 linaro-androidization-topic-rtc 7 linaro-androidization-topic-pm 11 linaro-androidization-topic-binder 6 linaro-androidization-topic-fs 8 linaro-androidization-topic-gpio 3 linaro-androidization-topic-misc 1
You can see actually the biggest series is in wireless, and that's because it seems to add a couple of wireless drivers there. USB gadget-related stuff is next.
If you look at say the ion topic
http://git.linaro.org/gitweb?p=people/andygreen/kernel-tilt.git%3Ba=shortlog...
it seems some of them could just be simplified down to a single patch or so, incorporating all the bugfixes and changes.
-Andy
Good afternoon Andy,
I am currently working on rebasing the STE GLK kernel to the androidization branch. Doing so I have the following questions:
1. You have made it clear that linaro-androidization-tracking is tracking Linus HEAD but is it also tracking kernel/common.git HEAD on the android side ?
2. What will be the rebase strategy ? Every week ? And when you do rebase, how do you intend to let the followers (like me) know ?
Best regards, Mathieu.
On 11-10-05 09:37 AM, Andy Green wrote:
Hi -
Recently at Linaro we've started a new tree linaro-androidization-tracking, which is pretty much "common-3.1"[1] at the moment on 3.1-rc8 basis.
http://git.linaro.org/gitweb?p=people/andygreen/kernel-tilt.git%3Ba=summary
We have already been running a kernel tree tracking Linus HEAD since 2.6.39, which has OMAP4 / Panda enablement stuff on top of Linus HEAD, so we have some experience with the rough and tumble.
What we missed having was an "all year round" Androidization tree that we could combine it with to casually create Android versions of the work we were doing on Vanilla Linus HEAD basis. We used common-3.0 for that for a while late in the kernel release cycle when it was tracking Linus HEAD itself and that was great. But common-3.0 like the name suggests is really a release tree, and it stops tracking at release, and a new one starts up only late in the next kernel release cycle.
Actually, it would be a big advantage for many folks to not be doing their Android kernel development on lagging releases, but by tracking Linus HEAD. They would have access to latest stuff already and they don't have to think about backport or later forwardport stuff to a new release, it's constantly tracking HEAD and being adapted.
One side-effect of using this tracking methodology is if they want release trees, they can just clone their tracking branch at the time Linus HEAD is at a release point and bam, a hopefully fully working release tree is there. Another very nice side-effect is they can do the bulk of the work once on a Vanilla Linus HEAD basis, and get and Android version of that work routinely by merging or rebasing in the Androidization tree - instead of doing and validating work twice on separate Vanilla and Android trees.
So linaro-androidization-tracking is our attempt at that Linus HEAD Androidization tree, moving it on regularly and fixing breakage as it happens throughout the cycle. It's generic same as common-, it should be usable for any arch or board to Androidize a vanilla kernel that is on the same Linus HEAD basis just by merge or rebase action.
It seemed this effort might be interesting for the guys that make the common- trees, since if there was a tracking Androidization tree, in fact common- releases for release trees like common-3.1 would just naturally fall out of it when Linus HEAD was at 3.1. So I'd be very happy to hear any opinions about that.
Another side-issue is we are also looking at refactoring the Androidization patchset into topic branches, at the moment they're kind of reflecting the history that they were applied on plus or minus some munging around. So, all the net core patches for example would be together in one series, then all the wireless ones in a series on top of that, etc. It seems they would be easier to maintain then, opinions on that approach are also welcome.
-Andy
[1] TI have a tree on omapzoom where we got the patches from
http://git.omapzoom.org/?p=kernel/omap.git%3Ba=shortlog%3Bh=refs/heads/p-and...
On 6 October 2011 06:14, Mathieu Poirier mathieu.poirier@linaro.org wrote:
Good afternoon Andy,
I am currently working on rebasing the STE GLK kernel to the androidization branch. Doing so I have the following questions:
- You have made it clear that linaro-androidization-tracking is
tracking Linus HEAD but is it also tracking kernel/common.git HEAD on the android side
It's not tracking it in terms of using it as a basis. We would follow it and pick new stuff that looked relevant. But right now, there is no common-3.1 branch we have access to that we can pick from.
- What will be the rebase strategy ? Every week ? And when you do
rebase, how do you intend to let the followers (like me) know ?
If you think about how you will consume the tree, your desire to use it at a particular basis will be driven by your own rebase schedule for your vanilla tree. So if you have your tracking tree going on -rc5, you are going to want -rc5 Androidization tree. If you don't have -rc5 tracking tree, you don't care if we have -rc5 Androidization tree or not.
What I suggest is for your tracking branch don't rebase against Linus repo direct but rebase against this
http://git.linaro.org/gitweb?p=people/andygreen/kernel-tilt.git%3Ba=shortlog...
it is the last pull that we did from Linus tree, all of our TILT trees and the Androidization tree will be using that as their basis. It's always pure Linus tree but the basis is exactly what we are using in our other trees.
Sometimes (ie, after -rc0) we will be blocked from pushing the Androidization tree for a while because it's regressed. The approach we will take then is not to push an update until the known regressions are at least contained or at best gone. However, your LT tracking tree is likely going to be in a similar position at that time for the same reasons.
Otherwise we'll be trying to provide it for each -rc at least and make those persistent using either tags or the branch archive repo we already run.
-Andy
-
Andy Green -
Mathieu Poirier -
Nicolas Pitre -
Tim Bird