Hello everyone,
As I've been working on the integration of the latest developments and fixes from upstream into the linaro-2.6.38 kernel lately, it became quickly evident that major merge conflicts were to be expected. The problem stems from the fact that we opened the 2.6.38 branch early i.e. around the 2.6.38-rc5 kernel. Many patches that were merged into the Linaro kernel have been subsequently modified by their respective maintainers until they were merged upstream, meaning that what we have now in mainline is different from what the Linaro kernel tree has. This creates several issues:
- It is hard to verify that what is in the Linaro tree is actually the latest and the best version of a patch.
- Merging additional fixes from upstream often ends up in merge conflicts requiring manual resolution, sometimes non-trivial ones, which is in itself a bug hazard.
- People wanting to contribute patches to the Linaro kernel potentially have to create a different patch than what they would submit upstream.
Given those issues, I decided to rebuild the linaro-2.6.38 branch from scratch to see where that would bring me. And as could be expected, the result looks nicer and it is much easier to work with than the current tree. For example, this allowed me to merge the latest OMAP support from mainline as is, including the latest fixes, without any need for backport work nor major conflict resolution. Another advantage is that the commit SHA1 references are now identical in most cases to what can be found into mainline.
So... my question is: should we switch to this rebuilt tree or not? There are drawbacks with such a move of course:
- All the testing done so far would be void. This is however not as bad as this may look as the rebuilt kernel contains fixes for existing bugs in the current tree, and the rebuilt kernel is using patches that have and still are being tested by a wider community.
- I didn't forward port a couple series of patches that are available in the current Linaro tree and not in mainline yet, including: * DT support (Grant Likely) * DVFS and PM for OMAP (Vishwanath Sripathy, Vishwanath BS) * Some ux500 fixups (Linus Walleij) * clock debug information (Yong Shen) * Samsung CPUIDLE (Amit Kachhap) So I would prefer if the people responsible for those patches did resubmit their patches once they apply to the new tree (that should be even easier now to apply patches that were meant for mainline).
- The history of the rebuilt tree is obviously different from the current tree's. This means special care when updating to the new tree with Git.
But overall I think there are more advantages than disadvantages for such a move. What other people think?
The current rebuilt tree can be seen at:
http://git.linaro.org/gitweb?p=kernel/linux-linaro-2.6.38.git%3Ba=shortlog%3...
or obtained from:
git://git.linaro.org/kernel/linux-linaro-2.6.38.git (the "rebuilt" branch)
Nicolas
The rebuilt branch is fine for me. My tree creation methodology is patterned after the Ubuntu kernel which means I rebase to new upstreams each release so I don't mind if my upstreams do as well.
John
On Sun, Mar 27, 2011 at 10:06 PM, Nicolas Pitre nicolas.pitre@linaro.org wrote:
Hello everyone,
As I've been working on the integration of the latest developments and fixes from upstream into the linaro-2.6.38 kernel lately, it became quickly evident that major merge conflicts were to be expected. The problem stems from the fact that we opened the 2.6.38 branch early i.e. around the 2.6.38-rc5 kernel. Many patches that were merged into the Linaro kernel have been subsequently modified by their respective maintainers until they were merged upstream, meaning that what we have now in mainline is different from what the Linaro kernel tree has. This creates several issues:
- It is hard to verify that what is in the Linaro tree is actually the latest and the best version of a patch.
- Merging additional fixes from upstream often ends up in merge conflicts requiring manual resolution, sometimes non-trivial ones, which is in itself a bug hazard.
- People wanting to contribute patches to the Linaro kernel potentially have to create a different patch than what they would submit upstream.
Given those issues, I decided to rebuild the linaro-2.6.38 branch from scratch to see where that would bring me. And as could be expected, the result looks nicer and it is much easier to work with than the current tree. For example, this allowed me to merge the latest OMAP support from mainline as is, including the latest fixes, without any need for backport work nor major conflict resolution. Another advantage is that the commit SHA1 references are now identical in most cases to what can be found into mainline.
So... my question is: should we switch to this rebuilt tree or not? There are drawbacks with such a move of course:
- All the testing done so far would be void. This is however not as bad as this may look as the rebuilt kernel contains fixes for existing bugs in the current tree, and the rebuilt kernel is using patches that have and still are being tested by a wider community.
- I didn't forward port a couple series of patches that are available in the current Linaro tree and not in mainline yet, including: * DT support (Grant Likely) * DVFS and PM for OMAP (Vishwanath Sripathy, Vishwanath BS) * Some ux500 fixups (Linus Walleij) * clock debug information (Yong Shen) * Samsung CPUIDLE (Amit Kachhap) So I would prefer if the people responsible for those patches did resubmit their patches once they apply to the new tree (that should be even easier now to apply patches that were meant for mainline).
- The history of the rebuilt tree is obviously different from the current tree's. This means special care when updating to the new tree with Git.
But overall I think there are more advantages than disadvantages for such a move. What other people think?
The current rebuilt tree can be seen at:
http://git.linaro.org/gitweb?p=kernel/linux-linaro-2.6.38.git%3Ba=shortlog%3...
or obtained from:
git://git.linaro.org/kernel/linux-linaro-2.6.38.git (the "rebuilt" branch)
Nicolas
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
I favor sticking close to mainline unless it irreparably breaks something.
Thanx, Paul
On Mon, Mar 28, 2011 at 12:06:31AM -0400, Nicolas Pitre wrote:
Hello everyone,
As I've been working on the integration of the latest developments and fixes from upstream into the linaro-2.6.38 kernel lately, it became quickly evident that major merge conflicts were to be expected. The problem stems from the fact that we opened the 2.6.38 branch early i.e. around the 2.6.38-rc5 kernel. Many patches that were merged into the Linaro kernel have been subsequently modified by their respective maintainers until they were merged upstream, meaning that what we have now in mainline is different from what the Linaro kernel tree has. This creates several issues:
It is hard to verify that what is in the Linaro tree is actually the latest and the best version of a patch.
Merging additional fixes from upstream often ends up in merge conflicts requiring manual resolution, sometimes non-trivial ones, which is in itself a bug hazard.
People wanting to contribute patches to the Linaro kernel potentially have to create a different patch than what they would submit upstream.
Given those issues, I decided to rebuild the linaro-2.6.38 branch from scratch to see where that would bring me. And as could be expected, the result looks nicer and it is much easier to work with than the current tree. For example, this allowed me to merge the latest OMAP support from mainline as is, including the latest fixes, without any need for backport work nor major conflict resolution. Another advantage is that the commit SHA1 references are now identical in most cases to what can be found into mainline.
So... my question is: should we switch to this rebuilt tree or not? There are drawbacks with such a move of course:
All the testing done so far would be void. This is however not as bad as this may look as the rebuilt kernel contains fixes for existing bugs in the current tree, and the rebuilt kernel is using patches that have and still are being tested by a wider community.
I didn't forward port a couple series of patches that are available in the current Linaro tree and not in mainline yet, including:
- DT support (Grant Likely)
- DVFS and PM for OMAP (Vishwanath Sripathy, Vishwanath BS)
- Some ux500 fixups (Linus Walleij)
- clock debug information (Yong Shen)
- Samsung CPUIDLE (Amit Kachhap)
So I would prefer if the people responsible for those patches did resubmit their patches once they apply to the new tree (that should be even easier now to apply patches that were meant for mainline).
The history of the rebuilt tree is obviously different from the current tree's. This means special care when updating to the new tree with Git.
But overall I think there are more advantages than disadvantages for such a move. What other people think?
The current rebuilt tree can be seen at:
http://git.linaro.org/gitweb?p=kernel/linux-linaro-2.6.38.git%3Ba=shortlog%3...
or obtained from:
git://git.linaro.org/kernel/linux-linaro-2.6.38.git (the "rebuilt" branch)
Nicolas
linaro-kernel mailing list linaro-kernel@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-kernel
On Sun, Mar 27, 2011 at 10:06 PM, Nicolas Pitre nicolas.pitre@linaro.org wrote: [...]> Given those issues, I decided to rebuild the linaro-2.6.38 branch from
scratch to see where that would bring me. And as could be expected, the result looks nicer and it is much easier to work with than the current tree. For example, this allowed me to merge the latest OMAP support from mainline as is, including the latest fixes, without any need for backport work nor major conflict resolution. Another advantage is that the commit SHA1 references are now identical in most cases to what can be found into mainline.
So... my question is: should we switch to this rebuilt tree or not? There are drawbacks with such a move of course:
- All the testing done so far would be void. This is however not as bad as this may look as the rebuilt kernel contains fixes for existing bugs in the current tree, and the rebuilt kernel is using patches that have and still are being tested by a wider community.
- I didn't forward port a couple series of patches that are available in the current Linaro tree and not in mainline yet, including: * DT support (Grant Likely)
I'm based on 2.6.38 in my devicetree/test branch anyway so it isn't a problem to republish my changes if so desired.
I do have concerns. It changes my current workflow on some of the DT enablement patches, and I'm mildly concerned about the loss of test coverage, but neither concern weighs heavily enough on my mind to raise an objection.
g.
On Mon, 2011-03-28 at 00:06 -0400, Nicolas Pitre wrote:
Given those issues, I decided to rebuild the linaro-2.6.38 branch from scratch to see where that would bring me. And as could be expected, the result looks nicer and it is much easier to work with than the current tree. For example, this allowed me to merge the latest OMAP support from mainline as is, including the latest fixes, without any need for backport work nor major conflict resolution. Another advantage is that the commit SHA1 references are now identical in most cases to what can be found into mainline.
Oof. So this will cause some pain with the Linaro + Android kernel tree, as it is a consumer of the Linaro 2.6.38 tree.
I could throw out the old tree and re-merge the new Linaro tree with the current Android tree and our fixes, but that could cause grief for folks pulling from the current tree.
Maybe as an alternative solution: you could diff the new tree from the old tree and apply those chunks to the old tree. That would get us to the same spot, but without the rebase. The patch log won't be great, but that could be fixed up when you open up the 2.6.39 branch (maybe naming it 2.6.39-rc would be best to avoid the same issue in the future).
But either way, I think the Android tree could probably could survive it.
thanks -john
On Monday 28 March 2011 20:22:42 john stultz wrote:
On Mon, 2011-03-28 at 00:06 -0400, Nicolas Pitre wrote:
Given those issues, I decided to rebuild the linaro-2.6.38 branch from scratch to see where that would bring me. And as could be expected, the result looks nicer and it is much easier to work with than the current tree. For example, this allowed me to merge the latest OMAP support from mainline as is, including the latest fixes, without any need for backport work nor major conflict resolution. Another advantage is that the commit SHA1 references are now identical in most cases to what can be found into mainline.
Oof. So this will cause some pain with the Linaro + Android kernel tree, as it is a consumer of the Linaro 2.6.38 tree.
I could throw out the old tree and re-merge the new Linaro tree with the current Android tree and our fixes, but that could cause grief for folks pulling from the current tree.
Maybe as an alternative solution: you could diff the new tree from the old tree and apply those chunks to the old tree. That would get us to the same spot, but without the rebase. The patch log won't be great, but that could be fixed up when you open up the 2.6.39 branch (maybe naming it 2.6.39-rc would be best to avoid the same issue in the future).
But either way, I think the Android tree could probably could survive it.
If you worry about the users of the Android tree more than the git history of it, there may be another option: Take the diff between the old and the new linaro-2.6.38 and apply that on the Android tree. After you have fixed up all conflicts you get from that, do a pull from the new tree and solve ignore all conflicts by reverting them in the merge.
That way, you have a tree that your downstream can pull.
Arnd
On Mon, Mar 28, 2011, Arnd Bergmann wrote:
If you worry about the users of the Android tree more than the git history of it, there may be another option: Take the diff between the old and the new linaro-2.6.38 and apply that on the Android tree. After you have fixed up all conflicts you get from that, do a pull from the new tree and solve ignore all conflicts by reverting them in the merge.
That way, you have a tree that your downstream can pull.
It feels kind of hackish to do this in the downstream Android tree rather than in linux-linaro; the latter would also avoid the issue in other trees outside of Linaro if any.
On Mon, 28 Mar 2011, john stultz wrote:
On Mon, 2011-03-28 at 00:06 -0400, Nicolas Pitre wrote:
Given those issues, I decided to rebuild the linaro-2.6.38 branch from scratch to see where that would bring me. And as could be expected, the result looks nicer and it is much easier to work with than the current tree. For example, this allowed me to merge the latest OMAP support from mainline as is, including the latest fixes, without any need for backport work nor major conflict resolution. Another advantage is that the commit SHA1 references are now identical in most cases to what can be found into mainline.
Oof. So this will cause some pain with the Linaro + Android kernel tree, as it is a consumer of the Linaro 2.6.38 tree.
Hmmm... Well...
I could throw out the old tree and re-merge the new Linaro tree with the current Android tree and our fixes, but that could cause grief for folks pulling from the current tree.
Maybe as an alternative solution: you could diff the new tree from the old tree and apply those chunks to the old tree. That would get us to the same spot, but without the rebase. The patch log won't be great, but that could be fixed up when you open up the 2.6.39 branch (maybe naming it 2.6.39-rc would be best to avoid the same issue in the future).
Well, yeah. Due to some unfortunate timing, I opened the linaro-2.6.38 earlier than I had wished. I normally open a new tree only when the equivalent upstream version is out.
Let's see... I currently have:
$ git diff --shortstat old_linaro-2.6.38..new_linaro-2.6.38 805 files changed, 55412 insertions(+), 25203 deletions(-)
$ git diff --shortstat old_linaro-2.6.38..v2.6.38 966 files changed, 15985 insertions(+), 40994 deletions(-)
$ git diff --shortstat v2.6.38..new_linaro-2.6.38 1586 files changed, 93151 insertions(+), 37933 deletions(-)
Given that I want to preserve the history, what I can do is to apply the old->new diff to the old branch as this is the smallest diff, and then merge the new branch on top to tie the new history to it. That should remove the need for any rebase in the technical sense of the word, but that would still cause quite a road bump next time you pull that. Is it worth it? To me this doesn't make a huge difference.
But either way, I think the Android tree could probably could survive it.
I guess this depends on the nature of the consumers you have for your tree.
Nicolas
On Mon, 28 Mar 2011, Nicolas Pitre wrote:
Let's see... I currently have:
$ git diff --shortstat old_linaro-2.6.38..new_linaro-2.6.38 805 files changed, 55412 insertions(+), 25203 deletions(-)
$ git diff --shortstat old_linaro-2.6.38..v2.6.38 966 files changed, 15985 insertions(+), 40994 deletions(-)
$ git diff --shortstat v2.6.38..new_linaro-2.6.38 1586 files changed, 93151 insertions(+), 37933 deletions(-)
Given that I want to preserve the history, what I can do is to apply the old->new diff to the old branch as this is the smallest diff, and then merge the new branch on top to tie the new history to it. That should remove the need for any rebase in the technical sense of the word, but that would still cause quite a road bump next time you pull that. Is it worth it? To me this doesn't make a huge difference.
So that's what I just did. No rebase needed, however expect quite a bump next time you pull/merge depending on your work area in the tree.
Nicolas
linaro-kernel@lists.linaro.org
-
Arnd Bergmann -
Grant Likely -
John Rigby -
john stultz -
Loïc Minier -
Nicolas Pitre -
Paul E. McKenney