(I'm aware that there is a thread on linaro-dev discussing this exact topic; this is a request for specific information, so I decided to start a new thread)
Hello, As has been pointed out elsewhere, it is very difficult to find the exact git tree that corresponds to a kernel release. Currently the problem I am having is that the 11.11 linaro kernel release (linux-linaro-lt-omap_3.1.0-1402.5~oneiric1) works well on my new board, but later kernels do not. While I can download the kernel tarball for 3.1.0-1402.5 from launchpad, I would much prefer to use the git tree that it was produced from. Can anybody tell me exactly which git tree was used to create that kernel, and which tag/branch I should be looking at?
Thanks,
On Tue, Jan 24, 2012 at 09:29:34AM -0500, Chris Lalancette wrote:
(I'm aware that there is a thread on linaro-dev discussing this exact topic; this is a request for specific information, so I decided to start a new thread)
Hello, As has been pointed out elsewhere, it is very difficult to find the exact git tree that corresponds to a kernel release. Currently the problem I am having is that the 11.11 linaro kernel release (linux-linaro-lt-omap_3.1.0-1402.5~oneiric1) works well on my new board, but later kernels do not. While I can download the kernel tarball for 3.1.0-1402.5 from launchpad, I would much prefer to use the git tree that it was produced from. Can anybody tell me exactly which git tree was used to create that kernel, and which tag/branch I should be looking at?
That's a really good question. The answer is that it's this tag and branch:
linux-release-2011-12 http://git.linaro.org/git/landing-teams/leb/ti/kernel.git
However, I think there should be a natural way for you to find that out as well. How could we package this metadata so you'd find it easily?
On Tue, Jan 24, 2012 at 12:59:33PM -0200, Christian Robottom Reis wrote:
On Tue, Jan 24, 2012 at 09:29:34AM -0500, Chris Lalancette wrote:
(I'm aware that there is a thread on linaro-dev discussing this exact topic; this is a request for specific information, so I decided to start a new thread)
Hello, As has been pointed out elsewhere, it is very difficult to find the exact git tree that corresponds to a kernel release. Currently the problem I am having is that the 11.11 linaro kernel release (linux-linaro-lt-omap_3.1.0-1402.5~oneiric1) works well on my new board, but later kernels do not. While I can download the kernel tarball for 3.1.0-1402.5 from launchpad, I would much prefer to use the git tree that it was produced from. Can anybody tell me exactly which git tree was used to create that kernel, and which tag/branch I should be looking at?
That's a really good question. The answer is that it's this tag and branch:
linux-release-2011-12 http://git.linaro.org/git/landing-teams/leb/ti/kernel.git
Sorry, for 11.11 that's tag linux-release-2011-11-1 -- you can see all the tags here:
http://git.linaro.org/gitweb?p=landing-teams/leb/ti/kernel.git%3Ba=summary
This is a recap of the discussion we did about source meta data for packages. While interesting it was fruitless, nothing has happened. I'd like to propose to put lava-friendly meta data in: /usr/share/lava/{srcpackagename}/source.json
The file should look like this (for bzr-based projects)
{ "branch_revision": 93556, "branch_url": "lp:gcc-linaro/4.4", "branch_vcs": "bzr", "commit_timestamp": "2010-09-07T14:49:43Z", "project_name": "linaro-gcc" }
And like this for git-based projects
{ "branch_revision": "69f19fbedf6b88eb314b22f1263e2624d4477ac8", "branch_url": "git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git", "branch_vcs": "git", "commit_timestamp": "2011-11-23T14:15:50Z", "project_name": "linux" }
If needed we can add support for svn (although I'd prefer to work with a bzr-svn import on launchpad in that case). The field commit_timestamp can be removed. It is optional and can be resolved automatically assuming the repository is still alive. Still it is highly recommended to include this information directly. The field project_name should refer to a launchpad project (when using bzr branches) or to a consistently-named upstream project (otherwise).
I'm not saying this will help to solve OPs problem but at least it will feed LAVA with loads of meta-information about source that runs on the tested boards. If someone likes this format it could be adopted by other tools.
Best regards Zygmunt Krynicki
On Tue, Jan 24, 2012 at 4:01 PM, Christian Robottom Reis kiko@linaro.org wrote:
On Tue, Jan 24, 2012 at 12:59:33PM -0200, Christian Robottom Reis wrote:
On Tue, Jan 24, 2012 at 09:29:34AM -0500, Chris Lalancette wrote:
(I'm aware that there is a thread on linaro-dev discussing this exact topic; this is a request for specific information, so I decided to start a new thread)
Hello, As has been pointed out elsewhere, it is very difficult to find the exact git tree that corresponds to a kernel release. Currently the problem I am having is that the 11.11 linaro kernel release (linux-linaro-lt-omap_3.1.0-1402.5~oneiric1) works well on my new board, but later kernels do not. While I can download the kernel tarball for 3.1.0-1402.5 from launchpad, I would much prefer to use the git tree that it was produced from. Can anybody tell me exactly which git tree was used to create that kernel, and which tag/branch I should be looking at?
That's a really good question. The answer is that it's this tag and branch:
linux-release-2011-12 http://git.linaro.org/git/landing-teams/leb/ti/kernel.git
Sorry, for 11.11 that's tag linux-release-2011-11-1 -- you can see all the tags here:
http://git.linaro.org/gitweb?p=landing-teams/leb/ti/kernel.git%3Ba=summary
Christian Robottom Reis, Engineering VP Brazil (GMT-3) | [+55] 16 9112 6430 | [+1] 612 216 4935 Linaro.org: Open Source Software for ARM SoCs
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On Tue, Jan 24, 2012 at 04:19:22PM +0100, Zygmunt Krynicki wrote:
This is a recap of the discussion we did about source meta data for packages. While interesting it was fruitless, nothing has happened. I'd like to propose to put lava-friendly meta data in: /usr/share/lava/{srcpackagename}/source.json
On an installed system image? What happens if, like Chris, somebody's just downloading the kernel release?
I'm not saying this will help to solve OPs problem but at least it
Ah. :-)
will feed LAVA with loads of meta-information about source that runs on the tested boards. If someone likes this format it could be adopted by other tools.
I think the general idea is good but probably doesn't apply as neatly to Android as it does for Ubuntu and other package-based Linux; is that right?
On 01/24/2012 10:01 AM, Christian Robottom Reis wrote:
On Tue, Jan 24, 2012 at 12:59:33PM -0200, Christian Robottom Reis wrote:
On Tue, Jan 24, 2012 at 09:29:34AM -0500, Chris Lalancette wrote:
(I'm aware that there is a thread on linaro-dev discussing this exact topic; this is a request for specific information, so I decided to start a new thread)
Hello, As has been pointed out elsewhere, it is very difficult to find the exact git tree that corresponds to a kernel release. Currently the problem I am having is that the 11.11 linaro kernel release (linux-linaro-lt-omap_3.1.0-1402.5~oneiric1) works well on my new board, but later kernels do not. While I can download the kernel tarball for 3.1.0-1402.5 from launchpad, I would much prefer to use the git tree that it was produced from. Can anybody tell me exactly which git tree was used to create that kernel, and which tag/branch I should be looking at?
That's a really good question. The answer is that it's this tag and branch:
linux-release-2011-12 http://git.linaro.org/git/landing-teams/leb/ti/kernel.gitSorry, for 11.11 that's tag linux-release-2011-11-1 -- you can see all the tags here:
http://git.linaro.org/gitweb?p=landing-teams/leb/ti/kernel.git;a=summary
Thanks, that is enormously helpful. Though doing a "diff -Nurp" between leb/ti/kernel.git (tag linux-release-2011-11-1) and the tarball that I pulled from launchpad (https://launchpad.net/~linaro-maintainers/+archive/overlay/+files/linux-lina...), these two trees aren't exactly the same. They are *mostly* the same, and most of the changes look like they'll be benign to me, but it is a bit disconcerting.
In terms of finding things in the future, I have to say that there is a bit of a forest of git trees in linaro. At the very least, I would make sure that the .dsc file in the released deb points to the correct tree+tag that it was generated from. Beyond that, I would recommend:
1) Attempt to reduce the number of trees on git.linaro.org. I understand that there is probably a lot going on, but the sheer number of trees makes it confusing. It might be a good idea to remove some of the very stale or no longer active trees.
2) Document on the wiki where the releases are built from, so there is a running record per release
Thanks again,
On 01/24/2012 09:50 AM, Chris Lalancette wrote:
Thanks, that is enormously helpful. Though doing a "diff -Nurp" between leb/ti/kernel.git (tag linux-release-2011-11-1) and the tarball that I pulled from launchpad (https://launchpad.net/~linaro-maintainers/+archive/overlay/+files/linux-lina...), these two trees aren't exactly the same. They are *mostly* the same, and most of the changes look like they'll be benign to me, but it is a bit disconcerting.
I think the difference you see here is because linux-linaro-lt-omap_3.1.0-1402.5%7Eoneiric1.tar.gz includes debian packaging and "ubuntu sauce" patches. I believe those come from:
http://git.linaro.org/gitweb?p=ubuntu/linux-linaro-oneiric.git;a=summary
However, I don't know how you find the proper commit in that repo that correlates to the tarball.
On Tue, Jan 24, 2012 at 10:50:51AM -0500, Chris Lalancette wrote:
In terms of finding things in the future, I have to say that there is a bit of a forest of git trees in linaro. At the very least, I would make sure that the .dsc file in the released deb points to the correct tree+tag that it was generated from.
That's not a bad idea. John, can we make that happen?
- Attempt to reduce the number of trees on git.linaro.org. I
understand that there is probably a lot going on, but the sheer number of trees makes it confusing. It might be a good idea to remove some of the very stale or no longer active trees.
What do Deepak and Loïc think of this in general?
- Document on the wiki where the releases are built from, so there
is a running record per release
Where would we record this, Deepak, Alexander, Loïc?
On Wed, Jan 25, 2012 at 7:14 PM, Christian Robottom Reis kiko@linaro.orgwrote:
- Attempt to reduce the number of trees on git.linaro.org. I
understand that there is probably a lot going on, but the sheer number of trees makes it confusing. It might be a good idea to remove some of the very stale or no longer active trees.
What do Deepak and Loïc think of this in general?
fwiw, i like the 'structured' layout used here http://arago-project.org/git/. you can keep the really critical/important trees on the top level, and then create sub-domains for 'people', 'android', ...
On Wed, Jan 25, 2012 at 04:14:10PM -0200, Christian Robottom Reis wrote:
On Tue, Jan 24, 2012 at 10:50:51AM -0500, Chris Lalancette wrote:
In terms of finding things in the future, I have to say that there is a bit of a forest of git trees in linaro. At the very least, I would make sure that the .dsc file in the released deb points to the correct tree+tag that it was generated from.
That's not a bad idea. John, can we make that happen?
- Attempt to reduce the number of trees on git.linaro.org. I
understand that there is probably a lot going on, but the sheer number of trees makes it confusing. It might be a good idea to remove some of the very stale or no longer active trees.
What do Deepak and Lo?c think of this in general?
- Document on the wiki where the releases are built from, so there
is a running record per release
Where would we record this, Deepak, Alexander, Lo?c?
For Ubuntu, I think the (only) correct way to track this is to represent linaro releases as actual releases in the package archive.
i.e., sources.list will look like
deb http://ports.ubuntu.com/ubuntu-ports precise main deb http://ppa.launchpad.net/linaro-maintainers/overlay linaro-12.02 main
And http://ppa.launchpad.net/linaro-maintainers/overlay/ubuntu/dists/linaro-12.0... and the things it points to would then be authoritative definition of what was in that release for evermore (or at least, the GPL-requisite 3 years).
Once we have this, we can document robustly how to find out, because the means to do so will be stable and not constantly changing.
Do you know if this is sensible/doable, or is launchpad very tied the ubuntu release names?
Cheers ---Dave
(dropping linaro-kernel@)
On Thu, Jan 26, 2012, Dave Martin wrote:
deb http://ports.ubuntu.com/ubuntu-ports precise main deb http://ppa.launchpad.net/linaro-maintainers/overlay linaro-12.02 main
And http://ppa.launchpad.net/linaro-maintainers/overlay/ubuntu/dists/linaro-12.0... and the things it points to would then be authoritative definition of what was in that release for evermore (or at least, the GPL-requisite 3 years).
[...]
Do you know if this is sensible/doable, or is launchpad very tied the ubuntu release names?
It's not possible with the current web UI of PPAs, but the Launchpad code is flexible in this area and it would likely be possible to do things like that, albeit it would be a bit weird to combine "distros" together (precise from Ubuntu with linaro-12.02 from Linaro). Launchpad gained over the last year improved support for "derivatives" but that means "forking" Ubuntu for Linaro and then constantly merging from it, having our own archive etc.; the only entry would be: deb http://archive.linaro.org linaro-12.02 main
This is something we envisioned doing in the early stages of Linaro, but Launchpad wasn't ready and it was costly to maintain, now it would be costly to switch; perhaps the incentive is strong enough though.
Another way to solve this could be to defer to some automated service for publishing our bits rather than uploading them ourselves; for instance we would point a robot at the git repo where the TI LT kernel for release is kept, and it would be assembling a source package from it and publishing that to some Vcs and to the release PPA. That way we would ensure we always now where we took the source from, how it was built etc. and can enforce any rule we like. This would be comparable to the automated merges of some bzr branches in Launchpad where a robot tries running the testsuite and then merges the branch in tip if that passes, but humans never push to tip.
Availibility of the original (git or not) source is a long standing problem which is partly due to the PPA rules (which are dropping some sources after being superseded for a while) and partly to relying on humans to document things like Vcs-Git properly.
On Wed, Jan 25, 2012 at 7:14 PM, Christian Robottom Reis kiko@linaro.org wrote:
On Tue, Jan 24, 2012 at 10:50:51AM -0500, Chris Lalancette wrote:
- Attempt to reduce the number of trees on git.linaro.org. I
understand that there is probably a lot going on, but the sheer number of trees makes it confusing. It might be a good idea to remove some of the very stale or no longer active trees.
What do Deepak and Loďc think of this in general?
I would propose to: * hide people/ trees and show them on people.git.linaro.org instead of main git.linaro.org. At best git:// urls would be valid still aftterwards * hide the android/ trees and don't show them anywhere else; keep git:// urls and/or http:// clone urls wprking too
This would definitely air to breath and we can continue to improve things from there.
- Document on the wiki where the releases are built from, so there
is a running record per release
Where would we record this, Deepak, Alexander, Loďc?
I am thinking ...
If it's about documenting released bits it's something that should come out of our release process. The release team can bring that together. For that the information needs to be shipped along- or inside the source artifact. As one example, ubuntu packages having a vcs-bzr field and a reasonable tagging/versioning scheme could probably work, but we have to look case by case.
Where are we putting it? One place that comes to my mind is obviously the release wiki page: https://wiki.linaro.org/Cycles/1201/Release/?
2012/1/27 Alexander Sack asac@linaro.org:
On Wed, Jan 25, 2012 at 7:14 PM, Christian Robottom Reis kiko@linaro.org wrote:
On Tue, Jan 24, 2012 at 10:50:51AM -0500, Chris Lalancette wrote:
- Attempt to reduce the number of trees on git.linaro.org. I
understand that there is probably a lot going on, but the sheer number of trees makes it confusing. It might be a good idea to remove some of the very stale or no longer active trees.
What do Deepak and Loďc think of this in general?
I would propose to: * hide people/ trees and show them on people.git.linaro.org instead of main git.linaro.org. At best git:// urls would be valid still aftterwards * hide the android/ trees and don't show them anywhere else; keep git:// urls and/or http:// clone urls wprking too
This would definitely air to breath and we can continue to improve things from there.
- Document on the wiki where the releases are built from, so there
is a running record per release
Where would we record this, Deepak, Alexander, Loďc?
I am thinking ...
If it's about documenting released bits it's something that should come out of our release process. The release team can bring that together. For that the information needs to be shipped along- or inside the source artifact. As one example, ubuntu packages having a vcs-bzr field and a reasonable tagging/versioning scheme could probably work, but we have to look case by case.
Where are we putting it? One place that comes to my mind is obviously the release wiki page: https://wiki.linaro.org/Cycles/1201/Release/?
Another one is https://wiki.linaro.org/Cycles/ProjectsSummary I'd like to generate this table automatically to keep it up-to-date, based on inputs from packages meta data (using additional user-defined fields to debian/control file).
On Fri, Jan 27, 2012, Alexander Sack wrote:
- hide people/ trees and show them on people.git.linaro.org instead
of main git.linaro.org. At best git:// urls would be valid still aftterwards
- hide the android/ trees and don't show them anywhere else; keep
git:// urls and/or http:// clone urls wprking too
This would definitely air to breath and we can continue to improve things from there.
The arago-project layout seems nicer, effectively just hiding subtrees rather than using separate DNSes; also, they are linked from the main repos to the other ones, whereas if we create a people.git.linaro.org, how will people discover it? It's best to avoid having 3 places/hostnames for people to host stuff.
As one example, ubuntu packages having avcs-bzr field and a reasonable tagging/versioning scheme could probably work, but we have to look case by case.
Problem is that a Vcs-* field is not consistently applied :-/ Sometimes it's not set, or doesn't point at the right repo, or points at a repo with a special tree where people have to do magic to get things done.
On Fri, Jan 27, 2012 at 11:40 AM, Loïc Minier loic.minier@linaro.org wrote:
On Fri, Jan 27, 2012, Alexander Sack wrote:
* hide people/ trees and show them on people.git.linaro.org instead of main git.linaro.org. At best git:// urls would be valid still aftterwards * hide the android/ trees and don't show them anywhere else; keep git:// urls and/or http:// clone urls wprking too
This would definitely air to breath and we can continue to improve things from there.
The arago-project layout seems nicer, effectively just hiding subtrees
Yeah. That makes sense. Happy if we get that.
As one example, ubuntu packages having a vcs-bzr field and a reasonable tagging/versioning scheme could probably work, but we have to look case by case.
Problem is that a Vcs-* field is not consistently applied :-/
Right. But I don't think it matters much, no?. Release Team will work with folks to sort out the details. And if they decide that release team will track Vcs- fields for mining the source info for ubuntu packages, we probably would ensure that it's consistently applied for the packages we care about :).
Sometimes it's not set, or doesn't point at the right repo, or points at a repo with a special tree where people have to do magic to get things done.
I see three main problems to solve:
1. How do I find the vcs component for this source component? 2. How can I hack on this tree and how does this tree work? 3. What trees/tags/components are in the LEB?
Point 1. can be solved through the release team working with component owners and sourcing the location of the tree used to produce a component/package
Point 2. can be solved by the tree owner through maintaining good documentation on the git repo main gitweb page that explains in simple instructions how things work. For example:
a. how the trees are produced b. what tag/branch syntax rules apply c. how to hack and contribute to this tree d. how to participate in meetings where this baseline is on topic to discuss....
Point 3. could be addressed by providing an explicit list of integrated/improved components alongside the LEB (on top of the usual android repo info etc.) to make this easier.
On Tue, Jan 24, 2012 at 10:50:51AM -0500, Chris Lalancette wrote:
On 01/24/2012 10:01 AM, Christian Robottom Reis wrote:
On Tue, Jan 24, 2012 at 12:59:33PM -0200, Christian Robottom Reis wrote:
On Tue, Jan 24, 2012 at 09:29:34AM -0500, Chris Lalancette wrote:
(I'm aware that there is a thread on linaro-dev discussing this exact topic; this is a request for specific information, so I decided to start a new thread)
Hello, As has been pointed out elsewhere, it is very difficult to find the exact git tree that corresponds to a kernel release. Currently the problem I am having is that the 11.11 linaro kernel release (linux-linaro-lt-omap_3.1.0-1402.5~oneiric1) works well on my new board, but later kernels do not. While I can download the kernel tarball for 3.1.0-1402.5 from launchpad, I would much prefer to use the git tree that it was produced from. Can anybody tell me exactly which git tree was used to create that kernel, and which tag/branch I should be looking at?
That's a really good question. The answer is that it's this tag and branch:
linux-release-2011-12 http://git.linaro.org/git/landing-teams/leb/ti/kernel.gitSorry, for 11.11 that's tag linux-release-2011-11-1 -- you can see all the tags here:
http://git.linaro.org/gitweb?p=landing-teams/leb/ti/kernel.git;a=summaryThanks, that is enormously helpful. Though doing a "diff -Nurp" between leb/ti/kernel.git (tag linux-release-2011-11-1) and the tarball that I pulled from launchpad (https://launchpad.net/~linaro-maintainers/+archive/overlay/+files/linux-lina...), these two trees aren't exactly the same. They are *mostly* the same, and most of the changes look like they'll be benign to me, but it is a bit disconcerting.
Here was my outline for a possible way to fix this.
http://lists.linaro.org/pipermail/linaro-dev/2012-January/009694.html
Do you have any comments on it? Your views are valuable, since you've experienced the problem we're trying to solve.
Cheers ---Dave
-
Alexander Sack -
Andy Doan -
Chris Lalancette -
Christian Robottom Reis -
Dave Martin -
Dechesne, Nicolas -
Fathi Boudra -
Loïc Minier -
Zygmunt Krynicki