On Tue, Jan 24, 2012 at 11:34:51AM +0000, Lee Jones wrote:
All released kernel source is available on git.linaro.org.
Specifically: Git: git://git.linaro.org/landing-teams/leb/<landing_team>/kernel.git HTML: http://git.linaro.org/git/landing-teams/leb/<landing_team>/kernel.git Gitweb: http://git.linaro.org/gitweb?p=landing-teams/leb/<landing_team>/kernel.git;a=summary
The Gitweb has a description of what you can expect from each tag.
Good, but I fear it's not enough.
The question was never whether the source exists somewhere in infospace, but whether a linaro consumer can find it and make use of it. Lets see how they get on in this typical scenario:
Suppose we have a board with linaro that we set up a while ago. We are experiencing some issues, and want to turn on some instrumentation in the kernel and/or add some patches of their own. This means we would like to reconfigure/change and rebuild the kernel that we have installed.
1) We know how to get source code on Ubuntu: run apt-get source
But, that specific package version may have disappeared from the server (Fixable by running apt-get upgrade, but then we get more problems ... see (4))
or
We get some source, but it's not for the expected kernel version (See http://ppa.launchpad.net/linaro-landing-team-samsung/linaro-samsung/ubuntu/p... It's actuall a 3.0.4 kernel. This may be a local mispackaging problem, but it's still wrong.)
or
We get some source, and it's the right version, but it's not much use to us because we might have some git branches we want to merge before building, so we really want to see the git repo, not a tarball.
2) apt-get source told us this package was in Git (via the Vcs-Git tag) at git://git.linaro.org/jcrigby/linux-linaro.git
But there is no such tree.
3) Feeling lucky, we fish about on git.linaro.org for alternatives
There's a bewildering variety of possible kernel trees for every platform:
e.g., for samsung:
http://git.linaro.org/gitweb?p=landing-teams/leb/samsung/kernel.git%3Ba=summ... http://git.linaro.org/gitweb?p=landing-teams/working/samsung/kernel.git%3Ba=... http://git.linaro.org/gitweb?p=people/angus/linux-samsung.git%3Ba=summary http://git.linaro.org/gitweb?p=people/tushar/linux-linaro-samsung.git%3Ba=su...
How do we know which one to look at?
I look at one with a tag matching my kernel version. I find none, except in generic trees that don't appear to be connected with my platform.
4) OK, we received a rumour about where the actual correct tree is, or just made a lucky guess let's try:
http://git.linaro.org/gitweb?p=landing-teams/leb/samsung/kernel.git%3Ba=summ...
The tree is nicely tagged with linaro releases.
But, we have no idea which release the installed kernel relates to, because we a) forgot, or b) ran apt-get upgrade to solve a disappeared- source problem in step (1), in which case the correspondence between installed packages and linaro releases has become indeterminate -- we are not linaro/ubuntu/debian experts, so we don't realise this is the case, but will just spend time searching for wrong things.
After somehow finding the kernel source for the release we installed and finding it doesn't match because we upgraded at some point...
...we only have the package name and version number to go on, because that's what debian/ubuntu packages have to identify them. Packages are not intrinsically mapped to a release; only the converse is true.
But, there is nothing in the git repository or package metadata to indicate how binary package versions map back to releases.
5) OK, we guessed or were told the correct release tag, and proceed to rebuild the binary packages.
The debian/ directory isn't in any of these repositories, so we can't build binary packages. Neither the build scripts or the kernel config are there.
To finally build valid binary packages, we need the appropriately- tagged debian*/ directories. And not just any ones, but those used for the release.
The chance of a linaro consumer making it from start to end of that process without falling over is minimal, and in my experience, it doesn't happen.
I think (hopefully) that my proposal robustly fixes this, but if it can be achieved more easily, I'm eager to know how.
Cheers ---Dave
On 24/01/12 10:18, Dave Martin wrote:
Hi again,
After a report of yet another instance of un-findable source for a kernel released from a landing team, it would be good if we could move forward with this.
Does anyone have any significant disagreements with the proposal below?
If not, I can try to write up a formal specification somewhere.
Can we then plan to implement it?
If anyone has any preference for the common prefix for tag names, please speak up (otherwise we will proceed with the "ubuntu/" prefix).
Cheers ---Dave
On Fri, Jan 13, 2012 at 01:32:35PM +0000, Dave Martin wrote:
For everyone who packages kernel trees:
I've had some questions about getting the source for linaro kernel packaged, and it seems that this is still not straightforward:
Getting the debian package source (i.e., flat tarball) for a binary kernel is possible, but only if it is a non-superseded version.
Working out which source package you need is non-obvious (You have to check for installed kernel packages, and guess which source package you need, based on that. Non-linaro folks may not understand the difference between the various meta- and real kernel packages and may get pretty confused along the way.)
Finding the git tree and the relevant commit to reproduce the source (including that for superseded kernels) is hard or impossible. It requires guesswork and/or specific knowledge about the way the relevant team manages their trees. For some platforms, it looks like there may be no single tree containing the packaged kernel at all, in which case you would also need more guesswork and/or scripts or help from the relevant landing team in order actually to reproduce a release.
Am I correct in these conclusions?
If so, here's a proposal -- I welcome people's comments (and please do say if any of these problems are actually solved already!):
For every binary kernel package (.deb) publicly released by any linaro team, including those produced by the platform team and the landing teams:
The source package's control file must contain a Vcs-Git entry
The Vcs-Git entry must reference a git tree which contains the
_exact_ source code which appears in the source package.
- Such a tree must exist and must be publicly readable. It does not
have to be on git.linaro.org (though this is the recommended place).
- Referring to git://git.linaro.org/ubuntu/linux-linaro.git is not
acceptable, unless that repo is populated with the real source for that specific kernel as well as the packaging.
- Manufacturing the source package from the contents of multiple
repositories or branches at source package upload time is not acceptable, unless the result is also recorded as a tagged commit in the repository referenced by the Vcs-Git entry in the debian/control file contained in that same commit. The commit must have full history: importing tarballs directly into a repository for the purpose of release tagging is not acceptable.
- Referring to a tree which does not contain the whole contents of
the debian source package (for example, debian/ and other packaging files/dirs are missing) is not acceptable.
Note: this means that the released binary packages must be reproducible from the tagged source using standard package build mechanisms, to the extent that the exact versions of build tools and other build-time dependencies used to build the originally released binaries are still avilable.
- Tagging of packaged kernels must be done in a standard way.
- I recommend <source package name>_<package version>, matching the
Source field of debian/control and the version number of the most recent debian/changelog entry respectively (which must both be present in the repository as a consequence of (2)). If we want to avoid namespace pollution, we probably want to add a prefix such as debian/ or ubuntu/ to the tag name to indicate that the tag describes the source for a published .deb package. If so, we must standardise that prefix so that it is identical in all out trees.
- Tree maintainers are of course free to add any other additional
tags for their own use if they want to.
All teams already do release tagging of some sort, but the lack of consistency creates difficulties when anyone from outside the team tries to understand that team's trees.
We _could_ standardise the following, but it is not essential:
- ubuntu/<release>: The tagged source for the _original_ kernel
which was distributed in <release> (where <release> is a linaro monthly release such as 11.12 or 12.01)
- No specific branch naming requirements exist.
Release tags do not necessarily need to appear on any branch.
We _could_ standardise the following, but it is not essential:
- ubuntu/latest - the tagged packaged source for the most recent
kernel release made from this tree
(In the above, we could choose a diferent prefix instead of ubuntu/, but as described in (3) ,this should be chosen globally and _not_ on a per-tree basis).
Cheers ---Dave
-- Lee Jones Linaro ST-Ericsson Landing Team Lead M: +44 77 88 633 515 Linaro.org ??? Open source software for ARM SoCs Follow Linaro: Facebook | Twitter | Blog