On 24 January 2012 13:32, Dave Martin <dave.martin@linaro.org> wrote:
On Tue, Jan 24, 2012 at 11:34:51AM +0000, Lee Jones wrote:
> All released kernel source is available on git.linaro.org.
>
> Specifically:
>   Git:
>     git://git.linaro.org/landing-teams/leb/<landing_team>/kernel.git
>   HTML:
>     http://git.linaro.org/git/landing-teams/leb/<landing_team>/kernel.git
>   Gitweb:
>     http://git.linaro.org/gitweb?p=landing-teams/leb/<landing_team>/kernel.git;a=summary
>
> The Gitweb has a description of what you can expect from each tag.


Good, but I fear it's not enough.

The question was never whether the source exists somewhere in
infospace, but whether a linaro consumer can find it and make use of it.
Lets see how they get on in this typical scenario:


Suppose we have a board with linaro that we set up a while ago.
We are experiencing some issues, and want to turn on some instrumentation
in the kernel and/or add some patches of their own.  This means we would
like to reconfigure/change and rebuild the kernel that we have installed.


1) We know how to get source code on Ubuntu: run apt-get source

But, that specific package version may have disappeared from the server
(Fixable by running apt-get upgrade, but then we get more problems ...
see (4))

       or

We get some source, but it's not for the expected kernel version
(See http://ppa.launchpad.net/linaro-landing-team-samsung/linaro-samsung/ubuntu/pool/main/l/linux-origen/linux-origen_3.0.0-1000.0samsung22.tar.gz
It's actuall a 3.0.4 kernel.  This may be a local mispackaging problem,
but it's still wrong.)

       or

We get some source, and it's the right version, but it's not much
use to us because we might have some git branches we want to merge before
building, so we really want to see the git repo, not a tarball.


2) apt-get source told us this package was in Git (via the Vcs-Git tag)
at git://git.linaro.org/jcrigby/linux-linaro.git

But there is no such tree.


3) Feeling lucky, we fish about on git.linaro.org for alternatives

There's a bewildering variety of possible kernel trees for every platform:

e.g., for samsung:

http://git.linaro.org/gitweb?p=landing-teams/leb/samsung/kernel.git;a=summary
http://git.linaro.org/gitweb?p=landing-teams/working/samsung/kernel.git;a=summary
http://git.linaro.org/gitweb?p=people/angus/linux-samsung.git;a=summary
http://git.linaro.org/gitweb?p=people/tushar/linux-linaro-samsung.git;a=summary

How do we know which one to look at?

I look at one with a tag matching my kernel version.  I find none, except
in generic trees that don't appear to be connected with my platform.


4) OK, we received a rumour about where the actual correct tree is,
or just made a lucky guess let's try:

http://git.linaro.org/gitweb?p=landing-teams/leb/samsung/kernel.git;a=summary

The tree is nicely tagged with linaro releases.

But, we have no idea which release the installed kernel relates to,
because we a) forgot, or b) ran apt-get upgrade to solve a disappeared-
source problem in step (1), in which case the correspondence between
installed packages and linaro releases has become indeterminate --
we are not linaro/ubuntu/debian experts, so we don't realise this is
the case, but will just spend time searching for wrong things.

After somehow finding the kernel source for the release we installed
and finding it doesn't match because we upgraded at some point...

...we only have the package name and version number to go on, because
that's what debian/ubuntu packages have to identify them.  Packages are
not intrinsically mapped to a release; only the converse is true.

But, there is nothing in the git repository or package metadata to
indicate how binary package versions map back to releases.


5) OK, we guessed or were told the correct release tag, and proceed to
rebuild the binary packages.

The debian/ directory isn't in any of these repositories, so we
can't build binary packages.  Neither the build scripts or the kernel
config are there.


To finally build valid binary packages, we need the appropriately-
tagged debian*/ directories.  And not just any ones, but those used
for the release.


The chance of a linaro consumer making it from start to end of that
process without falling over is minimal, and in my experience, it
doesn't happen.

I think (hopefully) that my proposal robustly fixes this, but if it
can be achieved more easily, I'm eager to know how.

For Android we have:

https://android-build.linaro.org/builds/~linaro-android/panda-12.01-release/

we should have the same thing for Ubuntu:

ubuntu-build.linaro.org

with the similar information.

 

Cheers
---Dave


> On 24/01/12 10:18, Dave Martin wrote:
> > Hi again,
> >
> > After a report of yet another instance of un-findable source for a
> > kernel released from a landing team, it would be good if we could move
> > forward with this.
> >
> > Does anyone have any significant disagreements with the proposal
> > below?
> >
> > If not, I can try to write up a formal specification somewhere.
> >
> > Can we then plan to implement it?
> >
> >
> > If anyone has any preference for the common prefix for tag names,
> > please speak up (otherwise we will proceed with the "ubuntu/" prefix).
> >
> > Cheers
> > ---Dave
> >
> > On Fri, Jan 13, 2012 at 01:32:35PM +0000, Dave Martin wrote:
> >> For everyone who packages kernel trees:
> >>
> >>
> >> I've had some questions about getting the source for linaro kernel
> >> packaged, and it seems that this is still not straightforward:
> >>
> >> Getting the debian package source (i.e., flat tarball) for a binary
> >> kernel is possible, but only if it is a non-superseded version.
> >>
> >> Working out which source package you need is non-obvious (You have to
> >> check for installed kernel packages, and guess which source package
> >> you need, based on that.  Non-linaro folks may not understand the
> >> difference between the various meta- and real kernel packages and may
> >> get pretty confused along the way.)
> >>
> >> Finding the git tree and the relevant commit to reproduce the source
> >> (including that for superseded kernels) is hard or impossible.  It
> >> requires guesswork and/or specific knowledge about the way the
> >> relevant team manages their trees.  For some platforms, it looks like
> >> there may be no single tree containing the packaged kernel at all, in
> >> which case you would also need more guesswork and/or scripts or help
> >> from the relevant landing team in order actually to reproduce a
> >> release.
> >>
> >> Am I correct in these conclusions?
> >>
> >>
> >> If so, here's a proposal -- I welcome people's comments (and please do
> >> say if any of these problems are actually solved already!):
> >>
> >>
> >> For every binary kernel package (.deb) publicly released by any linaro
> >> team, including those produced by the platform team and the landing
> >> teams:
> >>
> >>
> >> 1) The source package's control file must contain a Vcs-Git entry
> >>
> >>
> >> 2) The Vcs-Git entry must reference a git tree which contains the
> >> _exact_ source code which appears in the source package.
> >>
> >>  * Such a tree must exist and must be publicly readable.  It does not
> >> have to be on git.linaro.org (though this is the recommended place).
> >>
> >>  * Referring to git://git.linaro.org/ubuntu/linux-linaro.git is not
> >> acceptable, unless that repo is populated with the real source for
> >> that specific kernel as well as the packaging.
> >>
> >>  * Manufacturing the source package from the contents of multiple
> >> repositories or branches at source package upload time is not
> >> acceptable, unless the result is also recorded as a tagged commit in
> >> the repository referenced by the Vcs-Git entry in the debian/control
> >> file contained in that same commit.  The commit must have full
> >> history: importing tarballs directly into a repository for the purpose
> >> of release tagging is not acceptable.
> >>
> >>  * Referring to a tree which does not contain the whole contents of
> >> the debian source package (for example, debian/ and other packaging
> >> files/dirs are missing) is not acceptable.
> >
> > Note: this means that the released binary packages must be reproducible
> > from the tagged source using standard package build mechanisms, to the
> > extent that the exact versions of build tools and other build-time
> > dependencies used to build the originally released binaries are still
> > avilable.
> >
> >>
> >>
> >> 3) Tagging of packaged kernels must be done in a standard way.
> >>
> >>  * I recommend <source package name>_<package version>, matching the
> >> Source field of debian/control and the version number of the most
> >> recent debian/changelog entry respectively (which must both be present
> >> in the repository as a consequence of (2)).  If we want to avoid
> >> namespace pollution, we probably want to add a prefix such as debian/
> >> or ubuntu/ to the tag name to indicate that the tag describes the
> >> source for a published .deb package.  If so, we must standardise that
> >> prefix so that it is identical in all out trees.
> >>
> >>  * Tree maintainers are of course free to add any other additional
> >> tags for their own use if they want to.
> >>
> >> All teams already do release tagging of some sort, but the lack of
> >> consistency creates difficulties when anyone from outside the team
> >> tries to understand that team's trees.
> >>
> >>  * We _could_ standardise the following, but it is not essential:
> >>
> >>     * ubuntu/<release>: The tagged source for the _original_ kernel
> >> which was distributed in <release> (where <release> is a linaro
> >> monthly release such as 11.12 or 12.01)
> >>
> >>
> >> 4) No specific branch naming requirements exist.
> >>
> >>  * Release tags do not necessarily need to appear on any branch.
> >>
> >>  * We _could_ standardise the following, but it is not essential:
> >>
> >>     * ubuntu/latest - the tagged packaged source for the most recent
> >> kernel release made from this tree
> >>
> >>
> >> (In the above, we could choose a diferent prefix instead of ubuntu/,
> >> but as described in (3) ,this should be chosen globally and _not_ on a
> >> per-tree basis).
> >>
> >> Cheers
> >> ---Dave
>
>
> --
> Lee Jones
> Linaro ST-Ericsson Landing Team Lead
> M: +44 77 88 633 515
> Linaro.org ??? Open source software for ARM SoCs
> Follow Linaro: Facebook | Twitter | Blog

_______________________________________________
linaro-dev mailing list
linaro-dev@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-dev



--
Zach Pfeffer
Android Platform Team Lead, Linaro Platform Teams
Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://www.facebook.com/pages/Linaro
http://twitter.com/#!/linaroorg - http://www.linaro.org/linaro-blog