On Thu, 2 Jun 2011 10:24:12 +0300, Paul Sokolovsky paul.sokolovsky@linaro.org wrote:
Hello,
There was weird problem with android git mirror again (https://android-build.linaro.org/jenkins/job/patrik-ryd_lt-panda/4/consoleFu...):
error: revision tilt-linaro-android.38 in git.linaro.org/people/andygreen/kernel-tilt not found Archiving artifacts Finished: FAILURE
This was 100% reproducible, but that revision exists in the git repository on git.linaro.org, and actually it's being mirrored, as
git clone git://us-east-1.ec2-git-mirror.linaro.org/git.linaro.org/people/andygreen/kernel-tilt
Worked pretty well. But "repo sync" gave the error above, which well, pointed at some repo peculiarity.
There were similar issues already, with newly added git repos which lacked master branch. Adding master branch seemed to solve the issue. But kernel-tilt had master branch.
So, I for some time suspected that repo --mirror we use for mirroring doesn't make faithful enough mirror of git forest, and this time decided to dig out what is known about it.
First of all, while there's no --verbose or --debug switch for the repo, it's possible to make it dump git commands it executes by setting "REPO_TRACE=1" in the environment. I verified that very same git command, when run directly against git.linaro.org, works well, but leads to repo error quoted above if run against mirror.
I then googled for known issues with repo --mirror, couldn't find much, but there're few reports of obscure issues:
http://groups.google.com/group/repo-discuss/browse_thread/thread/b9752498836...
This is even more interesting, with Google people confirming that repo --mirror works within some bounds and limitations:
http://groups.google.com/group/repo-discuss/browse_thread/thread/401656c3ad0...
Finally, here's typical usage scenario for repo --mirror (which is pretty underdocumented):
http://www.excentral.org/archives/2011/02/24/android-repo-mirroring
So, like "repo help init" says, mirror created with repo --mirror should later be used with repo --reference, with both mirror and checkout using it to be located on local filesystem. I couldn't find referenced that repo --mirror is intended for use like we do - to serve it using git-daemon.
My next step was trying to do real git mirror (git clone --mirror), and already did it and prepared to move it into intended location, when I did ls -l in the mirror directory:
drwxr-xr-x 7 git-mirror nogroup 4096 2011-06-02 00:11 kernel-tilt.git drwxr-xr-x 7 git-mirror nogroup 4096 2011-06-01 12:30 kernel-tilt.git.git
!
Once I moved that ".git.git" repo away, original issue with repo sync resolved. I have no idea that double-.git repo was created. Based on the timestamp, it was created first, and later barely updated, with mirror and just git clone using the proper repo, while repo sync on the build slave managing to pick up wrong one. I wonder, if that (creation of 2 repos) could be a result of some race condition.
I bet it is.
And I wish this issue could be called resolved, but as quoted above, there're (more) issues are known with git --mirror, so I decided to put up this mail with the links, for the next time such issues may come.
As you pointed out to me fairly soon after you started, we don't actually need to run "repo sync" on the server at all -- it's not hard to figure out the "git clone" commands we need to run, and so maybe we (well, you :-p) should just do that.
Cheers, mwh