##- Please type your reply above this line -##

[IT Services] Update: Re: Daily builds are failing during repo init

You are registered as a CC on this support request (linaroithelp.zendesk.com/requests/109). Reply to this email to add a comment to the request.

----------------------------------------------

Paul Sokolovsky, May 20 08:37 (ADT)

Ben:

1) What could be explanation of need for lock - that script doesn't do any context-sensitive processing. 2) We should try to get rid of it at all

----------------------------------------------

Paul Sokolovsky, May 20 08:30 (ADT)

Vishal: this ticket is for issue reported by Jon, and other http access hiccups as confirmed by me.

"branch 'stable' has not been signed" is separate and tracked at https://bugs.launchpad.net/linaro-android-infrastructure/+bug/1181907

----------------------------------------------

Ben Copeland, May 20 08:22 (ADT)

Hi Paul,

I'm not too sure what I can do to speed up the http process. The bottleneck is down to the python script, We have had to put add "RewriteLock" to Apache otherwise users are getting these clone errors. The downside of this is it means it is preventing parallel requests.

When I start to clone http://android.git.linaro.org/git-ro/platform/manifest.git, watching apache log there are many 404 links "GET /git-ro/platform/manifest.git/objects/75/0313d9ddb35959103d75ff8c450f4d7b6596d9 HTTP/1.1" 404 506 "-" "git/1.8.1.2" at the start. These do stop, but its going process each http request per file, its going to be a lengthy wait along with this process lock.

Ben

----------------------------------------------

Vishal Bhoj, May 20 07:59 (ADT)

I still face the same issue which happens on android-build with this command: repo init -u http://android.git.linaro.org/git-ro/platform/manifest.git -b linaro_android_4.2.2 -m default.xml --repo-url=http://android.git.linaro.org/git-ro/tools/repo

Get http://android.git.linaro.org/git-ro/tools/repo/clone.bundle

fatal: branch 'stable' has not been signed

----------------------------------------------

Paul Sokolovsky, May 20 07:54 (ADT)

Reprod info for the issue I see:

$ time git clone git://android.git.linaro.org/platform/manifest.git [] real 0m7.706s

$ time git clone http://android.git.linaro.org/git-ro/platform/manifest.git [] real 14m17.214s

----------------------------------------------

Jon Medhurst (Tixy), May 20 07:53 (ADT)

I'm getting no problems now, thanks.

I had already synced up my local system using the git protocol, so I restored my Android directory from my Friday backup and did a repo sync. Dozens of git directories were updated by http:// URLs without any issues.

----------------------------------------------

Ben Copeland, May 20 07:34 (ADT)

Hello Jon,

We had some issues with the git server regarding the git-ro directory. I have made a similar change to the apache config which hopefully is linked to your problem.

Can you try again and let us know if you are still getting problems?

Thanks

Ben

----------------------------------------------

Paul Sokolovsky, May 20 07:34 (ADT)

All in all, I can confirm that there're issues with http:// access on android.git.linaro.org after migration - plain repo init takes very long, and I see errors like:

error: Unable to get pack index http://android.git.linaro.org/git-ro/platform/manifest.git/objects/pack/pack...

error: Unable to get pack index http://android.git.linaro.org/git-ro/platform/manifest.git/objects/pack/pack...

error: Unable to get pack index http://android.git.linaro.org/git-ro/platform/manifest.git/objects/pack/pack...

(Those are non-deterministic, re-trying them manually works.)

----------------------------------------------

Jon Medhurst (Tixy), May 20 07:10 (ADT)

On Mon, 2013-05-20 at 12:43 +0300, Paul Sokolovsky wrote:

Hello Jon,

On Mon, 20 May 2013 09:35:00 +0100 "Jon Medhurst (Tixy)" tixy@linaro.org wrote:

...

On Mon, 2013-05-20 at 09:15 +0530, Vishal Bhoj wrote:

...

Android builds are failing in repo init phase. I have opened a bug for the same: https://bugs.launchpad.net/bugs/1181907

repo init and sync worked for me locally. Can someone please have a look.

I get several errors like this, with various repos... error: Unable to find 51bd0a1525c7a722ad64a8954d90c6df0b8546a7 under http://android.git.linaro.org/git-ro/platform/ndk Cannot obtain needed blob 51bd0a1525c7a722ad64a8954d90c6df0b8546a7 while processing commit 7b3bda5d569a69c4f9b87225c5d21da4a41bf12e. error: Fetch failed.

This is different issue from one report by Vishal.

Oops, yes. I didn't read the bug, sorry. I jumped to conclusions after seeing Vishal and Bero's emails appeared at about the same time.

...

As I said in reply to bero's mail, I always seem to get these since the move to http URLs

Move to HTTP happened last couple of days during LCA13 in Hong Kong, do you want to say that you experienced the error as quoted above since then? Was it reported to some bug/ticket? (Do you have a reference?)

I didn't raise a bug as I had a workaround and I assumed if it was a widespread problem then it would get sorted out eventually.

The situation known to me is:

1. After flipping the switch in HK, git:// became unstable (because it

was throttled to stop wasting resources as it always did). It took some time to migrate everyone to http://.

If my memory serves me right, it wasn't long after this I noticed my problems, which is why it occurred to me to change the URLs in the manifests back to the old git:// (They'll get restored to http when the manifests are changed, which is why I notice it as a continuing problem.)

I perhaps only do a repo sync a couple of times a month as I don't really work much on Android, which is perhaps why a) I see this problem? b) I've not been motivated to get to the bottom of it.