On Thu, Aug 28, 2014 at 2:51 PM, Paul Sokolovsky
<paul.sokolovsky@linaro.org> wrote:
> On Thu, 28 Aug 2014 10:28:06 -0700
> John Stultz <john.stultz@linaro.org> wrote:
>
>> On Thu, Aug 28, 2014 at 10:05 AM, Paul Sokolovsky
>> <paul.sokolovsky@linaro.org> wrote:
>> So this is problematic, because there are folks out there in theWell, I can try to ping the users I know of, but the problematic spot
>> community who already use the git:// urls for fetching work from the
>> Linaro repos. (The 0day build/test bot, for instance..).
>
> So, it would be nice if they updated to use http://. We actually can be
> proactive and contact them regarding this change (we could use your
> help with that, or just with compiling list of parties who should
> be contacted).
is users I am not aware of.
So moving from what we had before to gitolite broke the git URLs that
>> We already went through one painful transition where our URLs got
>> scrambled, and I've had a few situations where folks have just
>> recently realized that we still had trees, but the URLs were just
>> different. So its quite frustrating to have to go through that again.
>
> I'm not sure which transition you mean, but the matter of deprecating
existed previously. I know folks tried to add compatibility urls, but
those URLs were slightly different then what we had previously. So
folks who were pulling regularly from our tree just stopped being able
to connect and get any updates.
Ok. I appreciate the transition. I was worried git:// url access was
> git:// and switching to http:// indeed comes not the first time. And
> previous time there were conservative (or skeptic) responses too, which
> made transition be far complete, and now we need to approach the
> same matter again, instead of having done it once and for all.
>
> But otherwise, the world is dynamic place and changes all the time. For
> example, in the summer 2011 aforementioned kernel.org was down for
> unbelievable 1.5 months, and what, people coped. But we're not going to
> do it like kernel.org and go down, but instead going to start as
> seamless as possible transition (I hope you didn't get any other idea
> from this mail). Just for that, we need some help of the users, first
> of all, internal users, as about their access and its stability we care
> the most.
about to be turned off.
Do we have a sense of who those users (IPs? which tree they are pulling?) are?
>> If we reduce the
>> git:// url load on the wort users, would that improve things enough?
>> Do you have stats on which trees are hardest hit?
>
> The case we have with git:// is that small number of users can hog
> almost all resources of a server. This can happen at release time and
> block work of Linaro engineers, something like that happened this time.
Ok. I was worried you were claiming that the git:// url might serve
>> > (*1) Unsupported in the current context means that "git://" URLs are
>> > not published in up-to-date information, and there's no warranty
>> > that any 3rd party will be able to complete a clone successfully
>> > using this protocol.
>>
>> So as someone who has sent git pull requests in the past with the git
>> urls, this is terrifying (and makes me hesitant to further use the
>> linaro infrastructure). Do you have a pointer to why the git urls
>> aren't coherent?
>
> Oh, I'm sorry for leaving ambiguity for such interpretation. I just
> meant that we are going to serve git:// to 3rd-party users on "best
> effort" basis, and apply measures to give priority to internal users.
> For example, if there's important build doing fetch, and an external
> user makes 5 (or maybe just 2) git:// connections, they may get
> connection reset. Note that this does not change status quo - for
> example, 2 days ago, *any* user who tried to connect was getting
> connection refused.
different (possibly stale) data then the https:// urls. If I was
making a pull request and someone only got half of the commits, that
would be a major infrastructure trust issue.
If its just the connections would be slow or refused, that's enough to
bother folks but not something that would make folks think or worry
our infrastructure was compromised.
So yea, kernel.org being down (and some git URLs changing) was a big
> So, kernel.org being down for 1.5 months is terrifying. Myself trying
> to build OABI toolchain and seeing all support being removed from
> everywhere, and finding aligned historical releases of toolchain
> parts almost impossible (while OABI hardware is still in use!) - that's
> terrifying. But I don't see anything terrifying with being frank
> about our git access policies and giving users a choice - either get
> reliable service with http:// or "best effort" with git://.
deal, but it was also due to a major compromise of the system, which
required a from-scratch rebuild. Hopefully things aren't (ever, we can
dream) so bad for us.
Ok. I'll try to make the change with my own uses, and I really think
removing the git:// url on the gitweb was the right call for the best
first step here (ie: stop any new users of the git urls).
My main concern was that we might break our current users just because
we had under-provisioned infrastructure. So as long as the git urls
continue to work for legacy users, I'm happy, and I can work on
changing my own uses, and trying to pester folks I know to change
their git remote urls.
Though I think having some tracking done via the connection logs and
notifying repo owners if their repo is a problem would be good in
helping get the worst offenders fixed up.
Also I think continuing discussion w/ the kernel.org folks to
understand their infrastructure would be good. They really started
taking things seriously after their compromise, and it would be good
for us to learn from their experience and take things similarly
seriously before any such problems arise for us.
Thanks again for the explanations here!
-john
_______________________________________________
linaro-dev mailing list
linaro-dev@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-dev