On Mon, 23 Apr 2012 12:26:19 +0200
Alexander Sack <asac@linaro.org> wrote:

> On Mon, Apr 23, 2012 at 11:44 AM, Paul Sokolovsky <
> paul.sokolovsky@linaro.org> wrote:
>
> > Hello,
> >
> > On Fri, we discovered that build started to take lot of time (30+
> > mins) on the CopyToSlave stage (one of the first on build startup)
> > to copy EULA-protected binary blobs to a build slave. This
> > adversely affected ability to restrict build/EULA handling
> > changes/RCs (few timed out as was reported by Fathi).
> >
> > This could be attributed only to
> > http://snapshots.linaro.org/android/binaries/snowball/ receiving
> > five 100MB drops over last 10 days, or 500% or so growth, which
> > uncovered poor throughput of CopyToSlave plugin. Due to
> > inconveniences it brings, I applied a quick dirty solution of
> > mirroring only the latest tarball from
> > http://snapshots.linaro.org/android/binaries/snowball/ . Let me
> > know if you see any issues. And that solution needs to be
> > cleaned/generalized of course.
> >
>
> This is about vendor.tar.bz2?
>
> Does this prevent us from reproducing builds that use older tarballs?

Potentially, yes, realistically, noone complained yet ;-). Script
doesn't delete anything, it just doesn't mirror old stuff, so unless
someone deletes old files (I did), they stay there.

> Can we make the mirroring smart and parse the info from the build
> config? e.g. mirror exactly the right tarball?

We can go deeper and deeper with applying adhoc workarounds which
shouldn't be there in the first place, or can instead think out and
implement access system which would allow our bot scripts to get access
to things they need directly.