On 23 April 2012 11:26, Alexander Sack asac@linaro.org wrote:
On Mon, Apr 23, 2012 at 11:44 AM, Paul Sokolovsky paul.sokolovsky@linaro.org wrote:
Hello,
On Fri, we discovered that build started to take lot of time (30+ mins) on the CopyToSlave stage (one of the first on build startup) to copy EULA-protected binary blobs to a build slave. This adversely affected ability to restrict build/EULA handling changes/RCs (few timed out as was reported by Fathi).
This could be attributed only to http://snapshots.linaro.org/android/binaries/snowball/ receiving five 100MB drops over last 10 days, or 500% or so growth, which uncovered poor throughput of CopyToSlave plugin. Due to inconveniences it brings, I applied a quick dirty solution of mirroring only the latest tarball from http://snapshots.linaro.org/android/binaries/snowball/ . Let me know if you see any issues. And that solution needs to be cleaned/generalized of course.
This is about vendor.tar.bz2?
Does this prevent us from reproducing builds that use older tarballs? Can we make the mirroring smart and parse the info from the build config? e.g. mirror exactly the right tarball?
There is all sorts of chatter about slow copies between masters and slaves (normally the discussion is slave -> master, but I am sure the problems are the same). The workaround is to perform the copy not using the plugin, which will be a pain.
The CopyToSlave plugin looks quite simple (https://svn.jenkins-ci.org/trunk/hudson/plugins/copy-to-slave/src/main/java/...) so it isn't obvious what the problem is. Other threads have pointed to setting TCP NODELAY:
https://issues.jenkins-ci.org/browse/JENKINS-3922 - https://github.com/tivv/ssh-slaves-plugin/commit/c2fcc257161f9f01c98883b3469...
https://issues.jenkins-ci.org/browse/JENKINS-7813
Perhaps the remoting.Pipe class needs some attention? I just cloned the Jenkins repository but didn't find the class. Perhaps I am missing something. Anyway, that is what I found.