Ricardo Salveti ricardo.salveti@linaro.org writes:
Hey folks,
I was checking today what we could do to optimize the job runs at LAVA, and noticed that the time it takes to download and cache the tarball is still the biggest bottleneck we have.
Checking job http://validation.linaro.org/lava-server/scheduler/job/23135, you can see at the top of the serial log that it took almost one hour just to download and cache the tarball.
Ugh.
As I believe this could also be related with the connection we have available at the lab, and as we're caching the tarballs already,
Heh well. Two points:
(a) we only cache the boot and rootfs tarballs currently. I doubt zsync will help there -- surely we'd need to keep the image around for that?
(b) we really shouldn't cache them for any jobs but health jobs :)
wouldn't it be possible to use zsync or a similar tool to speed up the download?
For the pre-built images we're already generating the zsync meta-data file together with the image itself, so would be nice to check if that would really make any difference from the lab perspective.
That said, I'm sure we could use zsync to improve download times in the dispatcher. I don't know much about it though -- do you have to explicitly point zsync at the version of the file you've already downloaded? We _could_ probably keep the latest version of each "kind" of image we downloaded (somehow... imagine me waving my hands furiously at this point). What would be utterly awesome of course would be some kind of automagic thing for squid that used zsync without us knowing about it :)
Cheers, mwh