[Linaro-validation] Old cache clean up on lava server

[Andy Doan]

On 06/07/2012 02:43 AM, Spring Zhang wrote:
> Hi, all,
>
> There's still a WI in squid proxy blueprint. That is to clean up the old
> cache. I jumped into linaro cache directory on v.l.o, and found it cost
> about 100GB.
>
> control:/linaro/images/cache$ du * -sh
> 328M ci.linaro.org <http://ci.linaro.org>
> 36M people.linaro.org <http://people.linaro.org>
> 3.4G    releases-linaro-org
> 10G releases.linaro.org <http://releases.linaro.org>
> 21G    snapshots-linaro-org
> 63G snapshots.linaro.org <http://snapshots.linaro.org>
>
> control:/linaro/images/cache$ ll
> total 32
> drwxr-xr-x  8 root root 4096 2012-05-11 23:26 ./
> drwxr-xr-x  7 root root 4096 2012-01-25 01:16 ../
> drwxr-xr-x  3 root root 4096 2012-03-30 12:07 ci.linaro.org/
> <http://ci.linaro.org/>
> drwxr-xr-x  3 root root 4096 2012-04-03 05:34 people.linaro.org/
> <http://people.linaro.org/>
> drwxr-xr-x  4 root root 4096 2012-05-24 16:27 releases-linaro-org/
> drwxr-xr-x 10 root root 4096 2012-05-09 14:31 releases.linaro.org/
> <http://releases.linaro.org/>
> drwxr-xr-x  3 root root 4096 2012-05-11 23:26 snapshots-linaro-org/
> drwxr-xr-x  6 root root 4096 2012-05-02 18:55 snapshots.linaro.org/
> <http://snapshots.linaro.org/>
>
> For some health check jobs, they need some cache. I think it's
> releases-linaro-org/ and snapshots-linaro-org/.
>
> So can we clean up others?

Clean up the others for now, but I actually just noticed a problem in 
our current code. The only thing that we should now be managing cache 
for are the root.tgz and boot.tgz files. However, we aren't caching 
pre-built images properly. In master.py we are still calling 
download_with_cache for pre-built images rather than downloading via the 
proxy.

I have a small local branch with a fix for this. I forgot about it, so 
I'm going to test on our staging instance to make sure it hasn't 
suffered from bit rot. Once that's fixed, I think we can deploy the fix, 
and remove all non boot/root tarballs from the cache.

Does this make sense?

-andy