On Fri, Aug 06, 2010 at 05:38:56PM +0100, Dave Martin wrote:
Would the overhead be significant even if the tarball wasn't compressed? I don't understand enough about tar's concatenate and delete performance to risk a guess.
tar's default internal blocksize is 512 bytes, so there would still be overhead but it would be less. I don't think tar really supports random access though, since tar files are sequential and monolithic; having many tarballs instead of just one may be better.
I don't know for sure if it does, but man tar does show --concatenate and --delete options so I think there's something there. Whether or not that's fast is another matter <wink>