On Thu, Jul 26, 2012, Michael Hope wrote:
I don't want a split solution where we use one thing for x86 and another for ARM and there's no way of doing PandaBoards in the cloud. Might be interesting in the future to treat the Pandas as a cloud like pool and see if there's a tool that gives a EC2 like API on top.
The EC2 instances would just be your x86 machines; if you need dedicated ones in Cambridge for benchmarks accuracy, that's fine. I don't think we care about the EC2 API here, it's more about the fact that Amazon is in the business of building efficient / competitive infrastructure and we're not: anything we build is bound to be expensive and a waste of our brains (hmmm brainnns). We're struggling for sysadmin time, vms in EC2 at least relieve us from the physical burden of admin and give us the possibility to upgrade/downgrade/revisit our virtual infrastructure.
I want the same storage solution for the x86 and ARM builds. It seems inefficient to push the build results up to S3, especially the ~100 MB binaries. 500 GB is $60/month on S3 and I don't think the sysadmin costs will be any different.
I don't know how much it costs to buy 500 GB of RAID storage, but I'm pretty sure it will take us more than a year to repay just the cost of the disks (disregarding the time to order, the time to set them up, the electricity costs, the cost of the RAID controller and of the server hosting it); then some disks might fail and we will have to replace them. Will we have 24/7 staff to replace the disks? etc.
If you're worried about the speed of the line, it's a worry no matter what; there's more pressure on the quality of the line if we host more things in Cambridge.
My tools are also bad and use the filesystem as the database. They'd need reworking to run against S3.
Why can't they use the local EBS filesystem on the instance they run on for the database side of things and S3 for the build publishing side of things? We could also use an EBS volume to store everything and expose it with Apache from EC2, just like we do for Android builds.
(There are solutions to pretend S3 is a filesystem, but frankly they suck, even more so with large files.)