Dr Clonelove:
HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD
Using Amazon's Elastic Computing Cloud ( and a few too many Star Wars analogies)
to create a scalable server infrastructure for School of Everything.
David O'Dwyer (t4 partnership)
Peter Brownell (Code Positive / School of Everything)
The AWS Start-up Event – London, Tuesday, 4 November

THE SILVER LINING
Why choose the cloud ?
● Pay for servers by the hour
● Add a new server in a few minutes
● Scale up and down as needed
http://flickr.com/photos/balakov/2468552226

BASIC STRUCTURE
● School of Everything built on Drupal
● Machine images run Debian
● MySQL database with MysqlProxy and Replication
● Custom machine boot scripts
● More details (soon) at http://schoolofeverything.com/tech

WHAT WE BUILT
● Minimal permanent setup, add new servers as needed
● 1 active primary webserver
● 1 primary database (read/write split clones coming)
● 2 off-cloud machines for monitoring and backups

THE END
● Work Backwards
●
●Scaling needs good discipline from the start
●Start with your plan for complete failure
●Build a backup system – and make sure you can test i
●Make sure you can rebuild everything from scratch
●

NOAH'S ARK PRINCIPLE:
Expect the worst and have two of everything
● Traditionally expensive.
● Adding new servers takes time.
● You may just end up with a lot of hardware doing nothing.
http://www.thebricktestament.com/genesis/the_flood/gn07_07_10.html

IT'S MUCH EASIER WITH CLONES
Virtual machines are cheap – but you still need to plan
● Configure one original, and then make many copies
● Only pay for the time you use
● Noah says: “It's not about having two of everything, it's about being able to”.
http://www.flickr.com/photos/jeremymates/2362399109/

LAUNCH, ORBIT, RECOVER
Get it up there, keep it up there, get it down safely and know where it is all the time.
Before you move into the cloud, make sure you have a plan.

CLOUD CITY
Never have faith in just one ally
● Set up a base off the cloud
● Plan for Apocalyptic failure first
● Monitor from outside
● Backup somewhere else

PLAN FOR ULTIMATE FAILURE
●Make sure you can rebuild your setup anywhere at any time

KNOW WHAT IS
GOING ON !
What do things look like:
- when they are working
- when they are not
We may not be afraid of the cloud, but we still
don't trust it (or any server for that matter).
Stats of normal operations help you know where
to look for issues, and plan for capacity.
We use Nagios and Munin.

STORAGE YOU CAN RELY ON
Persistence was our biggest issue when we first set up the project
● Initially used ElasticDrive to write to S3
● Now use ElasticBlocks
● (ElasticBlocks solved so many problems, we don't really need this slide anymore)

C2C (clone to clone) FILESYSTEM
Avoid central dependencies ( and extra server costs).
● Each server maintains a local copy of uploaded files
● GlusterFS runs on each machine sharing files as they arrive
● Primary webserver syncs files to persistent storage
● Drupal does not yet support S3 as a filesystem
Image by Gert Hansen, via vignettebricks.blogspot.com

Gluster File System

WHEN YOU CAN'T GET BATMAN
.......go with Robin.
● Round Robin DNS is the cheap solution for load balancing
● Each machine configures dynamic DNS on first startup
● Loads of future load balancing options

Want to learn more about Amazon Web Services ?
Sign up on School of Everything and tag yourself with AWS.
http://schoolofeverything.com/subject/AWS
http://schoolofeverything.com/
http://t4partnership.com