Dr Clonelove:
HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD
Using Amazon's Elastic Computing Cloud ( and a few too many Star Wars analogies)
to create a scalable server infrastructure for School of Everything.
David O'Dwyer (SupportFour)
Peter Brownell (Code Positive / School of Everything)

WHAT'S AHEAD
How we built a server cluster on Amazon EC2 for a Drupal based website.
It's all about how to add and remove servers on the fly.
The set up
Why use the cloud ?
An overview of our server infrastructure
How we got started with the cloud
Some sysadmin religious mumbo-jumbo
General best-practice for any cluster
Technology
The key tools to make it happen
Image by - http://www.flickr.com/photos/powerpig/

THE SILVER LINING
Why choose the cloud ?
Startups are poor
Pay for servers by the hour
Add a new server in a few minutes
Turn them off when not needed
http://flickr.com/photos/balakov/2468552226

SCALE DOWN
Be able to grow and shrink
The key word really is “Elastic”
Know how you will add and remove new servers
Photo credit: http://www.flickr.com/photos/34558813@N07/

BASIC STRUCTURE
Code built on Drupal
Machine images run Debian Lenny
MySQL database with Replication & MySQL proxy
Many custom scripts

TIME FOR AN EYE TEST
TIME FOR AN EYE TEST
TIME FOR AN EYE TEST
TIME FOR AN EYE TEST

THE BIG PICTURE

WHAT WE BUILT
Basic configuration – grow as needed
2 web-servers (small instance)
1 primary database (small instance)
2 slave databases
2 off-cloud machines for monitoring and backups

EVERYONA MAKES MISTAKES
Simple concepts to avoid ultimate disasta

THE END
Work Backwards
Non-persistence – all changes lost on power down
Must be able to rebuild everything - automaticallyish
Start with your plan for complete failure
Make sure you can test your backup system

ULTIMATE FAILURE
Assume that everything may just vanish
Be able to rebuild from backups
Document and blog
Fire Drills – practice makes perfect

CLOUD CITY
Never have faith in just one ally
Set up a base off the cloud
Keep a backup on the ground
Monitor from outside
Mail smarthost to avoid auto greylisting

IT'S A TRAP!
What does it look like –
working or not.
Don't trust anything!
Monitor as much as possible
- Clues for where problems arise
- Capacity planning
We use Nagios and Munin.

NOAH'S ARK PRINCIPLE:
Expect the worst and have two of everything
Traditionally expensive
Adding new servers takes time
You may just end up with a lot of hardware doing nothing
http://www.thebricktestament.com/genesis/the_flood/gn07_07_10.html

IT'S MUCH EASIER WITH CLONES
Virtual machines are cheap – but you still need to plan
Configure one original, and make many copies, in multiple zones
Only pay for the time you use
Noah says: “It's not having two of everything running, it's being able to”
http://www.flickr.com/photos/jeremymates/2362399109/

THE PIECES
Putting the cluster together

STORAGE YOU CAN RELY ON
How safe is your data
Local disk – EC2 Ephemeral Storage (/mnt)
EBS - Elastic Block Store
Point-in-time snapshots

CLONE 2 CLONE
FILESYSTEM
Avoid dependencies
Local copy of uploaded files
GlusterFS syncs files instantly
Primary webserver uses persistent storage
Image by - http://www.flickr.com/photos/powerpig/

Gluster File System

SMALLER IS BETTER
MySQL(tm) is MyFriend
Database replication – easy and reliable
Multiple small images as database servers
MySql proxy makes cluster transparent
Best practice guide: http://www.mysql.com/why-mysql/white-papers/mysql_wp_best_practices_mysql_ec2.php
MySQL is a registered trademark of MySQL inc, no it's Sun Microsystems, http://www.flickr.com/photos/powerpig/
no, woops looks like it's Oracle Inc. now. How weird is that ?
Anyway, we like it, and are happy to endorse them – however owns them.

Image by - http://www.flickr.com/photos/powerpig/
THE BALANCING ACT
Round Robin DNS is the cheap solution for load balancing
Each machine configures dynamic DNS on first startup
Investigating new amazon load balancing

WHAT WE SAID
Cloud infrastructure is not alien, it's just a little less solid – it can grow or shrink.
The cloud forces best practice – no time for slacker sysadmins!
Better safe than sorry
Remember Noah, avoid dependencies
Work backwards – rebuild, restore, backup, operate
Keep a base in the real world
Monitor everything
Managing growth
Loads of local storage, persistent elastic blocks, instant snapshots and S3
Synchronise user uploads with GlusterFS
Scale MySQL with replication and MySQL Proxy
Growing number of load balancing options
Mail
Mail smarthost off-cloud to avoid greylisting
Image by - http://www.flickr.com/photos/powerpig/

Want to learn more about Amazon Web Services ?
Sign up on School of Everything and tag yourself with AWS.
http://schoolofeverything.com/subject/AWS
http://schoolofeverything.com/
http://supportfour.com

Your humble presenters
The Church of Jar Jar paid a visit to the Star Wars Expo in London. They sang and danced,
the cast all queued up for a picture with us. Then they were thrown out.