An outline of School of Everything's approach to moving into the Cloud. Basic clustering best practice, and technical implementation overview.

1. Dr Clonelove:
HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD
Using Amazon's Elastic Computing Cloud ( and a few too many Star Wars analogies)
to create a scalable server infrastructure for School of Everything.
David O'Dwyer (SupportFour)
Peter Brownell (Code Positive / School of Everything)

2. WHAT'S AHEAD
How we built a server cluster on Amazon EC2 for a Drupal based website.
It's all about how to add and remove servers on the fly.
The set up
Why use the cloud ?
An overview of our server infrastructure
How we got started with the cloud
Some sysadmin religious mumbo-jumbo
General best-practice for any cluster
Technology
The key tools to make it happen
Image by - http://www.flickr.com/photos/powerpig/

3. THE SILVER LINING
Why choose the cloud ?
Startups are poor
Pay for servers by the hour
Add a new server in a few minutes
Turn them off when not needed
http://flickr.com/photos/balakov/2468552226

4. SCALE DOWN
Be able to grow and shrink
The key word really is “Elastic”
Know how you will add and remove new servers
Photo credit: http://www.flickr.com/photos/34558813@N07/

5. BASIC STRUCTURE
Code built on Drupal
Machine images run Debian Lenny
MySQL database with Replication & MySQL proxy
Many custom scripts

6. TIME FOR AN EYE TEST
TIME FOR AN EYE TEST
TIME FOR AN EYE TEST
TIME FOR AN EYE TEST

7. THE BIG PICTURE

8. WHAT WE BUILT
Basic configuration – grow as needed
2 web-servers (small instance)
1 primary database (small instance)
2 slave databases
2 off-cloud machines for monitoring and backups

9. EVERYONA MAKES MISTAKES
Simple concepts to avoid ultimate disasta

10. THE END
Work Backwards
Non-persistence – all changes lost on power down
Must be able to rebuild everything - automaticallyish
Start with your plan for complete failure
Make sure you can test your backup system

11. ULTIMATE FAILURE
Assume that everything may just vanish
Be able to rebuild from backups
Document and blog
Fire Drills – practice makes perfect

12. CLOUD CITY
Never have faith in just one ally
Set up a base off the cloud
Keep a backup on the ground
Monitor from outside
Mail smarthost to avoid auto greylisting

13. IT'S A TRAP!
What does it look like –
working or not.
Don't trust anything!
Monitor as much as possible
- Clues for where problems arise
- Capacity planning
We use Nagios and Munin.

14. NOAH'S ARK PRINCIPLE:
Expect the worst and have two of everything
Traditionally expensive
Adding new servers takes time
You may just end up with a lot of hardware doing nothing
http://www.thebricktestament.com/genesis/the_flood/gn07_07_10.html

15. IT'S MUCH EASIER WITH CLONES
Virtual machines are cheap – but you still need to plan
Configure one original, and make many copies, in multiple zones
Only pay for the time you use
Noah says: “It's not having two of everything running, it's being able to”
http://www.flickr.com/photos/jeremymates/2362399109/

16. THE PIECES
Putting the cluster together

17. STORAGE YOU CAN RELY ON
How safe is your data
Local disk – EC2 Ephemeral Storage (/mnt)
EBS - Elastic Block Store
Point-in-time snapshots

18. CLONE 2 CLONE
FILESYSTEM
Avoid dependencies
Local copy of uploaded files
GlusterFS syncs files instantly
Primary webserver uses persistent storage
Image by - http://www.flickr.com/photos/powerpig/

19. Gluster File System

20. SMALLER IS BETTER
MySQL(tm) is MyFriend
Database replication – easy and reliable
Multiple small images as database servers
MySql proxy makes cluster transparent
Best practice guide: http://www.mysql.com/why-mysql/white-papers/mysql_wp_best_practices_mysql_ec2.php
MySQL is a registered trademark of MySQL inc, no it's Sun Microsystems, http://www.flickr.com/photos/powerpig/
no, woops looks like it's Oracle Inc. now. How weird is that ?
Anyway, we like it, and are happy to endorse them – however owns them.

21. Image by - http://www.flickr.com/photos/powerpig/
THE BALANCING ACT
Round Robin DNS is the cheap solution for load balancing
Each machine configures dynamic DNS on first startup
Investigating new amazon load balancing

22. WHAT WE SAID
Cloud infrastructure is not alien, it's just a little less solid – it can grow or shrink.
The cloud forces best practice – no time for slacker sysadmins!
Better safe than sorry
Remember Noah, avoid dependencies
Work backwards – rebuild, restore, backup, operate
Keep a base in the real world
Monitor everything
Managing growth
Loads of local storage, persistent elastic blocks, instant snapshots and S3
Synchronise user uploads with GlusterFS
Scale MySQL with replication and MySQL Proxy
Growing number of load balancing options
Mail
Mail smarthost off-cloud to avoid greylisting
Image by - http://www.flickr.com/photos/powerpig/

23. Want to learn more about Amazon Web Services ?
Sign up on School of Everything and tag yourself with AWS.
http://schoolofeverything.com/subject/AWS
http://schoolofeverything.com/
http://supportfour.com

24. Your humble presenters
The Church of Jar Jar paid a visit to the Star Wars Expo in London. They sang and danced,
the cast all queued up for a picture with us. Then they were thrown out.