Dr Clonelove:
HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD




Using Amazon's Elastic Computing Cloud ( and a few too...
WHAT'S AHEAD
How we built a server cluster on Amazon EC2 for a Drupal based website.
It's all about how to add and remove ...
THE SILVER LINING
Why choose the cloud ?
Startups are poor
Pay for servers by the hour
Add a new server in a few minutes
T...
SCALE DOWN
Be able to grow and shrink
The key word really is “Elastic”
Know how you will add and remove new servers




  ...
BASIC STRUCTURE
Code built on Drupal
Machine images run Debian Lenny
MySQL database with Replication & MySQL proxy
Many cu...
TIME FOR AN EYE TEST
    TIME FOR AN EYE TEST
  TIME FOR AN EYE TEST
TIME FOR AN EYE TEST
THE BIG PICTURE
WHAT WE BUILT
Basic configuration – grow as needed

2 web-servers (small instance)
1 primary database (small instance)
2 s...
EVERYONA MAKES MISTAKES
Simple concepts to avoid ultimate disasta
THE END
Work Backwards
Non-persistence – all changes lost on power down
Must be able to rebuild everything - automatically...
ULTIMATE FAILURE
Assume that everything may just vanish
Be able to rebuild from backups
Document and blog
Fire Drills – pr...
CLOUD CITY
Never have faith in just one ally
Set up a base off the cloud
Keep a backup on the ground
Monitor from outside
...
IT'S A TRAP!
What does it look like –
working or not.

Don't trust anything!
Monitor as much as possible
 - Clues for wher...
NOAH'S ARK PRINCIPLE:
Expect the worst and have two of everything
Traditionally expensive
Adding new servers takes time
Yo...
IT'S MUCH EASIER WITH CLONES




   Virtual machines are cheap – but you still need to plan
   Configure one original, and...
THE PIECES
Putting the cluster together
STORAGE YOU CAN RELY ON
How safe is your data

Local disk – EC2 Ephemeral Storage (/mnt)
EBS - Elastic Block Store
Point-i...
CLONE 2 CLONE
  FILESYSTEM
Avoid dependencies

Local copy of uploaded files
GlusterFS syncs files instantly
Primary webser...
Gluster File System
SMALLER IS BETTER
                                     MySQL(tm) is MyFriend
                                     Database...
Image by - http://www.flickr.com/photos/powerpig/




THE BALANCING ACT
Round Robin DNS is the cheap solution for load bal...
WHAT WE SAID
Cloud infrastructure is not alien, it's just a little less solid – it can grow or shrink.
The cloud forces be...
Want to learn more about Amazon Web Services ?
Sign up on School of Everything and tag yourself with AWS.

        http://...
Your humble presenters




The Church of Jar Jar paid a visit to the Star Wars Expo in London. They sang and danced,
the c...
Upcoming SlideShare
Loading in …5
×

Best Practice for getting started with Cloud Computing (EC2, Drupal & MySQL)

10,256 views
9,790 views

Published on

An outline of School of Everything's approach to moving into the Cloud. Basic clustering best practice, and technical implementation overview.

Published in: Technology
3 Comments
12 Likes
Statistics
Notes
No Downloads
Views
Total views
10,256
On SlideShare
0
From Embeds
0
Number of Embeds
127
Actions
Shares
0
Downloads
391
Comments
3
Likes
12
Embeds 0
No embeds

No notes for slide

Best Practice for getting started with Cloud Computing (EC2, Drupal & MySQL)

  1. Dr Clonelove: HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD Using Amazon's Elastic Computing Cloud ( and a few too many Star Wars analogies) to create a scalable server infrastructure for School of Everything. David O'Dwyer (SupportFour) Peter Brownell (Code Positive / School of Everything)
  2. WHAT'S AHEAD How we built a server cluster on Amazon EC2 for a Drupal based website. It's all about how to add and remove servers on the fly. The set up Why use the cloud ? An overview of our server infrastructure How we got started with the cloud Some sysadmin religious mumbo-jumbo General best-practice for any cluster Technology The key tools to make it happen Image by - http://www.flickr.com/photos/powerpig/
  3. THE SILVER LINING Why choose the cloud ? Startups are poor Pay for servers by the hour Add a new server in a few minutes Turn them off when not needed http://flickr.com/photos/balakov/2468552226
  4. SCALE DOWN Be able to grow and shrink The key word really is “Elastic” Know how you will add and remove new servers Photo credit: http://www.flickr.com/photos/34558813@N07/
  5. BASIC STRUCTURE Code built on Drupal Machine images run Debian Lenny MySQL database with Replication & MySQL proxy Many custom scripts
  6. TIME FOR AN EYE TEST TIME FOR AN EYE TEST TIME FOR AN EYE TEST TIME FOR AN EYE TEST
  7. THE BIG PICTURE
  8. WHAT WE BUILT Basic configuration – grow as needed 2 web-servers (small instance) 1 primary database (small instance) 2 slave databases 2 off-cloud machines for monitoring and backups
  9. EVERYONA MAKES MISTAKES Simple concepts to avoid ultimate disasta
  10. THE END Work Backwards Non-persistence – all changes lost on power down Must be able to rebuild everything - automaticallyish Start with your plan for complete failure Make sure you can test your backup system
  11. ULTIMATE FAILURE Assume that everything may just vanish Be able to rebuild from backups Document and blog Fire Drills – practice makes perfect
  12. CLOUD CITY Never have faith in just one ally Set up a base off the cloud Keep a backup on the ground Monitor from outside Mail smarthost to avoid auto greylisting
  13. IT'S A TRAP! What does it look like – working or not. Don't trust anything! Monitor as much as possible - Clues for where problems arise - Capacity planning We use Nagios and Munin.
  14. NOAH'S ARK PRINCIPLE: Expect the worst and have two of everything Traditionally expensive Adding new servers takes time You may just end up with a lot of hardware doing nothing http://www.thebricktestament.com/genesis/the_flood/gn07_07_10.html
  15. IT'S MUCH EASIER WITH CLONES Virtual machines are cheap – but you still need to plan Configure one original, and make many copies, in multiple zones Only pay for the time you use Noah says: “It's not having two of everything running, it's being able to” http://www.flickr.com/photos/jeremymates/2362399109/
  16. THE PIECES Putting the cluster together
  17. STORAGE YOU CAN RELY ON How safe is your data Local disk – EC2 Ephemeral Storage (/mnt) EBS - Elastic Block Store Point-in-time snapshots
  18. CLONE 2 CLONE FILESYSTEM Avoid dependencies Local copy of uploaded files GlusterFS syncs files instantly Primary webserver uses persistent storage Image by - http://www.flickr.com/photos/powerpig/
  19. Gluster File System
  20. SMALLER IS BETTER MySQL(tm) is MyFriend Database replication – easy and reliable Multiple small images as database servers MySql proxy makes cluster transparent Best practice guide: http://www.mysql.com/why-mysql/white-papers/mysql_wp_best_practices_mysql_ec2.php MySQL is a registered trademark of MySQL inc, no it's Sun Microsystems, http://www.flickr.com/photos/powerpig/ no, woops looks like it's Oracle Inc. now. How weird is that ? Anyway, we like it, and are happy to endorse them – however owns them.
  21. Image by - http://www.flickr.com/photos/powerpig/ THE BALANCING ACT Round Robin DNS is the cheap solution for load balancing Each machine configures dynamic DNS on first startup Investigating new amazon load balancing
  22. WHAT WE SAID Cloud infrastructure is not alien, it's just a little less solid – it can grow or shrink. The cloud forces best practice – no time for slacker sysadmins! Better safe than sorry Remember Noah, avoid dependencies Work backwards – rebuild, restore, backup, operate Keep a base in the real world Monitor everything Managing growth Loads of local storage, persistent elastic blocks, instant snapshots and S3 Synchronise user uploads with GlusterFS Scale MySQL with replication and MySQL Proxy Growing number of load balancing options Mail Mail smarthost off-cloud to avoid greylisting Image by - http://www.flickr.com/photos/powerpig/
  23. Want to learn more about Amazon Web Services ? Sign up on School of Everything and tag yourself with AWS. http://schoolofeverything.com/subject/AWS http://schoolofeverything.com/ http://supportfour.com
  24. Your humble presenters The Church of Jar Jar paid a visit to the Star Wars Expo in London. They sang and danced, the cast all queued up for a picture with us. Then they were thrown out.

×