Your SlideShare is downloading. ×
Best Practice for getting started with Cloud Computing (EC2, Drupal & MySQL)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Best Practice for getting started with Cloud Computing (EC2, Drupal & MySQL)

9,348
views

Published on

An outline of School of Everything's approach to moving into the Cloud. Basic clustering best practice, and technical implementation overview.

An outline of School of Everything's approach to moving into the Cloud. Basic clustering best practice, and technical implementation overview.

Published in: Technology

3 Comments
12 Likes
Statistics
Notes
No Downloads
Views
Total Views
9,348
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
390
Comments
3
Likes
12
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Dr Clonelove: HOW I LEARNED TO STOP WORRYING AND LOVE THE CLOUD Using Amazon's Elastic Computing Cloud ( and a few too many Star Wars analogies) to create a scalable server infrastructure for School of Everything. David O'Dwyer (SupportFour) Peter Brownell (Code Positive / School of Everything)
  • 2. WHAT'S AHEAD How we built a server cluster on Amazon EC2 for a Drupal based website. It's all about how to add and remove servers on the fly. The set up Why use the cloud ? An overview of our server infrastructure How we got started with the cloud Some sysadmin religious mumbo-jumbo General best-practice for any cluster Technology The key tools to make it happen Image by - http://www.flickr.com/photos/powerpig/
  • 3. THE SILVER LINING Why choose the cloud ? Startups are poor Pay for servers by the hour Add a new server in a few minutes Turn them off when not needed http://flickr.com/photos/balakov/2468552226
  • 4. SCALE DOWN Be able to grow and shrink The key word really is “Elastic” Know how you will add and remove new servers Photo credit: http://www.flickr.com/photos/34558813@N07/
  • 5. BASIC STRUCTURE Code built on Drupal Machine images run Debian Lenny MySQL database with Replication & MySQL proxy Many custom scripts
  • 6. TIME FOR AN EYE TEST TIME FOR AN EYE TEST TIME FOR AN EYE TEST TIME FOR AN EYE TEST
  • 7. THE BIG PICTURE
  • 8. WHAT WE BUILT Basic configuration – grow as needed 2 web-servers (small instance) 1 primary database (small instance) 2 slave databases 2 off-cloud machines for monitoring and backups
  • 9. EVERYONA MAKES MISTAKES Simple concepts to avoid ultimate disasta
  • 10. THE END Work Backwards Non-persistence – all changes lost on power down Must be able to rebuild everything - automaticallyish Start with your plan for complete failure Make sure you can test your backup system
  • 11. ULTIMATE FAILURE Assume that everything may just vanish Be able to rebuild from backups Document and blog Fire Drills – practice makes perfect
  • 12. CLOUD CITY Never have faith in just one ally Set up a base off the cloud Keep a backup on the ground Monitor from outside Mail smarthost to avoid auto greylisting
  • 13. IT'S A TRAP! What does it look like – working or not. Don't trust anything! Monitor as much as possible - Clues for where problems arise - Capacity planning We use Nagios and Munin.
  • 14. NOAH'S ARK PRINCIPLE: Expect the worst and have two of everything Traditionally expensive Adding new servers takes time You may just end up with a lot of hardware doing nothing http://www.thebricktestament.com/genesis/the_flood/gn07_07_10.html
  • 15. IT'S MUCH EASIER WITH CLONES Virtual machines are cheap – but you still need to plan Configure one original, and make many copies, in multiple zones Only pay for the time you use Noah says: “It's not having two of everything running, it's being able to” http://www.flickr.com/photos/jeremymates/2362399109/
  • 16. THE PIECES Putting the cluster together
  • 17. STORAGE YOU CAN RELY ON How safe is your data Local disk – EC2 Ephemeral Storage (/mnt) EBS - Elastic Block Store Point-in-time snapshots
  • 18. CLONE 2 CLONE FILESYSTEM Avoid dependencies Local copy of uploaded files GlusterFS syncs files instantly Primary webserver uses persistent storage Image by - http://www.flickr.com/photos/powerpig/
  • 19. Gluster File System
  • 20. SMALLER IS BETTER MySQL(tm) is MyFriend Database replication – easy and reliable Multiple small images as database servers MySql proxy makes cluster transparent Best practice guide: http://www.mysql.com/why-mysql/white-papers/mysql_wp_best_practices_mysql_ec2.php MySQL is a registered trademark of MySQL inc, no it's Sun Microsystems, http://www.flickr.com/photos/powerpig/ no, woops looks like it's Oracle Inc. now. How weird is that ? Anyway, we like it, and are happy to endorse them – however owns them.
  • 21. Image by - http://www.flickr.com/photos/powerpig/ THE BALANCING ACT Round Robin DNS is the cheap solution for load balancing Each machine configures dynamic DNS on first startup Investigating new amazon load balancing
  • 22. WHAT WE SAID Cloud infrastructure is not alien, it's just a little less solid – it can grow or shrink. The cloud forces best practice – no time for slacker sysadmins! Better safe than sorry Remember Noah, avoid dependencies Work backwards – rebuild, restore, backup, operate Keep a base in the real world Monitor everything Managing growth Loads of local storage, persistent elastic blocks, instant snapshots and S3 Synchronise user uploads with GlusterFS Scale MySQL with replication and MySQL Proxy Growing number of load balancing options Mail Mail smarthost off-cloud to avoid greylisting Image by - http://www.flickr.com/photos/powerpig/
  • 23. Want to learn more about Amazon Web Services ? Sign up on School of Everything and tag yourself with AWS. http://schoolofeverything.com/subject/AWS http://schoolofeverything.com/ http://supportfour.com
  • 24. Your humble presenters The Church of Jar Jar paid a visit to the Star Wars Expo in London. They sang and danced, the cast all queued up for a picture with us. Then they were thrown out.