• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
reddit.com and Our Move to the Cloud
 

reddit.com and Our Move to the Cloud

on

  • 4,298 views

Presentation by Jeremy Edberg at the AWS Event in Los Angeles on ...

Presentation by Jeremy Edberg at the AWS Event in Los Angeles on
October 15, 2009. The presentation discusses the experiences and
strategies reddit.com used to migrate IT infrastructure to EC2 and S3.
Best Practices and lessons learned are also presented.

Statistics

Views

Total Views
4,298
Views on SlideShare
2,052
Embed Views
2,246

Actions

Likes
4
Downloads
0
Comments
0

13 Embeds 2,246

http://www.ranlevi.com 1118
http://cptl.corp.yahoo.co.jp 990
http://www.revolutioncloud.com 70
http://feedly.com 48
http://www.slideshare.net 6
http://digg.com 3
http://news.google.com 3
http://www.inoreader.com 2
http://www.google.co.il 2
http://translate.googleusercontent.com 1
http://summary 1
http://plus.url.google.com 1
http://www.feedspot.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    reddit.com and Our Move to the Cloud reddit.com and Our Move to the Cloud Presentation Transcript

    • Jeremy Edberg Text Operations Manager reddit.com http://www.revolutioncloud.com
    • What is reddit? • reddit is an online community http://www.revolutioncloud.com
    • Conde Nast owns reddit http://www.revolutioncloud.com
    • Conde Nast websites http://www.revolutioncloud.com
    • Timeline April 2006 -- S3 for logos September 2007 -- S3 for thumbnails November 2008 -- EC2 for batch processing May 2009 -- EC2 for entire site http://www.revolutioncloud.com
    • What led us to AWS (part 1) • Needed an easy way to distribute and upload our logo http://www.revolutioncloud.com
    • http://www.revolutioncloud.com
    • What led us to AWS (part 2) • Thumbnails! http://www.revolutioncloud.com
    • What led us to AWS (part 3) • Didn’t want to rent another cabinet http://www.revolutioncloud.com
    • What led us to AWS (part 3) • Didn’t want to rent another cabinet • Didn’t want to buy more servers http://www.revolutioncloud.com
    • What led us to AWS (part 3) • Didn’t want to rent another cabinet • Didn’t want to buy more servers New Servers New Servers http://www.revolutioncloud.com
    • Imaging and Racking Servers Is A (Sometimes Fun) Chore http://www.revolutioncloud.com
    • EC2 for Overflow • Used openvpn to create a secure link to our datacenter for batch processing http://www.revolutioncloud.com
    • Moving to EC2 • Started by migrating all data http://www.revolutioncloud.com
    • Moving to EC2 • Started by migrating all data • Got a complete stack running on EC2 http://www.revolutioncloud.com
    • Moving to EC2 • Started by migrating all data • Got a complete stack running on EC2 • Long Saturday night finishing the migration and “forklifting” the last bits of data http://www.revolutioncloud.com
    • Architecture http://www.revolutioncloud.com
    • Stats • 190 Virtual CPUs • 338GB of RAM • 9TB of Elastic Block Storage • 2TB of S3 Storage • 6.5 TB of Data Out / mo • 2TB of Data In / mo • 150M+ Pageviews and just one sysadmin! http://www.revolutioncloud.com
    • Benefits Data Center (per month) EC2 (per month) Servers: $6K Servers: $13K Cabinet (x3): $15K Storage: $1.5K Bandwidth: $2.5K Bandwidth: $1.1K Support: N/A Support: $1.2K Total: $23.5K Total: $16.8K 29% Cheaper! Based on Amazon public pricing, reddit open source Estimated Pricing code, and public configuration information http://www.revolutioncloud.com
    • Benefits • Don’t have to procure servers anymore http://www.revolutioncloud.com
    • Benefits • Don’t have to procure servers anymore • No racking or imaging servers anymore http://www.revolutioncloud.com
    • Benefits • Don’t have to procure servers anymore • No racking or imaging servers anymore • Time to market is faster http://www.revolutioncloud.com
    • Benefits • Don’t have to procure servers anymore • No racking or imaging servers anymore • Time to market is faster • Multiple physical locations with AZs http://www.revolutioncloud.com
    • Benefits • Don’t have to procure servers anymore • No racking or imaging servers anymore • Time to market is faster • Multiple physical locations with AZs • Get “free” upgrades http://www.revolutioncloud.com
    • Benefits • Don’t have to procure servers anymore • No racking or imaging servers anymore • Time to market is faster • Multiple physical locations with AZs • Get “free” upgrades • Elasticity! http://www.revolutioncloud.com
    • Pain Points http://www.revolutioncloud.com
    • Pain Points http://www.revolutioncloud.com
    • Pain Points Higher Latency Workaround: Fewer network calls, ask for more data at a time. http://www.revolutioncloud.com
    • Pain Points EBS sometimes slows down a bit Workaround: Use caching and replication with read slaves to avoid relying on a single disk. http://www.revolutioncloud.com
    • Pain Points Instances go away sometimes Workaround: Avoid single points of failure and make sure your servers have automated configuration. http://www.revolutioncloud.com
    • Pain Points Fixing these issues made our app more reliable and highly available. We are better off than when we started. http://www.revolutioncloud.com
    • Best Practices • Keep data in multiple Availability Zones http://www.revolutioncloud.com
    • Best Practices • Keep data in multiple Availability Zones • EBS for all persistent data http://www.revolutioncloud.com
    • Best Practices • Keep data in multiple Availability Zones • EBS for all persistent data • Snapshots http://www.revolutioncloud.com
    • Best Practices • Keep data in multiple Availability Zones • EBS for all persistent data • Snapshots • No secret keys on the instance http://www.revolutioncloud.com
    • Best Practices • Keep data in multiple Availability Zones • EBS for all persistent data • Snapshots • No secret keys on the instance • Different functions in different Security Groups http://www.revolutioncloud.com
    • Best Practices • A full stack in each zone http://www.revolutioncloud.com
    • Best Practices • A full stack in each zone • All data stored as key-value pairs http://www.revolutioncloud.com
    • Best Practices • A full stack in each zone • All data stored as key-value pairs • More use of queues http://www.revolutioncloud.com
    • Conclusion • AWS saves us a lot of money http://www.revolutioncloud.com
    • Conclusion • AWS saves us a lot of money • AWS gives us a lot of flexibility http://www.revolutioncloud.com
    • Conclusion • AWS saves us a lot of money • AWS gives us a lot of flexibility • Moving to AWS has forced us to build better applications and follow better IT practices. http://www.revolutioncloud.com
    • Conclusion • AWS saves us a lot of money • AWS gives us a lot of flexibility • Moving to AWS has forced us to build better applications and follow better IT practices. • http://code.reddit.com http://www.revolutioncloud.com
    • Revolution: Cloud To learn more, visit http://www.revolutioncloud.com http://www.revolutioncloud.com