reddit.com and Our Move to the Cloud

Jeremy Edberg Text Operations Manager reddit.com http://www.revolutioncloud.com

What is reddit? • reddit is an online community http://www.revolutioncloud.com

Conde Nast owns reddit http://www.revolutioncloud.com

Conde Nast websites http://www.revolutioncloud.com

Timeline April 2006 -- S3 for logos September 2007 -- S3 for thumbnails November 2008 -- EC2 for batch processing May 2009 -- EC2 for entire site http://www.revolutioncloud.com

What led us to AWS (part 1) • Needed an easy way to distribute and upload our logo http://www.revolutioncloud.com

http://www.revolutioncloud.com

What led us to AWS (part 2) • Thumbnails! http://www.revolutioncloud.com

What led us to AWS (part 3) • Didn’t want to rent another cabinet http://www.revolutioncloud.com

What led us to AWS (part 3) • Didn’t want to rent another cabinet • Didn’t want to buy more servers http://www.revolutioncloud.com

What led us to AWS (part 3) • Didn’t want to rent another cabinet • Didn’t want to buy more servers New Servers New Servers http://www.revolutioncloud.com

Imaging and Racking Servers Is A (Sometimes Fun) Chore http://www.revolutioncloud.com

EC2 for Overﬂow • Used openvpn to create a secure link to our datacenter for batch processing http://www.revolutioncloud.com

Moving to EC2 • Started by migrating all data http://www.revolutioncloud.com

Moving to EC2 • Started by migrating all data • Got a complete stack running on EC2 http://www.revolutioncloud.com

Moving to EC2 • Started by migrating all data • Got a complete stack running on EC2 • Long Saturday night ﬁnishing the migration and “forklifting” the last bits of data http://www.revolutioncloud.com

Architecture http://www.revolutioncloud.com

Stats • 190 Virtual CPUs • 338GB of RAM • 9TB of Elastic Block Storage • 2TB of S3 Storage • 6.5 TB of Data Out / mo • 2TB of Data In / mo • 150M+ Pageviews and just one sysadmin! http://www.revolutioncloud.com

Beneﬁts Data Center (per month) EC2 (per month) Servers: $6K Servers: $13K Cabinet (x3): $15K Storage: $1.5K Bandwidth: $2.5K Bandwidth: $1.1K Support: N/A Support: $1.2K Total: $23.5K Total: $16.8K 29% Cheaper! Based on Amazon public pricing, reddit open source Estimated Pricing code, and public conﬁguration information http://www.revolutioncloud.com

Beneﬁts • Don’t have to procure servers anymore http://www.revolutioncloud.com

Beneﬁts • Don’t have to procure servers anymore • No racking or imaging servers anymore http://www.revolutioncloud.com

Beneﬁts • Don’t have to procure servers anymore • No racking or imaging servers anymore • Time to market is faster http://www.revolutioncloud.com

Beneﬁts • Don’t have to procure servers anymore • No racking or imaging servers anymore • Time to market is faster • Multiple physical locations with AZs http://www.revolutioncloud.com

Beneﬁts • Don’t have to procure servers anymore • No racking or imaging servers anymore • Time to market is faster • Multiple physical locations with AZs • Get “free” upgrades http://www.revolutioncloud.com

Beneﬁts • Don’t have to procure servers anymore • No racking or imaging servers anymore • Time to market is faster • Multiple physical locations with AZs • Get “free” upgrades • Elasticity! http://www.revolutioncloud.com

Pain Points http://www.revolutioncloud.com

Pain Points Higher Latency Workaround: Fewer network calls, ask for more data at a time. http://www.revolutioncloud.com

Pain Points EBS sometimes slows down a bit Workaround: Use caching and replication with read slaves to avoid relying on a single disk. http://www.revolutioncloud.com

Pain Points Instances go away sometimes Workaround: Avoid single points of failure and make sure your servers have automated conﬁguration. http://www.revolutioncloud.com

Pain Points Fixing these issues made our app more reliable and highly available. We are better off than when we started. http://www.revolutioncloud.com

Best Practices • Keep data in multiple Availability Zones http://www.revolutioncloud.com

Best Practices • Keep data in multiple Availability Zones • EBS for all persistent data http://www.revolutioncloud.com

Best Practices • Keep data in multiple Availability Zones • EBS for all persistent data • Snapshots http://www.revolutioncloud.com

Best Practices • Keep data in multiple Availability Zones • EBS for all persistent data • Snapshots • No secret keys on the instance http://www.revolutioncloud.com

Best Practices • Keep data in multiple Availability Zones • EBS for all persistent data • Snapshots • No secret keys on the instance • Different functions in different Security Groups http://www.revolutioncloud.com

Best Practices • A full stack in each zone http://www.revolutioncloud.com

Best Practices • A full stack in each zone • All data stored as key-value pairs http://www.revolutioncloud.com

Best Practices • A full stack in each zone • All data stored as key-value pairs • More use of queues http://www.revolutioncloud.com

Conclusion • AWS saves us a lot of money http://www.revolutioncloud.com

Conclusion • AWS saves us a lot of money • AWS gives us a lot of ﬂexibility http://www.revolutioncloud.com

Conclusion • AWS saves us a lot of money • AWS gives us a lot of ﬂexibility • Moving to AWS has forced us to build better applications and follow better IT practices. http://www.revolutioncloud.com

Conclusion • AWS saves us a lot of money • AWS gives us a lot of ﬂexibility • Moving to AWS has forced us to build better applications and follow better IT practices. • http://code.reddit.com http://www.revolutioncloud.com

Revolution: Cloud To learn more, visit http://www.revolutioncloud.com http://www.revolutioncloud.com

reddit.com and Our Move to the Cloud

by Revolution Cloud on Oct 21, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 67

Statistics

reddit.com and Our Move to the Cloud — Presentation Transcript

http://www.revolutioncloud.com	61
http://www.slideshare.net	6