Jeremy Edberg
Text
Operations Manager
reddit.com
http://www.revolutioncloud.com

What is reddit?
• reddit is an online community
http://www.revolutioncloud.com

Conde Nast owns reddit
http://www.revolutioncloud.com

Conde Nast websites
http://www.revolutioncloud.com

Timeline
April 2006 -- S3 for logos
September 2007 -- S3 for thumbnails
November 2008 -- EC2 for batch processing
May 2009 -- EC2 for entire site
http://www.revolutioncloud.com

What led us to AWS
(part 1)
• Needed an easy way to distribute and
upload our logo
http://www.revolutioncloud.com

http://www.revolutioncloud.com

What led us to AWS
(part 2)
• Thumbnails!
http://www.revolutioncloud.com

What led us to AWS
(part 3)
• Didn’t want to rent another cabinet
http://www.revolutioncloud.com

What led us to AWS
(part 3)
• Didn’t want to rent another cabinet
• Didn’t want to buy more servers
http://www.revolutioncloud.com

What led us to AWS
(part 3)
• Didn’t want to rent another cabinet
• Didn’t want to buy more servers
New Servers
New Servers
http://www.revolutioncloud.com

Imaging and Racking Servers Is
A (Sometimes Fun) Chore
http://www.revolutioncloud.com

EC2 for Overflow
• Used openvpn to create a secure link to
our datacenter for batch processing
http://www.revolutioncloud.com

Moving to EC2
• Started by migrating all data
http://www.revolutioncloud.com

Moving to EC2
• Started by migrating all data
• Got a complete stack running on EC2
http://www.revolutioncloud.com

Moving to EC2
• Started by migrating all data
• Got a complete stack running on EC2
• Long Saturday night finishing the migration
and “forklifting” the last bits of data
http://www.revolutioncloud.com

Architecture
http://www.revolutioncloud.com

Stats
• 190 Virtual CPUs
• 338GB of RAM
• 9TB of Elastic Block Storage
• 2TB of S3 Storage
• 6.5 TB of Data Out / mo
• 2TB of Data In / mo
• 150M+ Pageviews and just one sysadmin!
http://www.revolutioncloud.com

Benefits
Data Center (per month) EC2 (per month)
Servers: $6K Servers: $13K
Cabinet (x3): $15K Storage: $1.5K
Bandwidth: $2.5K Bandwidth: $1.1K
Support: N/A Support: $1.2K
Total: $23.5K Total: $16.8K
29% Cheaper!
Based on Amazon public pricing, reddit open source
Estimated Pricing code, and public configuration information
http://www.revolutioncloud.com

Benefits
• Don’t have to procure servers anymore
http://www.revolutioncloud.com

Benefits
• Don’t have to procure servers anymore
• No racking or imaging servers anymore
http://www.revolutioncloud.com

Benefits
• Don’t have to procure servers anymore
• No racking or imaging servers anymore
• Time to market is faster
http://www.revolutioncloud.com

Benefits
• Don’t have to procure servers anymore
• No racking or imaging servers anymore
• Time to market is faster
• Multiple physical locations with AZs
http://www.revolutioncloud.com

Benefits
• Don’t have to procure servers anymore
• No racking or imaging servers anymore
• Time to market is faster
• Multiple physical locations with AZs
• Get “free” upgrades
http://www.revolutioncloud.com

Benefits
• Don’t have to procure servers anymore
• No racking or imaging servers anymore
• Time to market is faster
• Multiple physical locations with AZs
• Get “free” upgrades
• Elasticity!
http://www.revolutioncloud.com

Pain Points
http://www.revolutioncloud.com

Pain Points
http://www.revolutioncloud.com

Pain Points
Higher Latency
Workaround: Fewer network calls, ask for
more data at a time.
http://www.revolutioncloud.com

Pain Points
EBS sometimes slows down a bit
Workaround: Use caching and replication
with read slaves to avoid relying on a single
disk.
http://www.revolutioncloud.com

Pain Points
Instances go away sometimes
Workaround: Avoid single points of failure
and make sure your servers have automated
configuration.
http://www.revolutioncloud.com

Pain Points
Fixing these issues made
our app more reliable and
highly available. We are
better off than when we
started.
http://www.revolutioncloud.com

Best Practices
• Keep data in multiple Availability Zones
http://www.revolutioncloud.com

Best Practices
• Keep data in multiple Availability Zones
• EBS for all persistent data
http://www.revolutioncloud.com

Best Practices
• Keep data in multiple Availability Zones
• EBS for all persistent data
• Snapshots
http://www.revolutioncloud.com

Best Practices
• Keep data in multiple Availability Zones
• EBS for all persistent data
• Snapshots
• No secret keys on the instance
http://www.revolutioncloud.com

Best Practices
• Keep data in multiple Availability Zones
• EBS for all persistent data
• Snapshots
• No secret keys on the instance
• Different functions in different Security
Groups
http://www.revolutioncloud.com

Best Practices
• A full stack in each zone
http://www.revolutioncloud.com

Best Practices
• A full stack in each zone
• All data stored as key-value pairs
http://www.revolutioncloud.com

Best Practices
• A full stack in each zone
• All data stored as key-value pairs
• More use of queues
http://www.revolutioncloud.com

Conclusion
• AWS saves us a lot of money
http://www.revolutioncloud.com

Conclusion
• AWS saves us a lot of money
• AWS gives us a lot of flexibility
http://www.revolutioncloud.com

Conclusion
• AWS saves us a lot of money
• AWS gives us a lot of flexibility
• Moving to AWS has forced us to build
better applications and follow better IT
practices.
http://www.revolutioncloud.com

Conclusion
• AWS saves us a lot of money
• AWS gives us a lot of flexibility
• Moving to AWS has forced us to build
better applications and follow better IT
practices.
• http://code.reddit.com
http://www.revolutioncloud.com

Revolution: Cloud
To learn more, visit
http://www.revolutioncloud.com
http://www.revolutioncloud.com