Wired.com and Reddit.com Presentation at AWS Event in NYC

Rajiv Pant / Paul Fisher
VP Technology, Conde Nast Digital
Revolution:Cloud , Co-Author
www.revolutioncloud.com
Manager of Technology, Wired.com Revolution:Cloud , Co-Author www.revolutioncloud.com www.revolutioncloud.com

Conde Nast Digital

Conde Nast websites

— About Wired.com
High-Traffic Technology News Website
Millions of Page Views each Day
Numerous Site Components, implemented in Java, Grails, and PHP
Many Disparate Features, including Admin, CMS, and User-Facing Functionality
News Content, Product Reviews, Widgets, etc.

— About Wired.com

— Leveraging of EC2
Rapid Prototyping and Deployment
Wired Widgets

Wired Widgets

Wired Widgets
Dataset Wiki

Wired Widgets
Dataset Wiki
Production CMS Components — Post-Process content and deploy images to S3

These Projects utilized S3 and EC2
Rapid Development via Grails
Production Deployment and integration with stack via CNAME
Critical CMS Workflow Components Rely on S3 for content deployment and user-facing asset access.

Other Administrative Tools — Preparing and Editing Production Content
Product Reviews Application
Began as Simple Admin Tool
Short Timeline and “In-Book” Deadline
Selected EC2 to avoid delays — set-up production environment within a day

EC2 simplified Development and Staging
Deployed and Tested application by duplicating image and environment
Product Reviews Application posed integration challenges:
Major Site Component
Short Timeframe to Go Live

Product Reviews:
Developed Admin Tool and User-Facing Application using Grails
Integrated application via Proxying from internal hosting infrastructure
Solution allowed us to leverage EC2 without limiting functionality or seamlessness of integration

Product Reviews:

Product Reviews — Admin Tool:

What is reddit?
reddit is an online community

Timeline
April 2006 -- S3 for logos
September 2007 -- S3 for thumbnails
November 2008 -- EC2 for batch processing
May 2009 -- EC2 for entire site

What led us to AWS (part 1)
Needed an easy way to distribute and upload our logo

Thumbnails!

Didn’t want to rent another cabinet

Didn’t want to buy more servers

Didn’t want to buy more servers
New Servers New Servers

Imaging and Racking Servers Is A (Sometimes Fun) Chore

EC2 for Overflow
Used openvpn to create a secure link to our datacenter for batch processing

Moving to EC2
Started by migrating all data

Moving to EC2
Got a complete stack running on EC2

Moving to EC2
Got a complete stack running on EC2
Long Saturday night finishing the migration and “forklifting” the last bits of data

Architecture

Stats
190 Virtual CPUs
338GB of RAM
9TB of Elastic Block Storage
2TB of S3 Storage
6.5 TB of Data Out / mo
2TB of Data In / mo
150M+ Pageviews and just one sysadmin!

Benefits Estimated Pricing Based on public Amazon pricing, reddit open source code, and public configuration information Data Center (per month) Servers: $6K Cabinet (x3): $15K Bandwidth: $2.5K Support: N/A Total: $23.5K EC2 (per month) Servers: $13K Storage: $1.5K Bandwidth: $1.1K Support: $1.2K Total: $16.8K 29% Cheaper!

Benefits
Don’t have to procure servers anymore

Benefits
No racking or imaging servers anymore

Benefits
Time to market is faster

Benefits
Multiple physical locations with AZs

Benefits
Get “free” upgrades

Benefits
Get “free” upgrades
Elasticity!

Pain Points

Pain Points
Higher Latency
Workaround: Fewer network calls, ask for more data at a time.

Pain Points
EBS sometimes slows down a bit
Workaround: Use caching and replication with read slaves to avoid relying on a single disk.

Pain Points
Instances go away sometimes
Workaround: Avoid single points of failure and make sure your servers have automated configuration.

Pain Points
Fixing these issues made our app more reliable and highly available. We are better off than when we started.

Best Practices
Keep data in multiple Availability Zones

Best Practices
EBS for all persistent data

Best Practices
Snapshots

Best Practices
Snapshots
No secret keys on the instance

Best Practices
Snapshots
No secret keys on the instance
Different functions in different Security Groups

Best Practices
A full stack in each zone

Best Practices
All data stored as key-value pairs

Best Practices
All data stored as key-value pairs
More use of queues

Conclusion
AWS saves us a lot of money

Conclusion
AWS gives us a lot of flexibility

Conclusion
Moving to AWS has forced us to build better applications and follow better IT practices.

Conclusion
Moving to AWS has forced us to build better applications and follow better IT practices.
http://code.reddit.com

Revolution: Cloud
To learn more, visit
http://www.revolutioncloud.com

Wired.com and Reddit.com Presentation at AWS Event in NYC

by Revolution Cloud on Oct 20, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 5

Statistics

Wired.com and Reddit.com Presentation at AWS Event in NYC — Presentation Transcript