AWS Customer Presentation - Conde Nast

1,438 views
1,244 views

Published on

Customer Presentation by Conde Nast at the AWS Cloud for the Enterprise Event in NYC on October 19, 2009

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,438
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • reddit is a subsidiary of Conde Nast. They are a multi-billion doallar media conglomorate that owns TV, newspapers, and some of the biggest brand names in publishing, such as Vogue, GQ, etc
  • They also have a lot of webistes (other than reddit). currently doing an EC2 proof of concept with Vogue UK and wired.com is also using EC2 for some of their site functionality. But reddit is by far the biggest user of EC2 in Conde Nast.
  • community where people come together share and discuss interesting things on the internet such as links to other stuff or their own content.
  • Been AWS user since almost the begining
  • When we started, we had a custom logo almost every day.
  • Unfortunately, allowing the designer SVN access was untenable. So we signed up for S3, and everything was great!
  • Now have over 20 million items in a single s3 bucket.
  • This is a graph of actual costs vs. what we could have spent on EC2. you can see the spike every time we had to get a new cabinet full of servers
  • So we had to make a decision
  • This was before Amazon offered the Virtual Private Cloud service. Ran like this for about 7 months, slowing adding new machines as necessary. This went so well we decided to move our whole site to EC2
  • started by migrating all the data that could be done ahead of time set up replication to keep the data up to date
  • multiple zones beta/staging is elastic app servers are elastic
  • What our datacenter would have cost vs. what we pay now 29% cheaper!
  • Time to market is faster because I no longer have to wait for servers or set them up. i can get capacity when i need it
  • As Amazon upgrades their hardware, I can move up to better stuff.
  • Don’t have to pay for capacity we don’t need. I can shut down servers at night, or if a product doesn’t take off
  • But moving to EC2 wasn’t all roses
  • Virtualized hardware just doesn’t get sub-millisecond response times. We had to rethink how we used memcached, making less calls for more data at a time
  • Due to the redundant and network based nature, sometimes the underlying drive has to remirror or the network may be momentarily unavailable. This can be avoided by having read slaves and caching It forced me to add read slaves I should have had anyway
  • It is a new paradigm you have to get used to. Ideally the machine can be told to boot and then be ready with no user intervention.
  • I can’t stress this enough.
  • Eric Hammond’s runurl to trigger a push of the keys to only authorized hosts.
  • get a whole application stack in at least 2 zones
  • aka. nosql. We’re mostly there, just a few more bits
  • A more functional style of programming is generally more reliable in exchange for eventual consistancy
  • We are open source, so go to code.reddit.com to see our source.
  • This the website of our book which we hope to start filling with lots of useful stuff soon. stay tuned. thank you.
  • AWS Customer Presentation - Conde Nast

    1. 1. Rajiv Pant / Paul Fisher <ul><li>VP Technology, Conde Nast Digital </li></ul><ul><li>Revolution:Cloud , Co-Author </li></ul><ul><li>www.revolutioncloud.com </li></ul>Manager of Technology, Wired.com Revolution:Cloud , Co-Author www.revolutioncloud.com www.revolutioncloud.com
    2. 2. Conde Nast Digital
    3. 3. Conde Nast websites
    4. 4. — About Wired.com <ul><li>High-Traffic Technology News Website </li></ul><ul><li>Millions of Page Views each Day </li></ul><ul><li>Numerous Site Components, implemented in Java, Grails, and PHP </li></ul><ul><li>Many Disparate Features, including Admin, CMS, and User-Facing Functionality </li></ul><ul><li>News Content, Product Reviews, Widgets, etc. </li></ul>
    5. 5. — About Wired.com
    6. 6. — Leveraging of EC2 <ul><li>Rapid Prototyping and Deployment </li></ul><ul><li>Wired Widgets </li></ul>
    7. 7. <ul><li>Rapid Prototyping and Deployment </li></ul><ul><li>Wired Widgets </li></ul>— Leveraging of EC2
    8. 8. <ul><li>Rapid Prototyping and Deployment </li></ul><ul><li>Wired Widgets </li></ul><ul><li>Dataset Wiki </li></ul>— Leveraging of EC2
    9. 9. <ul><li>Rapid Prototyping and Deployment </li></ul><ul><li>Wired Widgets </li></ul><ul><li>Dataset Wiki </li></ul>— Leveraging of EC2
    10. 10. <ul><li>Rapid Prototyping and Deployment </li></ul><ul><li>Wired Widgets </li></ul><ul><li>Dataset Wiki </li></ul>— Leveraging of EC2
    11. 11. <ul><li>Rapid Prototyping and Deployment </li></ul><ul><li>Wired Widgets </li></ul><ul><li>Dataset Wiki </li></ul>— Leveraging of EC2
    12. 12. <ul><li>Rapid Prototyping and Deployment </li></ul><ul><li>Wired Widgets </li></ul><ul><li>Dataset Wiki </li></ul><ul><li>Production CMS Components — Post-Process content and deploy images to S3 </li></ul>— Leveraging of EC2
    13. 13. <ul><li>These Projects utilized S3 and EC2 </li></ul><ul><li>Rapid Development via Grails </li></ul><ul><li>Production Deployment and integration with stack via CNAME </li></ul><ul><li>Critical CMS Workflow Components Rely on S3 for content deployment and user-facing asset access. </li></ul>— Leveraging of EC2
    14. 14. <ul><li>Other Administrative Tools — Preparing and Editing Production Content </li></ul><ul><li>Product Reviews Application </li></ul><ul><ul><li>Began as Simple Admin Tool </li></ul></ul><ul><ul><li>Short Timeline and “In-Book” Deadline </li></ul></ul><ul><ul><li>Selected EC2 to avoid delays — set-up production environment within a day </li></ul></ul>— Leveraging of EC2
    15. 15. <ul><li>EC2 simplified Development and Staging </li></ul><ul><li>Deployed and Tested application by duplicating image and environment </li></ul><ul><li>Product Reviews Application posed integration challenges: </li></ul><ul><ul><li>Major Site Component </li></ul></ul><ul><ul><li>Short Timeframe to Go Live </li></ul></ul>— Leveraging of EC2
    16. 16. <ul><li>Product Reviews: </li></ul><ul><li>Developed Admin Tool and User-Facing Application using Grails </li></ul><ul><li>Integrated application via Proxying from internal hosting infrastructure </li></ul><ul><li>Solution allowed us to leverage EC2 without limiting functionality or seamlessness of integration </li></ul>— Leveraging of EC2
    17. 17. <ul><li>Product Reviews: </li></ul>— Leveraging of EC2
    18. 18. <ul><li>Product Reviews: </li></ul>— Leveraging of EC2
    19. 19. <ul><li>Product Reviews: </li></ul>— Leveraging of EC2
    20. 20. <ul><li>Product Reviews — Admin Tool: </li></ul>— Leveraging of EC2
    21. 21. What is reddit? <ul><li>reddit is an online community </li></ul>
    22. 22. Timeline <ul><li>April 2006 -- S3 for logos </li></ul><ul><li>September 2007 -- S3 for thumbnails </li></ul><ul><li>November 2008 -- EC2 for batch processing </li></ul><ul><li>May 2009 -- EC2 for entire site </li></ul>
    23. 23. What led us to AWS (part 1) <ul><li>Needed an easy way to distribute and upload our logo </li></ul>
    24. 25. What led us to AWS (part 2) <ul><li>Thumbnails! </li></ul>
    25. 26. What led us to AWS (part 3) <ul><li>Didn’t want to rent another cabinet </li></ul>
    26. 27. What led us to AWS (part 3) <ul><li>Didn’t want to rent another cabinet </li></ul><ul><li>Didn’t want to buy more servers </li></ul>
    27. 28. What led us to AWS (part 3) <ul><li>Didn’t want to rent another cabinet </li></ul><ul><li>Didn’t want to buy more servers </li></ul>New Servers New Servers
    28. 29. Imaging and Racking Servers Is A (Sometimes Fun) Chore
    29. 30. EC2 for Overflow <ul><li>Used openvpn to create a secure link to our datacenter for batch processing </li></ul>
    30. 31. Moving to EC2 <ul><li>Started by migrating all data </li></ul>
    31. 32. <ul><li>Started by migrating all data </li></ul><ul><li>Got a complete stack running on EC2 </li></ul>Moving to EC2
    32. 33. <ul><li>Started by migrating all data </li></ul><ul><li>Got a complete stack running on EC2 </li></ul><ul><li>Long Saturday night finishing the migration and “forklifting” the last bits of data </li></ul>Moving to EC2
    33. 34. Architecture
    34. 35. Stats <ul><li>190 Virtual CPUs </li></ul><ul><li>338GB of RAM </li></ul><ul><li>9TB of Elastic Block Storage </li></ul><ul><li>2TB of S3 Storage </li></ul><ul><li>6.5 TB of Data Out / mo </li></ul><ul><li>2TB of Data In / mo </li></ul><ul><li>150M+ Pageviews and just one sysadmin! </li></ul>
    35. 36. Benefits Estimated Pricing Based on public Amazon pricing, reddit open source code, and public configuration information Data Center (per month) Servers: $6K Cabinet (x3): $15K Bandwidth: $2.5K Support: N/A Total: $23.5K EC2 (per month) Servers: $13K Storage: $1.5K Bandwidth: $1.1K Support: $1.2K Total: $16.8K 29% Cheaper!
    36. 37. Benefits <ul><li>Don’t have to procure servers anymore </li></ul>
    37. 38. Benefits <ul><li>Don’t have to procure servers anymore </li></ul><ul><li>No racking or imaging servers anymore </li></ul>
    38. 39. Benefits <ul><li>Don’t have to procure servers anymore </li></ul><ul><li>No racking or imaging servers anymore </li></ul><ul><li>Time to market is faster </li></ul>
    39. 40. Benefits <ul><li>Don’t have to procure servers anymore </li></ul><ul><li>No racking or imaging servers anymore </li></ul><ul><li>Time to market is faster </li></ul><ul><li>Multiple physical locations with AZs </li></ul>
    40. 41. Benefits <ul><li>Don’t have to procure servers anymore </li></ul><ul><li>No racking or imaging servers anymore </li></ul><ul><li>Time to market is faster </li></ul><ul><li>Multiple physical locations with AZs </li></ul><ul><li>Get “free” upgrades </li></ul>
    41. 42. Benefits <ul><li>Don’t have to procure servers anymore </li></ul><ul><li>No racking or imaging servers anymore </li></ul><ul><li>Time to market is faster </li></ul><ul><li>Multiple physical locations with AZs </li></ul><ul><li>Get “free” upgrades </li></ul><ul><li>Elasticity! </li></ul>
    42. 43. Pain Points
    43. 44. Pain Points
    44. 45. Pain Points <ul><li>Higher Latency </li></ul><ul><li>Workaround: Fewer network calls, ask for more data at a time. </li></ul>
    45. 46. Pain Points <ul><li>EBS sometimes slows down a bit </li></ul><ul><li>Workaround: Use caching and replication with read slaves to avoid relying on a single disk. </li></ul>
    46. 47. Pain Points <ul><li>Instances go away sometimes </li></ul><ul><li>Workaround: Avoid single points of failure and make sure your servers have automated configuration. </li></ul>
    47. 48. Pain Points <ul><li>Fixing these issues made our app more reliable and highly available. We are better off than when we started. </li></ul>
    48. 49. Best Practices <ul><li>Keep data in multiple Availability Zones </li></ul>
    49. 50. Best Practices <ul><li>Keep data in multiple Availability Zones </li></ul><ul><li>EBS for all persistent data </li></ul>
    50. 51. Best Practices <ul><li>Keep data in multiple Availability Zones </li></ul><ul><li>EBS for all persistent data </li></ul><ul><li>Snapshots </li></ul>
    51. 52. Best Practices <ul><li>Keep data in multiple Availability Zones </li></ul><ul><li>EBS for all persistent data </li></ul><ul><li>Snapshots </li></ul><ul><li>No secret keys on the instance </li></ul>
    52. 53. Best Practices <ul><li>Keep data in multiple Availability Zones </li></ul><ul><li>EBS for all persistent data </li></ul><ul><li>Snapshots </li></ul><ul><li>No secret keys on the instance </li></ul><ul><li>Different functions in different Security Groups </li></ul>
    53. 54. Best Practices <ul><li>A full stack in each zone </li></ul>
    54. 55. Best Practices <ul><li>A full stack in each zone </li></ul><ul><li>All data stored as key-value pairs </li></ul>
    55. 56. Best Practices <ul><li>A full stack in each zone </li></ul><ul><li>All data stored as key-value pairs </li></ul><ul><li>More use of queues </li></ul>
    56. 57. Conclusion <ul><li>AWS saves us a lot of money </li></ul>
    57. 58. Conclusion <ul><li>AWS saves us a lot of money </li></ul><ul><li>AWS gives us a lot of flexibility </li></ul>
    58. 59. Conclusion <ul><li>AWS saves us a lot of money </li></ul><ul><li>AWS gives us a lot of flexibility </li></ul><ul><li>Moving to AWS has forced us to build better applications and follow better IT practices. </li></ul>
    59. 60. Conclusion <ul><li>AWS saves us a lot of money </li></ul><ul><li>AWS gives us a lot of flexibility </li></ul><ul><li>Moving to AWS has forced us to build better applications and follow better IT practices. </li></ul><ul><li>http://code.reddit.com </li></ul>
    60. 61. Revolution: Cloud <ul><li>To learn more, visit </li></ul><ul><li>http://www.revolutioncloud.com </li></ul>

    ×