Drinking our own Champagne: How Woot, an Amazon subsidiary, uses AWS (ARC212) | AWS re:Invent 2013

5,404 views
5,158 views

Published on

Woot, an Amazon subsidiary, specializes in offering great new product deals every day. Woot's deeply discounted deals; and signature events like the 'Woot Off 'and 'Bag of Crap' sales launch at specific times throughout the day, and the resulting spiky traffic patterns are highly correlated to revenue.
In this session, we offer an unvarnished perspective into how Woot uses services such as Amazon DynamoDB, EC2, ELB, CloudSearch, CloudFront, and SES. Learn how to architect for security and PCI for a retail website running on AWS. Dig into the technical details of a data-store comparison between DynamoDB, Mongo, Oracle, and SQLServer, to find the right solution for unique workloads. Join us as we share our musings and real-lessons learned from using a cocktail of AWS services. We encourage you to attend even if none of this makes sense or is interesting. Don't miss the opportunity to hang out with Mortimer the Woot monkey and his crew and to walk away with one of our legendary flying monkeys.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,404
On SlideShare
0
From Embeds
0
Number of Embeds
17
Actions
Shares
0
Downloads
21
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Drinking our own Champagne: How Woot, an Amazon subsidiary, uses AWS (ARC212) | AWS re:Invent 2013

  1. 1. Drinking Our Own Champagne: How Woot, an Amazon subsidiary, uses AWS technologies Vivek Sagi, CTO, Woot - vsagi@woot.com Dan Pinkard, Systems Manager, Woot – dpinkard@woot.com November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. Introduction to Woot
  3. 3. Woot Trivia (Question 1) Q: What is Woot? A. A rabid monkey only found in the Amazon jungle B. A daily deal site, flash site and vibrant deal community C. The name of a dance performed by hungry SDEs D. A retail website selling toys 11/15/2013 Amazon Confidential 3
  4. 4. Woot Trivia (Question 2) Q: Name the event on the Woot website where we sell a random collection of leftover inventory and “surprises”? A. Bunch of Carrots No Canadians were harmed in the making of this presentation B. Bitter Old Canadians C. Baldness on Curly D. Bag of Crap 11/15/2013 Amazon Confidential 4
  5. 5. Woot Trivia (Question 3) Q: When was Woot acquired by Amazon? A. 2004 B. 2008 C. 2010 D. None of the above because Woot is only a figment of a six year old’s imagination 11/15/2013 Amazon Confidential 5
  6. 6. Ye Olde Woot 11/15/2013 Amazon Confidential 6
  7. 7. Modern Day Woot Growth of new categories Introduction of Woot! Plus flash sales 11/15/2013 11/15/2013 Amazon Confidential Amazon Confidential 7 7
  8. 8. It’s All About the Benjamins
  9. 9. Woot’s technology stack
  10. 10. Woot (Pre-Acquisition) S3 EC2 • Traditional co-location (since 2003) – 8 servers – Monolithic application • Amazon S3 for images from 2006 • Woot Deals launch in 2009 – Amazon EC2 hosted (public) – SimpleDB MongoDB + SOLR – Amazon SQS to relay to Woot Retail ELB SQS
  11. 11. Woot (Post-Acquisition) S3 EC2 • Migrated to Amazon VPC (2011) – – – – – 2 months to prep VPC VPN to colo (for secure data migration) P2V for SQL servers Migrate MongoDB replicas 18 hours to move production services ELB SQS Route 53 CloudFront SES VPC RDS • Went from 8 servers to 20
  12. 12. Application Rewrite in 2012 • Business Goals – Support multiple deal types (Daily, Flash, VIP) – Changes must be visible quickly • Technology Goals – – – – Services-oriented architecture Improve site reliability Optimize for cloud deployments Language and platform neutral
  13. 13. Woot Current Technologies • Core Technologies – – – – C#, ASP.Net, MVC4, WCF, MongoDB (All Content), SQL Server (Transactional), DynamoDB (Shopping Cart) Memcached RabbitMQ • Reporting & Operations Technologies – – – – – Node.JS, Perl, Python, Ruby, PowerShell Durango, Rails Oracle, MySQL Redis Bind9, Postfix
  14. 14. Woot’s AWS footprint
  15. 15. AWS Footprint S3 EC2 ELB RDS SQS CloudSearch Route 53 CloudFront DynamoDB SES VPC IAM
  16. 16. Network Infrastructure • Multi-node VPN • 3 accounts • 3 environments Staging Production Development
  17. 17. Monitoring • System Center + AWS Management Pack – .Net aware application monitoring – Ties to external monitors – Correlates security and system events
  18. 18. Monitoring • System Center + AWS Management Pack • OpenNMS – Service discovery – SNMP integration – Multi-AZ perspective
  19. 19. Monitoring • System Center + AWS Management Pack • OpenNMS • StatsD + Graphite + Skyline – Great dashboards – Measure everything – High resolution – Flexible graphing options
  20. 20. Monitoring • • • • System Center + AWS Management Pack OpenNMS StatsD + Graphite + Skyline Amazon CloudWatch – Most direct source of truth – Systems performance and usage metrics
  21. 21. Monitoring • • • • • System Center + AWS Management Pack OpenNMS StatsD + Graphite + Skyline Amazon CloudWatch Secret Sauce(s) – Graphite as graphing platform – Integrate Amazon CloudWatch – Track AMPQ stats
  22. 22. Woot SOA Overview SG – Retail Public SG – Retail Private SG - Catalog Public SG - Catalog Private SG – Content Public SG - Content Private SG – Users Public SG – Users Private CloudFront Public
  23. 23. SOA Benefits for Woot • • • • • Scalability Organizational flexibility Security Faster time to market Improved reliability
  24. 24. How do we choose a technology?
  25. 25. Woot Throwdown Tenets • • • • Don’t make assumptions; all options on the table AWS is not the default choice Simulate real-world behavior Compare as many facets as possible - Driver support - Resiliency - Performance - Support - Scalability - Cost
  26. 26. Throw Down 1: Load Balancers Contenders Pros Cons Varnish Flexibility No SSL NginX Lightweight and fast No HTTP1/1 to origin Apache HTTP1/1 in/out Hard to tune Elastic Load Balancing Cloud Aware Limited logging
  27. 27. Throw Down 2: Cart Persistence Storage Woot “Cart Service” – Simple – Host a bag of crap – Fast – Scalable to follow growth – Resilient – Lots of small data facts – Secure patterns
  28. 28. Cart Persistent Storage Document Databases Contenders Client API High availability Performance Ramp-up time Tools Cost Operational management Debugging Ad-hoc queries (analytics) Support Documentation Total Relational Databases DynamoDB MongoDB Oracle SQL Server 4 5 2 5 5 5 2 5 5 4 -- -- 3 5 1 2 4 4 4 5 5 1 3 3 5 4 3 3 4 5 5 5 3 4 5 5 5 5 4 4 5 5 5 5 48 47 -- --
  29. 29. Cart Persistent Storage Document Databases Criteria Client API High availability Performance Ramp-up time Tools Cost Operational management Debugging Ad-hoc queries (analytics) Support Documentation Total Relational Databases DynamoDB MongoDB Oracle SQL Server 4 5 2 5 5 5 2 5 5 4 -- -- 3 5 1 2 4 4 4 5 5 1 3 3 5 4 3 3 4 5 5 5 3 4 5 5 5 5 4 4 5 5 5 5 48 47 -- --
  30. 30. Cart Persistent Storage Document Databases Relational Databases DynamoDB Client API High availability Performance Ramp-up time Tools Cost Operational management Debugging Ad-hoc queries (analytics) Support Documentation Total MongoDB Oracle SQL Server 4 5 2 5 5 5 2 5 5 4 -- -- 3 5 1 2 4 4 4 5 5 1 3 3 5 4 3 3 4 5 5 5 3 4 5 5 5 5 4 4 5 5 5 5 48 47 -- --
  31. 31. Cart Throw Down Results - MongoDB and DynamoDB were our finalists - DynamoDB outperformed MongoDB in our performance tests - We could launch and scale DynamoDB with a button push - We could host a bag of crap
  32. 32. Cart Persistent Storage: Cost Analysis Cost ($/Month) 3500 3000 2500 2000 1500 1000 500 0 DynamoDB MongoDB Oracle RDS SQL Server
  33. 33. Cart Persistent Storage: Performance Operations / second 10000 2060 1348 1000 100 64 107 42 70 10 1 Add Items Get DynamoDB MongoDB Update Items
  34. 34. DynamoDB – Rough Edges • Only basic index support (primary, secondary) – Use searching technologies such as Amazon CloudSearch to index data • Ad-hoc query support not so simple – Export data we want to query into a data warehouse
  35. 35. Cart in Production • • • • • 20,000,000,000 requests served Sub 10ms response times 0 errors reported since launch No maintenance No SLA misses
  36. 36. Recovering from a cloud outage
  37. 37. Triage of an outage: July 2, 2012 • Massive Storm + Maintenance Problem = No Power in an Availability Zone • Availability Zone hosted Woot’s primary SQL and AD role masters • Amazon EBS failures prevented recovery • AD DNS collapsed
  38. 38. Lessons Learnt • Data integrity means more Availability Zones – Mirror/replicate to other Availability Zones – Lots of hosts for MongoDB – Dedicated hosts for irregular patterns • Backups • Long-running queries • Specialized indexes
  39. 39. Lessons Learnt • Data integrity means more Availability Zones • Queue everything – – – – Decouple systems Save state information Inspection of service operation Recoverable
  40. 40. Lessons Learnt • Data integrity means more Availability Zones • Queue everything • Hosts are disposable – – – – Don’t store unique data Push-button launches Centralize configuration Dynamic role assignment
  41. 41. Lessons Learnt • • • • Data integrity means more Availability Zones Queue everything Hosts are disposable Unique solutions – Different failure modes than traditional data center – Combine off-the shelf items for resiliency
  42. 42. Final thoughts on the cloud
  43. 43. Security • Same problems as anywhere else – Centralize logging – Centralize accounts • Application-level logging – Gain visibility – Follow state • Use application tiers – SOA separation
  44. 44. Best Practices • CloudSearch Best Practices – Don’t make too many “cs-describe-domain” calls – Perform updates in batches
  45. 45. Best Practices • CloudSearch Best Practices – Don’t make too many “cs-describe-domain” calls – Perform updates in batches • EC2+EBS Best Practices – Deploy EBS-optimized hosts for required bandwidth – Provisioned IOPs to ensure lower latency for disks
  46. 46. Best Practices • CloudSearch Best Practices – Don’t make too many “cs-describe-domain” calls. – Perform updates in batches • EC2+EBS Best Practices – Deploy EBS Optimized hosts for required bandwidth – Provisioned IOPs to ensure lower latency for disks
  47. 47. Best Practices • CloudSearch Best Practices – Don’t make too many “cs-describe-domain” calls – Perform updates in batches • EC2+EBS Best Practices – EBS-optimized hosts for more bandwidth – Provisioned IOPs to ensure lower latency • Scaling Best Practices – No reward for using too much or too little
  48. 48. Woot’s AWS Experience • Tools continue to improve - AWS Dashboard - Expanded CloudWatch - AWS Trusted Advisor - Premium Support • AWS advocates for users • Capacity really is elastic • The more we see, the better it looks
  49. 49. Please give us your feedback on this presentation ARC212 As a thank you, we will select prize winners daily for completed surveys!

×