• Save
AWS Summit 2011: Closing Keynote : The Story of Amazon.com's Move to the AWS Cloud
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

AWS Summit 2011: Closing Keynote : The Story of Amazon.com's Move to the AWS Cloud

on

  • 2,398 views

 

Statistics

Views

Total Views
2,398
Views on SlideShare
2,397
Embed Views
1

Actions

Likes
1
Downloads
23
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

AWS Summit 2011: Closing Keynote : The Story of Amazon.com's Move to the AWS Cloud Presentation Transcript

  • 1. amazon.com’sJourney to the Cloud Jon Jenkins jjenkin@amazon.com AWS Summit June 13, 2011
  • 2. 1995 - 20101995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 3. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 4. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 5. +1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 6. First real data center1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 7. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 8. Distribution Center Isolation1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 9. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 10. Decouple Service Oriented Architecture Scale Horizontally Increase Speed of Execution Develop Iteratively Seek Simplicity1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 11. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 12. What could we do with just S3?1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 13. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 14. IMDB Widget Architecture Amazon Amazon Web Server IMDB Service Call Customer IMDB Service IMDB Database Render Process1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 15. The Problem • Release process is coupled • Runtime latency & scale requirements • Service integration issues The Solution • Use S3 as a service • Store raw HTML for the feature in S31995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 16. Before Amazon Amazon Web Server IMDB Service Call Customer IMDB Service IMDB Database Render Process After Amazon IMDB Amazon Web Server Generic S3 HTML S3 Customer Puller HTML Store1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 17. Results • Reduced page latency • IMDB doesn’t worry about scaling • Reduced web server CPU utilization • Improved availability through reduced dependencies • Simplified release model • AJAX readiness1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 18. What about a more complex case?1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 19. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 20. The Problem • The system has lots of moving parts • It must run in an external data center • It must scale up quickly • Development team is two people The Solution • Use as many AWS services as possible1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 21. CloudWatch SQS (latency, test case Alarm Rule results, status codes) Config Engines Store Agents Notification Notification Web Portal Scheduler S3 Configuration (primary/ EC2 secondary) EC2 Web Service SDB ... EC2 RDS1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 22. Results • Very few dev resources required • Launched without having to negotiate any new datacenter co-lo presence • True external performance metrics • We can test site features in development that have not yet launched • The system scales horizontally to large amounts of traffic1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 23. What about amazon.com web servers?1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 24. Typical Weekly Traffic to amazon.com Sunday Monday Tuesday Wednesday Thursday Friday Saturday1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 25. Typical Weekly Traffic to amazon.com Sunday Monday Tuesday Wednesday Thursday Friday Saturday1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 26. Typical Weekly Traffic to amazon.com 39% 61% Sunday Monday Tuesday Wednesday Thursday Friday Saturday1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 27. November Traffic for amazon.com1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 28. November Traffic for amazon.com 76% 24%1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 29. The Problem • Retail web site hardware is underutilized • Traffic spikes require heroic effort • Scaling is non-linear The Solution • Migrate the entire www.amazon.com web server fleet to AWS1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 30. amazon.com AWS Availability Zone 1 ... EC2 www 1 EC2 www N Load Balancer Customer ... Availability Zone N VPC ... Databases Services EC2 www 1 EC2 www N1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 31. November 10, 20101995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 32. Results • All traffic for www.amazon.com is now served from AWS • We can dynamically scale the fleet in increments as small as a single host • Traffic spikes can be handled with ease1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 33. What about a DB use case?1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 34. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 35. Basic Order Storage Architecture Order Database Web Servers Ordering Service1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 36. Basic Order Storage Architecture Order Database Web Servers Ordering Service Scaling Pattern 1 Scaling Pattern 2 Order Order Database DatabaseWeb Servers Ordering Service Web Servers Ordering Service1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 37. The Problem • Cumulative data impacts scale • No database scaling pattern is ideal • Databases infrastructure is expensive The Solution • Create a tiered storage system with AWS1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 38. Order Database Web Servers Ordering Service S31995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 39. Results • 670 million (4TB) orders now stored in S3 • We are spending way less on DB hosts • Sets us up for migration to RDS / SDB1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 40. Lessons learned1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 41. Business Lessons • Less time spent on capacity planning • Fewer conversations with finance • More innovation • Happier developers • I get credit for AWS price reductions • Be sure to consider compliance issues • No more lease returns!1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 42. Technical Lessons • Start with simple applications • Iterate toward your desired end-state • Identify reusable components • Engage security early and treat them as partners • Migrate to the cloud in concert with your other architectural objectives • The cloud can’t cover up sloppy engineering1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
  • 43. Q&A1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011