Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
AWS Cloud Tour 2011, Australia<br />Closing Keynote:<br />How Amazon.com<br />Migrated to AWS<br />Jon Jenkins<br />Direct...
amazon.com’sJourney to the Cloud<br />Jon Jenkins<br />jjenkin@amazon.com<br />Twitter - #awstour<br />AWS Cloud Tour<br />
≠<br />
1995 - 2011<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006  ...
1995    1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008  ...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
+<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007   ...
First real data center<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
Distribution Center Isolation<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2004  ...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
“We have 50 million lines of C++ code. No, it's more than that now. I don't know what it is anymore. It was 50 million las...
Decouple<br />Service Oriented Architecture<br />Scale Horizontally<br />Increase Speed of Execution<br />Develop Iterativ...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
What could we do with just S3?<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2004 ...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
The Problem<br /><ul><li>Release process is coupled
Runtime latency & scale requirements
Service integration issues</li></ul>The Solution<br /><ul><li>Use S3 as a service
Store raw HTML for the feature in S3</li></ul>1995     1996     1997     1998     1999     2000     2001     2002     2003...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
Results<br /><ul><li>Reduced page latency
IMDB doesn’t worry about scaling
Reduced web server CPU utilization
Improved availability through reduced dependencies
Simplified release model
AJAX readiness</li></ul>1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005    ...
What about a more complex case?<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2004...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
The Problem<br /><ul><li>The system has lots of moving parts
It must run in an external data center
It must scale up quickly
Development team is two people</li></ul>The Solution<br /><ul><li>Use as many AWS services as possible</li></ul>1995     1...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
Results<br /><ul><li>Very few dev resources required
Launched without having to negotiate any new datacenter co-lo presence
True external performance metrics
We can test site features in development that have not yet launched
The system scales horizontally to large amounts of traffic</li></ul>1995     1996     1997     1998     1999     2000     ...
What about amazon.com web servers?<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
39%<br />61%<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006 ...
1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006     2007     2008 ...
76%<br />24%<br />1995     1996     1997     1998     1999     2000     2001     2002     2003     2004     2005     2006 ...
The Problem<br /><ul><li>Retail web site hardware is underutilized
Upcoming SlideShare
Loading in …5
×

2011 AWS Tour Australia, Closing Keynote: How Amazon.com migrated to AWS, by Jon Jenkins

6,881 views

Published on

Published in: Spiritual, Technology, Business
  • Be the first to comment

2011 AWS Tour Australia, Closing Keynote: How Amazon.com migrated to AWS, by Jon Jenkins

  1. 1. AWS Cloud Tour 2011, Australia<br />Closing Keynote:<br />How Amazon.com<br />Migrated to AWS<br />Jon Jenkins<br />Director, Software Development<br />Amazon.com<br />
  2. 2. amazon.com’sJourney to the Cloud<br />Jon Jenkins<br />jjenkin@amazon.com<br />Twitter - #awstour<br />AWS Cloud Tour<br />
  3. 3. ≠<br />
  4. 4. 1995 - 2011<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  5. 5. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  6. 6. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  7. 7. +<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  8. 8. First real data center<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  9. 9. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  10. 10. Distribution Center Isolation<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  11. 11. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  12. 12. “We have 50 million lines of C++ code. No, it's more than that now. I don't know what it is anymore. It was 50 million last Christmas, nine months ago, and was expanding at 8 million lines a quarter. The expansion rate was increasing as well. Ouch.”<br />Amazon SDE, internal blog post<br />September 2004<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  13. 13. Decouple<br />Service Oriented Architecture<br />Scale Horizontally<br />Increase Speed of Execution<br />Develop Iteratively<br />Seek Simplicity<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  14. 14. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  15. 15. What could we do with just S3?<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  16. 16. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  17. 17. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />IMDB Widget Architecture<br />
  18. 18. The Problem<br /><ul><li>Release process is coupled
  19. 19. Runtime latency & scale requirements
  20. 20. Service integration issues</li></ul>The Solution<br /><ul><li>Use S3 as a service
  21. 21. Store raw HTML for the feature in S3</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  22. 22. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />Before<br />After<br />
  23. 23. Results<br /><ul><li>Reduced page latency
  24. 24. IMDB doesn’t worry about scaling
  25. 25. Reduced web server CPU utilization
  26. 26. Improved availability through reduced dependencies
  27. 27. Simplified release model
  28. 28. AJAX readiness</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  29. 29. What about a more complex case?<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  30. 30. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  31. 31. The Problem<br /><ul><li>The system has lots of moving parts
  32. 32. It must run in an external data center
  33. 33. It must scale up quickly
  34. 34. Development team is two people</li></ul>The Solution<br /><ul><li>Use as many AWS services as possible</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  35. 35. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  36. 36. Results<br /><ul><li>Very few dev resources required
  37. 37. Launched without having to negotiate any new datacenter co-lo presence
  38. 38. True external performance metrics
  39. 39. We can test site features in development that have not yet launched
  40. 40. The system scales horizontally to large amounts of traffic</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  41. 41. What about amazon.com web servers?<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  42. 42. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  43. 43. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  44. 44. 39%<br />61%<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  45. 45. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  46. 46. 76%<br />24%<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  47. 47. The Problem<br /><ul><li>Retail web site hardware is underutilized
  48. 48. Traffic spikes require heroic effort
  49. 49. Scaling is non-linear</li></ul>The Solution<br /><ul><li>Migrate the entire www.amazon.com web server fleet to AWS</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  50. 50. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  51. 51. November 10, 2010<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  52. 52. Results<br /><ul><li>All traffic for www.amazon.com is now served from AWS
  53. 53. We can dynamically scale the fleet in increments as small as a single host
  54. 54. Traffic spikes can be handled with ease</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  55. 55. What about a DB use case?<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  56. 56. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  57. 57. Basic Order Storage Architecture<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  58. 58. Basic Order Storage Architecture<br />Scaling Pattern 1<br />Scaling Pattern 2<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  59. 59. The Problem<br /><ul><li>Cumulative data impacts scale
  60. 60. No database scaling pattern is ideal
  61. 61. Databases infrastructure is expensive</li></ul>The Solution<br /><ul><li>Create a tiered storage system with AWS</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  62. 62. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20102011<br />
  63. 63. Results<br /><ul><li>670 million (4TB) orders now stored in S3
  64. 64. We are spending way less on DB hosts
  65. 65. Sets us up for migration to RDS / SDB</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 20102011<br />
  66. 66. Lessons learned<br />1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  67. 67. Business Lessons<br /><ul><li>Less time spent on capacity planning
  68. 68. Fewer conversations with finance
  69. 69. More innovation
  70. 70. Happier developers
  71. 71. I get credit for AWS price reductions
  72. 72. Be sure to consider compliance issues
  73. 73. No more lease returns!</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  74. 74. Technical Lessons<br /><ul><li>Start with simple applications
  75. 75. Iterate toward your desired end-state
  76. 76. Identify reusable components
  77. 77. Engage security early and treat them as partners
  78. 78. Migrate to the cloud in concert with your other architectural objectives
  79. 79. The cloud can’t cover up sloppy engineering</li></ul>1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011<br />
  80. 80. amazon.com’sJourney to the Cloud<br />Jon Jenkins<br />jjenkin@amazon.com<br />Twitter - #awstour<br />AWS Cloud Tour<br />

×