Why Startups Are Still On AWS

3,154 views

Published on

Published in: Technology, Business
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,154
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
37
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

Why Startups Are Still On AWS

  1. 1. WHY STARTUPS ARE (STILL) ON AMAZON WEB SERVICES Danilo Poccia | Solutions Architect
  2. 2. Startups? http://www.flickr.com/photos/klearchos/5857451393/
  3. 3. « A startup is a human institution designed to create a new product or service under conditions of extreme uncertainty. » Eric Ries, The Lean Startup
  4. 4. Startups fail?
  5. 5. Assume customer is known Assume features are known Assume solution is known
  6. 6. Waterfall approach You know the problem and the solution
  7. 7. TEST LAUNCHBUILDSPEC Known set of requirements Known ways to satisfy them
  8. 8. Agile methodologies You know the problem, not the solution
  9. 9. TEST LAUNCHBUILDSPEC Known set of requirements Unknown ways to satisfy them VALID?
  10. 10. You don't know precisely what problem you solve
  11. 11. Lean startups: LEARN & ADAPT
  12. 12. 1.  Focus on a simple implementation of your idea
  13. 13. 1.  Focus on a simple implementation of your idea 2.  Start with a minimal core set of features
  14. 14. 1.  Focus on a simple implementation of your idea 2.  Start with a minimal core set of features 3.  Release and listen to your users
  15. 15. 1.  Focus on a simple implementation of your idea 2.  Start with a minimal core set of features 3.  Release and listen to your users 4.  Question your initial assumptions based on feedback
  16. 16. 1.  Focus on a simple implementation of your idea 2.  Start with a minimal core set of features 3.  Release and listen to your users 4.  Question your initial assumptions based on feedback 5.  Rinse and repeat
  17. 17. MVP Minimum Viable Product
  18. 18. MVP Smallest thing I can do to test my idea?
  19. 19. « If you're not embarrassed when you ship your first version you waited too long » Matt Mullenweg CEO & Founder of WordPress
  20. 20. Staying lean is creating the smallest viable product and then iterate around it You don't know precisely your user's needs
  21. 21. RELEASE EVALBUILDIDEA Unknown set of requirements Unknown ways to satisfy them ITERATE OR PIVOT
  22. 22. RELEASE EVALBUILDIDEA ITERATE OR PIVOT
  23. 23. YOUR problem you have the idea you don't have the resources
  24. 24. It should be cheap and validate ideas
  25. 25. SPEED AND AGILITY Experiment Often Fail quickly at a low cost More Innovation Experiment Infrequently Failure is expensive Less Innovation “ON-PREMISE”
  26. 26. RELEASE EVALBUILDIDEA ITERATE OR PIVOT
  27. 27. you created a fantastic webapp http://www.flickr.com/photos/scobleizer/3985020876/
  28. 28. people love your app http://www.flickr.com/photos/grantrobertson/
  29. 29. and everyone wants to use it!
  30. 30. http://www.flickr.com/photos/livingos/4480894461/
  31. 31. the traditional way…
  32. 32. invest on infrastructure
  33. 33. capacity planning… http://www.flickr.com/photos/mutsmuts/ …capacity guessing
  34. 34. once it's deployed… maintenance? monitoring? log analysis? test environments? http://www.old-computers.com/news
  35. 35. Unable to serve customers Infrastructure Cost $ time Large Capital Expenditure Opportunity Cost Predicted Demand Traditional Hardware Actual Demand Automated Virtualization
  36. 36. with AWS…
  37. 37. “When we started musiXmatch in 2010, we wanted to focus on building a great user experience for our users and our customers", says Francesco Delfino, cofounder, "We choose Amazon Web Services because it allowed us to freely define and fine tune our server architecture, while shielding us from common hardware issues. During these years, AWS sustained our growth providing the resources we needed as soon as our traffic increased: there is no bigger cost for a startup than missing the opportunity to scale exactly when you need it.”
  38. 38. “Amazon Web Services gives us the power to scale our infrastructure without worrying about capacity limits. Our infrastructure is distributed and currently runs on 3 regions.” www.spreaker.com
  39. 39. RELEASE EVALBUILDIDEA ITERATE OR PIVOT
  40. 40. Data based decision making
  41. 41. 1.  Collect as much data as you can
  42. 42. 1.  Collect as much data as you can 2.  Do A/B testing
  43. 43. 1.  Collect as much data as you can 2.  Do A/B testing 3.  Drive your development by user's feedback
  44. 44. Store valuable data sources Server logs, click streams, application events, …
  45. 45. "Hadoop  is  a  reliable  storage  and  data  analysis  system"   HDFS   MapReduce  
  46. 46. Deploying  a  Hadoop  cluster  is  hard   h5p://eddie.niese.net/20090313/dont-­‐pity-­‐incompetence/  
  47. 47. Amazon Elastic MapReduce Hadoop + The AWS Cloud
  48. 48. Elastic Data Warehouse Expand to 25 instances Data Warehouse (Steady State) Data Warehouse (Batch Processing) Shrink to 9 instances Data Warehouse (Steady State)
  49. 49. 3.5 billion records, 71 million unique cookies, 1.7 million targeted ads required per day Targeted Ad User recently purchased a sports movie and is searching for video games (1.7 Million per day) "   Leveraged AWS and Elastic MapReduce §  100 node cluster on demand §  Processing time dropped from 2+ days to 8 hours §  Increased ROAS by 500%
  50. 50. We use EC2, Auto Scaling and Elastic Load Balancing to deliver ads along side our main infrastructure. We automatically absorb peaks through scripts monitoring our local farm, and change entries in the Route 53 (DNS) when needed, allowing more traffic towards AWS load balancers. www.dotandmedia.com
  51. 51. RELEASE EVALBUILDIDEA ITERATE OR PIVOT
  52. 52. Iterate Enhance your product, get more feedback to prepare next releases
  53. 53. The platform grows with you Add or remove components as needed Scale up – scale down Pay as you go
  54. 54. Pivot If no traction, pivot to address a different vertical, a different problem
  55. 55. POSSIBLE OFFERING POSSIBLE OFFERING POSSIBLE OFFERING POSSIBLE OFFERING PROBLEM SPACE
  56. 56. POSSIBLE OFFERING POSSIBLE OFFERING POSSIBLE OFFERING PROBLEM SPACE POSSIBLE OFFERING
  57. 57. Getting the right features matters less Knowing what's not working, quickly, matters more
  58. 58. Getting the right features matters less Knowing what's not working, quickly, matters more
  59. 59. Experiment and Innovate
  60. 60. Lean startup goal is to accelerate the cycle
  61. 61. CONTINUOUS INTEGRATION
  62. 62. DEVELOPER
  63. 63. SOURCE CODE REPOSITORY
  64. 64. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER
  65. 65. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER
  66. 66. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER PICK TASKS
  67. 67. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SUBMIT CODE
  68. 68. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SCHEDULE BUILD
  69. 69. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER RECURENT BUILDS
  70. 70. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE FETCH
  71. 71. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE QUALITY TESTS TEST RESULTS
  72. 72. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER BUILD OUTPUT
  73. 73. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER DOCS BINARIES / PACKAGES
  74. 74. SOURCE CODE REPOSITORY DNS CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER BUILDS
  75. 75. PAIN POINTS: •  UNIT TESTS INCOMPLETE •  MOCKS MAINTENANCE •  TEST ENV EXPENSIVE •  TEST ENV ≠ PROD
  76. 76. KEY = ITERATION
  77. 77. ITERATIVELY MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF YOUR USERS
  78. 78. ON-DEMAND PAY AS YOU GO ELASTIC
  79. 79. AWS CLOUDFORMATION STACK-BASED DEPLOYMENT SERVICE
  80. 80. CLOUDFORMATION TEMPLATE
  81. 81. { "Description" : "Create RDS with username and password", "Resources" : { "MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "500", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : "MyName", "MasterUserPassword" : "MyPassword" } } } }
  82. 82. "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : "https://s3.amazonaws.com/my-builds/build-v4.zip" } } }
  83. 83. { "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, }
  84. 84. CLOUDFORMATION TEMPLATE PROCEDURAL DEFINITION Create it programmatically KNOWN CONFIGURATION Store stack configuration in source control PARAMETER DRIVEN Dynamic and user-driven templates COLLABORATION Share templates with ease as just files
  85. 85. Template ELBs to front secondary cache ~100 Nginx secondary cache servers 2-3 Nginx mid-tier cache servers Stack CLOUDFORMATION TEMPLATE VIDEO CACHING INFRASTRUCTURE
  86. 86. APPLICATION VERSIONS & INFRASTRUCTURE VERSIONS
  87. 87. CLOUDFORMATION TEMPLATE
  88. 88. TEST ENVIRONMENTS
  89. 89. 30,000 REQUESTS / SECOND 1 TB TRAFFIC / DAY
  90. 90. “…AWS  seemed  to  be  the  best  solu?on  available   to  allow  a  small,  independent  company  to  rapidly   develop  and  test  a  completely  new  infrastructure,   and  host  it.      We  also  loved  the  flexibility  that  AWS  allowed  us,   when  spinning  up  smaller  test  environments,  for   beta  trials,  QA,  localiza?on,  and  during   development.  The  low  ini?al  cost  was  also   crucial.”     Alex  Evans,  CTO    
  91. 91. CONTINUOUS INTEGRATION CONTINUOUS DEPLOYMENT
  92. 92. CONTINUOUS DEPLOYMENT SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO PRODUCTION.
  93. 93. SOFTWARE DEPLOY ≠ PRODUCT LAUNCH
  94. 94. 1.5 BILLION PAGE VIEWS OCTOBER 2012 $83 MILLION IN TRANSACTIONS 4.2 MILLION ITEMS SOLD
  95. 95. 30 DEPLOYS PER DAY = 1 DEPLOY EVERY 20 MINUTES
  96. 96. « Complexity arises when the dependencies among the elements become important. » Scott E. Page, John H. Miller, Complex Adaptive Systems: An Introduction to Computational Models of Social Life
  97. 97. “Production is truly the only place you can validate your code.”
  98. 98. HTTPS://GITHUB.COM/ETSY/DEPLOYINATOR HTTP://CODEASCRAFT.ETSY.COM/
  99. 99. HTTP://SORCERY.SMUGMUG.COM/
  100. 100. AWS OPSWORKS MANAGING THE COMPLETE APPLICATION LIFECYCLE
  101. 101. MODEL, CONTROL AND AUTOMATE AT ANY SCALE AND COMPLEXITY
  102. 102. A stack represents your application. One stack might be used for staging and another for production. A layer defines how to setup and configure a set of instances and related resources such as volumes and software. Tell OpsWorks where it can find your code and define any additional deployment tasks. OpsWorks will take care of deploying your app. Scale your stack based on time or load. Clone your production stack to a different region. Automate workflows for common tasks. STACK | LAYER | APP | INSTANCE GETTING STARTED WITH OPSWORKS
  103. 103. YOUR STACKS IN THE DASHBOARD
  104. 104. STACK OVERVIEW
  105. 105. LAYERS IN A STACK
  106. 106. INSTANCES IN A STACK
  107. 107. APPS IN A STACK
  108. 108. DEPLOYMENTS & COMMANDS
  109. 109. YOU CAN BRING YOUR OWN CHEF RECIPES OR LEVERAGE HUNDREDS OF COMMUNITY-BUILT CONFIGURATIONS
  110. 110. 14 BILLION REQUESTS/MONTH 50 000 DATABASE UPDATES / SEC NO CACHE
  111. 111. “AWS OpsWorks gives us the tools we need to automate operations. We can scale Monster World, one of the largest Facebook games, to millions of users without ever needing more than two backend developers” Jesper Richter-Reichhelm Head of engineering – Wooga
  112. 112. THERE IS NO ADDITIONAL CHARGE FOR USING CLOUDFORMATION OR OPSWORKS
  113. 113. YOU PAY ONLY FOR THE AWS RESOURCES NEEDED TO STORE AND RUN YOUR APPLICATIONS
  114. 114. A/B TESTING
  115. 115. LOAD TESTING
  116. 116. USING AMAZON EC2 TO SIMULATE 2.4 MILLION PLAYERS
  117. 117. DATA-DRIVEN ARCHITECTURES
  118. 118. METRICS @ETSY
  119. 119. COST-ORIENTED ARCHITECTURES
  120. 120. PHP+APACHE+VARNISH NGINX+NODEJS
  121. 121. 11.6s Mean time between deployments (weekday) 1,079 Max number of deployments in a single hour 10,000 Mean number of hosts simultaneously receiving a deployment 30,000 Max number of hosts simultaneously receiving a deployment DEPLOYMENTS AT AMAZON.COM
  122. 122. CONTINUOUS DEPLOYMENT = CONTINUOUS EXPERIMENTATION
  123. 123. CONTINUOUS DEPLOYMENT = CONTINUOUS IMPROVEMENT
  124. 124. « FASTEST TIME TO MARKET
  125. 125. www.chili-tv.it Main advantages to use Amazon Web Services are: -  Quick time to market; AWS provides possibility to set up new architectures in few clicks, allowing us to concentrate in services and features development -  Flexibility and Scalability; AWS provides all the tools to manage big co-marketing activities, giving us scalability to handle load peaks and flexiility to test and perform sizing and profiling as we need
  126. 126. “The  AWS  Cloud  brings  business  agility  as  Shell   is  able  to  deploy  services  much  more  quickly”     Johan  Krebbers    Vice  President  of  Architecture  
  127. 127. “AWS has enabled Soundtracker to perform nimble development” says Daniele Calabrese, Founder and CEO, “allowing our service to scale quickly, effectively, and seamlessly.”
  128. 128. focus on your business
  129. 129. « Money is a renewable resource. Time is not » Adil Wali CTO of ModCloth
  130. 130. Stefano Pochet, founder and CEO of Nealab, explains, “Amazon Web Services fits the need for flexibility and reliability required by high-traffic Web applications. In addition, it makes it easier and cheaper to maintain our entire infrastructure.” … “AWS has allowed us to forget about hardware and focus on software. For us, AWS has really made Web development easier.”

×