Why Startups Are Still On AWS
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,113
On Slideshare
2,113
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
27
Comments
0
Likes
7

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. WHY STARTUPS ARE (STILL) ON AMAZON WEB SERVICES Danilo Poccia | Solutions Architect
  • 2. Startups? http://www.flickr.com/photos/klearchos/5857451393/
  • 3. « A startup is a human institution designed to create a new product or service under conditions of extreme uncertainty. » Eric Ries, The Lean Startup
  • 4. Startups fail?
  • 5. Assume customer is known Assume features are known Assume solution is known
  • 6. Waterfall approach You know the problem and the solution
  • 7. TEST LAUNCHBUILDSPEC Known set of requirements Known ways to satisfy them
  • 8. Agile methodologies You know the problem, not the solution
  • 9. TEST LAUNCHBUILDSPEC Known set of requirements Unknown ways to satisfy them VALID?
  • 10. You don't know precisely what problem you solve
  • 11. Lean startups: LEARN & ADAPT
  • 12. 1.  Focus on a simple implementation of your idea
  • 13. 1.  Focus on a simple implementation of your idea 2.  Start with a minimal core set of features
  • 14. 1.  Focus on a simple implementation of your idea 2.  Start with a minimal core set of features 3.  Release and listen to your users
  • 15. 1.  Focus on a simple implementation of your idea 2.  Start with a minimal core set of features 3.  Release and listen to your users 4.  Question your initial assumptions based on feedback
  • 16. 1.  Focus on a simple implementation of your idea 2.  Start with a minimal core set of features 3.  Release and listen to your users 4.  Question your initial assumptions based on feedback 5.  Rinse and repeat
  • 17. MVP Minimum Viable Product
  • 18. MVP Smallest thing I can do to test my idea?
  • 19. « If you're not embarrassed when you ship your first version you waited too long » Matt Mullenweg CEO & Founder of WordPress
  • 20. Staying lean is creating the smallest viable product and then iterate around it You don't know precisely your user's needs
  • 21. RELEASE EVALBUILDIDEA Unknown set of requirements Unknown ways to satisfy them ITERATE OR PIVOT
  • 22. RELEASE EVALBUILDIDEA ITERATE OR PIVOT
  • 23. YOUR problem you have the idea you don't have the resources
  • 24. It should be cheap and validate ideas
  • 25. SPEED AND AGILITY Experiment Often Fail quickly at a low cost More Innovation Experiment Infrequently Failure is expensive Less Innovation “ON-PREMISE”
  • 26. RELEASE EVALBUILDIDEA ITERATE OR PIVOT
  • 27. you created a fantastic webapp http://www.flickr.com/photos/scobleizer/3985020876/
  • 28. people love your app http://www.flickr.com/photos/grantrobertson/
  • 29. and everyone wants to use it!
  • 30. http://www.flickr.com/photos/livingos/4480894461/
  • 31. the traditional way…
  • 32. invest on infrastructure
  • 33. capacity planning… http://www.flickr.com/photos/mutsmuts/ …capacity guessing
  • 34. once it's deployed… maintenance? monitoring? log analysis? test environments? http://www.old-computers.com/news
  • 35. Unable to serve customers Infrastructure Cost $ time Large Capital Expenditure Opportunity Cost Predicted Demand Traditional Hardware Actual Demand Automated Virtualization
  • 36. with AWS…
  • 37. “When we started musiXmatch in 2010, we wanted to focus on building a great user experience for our users and our customers", says Francesco Delfino, cofounder, "We choose Amazon Web Services because it allowed us to freely define and fine tune our server architecture, while shielding us from common hardware issues. During these years, AWS sustained our growth providing the resources we needed as soon as our traffic increased: there is no bigger cost for a startup than missing the opportunity to scale exactly when you need it.”
  • 38. “Amazon Web Services gives us the power to scale our infrastructure without worrying about capacity limits. Our infrastructure is distributed and currently runs on 3 regions.” www.spreaker.com
  • 39. RELEASE EVALBUILDIDEA ITERATE OR PIVOT
  • 40. Data based decision making
  • 41. 1.  Collect as much data as you can
  • 42. 1.  Collect as much data as you can 2.  Do A/B testing
  • 43. 1.  Collect as much data as you can 2.  Do A/B testing 3.  Drive your development by user's feedback
  • 44. Store valuable data sources Server logs, click streams, application events, …
  • 45. "Hadoop  is  a  reliable  storage  and  data  analysis  system"   HDFS   MapReduce  
  • 46. Deploying  a  Hadoop  cluster  is  hard   h5p://eddie.niese.net/20090313/dont-­‐pity-­‐incompetence/  
  • 47. Amazon Elastic MapReduce Hadoop + The AWS Cloud
  • 48. Elastic Data Warehouse Expand to 25 instances Data Warehouse (Steady State) Data Warehouse (Batch Processing) Shrink to 9 instances Data Warehouse (Steady State)
  • 49. 3.5 billion records, 71 million unique cookies, 1.7 million targeted ads required per day Targeted Ad User recently purchased a sports movie and is searching for video games (1.7 Million per day) "   Leveraged AWS and Elastic MapReduce §  100 node cluster on demand §  Processing time dropped from 2+ days to 8 hours §  Increased ROAS by 500%
  • 50. We use EC2, Auto Scaling and Elastic Load Balancing to deliver ads along side our main infrastructure. We automatically absorb peaks through scripts monitoring our local farm, and change entries in the Route 53 (DNS) when needed, allowing more traffic towards AWS load balancers. www.dotandmedia.com
  • 51. RELEASE EVALBUILDIDEA ITERATE OR PIVOT
  • 52. Iterate Enhance your product, get more feedback to prepare next releases
  • 53. The platform grows with you Add or remove components as needed Scale up – scale down Pay as you go
  • 54. Pivot If no traction, pivot to address a different vertical, a different problem
  • 55. POSSIBLE OFFERING POSSIBLE OFFERING POSSIBLE OFFERING POSSIBLE OFFERING PROBLEM SPACE
  • 56. POSSIBLE OFFERING POSSIBLE OFFERING POSSIBLE OFFERING PROBLEM SPACE POSSIBLE OFFERING
  • 57. Getting the right features matters less Knowing what's not working, quickly, matters more
  • 58. Getting the right features matters less Knowing what's not working, quickly, matters more
  • 59. Experiment and Innovate
  • 60. Lean startup goal is to accelerate the cycle
  • 61. CONTINUOUS INTEGRATION
  • 62. DEVELOPER
  • 63. SOURCE CODE REPOSITORY
  • 64. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER
  • 65. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER
  • 66. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER PICK TASKS
  • 67. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SUBMIT CODE
  • 68. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SCHEDULE BUILD
  • 69. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER RECURENT BUILDS
  • 70. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE FETCH
  • 71. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE QUALITY TESTS TEST RESULTS
  • 72. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER BUILD OUTPUT
  • 73. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER DOCS BINARIES / PACKAGES
  • 74. SOURCE CODE REPOSITORY DNS CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER BUILDS
  • 75. PAIN POINTS: •  UNIT TESTS INCOMPLETE •  MOCKS MAINTENANCE •  TEST ENV EXPENSIVE •  TEST ENV ≠ PROD
  • 76. KEY = ITERATION
  • 77. ITERATIVELY MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF YOUR USERS
  • 78. ON-DEMAND PAY AS YOU GO ELASTIC
  • 79. AWS CLOUDFORMATION STACK-BASED DEPLOYMENT SERVICE
  • 80. CLOUDFORMATION TEMPLATE
  • 81. { "Description" : "Create RDS with username and password", "Resources" : { "MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "500", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : "MyName", "MasterUserPassword" : "MyPassword" } } } }
  • 82. "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : "https://s3.amazonaws.com/my-builds/build-v4.zip" } } }
  • 83. { "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, }
  • 84. CLOUDFORMATION TEMPLATE PROCEDURAL DEFINITION Create it programmatically KNOWN CONFIGURATION Store stack configuration in source control PARAMETER DRIVEN Dynamic and user-driven templates COLLABORATION Share templates with ease as just files
  • 85. Template ELBs to front secondary cache ~100 Nginx secondary cache servers 2-3 Nginx mid-tier cache servers Stack CLOUDFORMATION TEMPLATE VIDEO CACHING INFRASTRUCTURE
  • 86. APPLICATION VERSIONS & INFRASTRUCTURE VERSIONS
  • 87. CLOUDFORMATION TEMPLATE
  • 88. TEST ENVIRONMENTS
  • 89. 30,000 REQUESTS / SECOND 1 TB TRAFFIC / DAY
  • 90. “…AWS  seemed  to  be  the  best  solu?on  available   to  allow  a  small,  independent  company  to  rapidly   develop  and  test  a  completely  new  infrastructure,   and  host  it.      We  also  loved  the  flexibility  that  AWS  allowed  us,   when  spinning  up  smaller  test  environments,  for   beta  trials,  QA,  localiza?on,  and  during   development.  The  low  ini?al  cost  was  also   crucial.”     Alex  Evans,  CTO    
  • 91. CONTINUOUS INTEGRATION CONTINUOUS DEPLOYMENT
  • 92. CONTINUOUS DEPLOYMENT SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO PRODUCTION.
  • 93. SOFTWARE DEPLOY ≠ PRODUCT LAUNCH
  • 94. 1.5 BILLION PAGE VIEWS OCTOBER 2012 $83 MILLION IN TRANSACTIONS 4.2 MILLION ITEMS SOLD
  • 95. 30 DEPLOYS PER DAY = 1 DEPLOY EVERY 20 MINUTES
  • 96. « Complexity arises when the dependencies among the elements become important. » Scott E. Page, John H. Miller, Complex Adaptive Systems: An Introduction to Computational Models of Social Life
  • 97. “Production is truly the only place you can validate your code.”
  • 98. HTTPS://GITHUB.COM/ETSY/DEPLOYINATOR HTTP://CODEASCRAFT.ETSY.COM/
  • 99. HTTP://SORCERY.SMUGMUG.COM/
  • 100. AWS OPSWORKS MANAGING THE COMPLETE APPLICATION LIFECYCLE
  • 101. MODEL, CONTROL AND AUTOMATE AT ANY SCALE AND COMPLEXITY
  • 102. A stack represents your application. One stack might be used for staging and another for production. A layer defines how to setup and configure a set of instances and related resources such as volumes and software. Tell OpsWorks where it can find your code and define any additional deployment tasks. OpsWorks will take care of deploying your app. Scale your stack based on time or load. Clone your production stack to a different region. Automate workflows for common tasks. STACK | LAYER | APP | INSTANCE GETTING STARTED WITH OPSWORKS
  • 103. YOUR STACKS IN THE DASHBOARD
  • 104. STACK OVERVIEW
  • 105. LAYERS IN A STACK
  • 106. INSTANCES IN A STACK
  • 107. APPS IN A STACK
  • 108. DEPLOYMENTS & COMMANDS
  • 109. YOU CAN BRING YOUR OWN CHEF RECIPES OR LEVERAGE HUNDREDS OF COMMUNITY-BUILT CONFIGURATIONS
  • 110. 14 BILLION REQUESTS/MONTH 50 000 DATABASE UPDATES / SEC NO CACHE
  • 111. “AWS OpsWorks gives us the tools we need to automate operations. We can scale Monster World, one of the largest Facebook games, to millions of users without ever needing more than two backend developers” Jesper Richter-Reichhelm Head of engineering – Wooga
  • 112. THERE IS NO ADDITIONAL CHARGE FOR USING CLOUDFORMATION OR OPSWORKS
  • 113. YOU PAY ONLY FOR THE AWS RESOURCES NEEDED TO STORE AND RUN YOUR APPLICATIONS
  • 114. A/B TESTING
  • 115. LOAD TESTING
  • 116. USING AMAZON EC2 TO SIMULATE 2.4 MILLION PLAYERS
  • 117. DATA-DRIVEN ARCHITECTURES
  • 118. METRICS @ETSY
  • 119. COST-ORIENTED ARCHITECTURES
  • 120. PHP+APACHE+VARNISH NGINX+NODEJS
  • 121. 11.6s Mean time between deployments (weekday) 1,079 Max number of deployments in a single hour 10,000 Mean number of hosts simultaneously receiving a deployment 30,000 Max number of hosts simultaneously receiving a deployment DEPLOYMENTS AT AMAZON.COM
  • 122. CONTINUOUS DEPLOYMENT = CONTINUOUS EXPERIMENTATION
  • 123. CONTINUOUS DEPLOYMENT = CONTINUOUS IMPROVEMENT
  • 124. « FASTEST TIME TO MARKET
  • 125. www.chili-tv.it Main advantages to use Amazon Web Services are: -  Quick time to market; AWS provides possibility to set up new architectures in few clicks, allowing us to concentrate in services and features development -  Flexibility and Scalability; AWS provides all the tools to manage big co-marketing activities, giving us scalability to handle load peaks and flexiility to test and perform sizing and profiling as we need
  • 126. “The  AWS  Cloud  brings  business  agility  as  Shell   is  able  to  deploy  services  much  more  quickly”     Johan  Krebbers    Vice  President  of  Architecture  
  • 127. “AWS has enabled Soundtracker to perform nimble development” says Daniele Calabrese, Founder and CEO, “allowing our service to scale quickly, effectively, and seamlessly.”
  • 128. focus on your business
  • 129. « Money is a renewable resource. Time is not » Adil Wali CTO of ModCloth
  • 130. Stefano Pochet, founder and CEO of Nealab, explains, “Amazon Web Services fits the need for flexibility and reliability required by high-traffic Web applications. In addition, it makes it easier and cheaper to maintain our entire infrastructure.” … “AWS has allowed us to forget about hardware and focus on software. For us, AWS has really made Web development easier.”