AWS Summit Tel Aviv - Startup Track - Continuous Deployment

1,294 views

Published on

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,294
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
33
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

AWS Summit Tel Aviv - Startup Track - Continuous Deployment

  1. 1. AWS Summit 2013 Tel Aviv Oct 16 – Tel Aviv, Israel Continuous Deployment Tiago Henriques Solutions Architect, Amazon Web Services
  2. 2. CONTINUOUS INTEGRATION
  3. 3. DEVELOPER
  4. 4. SOURCE CODE REPOSITORY
  5. 5. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER
  6. 6. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER
  7. 7. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER PICK TASKS
  8. 8. SOURCE CODE REPOSITORY SUBMIT CODE CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER
  9. 9. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER SCHEDULE BUILD PROJECT MANAGEMENT SERVER
  10. 10. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER RECURRENT BUILDS PROJECT MANAGEMENT SERVER
  11. 11. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER CODE FETCH PROJECT MANAGEMENT SERVER
  12. 12. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER CODE QUALITY TESTS PROJECT MANAGEMENT SERVER TEST RESULTS
  13. 13. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER BUILD OUTPUT PROJECT MANAGEMENT SERVER
  14. 14. SOURCE CODE REPOSITORY DOCS CONTINUOUS INTEGRATION SERVER BINARIES & PACKAGES PROJECT MANAGEMENT SERVER
  15. 15. SOURCE CODE REPOSITORY BUILDS CONTINUOUS INTEGRATION SERVER DNS PROJECT MANAGEMENT SERVER
  16. 16. PAIN POINTS • • • • • UNIT TESTS INCOMPLETE MOCK TESTS MAINTENANCE EXPENSIVE TEST ENVIRONMENT TEST ENVIRONMENT ≠ PRODUCTION DEPLOYMENT CYCLES
  17. 17. ON-DEMAND ELASTIC PAY AS YOU GO
  18. 18. = PROGRAMMABLE PLATFORM
  19. 19. IF YOU CAN PROGRAM IT YOU CAN AUTOMATE IT
  20. 20. AWS CLOUDFORMATION STACK-BASED DEPLOYMENT SERVICE
  21. 21. CLOUDFORMATION TEMPLATE
  22. 22. { "Description" : "Create RDS with username and password", "Resources" : { "MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "500", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : "MyName", "MasterUserPassword" : "MyPassword" } } } }
  23. 23. "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : "https://s3.amazonaws.com/my-builds/build-v4.zip" } }
  24. 24. { "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, }
  25. 25. PROCEDURAL DEFINITION KNOWN CONFIGURATION Create it programmatically Store stack configuration in source control CLOUDFORMATION TEMPLATE PARAMETER DRIVEN Dynamic and user-driven templates COLLABORATION Share templates with ease as just files
  26. 26. APPLICATION VERSIONS + INFRASTRUCTURE VERSIONS
  27. 27. CLOUDFORMATION TEMPLATE
  28. 28. HTTP://SORCERY.SMUGMUG.COM/
  29. 29. TEST ENVIRONMENTS
  30. 30. 30,000 REQUESTS / SECOND 1 TB TRAFFIC / DAY
  31. 31. “…AWS seemed to be the best solution available to allow a small, independent company to rapidly develop and test a completely new infrastructure, and host it. We also loved the flexibility that AWS allowed us, when spinning up smaller test environments, for beta trials, QA, localization, and during development. The low initial cost was also crucial.” Alex Evans, CTO
  32. 32. LOAD TESTING
  33. 33. USING AMAZON EC2 TO SIMULATE 2.4 MILLION PLAYERS
  34. 34. CONTINUOUS DEPLOYMENT SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO PRODUCTION
  35. 35. KEY = ITERATION
  36. 36. ITERATION = MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF YOUR USERS
  37. 37. DEPLOYMENTS AT AMAZON.COM 11.6s 1,079 10,000 30,000 Mean time between deployments (weekday) Max number of deployments in a single hour Mean number of hosts simultaneously receiving a deployment Max number of hosts simultaneously receiving a deployment
  38. 38. SOFTWARE DEPLOY ≠ PRODUCT LAUNCH
  39. 39. 1.5 BILLION PAGE VIEWS $83 MILLION IN TRANSACTIONS 4.2 MILLION ITEMS SOLD OCTOBER 2012
  40. 40. 30 DEPLOYS PER DAY 1 DEPLOY EVERY 20 MINUTES
  41. 41. "Production is truly the only place you can validate your code."
  42. 42. AWS OPSWORKS INTEGRATED APPLICATION MANAGEMENT
  43. 43. 14 BILLION REQUESTS/MONTH 50 000 DATABASE UPDATES / SEC NO CACHE
  44. 44. A / B TESTING
  45. 45. DATA-DRIVEN ARCHITECTURES
  46. 46. METRICS @ETSY
  47. 47. COST-ORIENTED ARCHITECTURES
  48. 48. PHP+APACHE+VARNISH NGINX+NODEJS
  49. 49. CONTINUOUS DEPLOYMENT CONTINUOUS INTEGRATION
  50. 50. CONTINUOUS DEPLOYMENT = CONTINUOUS EXPERIMENTATION
  51. 51. CONTINUOUS DEPLOYMENT = CONTINUOUS IMPROVEMENT
  52. 52. INNOVATE
  53. 53. « Want to increase innovation? Lower the cost of failure » Joi Ito
  54. 54. SPEED AND AGILITY “ON-PREMISE” Experiment Infrequently Experiment Often Failure is expensive Fail quickly at a low cost Less Innovation More Innovation
  55. 55. CloudOn Dev / Test Flows AWS IL Conference Oct 16 2013
  56. 56. What is CloudOn? CloudOn is a versatile productivity solution that allows you to connect with cloud storage providers to create, review, edit and share your Micosoft Office files across different devices. Proprietary and Confidential | 82
  57. 57. The CloudOn Production Environment Provisioning Storage Social Sessions DB File Sharing API RDS Proprietary and Confidential | 83
  58. 58. Production Deployment Flow  Bootsrapping Launch Instances install 3rd party infra (Java, Tomcat, etc) Configure roles  Deploy CloudOn Java services    (Not using OpsWorks yet) Proprietary and Confidential | 84
  59. 59. Dev / Test Deployment Needs Developer Needs    Push-button deploy by developers Deploy any branch Several deployments in parallel Automated Regression Needs    Deploy specific release candidate combos Run parallel regressions Low Cost Proprietary and Confidential | 85
  60. 60. The CloudOn Dev/Test Deployment Model All In One Proprietary and Confidential | 86
  61. 61. All In One (AIO) - Deploy    A single instance hosting all CloudOn services Launches via Jenkins Job All In One Allows to override defaults:        Branch and build Instance type Regions Settings Tagged by the developer name Each CloudOn service runs on a different port, no collision Allows re-deploy on existing AIO Proprietary and Confidential | 87
  62. 62. All In One – External Services What to do with the external, AWS services?       RDS DynamoDB ElastiCache RDS SQS SES Proprietary and Confidential | 88
  63. 63. All In One – SQS, DynamoDB, SES     Using a shared dev account - SQS, DynamoDB, SES New Queue, Tables created for each AIO Keyed by the instance private IP Old Queues and Tables (from obsolete AIOs) deleted upon creation of the newest AIO Proprietary and Confidential | 89
  64. 64. All In One – RDS, Memcached     Using a local mySQL, memcached on the machine Allows each instance to have its own data Allows exploring private schema changes (No cost) Proprietary and Confidential | 90
  65. 65. All In One – Cost     Medium instance (default, can be changed) Most AIOs are automatically destroyed at midnight (unless requested not to at creation time) Weekly report on long-running AIOs (still find rogue AIOs running for too long…) Proprietary and Confidential | 91
  66. 66. All In One – Additional Benefits • • • • • • Push-button deploy by developers On demand Easy end-to-end integration test on AWS Any branch and version Allows operational testing Allows sandboxing for QA, partners Proprietary and Confidential | 92
  67. 67. Thanks! Proprietary and Confidential | 93

×