AWS Summit Milan - Continuous Deployment

881 views

Published on

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
881
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
60
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

AWS Summit Milan - Continuous Deployment

  1. 1. Continuous Deployment Pierre Gilot Solutions Architect, Amazon Web Services
  2. 2. CONTINUOUS INTEGRATION
  3. 3. DEVELOPER
  4. 4. SOURCE CODE REPOSITORY
  5. 5. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER
  6. 6. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER
  7. 7. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER PICK TASKS
  8. 8. SOURCE CODE REPOSITORY SUBMIT CODE CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER
  9. 9. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER SCHEDULE BUILD PROJECT MANAGEMENT SERVER
  10. 10. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER RECURRENT BUILDS PROJECT MANAGEMENT SERVER
  11. 11. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER CODE FETCH PROJECT MANAGEMENT SERVER
  12. 12. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER CODE QUALITY TESTS PROJECT MANAGEMENT SERVER TEST RESULTS
  13. 13. SOURCE CODE REPOSITORY CONTINUOUS INTEGRATION SERVER BUILD OUTPUT PROJECT MANAGEMENT SERVER
  14. 14. SOURCE CODE REPOSITORY DOCS CONTINUOUS INTEGRATION SERVER BINARIES & PACKAGES PROJECT MANAGEMENT SERVER
  15. 15. SOURCE CODE REPOSITORY BUILDS CONTINUOUS INTEGRATION SERVER DNS PROJECT MANAGEMENT SERVER
  16. 16. PAIN POINTS • • • • • UNIT TESTS INCOMPLETE MOCK TESTS MAINTENANCE EXPENSIVE TEST ENVIRONMENT TEST ENVIRONMENT ≠ PRODUCTION DEPLOYMENT CYCLES
  17. 17. ON-DEMAND ELASTIC PAY AS YOU GO
  18. 18. = PROGRAMMABLE PLATFORM
  19. 19. IF YOU CAN PROGRAM IT YOU CAN AUTOMATE IT
  20. 20. HTTP://SORCERY.SMUGMUG.COM/
  21. 21. AWS CLOUDFORMATION STACK-BASED DEPLOYMENT SERVICE
  22. 22. CLOUDFORMATION TEMPLATE
  23. 23. { "Description" : "Create RDS with username and password", "Resources" : { "MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "500", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : "MyName", "MasterUserPassword" : "MyPassword" } } } }
  24. 24. "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : "https://s3.amazonaws.com/my-builds/build-v4.zip" } }
  25. 25. { "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, }
  26. 26. PROCEDURAL DEFINITION KNOWN CONFIGURATION Create it programmatically Store stack configuration in source control CLOUDFORMATION TEMPLATE PARAMETER DRIVEN Dynamic and user-driven templates COLLABORATION Share templates with ease as just files
  27. 27. APPLICATION VERSIONS + INFRASTRUCTURE VERSIONS
  28. 28. CLOUDFORMATION TEMPLATE
  29. 29. TEST ENVIRONMENTS
  30. 30. 30,000 REQUESTS / SECOND 1 TB TRAFFIC / DAY
  31. 31. “…AWS seemed to be the best solution available to allow a small, independent company to rapidly develop and test a completely new infrastructure, and host it. We also loved the flexibility that AWS allowed us, when spinning up smaller test environments, for beta trials, QA, localization, and during development. The low initial cost was also crucial.” Alex Evans, CTO
  32. 32. LOAD TESTING
  33. 33. USING AMAZON EC2 TO SIMULATE 2.4 MILLION PLAYERS
  34. 34. CONTINUOUS DEPLOYMENT SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO PRODUCTION
  35. 35. KEY = ITERATION
  36. 36. ITERATION = MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF YOUR USERS
  37. 37. DEPLOYMENTS AT AMAZON.COM 11.6s 1,079 10,000 30,000 Mean time between deployments (weekday) Max number of deployments in a single hour Mean number of hosts simultaneously receiving a deployment Max number of hosts simultaneously receiving a deployment
  38. 38. SOFTWARE DEPLOY ≠ PRODUCT LAUNCH
  39. 39. 1.5 BILLION PAGE VIEWS $83 MILLION IN TRANSACTIONS 4.2 MILLION ITEMS SOLD OCTOBER 2012
  40. 40. 30 DEPLOYS PER DAY 1 DEPLOY EVERY 20 MINUTES
  41. 41. "Production is truly the only place you can validate your code."
  42. 42. A / B TESTING
  43. 43. 14 BILLION REQUESTS/MONTH 50 000 DATABASE UPDATES / SEC NO CACHE
  44. 44. DATA-DRIVEN ARCHITECTURES
  45. 45. METRICS @ETSY
  46. 46. COST-ORIENTED ARCHITECTURES
  47. 47. PHP+APACHE+VARNISH NGINX+NODEJS
  48. 48. # USERS APPLICATION GENERATED REVENUE APPLICATION COST $ COST
  49. 49. CONTINUOUS DEPLOYMENT CONTINUOUS INTEGRATION
  50. 50. CONTINUOUS DEPLOYMENT = CONTINUOUS EXPERIMENTATION
  51. 51. CONTINUOUS DEPLOYMENT = CONTINUOUS IMPROVEMENT
  52. 52. INNOVATE
  53. 53. « Want to increase innovation? Lower the cost of failure » Joi Ito
  54. 54. SPEED AND AGILITY “ON-PREMISE” Experiment Infrequently Experiment Often Failure is expensive Fail quickly at a low cost Less Innovation More Innovation
  55. 55. Continuous Deployment at Marco Pracucci
  56. 56. Platform for Creating and Sharing Audio 8 people (4 developers) 2M unique listeners / month 3.5M audio tracks created 500 reqs / sec
  57. 57. we run 80% of our traffic on AWS about 60 EC2 instances and 50 TB on S3
  58. 58. we deploy frequently [even on friday] 51 deploys in the last 30 days
  59. 59. how we deploy the web app
  60. 60. Deploy Task
  61. 61. code repository fab www.deploy 1. ssh EC2 instances
  62. 62. code repository 2. checkout fab www.deploy EC2 instances
  63. 63. Checklist code repository - local cache warmup - db schema migrations fab www.deploy - check consistency 3. warmup EC2 instances
  64. 64. code repository fab www.deploy 4. switch code everything ok? EC2 instances
  65. 65. Checklist code repository - iptables –dport 80 –j DROP - ln –sfn /var/www/new /var/www/curr fab www.deploy - clear bytecode cache 4. switch code - iptables –dport 80 –j ACCEPT EC2 instances
  66. 66. generate report fab www.deploy notify team on IRC
  67. 67. which instances to deploy? we do auto-scaling, they change dinamically
  68. 68. we exploited DNS
  69. 69. Split infrastructure into roles EC2 instances api1 worker1 www1 api2 www3 role: api worker2 { www2 { { api3 role: web role: worker-web
  70. 70. Tag each instance with roles [space separated roles for multi-role instances]
  71. 71. Map names and roles to DNS https://github.com/airbnb/zonify Creates Route 53 DNS entries for all instances and roles
  72. 72. Address a single instance
  73. 73. Address instances by role
  74. 74. Easy to integrate with Fabric
  75. 75. Do you remember the starting point? Internally does fab –R web www.deploy
  76. 76. few good reasons Easy, Fast, Robust
  77. 77. few good reasons Easy, Fast, Robust Warning: an attacker can easier get your instances IPs (use an unguessable DNS zone name)
  78. 78. Thank you www.spreaker.com/marco @pracucci
  79. 79. Thank You Pierre Gilot Solutions Architect, Amazon Web Services

×