Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS Webcast - Build Agile Applications in AWS Cloud for Government

2,318 views

Published on

Join this webinar for a deep dive on the speed and agility of the AWS cloud which allows you to access resources faster.

Published in: Technology

AWS Webcast - Build Agile Applications in AWS Cloud for Government

  1. 1. Building Agile Applications using Continuous Integration & Deployment on AWS Leo Zhadanovsky Senior Solutions Architect
  2. 2. What is Continuous Integration? Changes to code automatically deployed to mainline branch • After passing unit and mock tests Makes changes to code, and deployments iterative, not monolithic Bugs are detected quickly Allows rapid development Helps automate deployments
  3. 3. DEVELOPER
  4. 4. SOURCE CODE REPOSITORY
  5. 5. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER
  6. 6. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER
  7. 7. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER PICK TASKS
  8. 8. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SUBMIT CODE
  9. 9. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SCHEDULE BUILD
  10. 10. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER RECURRENT BUILDS
  11. 11. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE FETCH
  12. 12. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE QUALITY TESTS TEST RESULTS
  13. 13. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER BUILD OUTPUT
  14. 14. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER DOCS BINARIES & PACKAGES DEV FACING NOTIFICATIONS
  15. 15. SOURCE CODE REPOSITORY DNS CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER BUILDS
  16. 16. New AWS Managed Options CodeDeploy • Deploy your code to hundreds or thousands of instances CodeCommit • AWS Managed Git Hosting CodePipelines • AWS Managed Continuous Integration System
  17. 17. Cloud Software Development Lifecycle 10/13/14 17 MonitorProvisionDeployTestBuildCode Elastic Beanstalk OpsWorks Cloud Watch Cloud Formation ?
  18. 18. Why? Customers are asking Share Amazon tools and processes with customers Showcase and integrate AWS partner tools 9/25/14 18
  19. 19. Source Control Private Git repositories hosted on Amazon S3 Full Git compatibility (use with existing tools) All the benefits of the cloud (scalable, durable, reliable, low pay as you go pricing) No size limits on repositories (store binary files) Online code tools with browse, edit, diff 10/13/14 19
  20. 20. Continuous Delivery Customizable release automation, with integrated build and test Model and visualize custom release workflow (source  build  beta  gamma  prod) Automate builds, tests, and deployments Enforce custom rules, approvals, and gates Integrate with third-party and custom tools 10/13/14 21
  21. 21. Deployment Coordinate software updates to fleets of EC2 instances Rolling updates for no downtime Deployment health checks and easy rollback Auto Scaling integration Works with any application Reuse existing setup tools (Bash, Powershell, Chef, Puppet…) 10/13/14 23
  22. 22. 9/25/14 Slides not intended for 24
  23. 23. Cloud Software Development Lifecycle 10/13/14 25 MonitorProvisionDeployTestBuildCode Elastic Beanstalk OpsWorks Cloud Watch Cloud Formation Code Deploy Code Commit Code Pipeline
  24. 24. PAIN POINTS • UNIT TESTS INCOMPLETE • MOCK TESTS MAINTENANCE • EXPENSIVE TEST ENVIRONMENT • TEST ENVIRONMENT ≠ PRODUCTION • DEPLOYMENT CYCLES
  25. 25. ON-DEMAND PAY AS YOU GO ELASTIC
  26. 26. = PROGRAMMABLE PLATFORM
  27. 27. A lot of options… Configuration Management Systems • Puppet • Chef • Saltstack Deployment Frameworks • Elastic Beanstalk • OpsWorks • Ansible • Fabric • Capistrano Infrastructure Management • CloudFormation
  28. 28. Bake an AMI Configure dynamically Time consuming configuration (startup time) Static configurations (less change management) Bootstrapping
  29. 29. Bake an AMI Configure dynamically Continuous deployment (latest code) Environment specific (dev- test-prod) Bootstrapping
  30. 30. Obama for America awsofa.info
  31. 31. Web-Scale Applications
  32. 32. 500k+ IOPS DB Systems
  33. 33. Services API
  34. 34. Typical Charts
  35. 35. Ingredients Ubuntu nginx boundary Unity jQuery SQLServer hbase NewRelic EC2 node.js Cybersource hive ElasticSearch Ruby Twilio EE S3 ELB boto Magento PHP EMR SES Route53 SimpleDB Campfire nagios Paypal CentOS CloudSearch levelDB mongoDB python securitygroups Usahidhi PostgresSQL Github apache bootstrap SNS OpsView Jekyll RoR EBS FPS VPC Mashery Vertica RDS Optimizely MySQL puppet tsunamiUDP R asgard cloudwatch ElastiCache cloudopt SQS cloudinit DirectConnect BSD rsync STS Objective-C DynamoDB
  36. 36. Infrastructure, Configuration Management & Monitoring Ubuntu nginx boundary Unity jQuery SQLServer hbase NewRelic EC2 node.js Cybersource hive ElasticSearch Ruby Twilio EE S3 ELB boto Magento PHP EMR SES Route53 SimpleDB Campfire nagios Paypal CentOS CloudSearch levelDB mongoDB python securitygroups Usahidhi PostgresSQL Github apache bootstrap SNS OpsView Jekyll RoR EBS FPS VPC Mashery Vertica RDS Optimizely MySQL puppet tsunamiUDP R asgard cloudwatch ElastiCache cloudopt SQS cloudinit DirectConnect BSD rsync STS Objective-C DynamoDB
  37. 37. One thing that is difficult to prepare for…
  38. 38. They had this built for the previous 3 months, a on the East Coast. We built this part in 9 hours to be safe. AWS + Puppet + Netflix Asgard + WAN Optimization Software + DevOps = Cross-Continent Fault- Tolerance On-Demand
  39. 39. Mozilla Foundation
  40. 40. Webmaker.org circa 2012 • Included Apps, non-SOA: Thimble, Popcorn, Goggles • ~20 pushes of new software in 2012 • Operations and Development interacted mostly through bugzilla tickets for deploys. • Hosting in physical datacenter at Mozilla Webmaker.org circa early 2013 • Deciding to go 12-factor, SOA in app layer • Weekly pushes of Popcorn on train model • Operations and Development interacted mostly through bugzilla tickets for deploys.
  41. 41. April 2013 Webmaker begins rebuilding entire platform SOA, 12-factor in node.js exclusively Moving apps into AWS and DevOps / CI
  42. 42. Since April 2013…. Openbadges, Webmaker combine for: 1339 Pushes Pushes Per Day to Staging / Prod Pushes Per Day (Staging and Prod)
  43. 43. Who? • ~30 Paid Developers • Hundreds of Students • Thousands of Contributors • One DevOps / Internet Jedi • Multiple Teams How? • Puppet, Jenkins, Fabric • Tight feedback loops: Newrelic, Opsview • Culture Shift • Staging Envs • Brave devs iterate, keeping work in-context • Visible Ops • Cross-train developers in operations
  44. 44. What changed?
  45. 45. 1) Know How You Were Doing Before 2) Know What Changed When 3) Know How You Are Now Doing = The confidence to try more things and try them faster, with minimum viable planning.
  46. 46. Deployment Pipeline
  47. 47. CLOUDFORMATION TEMPLATE
  48. 48. { "Description" : "Create RDS with username and password", "Resources" : { "MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "500", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : "MyName", "MasterUserPassword" : "MyPassword" } } } }
  49. 49. "AWS::CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : "https://s3.amazonaws.com/my-builds/build-v4.zip" } }
  50. 50. { "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } }, }
  51. 51. CLOUDFORMATION TEMPLATE PROCEDURAL DEFINITION Create it programmatically KNOWN CONFIGURATION Store stack configuration in source control PARAMETER DRIVEN Dynamic and user-driven templates COLLABORATION Share templates with ease as just files
  52. 52. APPLICATION VERSIONS + INFRASTRUCTURE VERSIONS
  53. 53. CLOUDFORMATION TEMPLATE
  54. 54. LOAD TESTING
  55. 55. Benchmarking
  56. 56. US-East Cache Node Performance 25.3 Gbps
  57. 57. Only ~42Mbps Impact on US-East FMS Origin Servers
  58. 58. CONTINUOUS DEPLOYMENT SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO PRODUCTION.
  59. 59. KEY = ITERATION
  60. 60. ITERATION = MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF YOUR USERS
  61. 61. 11.6s Mean time between deployments (weekday) 1,079 Max number of deployments in a single hour 10,000 Mean number of hosts simultaneously receiving a deployment 30,000 Max number of hosts simultaneously receiving a deployment DEPLOYMENTS AT AMAZON.COM
  62. 62. SOFTWARE DEPLOY ≠ PRODUCT LAUNCH
  63. 63. 1.5 BILLION PAGE VIEWS OCTOBER 2012 $83 MILLION IN TRANSACTIONS 4.2 MILLION ITEMS SOLD
  64. 64. 30 DEPLOYS PER DAY 1 DEPLOY EVERY 20 MINUTES
  65. 65. "Production is truly the only place you can validate your code."
  66. 66. DATA-DRIVEN ARCHITECTURES
  67. 67. METRICS @ETSY
  68. 68. METRICS @OBAMA FOR AMERICA
  69. 69. COST-ORIENTED ARCHITECTURES
  70. 70. PHP+APACHE+VARNISH NGINX+NODEJS
  71. 71. CONTINUOUS INTEGRATION CONTINUOUS DEPLOYMENT
  72. 72. CONTINUOUS DEPLOYMENT = CONTINUOUS EXPERIMENTATION
  73. 73. CONTINUOUS DEPLOYMENT = CONTINUOUS IMPROVEMENT
  74. 74. INNOVATE
  75. 75. « Want to increase innovation? Lower the cost of failure » Joi Ito
  76. 76. SPEED AND AGILITY Experiment Often Fail quickly at a low cost More Innovation Experiment Infrequently Failure is expensive Less Innovation “ON-PREMISE”

×