• Save
AWS Summit Stockholm 2014 – T4 – Continuous integration on AWS
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

AWS Summit Stockholm 2014 – T4 – Continuous integration on AWS

  • 2,328 views
Uploaded on

With AWS companies now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100% API driven enables......

With AWS companies now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100% API driven enables businesses to use lean methodologies and realize these benefits. This in turn leads to greater success for those who make use of these practices. In this session we'll talk about some key concepts and design patterns for Continuous Deployment and Continuous Integration, two elements of lean development of applications and infrastructures.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,328
On Slideshare
2,325
From Embeds
3
Number of Embeds
1

Actions

Shares
Downloads
42
Comments
0
Likes
1

Embeds 3

http://www.slideee.com 3

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • All canonical software life cycles were designed by people who build installed software
  • The world’s handmade and vintage marketplace.
  • The world’s handmade and vintage marketplace.
  • Deploying code is the very first thing engineers learn to do at Etsy.
  • GUIs / algos
  • +24% on magic wand sales !
  • No authorization required, Detect problems quickly, gain confidence
  • Pour illustrer le launch early

Transcript

  • 1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Continuous Integration and Deployment on AWS Martin Elwin Manager, Solutions Architecture
  • 2. CONTINUOUS INTEGRATION
  • 3. DEVELOPER
  • 4. SOURCE CODE REPOSITORY
  • 5. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER
  • 6. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER
  • 7. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER PICK TASKS
  • 8. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SUBMIT CODE
  • 9. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE FETCH
  • 10. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER BUILD OUTPUT
  • 11. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER DOCS BINARIES & PACKAGES
  • 12. SOURCE CODE REPOSITORY DNS CONTINUOUS INTEGRATION SERVER PROJECT MANAGEMENT SERVER BUILDS
  • 13. PAIN POINTS • UNIT TESTS INCOMPLETE • MOCK MAINTENANCE • EXPENSIVE TEST ENVIRONMENT • TEST ENVIRONMENT ≠ PRODUCTION
  • 14. ON-DEMAND PAY AS YOU GO ELASTIC
  • 15. = PROGRAMMABLE PLATFORM
  • 16. … ec2.runInstances(new RunInstancesRequest() .withImageId("ami-4b814f22") .withInstanceType("m1.small") .withMinCount(1) .withMaxCount(1) .withKeyName("mykey") .withSecurityGroups("httpssh")); … JAVA SDK
  • 17. … var runInstanceRequest = new RunInstancesRequest() { ImageId = "ami-4b814f22", InstanceType = "m1.small", MinCount = 1, MaxCount = 1, KeyName = "mykey" }; runInstanceRequest.SecurityGroup.Add("httpssh"); ec2.RunInstances(runInstanceRequest); … .NET SDK
  • 18. … ec2.run_instances( 'ami-4b814f22', key_name='mykey', instance_type='m1.small', min_count=1, max_count=1, security_groups=['httpssh']) … PYTHON SDK
  • 19. … ec2.instances.create( :image_id => 'ami-4b814f22', :instance_type => 'm1.small', :count => 1, :security_groups => 'httpssh', :key_pair => ec2.key_pairs['mykey']) … RUBY SDK
  • 20. … $ec2 -> run_instances( 'ami-4b814f22', 1, 1, array( 'InstanceType' => 'm1.small', 'KeyName' => 'mykey', 'SecurityGroup' => 'httpssh') ); … PHP SDK
  • 21. … var params = { ImageId: 'ami-4b814f22', InstanceType: 'm1.small', MinCount: 1, MaxCount: 1, SecurityGroups: ['httpssh'] }; ec2.runInstances(params, function(err, res) { … }); … JAVASCRIPT SDK
  • 22. aws ec2 run-instances --image-id ami-4b814f22 --min-count 1 --max-count 1 --key-name mykey --security-groups httpssh UNIFIED CLI
  • 23. New-EC2Instance -ImageId ami-4b814f22 -MinCount 1 -MaxCount 1 -KeyName mykey -SecurityGroupId sg-9cf9e5d9 -InstanceType m1.small POWERSHELL CLI
  • 24. https://ec2.amazonaws.com/ ?Action=RunInstances &ImageId=ami-4b814f22 &MaxCount=1 &MinCount=1 &SecurityGroup.1=httpssh &KeyName=mykey &<AUTHPARAMS> AWS REST API
  • 25. IF YOU CAN PROGRAM IT YOU CAN AUTOMATE IT
  • 26. AWS CLOUDFORMATION STACK-BASED DEPLOYMENT SERVICE
  • 27. AWS CLOUDFORMATION TEMPLATE
  • 28. { "Description" : "Create a small instance", "Resources" : { "MyInstance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "ImageId" : "ami-4b814f22", "KeyName" : "mykey", "InstanceType" : "m1.small", "SecurityGroups" : ["httpssh"] } } } }
  • 29. CLOUDFORMATION TEMPLATE DECLARATIVE DEFINITION Create it programmatically KNOWN CONFIGURATION Store stack configuration in source control PARAMETER DRIVEN Dynamic and user-driven templates COLLABORATION Share templates with ease as just files
  • 30. APPLICATION VERSIONS + INFRASTRUCTURE VERSIONS
  • 31. AWS CLOUDFORMATION TEMPLATE
  • 32. Continuous Deployment at Viaplay
  • 33. • Viaplay is Nordic’s leading online video service for TV, sport, film and kids' content. • The service is available on PC, Mac, Android and iOS mobile phones, tablets, the Viaplay box, LG Smart TV, Samsung Smart TV, Philips Smart TV, Panasonic Smart TV, Xbox 360, PlayStation®3 and PlayStation®4.
  • 34. Preparations for the Olympics • New content / video / subscriber management platform • API:s completely rebuilt in Node.js in a service oriented architecture • All clients rebuilt (Web, Smartphones, Smart TV:s, Gaming consoles, Set Top Box) • Moved hosting of API:s to AWS
  • 35. System Setup • Almost everything described in CloudFormation templates: Network (VPC), Load Balancers, Cache clusters, DNS entries, Autoscaling groups • The same templates for all different environments • Every application has its own CloudFormation stack with DNS entries, Elastic Load Balancers and an Auto Scaling Group
  • 36. Auto Scaling • An ASG contains a variable number of instances running the same AMI • To enable auto scaling, the AMI must start automatically • Application servers starts with our custom AMI containing everything except the application build
  • 37. Auto Scaling • Build and environment are set in tags and UserData. UserData is run at instance startup, tags are readable from the instance but more important clearly visible in the console. • When an EC2 instance is launched it fetches the build from S3, installs and starts it • As soon as the ELB health check is OK, the instance gets traffic
  • 38. Deploy • Autoscaling has already made us create AMIs that starts the application at instance startup • Deploy is done by changing parameters to the stack • Setting new UserData on the ASG/EC2 launch config requires exchange of instances. CloudFormation does that for us with the Rolling Update policy
  • 39. Lessons learnt • Rolling update takes time and is practically unstoppable during update. We are currently rewriting this so that we start a new ASG that completely replaces the old one • To be able to do a change in a stack, your IAM user needs to have the rights to change all resources in the template. All developers must be able to deploy their code, but we do not want all developers to be Power Users.
  • 40. Custom Admin Application • Our admin application enables CloudFormation updates without giving Power User rights to all developers • We can keep track of who has done what in an easy way • The application also makes sure that deploys are done in exactly the same way every time • Built on top of the AWS Javascript SDK
  • 41. Custom Admin Application • Our admin application enables CloudFormation updates without giving Power User rights to all developers • We can keep track of who has done what in an easy way • The application also makes sure that deploys are done in exactly the same way every time • Built on top of the AWS Javascript SDK
  • 42. Deploy From a Web Interface • Build OK from Jenkins -> copied to S3 builds bucket • Deploy tool lists all files from the builds bucket
  • 43. Deploy From a Web Interface • Write a short description of the deploy for the release email
  • 44. Deploy From a Web Interface • The admin tool updates the CloudFormation stack with the new build parameter • The deploy notifier application sends an email with information on all commits done between the current and the new build
  • 45. Deploy From a Web Interface • If everything looks good on stage - click ”thumbs up” to copy the build to the production bucket, else ”thumbs down” to reject the build
  • 46. Ready for the Olympics • Everything deployed during the winter • Last user migrated in January • Opening ceremony at Friday Feb 7
  • 47. Ready for the Olympics • 39 deploys during the week starting Feb 3 • 11 deploys in 6 different applications on the day of the opening ceremony • All went fine
  • 48. Thanks! johan.waernulf@viaplay.com
  • 49. CONTINUOUS DEPLOYMENT SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO PRODUCTION
  • 50. KEY = ITERATION
  • 51. ITERATION = MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF YOUR USERS
  • 52. 11.6s Mean time between deployments (weekday) 1,079 Max number of deployments in a single hour 10,000 Mean number of hosts simultaneously receiving a deployment 30,000 Max number of hosts simultaneously receiving a deployment DEPLOYMENTS AT AMAZON.COM
  • 53. SOFTWARE DEPLOY ≠ PRODUCT LAUNCH
  • 54. 1.8 BILLION PAGE VIEWS $109 MILLION IN SALES 5.5 MILLION ITEMS SOLD
  • 55. 30 DEPLOYS PER DAY 1 DEPLOY EVERY 20 MINUTES
  • 56. "Production is truly the only place you can validate your code."
  • 57. RUN-TIME CONFIGURATION OF ACTIVE FEATURES
  • 58. AWS OPSWORKS INTEGRATED APPLICATION MANAGEMENT
  • 59. 14 BILLION REQUESTS/MONTH 50 000 DATABASE UPDATES/SEC
  • 60. A / B TESTING
  • 61. DATA-DRIVEN ARCHITECTURES
  • 62. METRICS @ETSY
  • 63. CONTINUOUS INTEGRATION CONTINUOUS DEPLOYMENT
  • 64. CONTINUOUS DEPLOYMENT = CONTINUOUS EXPERIMENTATION
  • 65. CONTINUOUS DEPLOYMENT = CONTINUOUS IMPROVEMENT
  • 66. « Want to increase innovation? Lower the cost of failure » Joi Ito
  • 67. SPEED AND AGILITY Experiment Often Fail quickly at a low cost More Innovation Experiment Infrequently Failure is expensive Less Innovation “ON-PREMISES”
  • 68. wow so deploy much innovate such agility very continuous
  • 69. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Thanks! Continuous Integration and Deployment on AWS Martin Elwin Manager, Solutions Architecture