Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Dive - CI/CD on AWS

184 views

Published on

Deep Dive - CI/CD on AWS

  • Be the first to comment

  • Be the first to like this

Deep Dive - CI/CD on AWS

  1. 1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CI/CD on AWS Danilo Poccia Principal Evangelist, Serverless AWS @danilop S e s s i o n I D Flynn Bundy Consultant, DevOps AWS @bundyfx
  2. 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Development transformation at Amazon: 2001–2002 monolithic application + teams 2001 Lesson learned: decompose for agility 2002 microservices + 2 pizza teams
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Full ownership Full accountability “DevOps” Focused innovation Two-pizza teams
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Monolith development lifecycle monitorreleasetestbuild developers delivery pipelines services
  5. 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Microservice development lifecycle ??? developers delivery pipelines services
  6. 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Microservice development lifecycle developers services monitorreleasetestbuild delivery pipelines monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T ListenIterate Experiment Innovation Flywheel Experiments power the engine of rapid innovation
  8. 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What changes do you need to make to adopt these best practices? Serverless No provisioning/management Automatic scaling Pay for value billing Availability and resiliency Microservices Componentization Business capabilities Products not projects Infrastructure automation DevOps Cultural philosophies Cross-disciplinary teams CI/CD Automation tools DEV OPS Architectural patterns Operational Model Software Delivery
  9. 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Release process stages Source Build Test Production
  10. 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Release process stages Source Build Test Production
  11. 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Release process stages Source Build Test Production
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Release process stages Source Build Test Production • Integration tests with other systems • Load testing • UI tests • Security testing • Check-in source code such as .java files • Peer review new code • Compile code • Unit tests • Style checkers • Create container images and function deployment packages • Deployment to production environments • Monitor code in production to quickly detect errors
  13. 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications
  14. 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications Infrastructure as code
  15. 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Infrastructure as code Declarative I tell you what I need I tell you what to do Imperative
  16. 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Infrastructure as code goals 1. Make infrastructure changes repeatable and predictable 2. Release infrastructure changes using the same tools as code changes 3. Replicate production environment in a staging environment to enable continuous testing
  17. 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Release infrastructure-as-code “Master” branch Prepare template Create & execute change set Create & execute change set
  18. 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Model function environments with AWS Serverless Application Model (SAM) • Open source framework for building serverless applications on AWS • Shorthand syntax to express functions, APIs, databases, and event source mappings • Transforms and expands SAM syntax into AWS CloudFormation syntax on deployment • Supports all AWS CloudFormation resource types https://aws.amazon.com/serverless/sam/
  19. 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Model container environments with AWS Cloud Development Kit (CDK) Developer Preview • Open source framework to define cloud infrastructure in TypeScript, Java, C#, … • Provides library of higher-level resource types (“construct” classes) that have AWS best practices built in by default, packaged as npm modules • Provisions resources with CloudFormation • Supports all CloudFormation resource types AWS CDK https://awslabs.github.io/aws-cdk
  20. 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CDK template import ec2 = require('@aws-cdk/aws-ec2'); import ecs = require('@aws-cdk/aws-ecs'); import cdk = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run();
  21. 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T import ec2 = require('@aws-cdk/aws-ec2'); import ecs = require('@aws-cdk/aws-ecs'); import cdk = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run(); CDK template
  22. 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CDK template import ec2 = require('@aws-cdk/aws-ec2'); import ecs = require('@aws-cdk/aws-ecs'); import cdk = require('@aws-cdk/cdk'); class BonjourFargate extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 }); const cluster = new ecs.Cluster(this, 'Cluster', { vpc }); new ecs.LoadBalancedFargateService( this, "FargateService", { cluster, image: ecs.DockerHub.image("amazon/amazon-ecs-sample"), }); } } const app = new cdk.App(); new BonjourFargate(app, 'Bonjour'); app.run();
  23. 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CDK template applets: MyHelloWorldService: type: @aws-cdk/aws-ecs:LoadBalancedFargateServiceApplet properties: image: 'amazon/amazon-ecs-sample’ $ cdk --app ./my-applet.yaml deploy
  24. 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Model pipelines with AWS CDK • Minimize copy-and-paste by using object-oriented language • Define microservice pipeline “shape” in one class, then re-use it across many pipelines • CDK includes many high-level constructs for modeling a CodePipeline pipeline, including automatically configuring IAM role policies
  25. 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CDK pipelines: Construct export class MyMicroservicePipeline extends cdk.Construct { constructor(parent: cdk.Construct, name: string, props: MyMicroservicePipelineProps) { super(parent, name); const pipeline = new codepipeline.Pipeline(this, 'Pipeline', { pipelineName: props.serviceName, }); const githubAccessToken = new cdk.SecretParameter(this, 'GitHubToken', { ssmParameter: 'GitHubToken' }); new codepipeline.GitHubSourceAction(this, 'GitHubSource', { stage: pipeline.addStage('Source'), owner: 'myorg', repo: props.serviceName, oauthToken: githubAccessToken.value }); …
  26. 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CDK pipelines: Stack import cdk = require('@aws-cdk/cdk'); import { MyMicroservicePipeline } from './pipeline'; class MyMicroservicePipelinesStack extends cdk.Stack { constructor(parent: cdk.App, name: string, props?: cdk.StackProps) { super(parent, name, props); new MyMicroservicePipeline(this, 'Pipeline1', { 'serviceName': 'Microservice1' }); new MyMicroservicePipeline(this, 'Pipeline2', { 'serviceName': 'Microservice2' }); new MyMicroservicePipeline(this, 'Pipeline3', { 'serviceName': 'Microservice3' }); new MyMicroservicePipeline(this, 'Pipeline4', { 'serviceName': 'Microservice4' }); } } const app = new cdk.App(); new MyMicroservicePipelinesStack(app, 'MyMicroservicePipelines'); app.run();
  27. 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Use CDK CLI to synthesize and deploy CDK templates npm install -g aws-cdk cdk init app --language typescript cdk synth cdk deploy
  28. 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications Infrastructure as code
  29. 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications
  30. 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications Continuous integration
  31. 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Continuous integration goals Source Build Test Production
  32. 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Continuous integration goals 1. Automatically kick off a new release when new code is checked in 2. Build and test code in a consistent, repeatable environment 3. Continually have an artifact ready for deployment 4. Continually close feedback loop when build fails
  33. 33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodePipeline • Continuous delivery service for fast and reliable application updates • Model and visualize your software release process • Builds, tests, and deploys your code every time there is a code change • Integrates with third-party tools and AWS
  34. 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodePipeline: Supported sources Pick branch AWS CodeCommit GitHub Pick object or folder Amazon S3 Pick Docker tag Amazon ECR Automatically kick off release and pull latest source code
  35. 35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodePipeline: ECR source action Source code: “master” branch ECR repository: “release” tag
  36. 36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodePipeline: Supported triggers Automatically kick off release Amazon CloudWatch Events • Scheduled (nightly release) • AWS Health events (Fargate platform retirement) Available in CloudWatch Events console, API, SDK, CLI, and AWS CloudFormation Webhooks • DockerHub • Quay • Artifactory Available in CodePipeline API, SDK, CLI, and CloudFormation
  37. 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodeBuild • Fully managed build service that compiles source code, runs tests, and produces software packages • Scales continuously and processes multiple builds concurrently • No build servers to manage • Pay by the minute, only for the compute resources you use • Monitor builds through CloudWatch Events
  38. 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodeBuild • Each build runs in a new Docker container for a consistent, immutable environment • Docker and AWS CLI are installed in every official CodeBuild image • Provide custom build environments suited to your needs through the use of Docker images
  39. 39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodeBuild: Lambda buildspec version: 0.2 phases: build: commands: - npm ci - npm test - > aws cloudformation package --template-file template.yaml --output-template packaged.yaml --s3-bucket $BUCKET artifacts: type: zip files: - packaged.yaml
  40. 40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodeBuild: Lambda buildspec using SAM CLI version: 0.2 phases: install: commands: - pip install --upgrade awscli aws-sam-cli build: commands: - sam build - sam package --s3-bucket $BUCKET --output-template-file packaged.yaml artifacts: type: zip files: - packaged.yaml
  41. 41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodeBuild: Docker buildspec version: 0.2 phases: build: commands: - $(aws ecr get-login --no-include-email) - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG . - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $ECR_REPO:$IMAGE_TAG - docker push $ECR_REPO:$IMAGE_TAG
  42. 42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Continuous integration goals 1. Automatically kick off a new release when new code is checked in 2. Build and test code in a consistent, repeatable environment 3. Continually have an artifact ready for deployment 4. Continually close feedback loop when build fails
  43. 43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications Continuous integration
  44. 44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications
  45. 45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications Continuous deployment
  46. 46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Continuous deployment goals Source Build Test Production
  47. 47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Continuous deployment goals 1. Automatically deploy new changes to staging environments for testing 2. Deploy to production safely without impacting customers 3. Deliver to customers faster: Increase deployment frequency, and reduce change lead time and change failure rate
  48. 48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS CodeDeploy • Automates code deployments to any instance and Lambda • Handles the complexity of updating your applications • Avoid downtime during application deployment • Roll back automatically if failure detected • Deploy to Amazon EC2, Lambda, or on-premises servers
  49. 49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-Lambda deployments • Shifts traffic using Lambda function weighted aliases • Choose canary (“shift 10% of traffic for 10 minutes, then shift rest”) or linear (“shift 10% more traffic every 10 minutes”) • Validation “hooks” enable testing at each stage of the deployment • Fast rollback in seconds if case of hook failure or CloudWatch alarms • Monitor deployment status and history via console, API, Amazon Simple Notification Service (Amazon SNS) notifications, and CloudWatch Events
  50. 50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy – Lambda deployments Enable in your serverless application template Resources: GetFunction: Type: AWS::Serverless::Function Properties: DeploymentPreference: Type: Canary10Percent10Minutes Alarms: - !Ref ErrorsAlarm Hooks: PreTraffic: !Ref PreTrafficHook Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes AllAtOnce
  51. 51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted alias “live” v1 Lambda function code 100%
  52. 52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted alias “live” v1 code100% Run PreTraffic hook against v2 code before it receives traffic v2 code0%
  53. 53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted alias “live” v1 code90% Wait for 10 minutes, roll back in case of alarm v2 code10%
  54. 54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted alias “live” v1 code0% Run PostTraffic hook and complete deployment v2 code100%
  55. 55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T API Gateway canary stage API Gateway Production stage v1 code v2 code 99.5% 0.5% Canary stage
  56. 56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-ECS blue-green deployments • Provisions “green” tasks, then flips traffic at the load balancer • Validation “hooks” enable testing at each stage of the deployment • Fast rollback to “blue” tasks in seconds if case of hook failure or CloudWatch alarms • Monitor deployment status and history via console, API, Amazon SNS notifications, and CloudWatch Events • Use “CodeDeploy-ECS” deploy action in CodePipeline or “aws ecs deploy” command in Jenkins
  57. 57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-ECS appspec version: 1.0 Resources: - TargetService: Type: AWS::ECS::Service Properties: - TaskDefinition: "my_task_definition:8" LoadBalancerInfos: - ContainerName: "SampleApp" ContainerPort: 80 Hooks: - BeforeInstall: "LambdaFunctionToExecuteAnythingBeforeNewRevisionInstalltion" - AfterInstall: "LambdaFunctionToExecuteAnythingAfterNewRevisionInstallation" - AfterAllowTestTraffic: "LambdaFunctionToValidateAfterTestTrafficShift" - BeforeAllowTraffic: "LambdaFunctionToValidateBeforeTrafficShift" - AfterAllowTraffic: "LambdaFunctionToValidateAfterTrafficShift"
  58. 58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-ECS blue-green deployment 100% Prod traffic
  59. 59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-ECS blue-green deployment Target group 2 100% Prod traffic Test traffic listener (port 9000)
  60. 60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-ECS blue-green deployment Green tasks: v2 code 100% Prod traffic Provision green tasks
  61. 61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-ECS blue-green deployment 100% Test traffic 100% Prod traffic Run hook against test endpoint before green tasks receive prod traffic
  62. 62. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-ECS blue-green deployment 100% Prod traffic Flip traffic to green tasks, rollback in case of alarm 0% Prod traffic
  63. 63. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T CodeDeploy-ECS blue-green deployment 100% Prod traffic Drain blue tasks
  64. 64. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Continuous deployment goals 1. Automatically deploy new changes to staging environments for testing 2. Deploy to production safely without impacting customers 3. Deliver to customers faster: Increase deployment frequency, and reduce change lead time and change failure rate
  65. 65. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications Continuous deployment
  66. 66. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Pillars of releasing modern applications
  67. 67. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  68. 68. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Capital One – Credit Offers API serverless architecture Affiliates www.capitalone.com/ credit-cards/prequalify AWS Cloud Capital One API Gateway VPC Lambda Function Traces Logs Production Support Command Center COAT Credit Offers API Team Lambda Function S3 Bucket TTL Third-Party API
  69. 69. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Capital One – Credit Offers API CI/CD pipeline Continuous Improvement, Continuous Delivery! GitHub LGTM Bot Jenkins AWS SAM S3 Bucket (Versioning) Lambda Function DeploymentType: dev: AllAtOnce qa: AllAtOnce qaw: AllAtOnce prod: Canary10Percent10Minutes prodw: Canary10Percent10Minutes canary5xxGetProductsAlarm: Type: AWS::CloudFormation::Alarm Properties: AlarmActions: - !FindInMap: - params - AdminSNSTopic - !Ref Environment AlarmDescription: 500 error from product listing Lambda. ComparisonOperator: GreatherThanOrEqualTothreshold Period: 300 Statistic: Sum Threshold: 1 EvaluationPeriod: 1
  70. 70. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Capital One – Benefits from taking the API serverless Performance gains From the time the request is received by lambda to the time to send the response back 70% Cost savings By removing EC2, ELB and RDS from our solution 90% Increase in team velocity Reduce investment in team’s time on DevOps and dedicate back to feature development! 30%
  71. 71. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  72. 72. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Takeaways 1. Manage your infrastructure as code 2. Frequently build and integrate your code to get a first feedback 3. Continuously release in production using canary releases with monitoring and automated rollbacks 4. Use canary releases to get both technical and business feedback
  73. 73. Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Danilo Poccia @danilop Flynn Bundy @bundyfx
  74. 74. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

×