Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CI/CD with AWS Developer Tools and Fargate

174 views

Published on

CI/CD for a Docker Node.JS application using Code* services. This session will walkthrough what a solution like this would look like, what Code* services are used, how your build will work, and how deploys will work. The purpose of this session is to allow customers to see how to deploy their containerized applications in Amazon Elastic Container Service (ECS) Fargate using our CI/CD solutions. Come with your questions and pain points. We will also talk about how to use Bitbucket as your source control rather than Code Commit for the many customers already using BitBucket and Jenkins.

  • Be the first to comment

CI/CD with AWS Developer Tools and Fargate

  1. 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lenworth Henry Solutions Architect, Amazon Web Services 194353 CI/CD with AWS Developer Tools and Fargate
  2. 2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. What To Expect from This Session • Review Continuous Integration, Delivery, and Deployment • Introducing the AWS suite of Continuous Integration Continuous Deployment services • Using Docker Images, Amazon ECS Fargate, and Code Build, Code Pipeline, Code Commit and Amazon ECR for CI/CD • Building Docker Container Images with AWS CodeBuild • Orchestrating Deployment Pipelines with AWS CodePipeline • Demo
  3. 3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Continuous Integration, Delivery, and Deployment
  4. 4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The Basic Challenge
  5. 5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  6. 6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How can we quickly and reliably deliver good ideas to our customers?
  7. 7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Goals • Frequency reduces difficulty • Latency between check-in and production is waste • Consistency improves confidence • Automation over toil • Empowered developers make happier teams • Smaller batch sizes are easier to debug • Faster delivery improves software development practices
  8. 8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Source Build Test Production • Version Control • Branching • Code Review • Compilation • Unit Tests • Static Analysis • Packaging • Integration Tests • Load Tests • Security Tests • Acceptance Tests • Deployment • Monitoring • Measuring • Validation
  9. 9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Continuous Integration Continuous Delivery Continuous Deployment Source Build Test Production Feedback
  10. 10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services AWS CodeCommit AWS CodeBuild AWS CodeDeploy AWS CodePipeline
  11. 11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Software Release Steps: Source Build Test Production
  12. 12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Software Release Steps: Source Build Test Production
  13. 13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Software Release Steps: Source Build Test Production AWS CodeCommit
  14. 14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Software Release Steps: Source Build Test Production AWS CodeBuild
  15. 15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Software Release Steps: Source Build Test Production Third Party Tooling
  16. 16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Software Release Steps: Source Build Test Production Third Party Tooling
  17. 17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Software Release Steps: Source Build Test Production AWS CodeDeploy
  18. 18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Software Release Steps: Source Build Test Production EC2 On-Prem AWS CodeDeploy
  19. 19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Source Build Test Production Software Release Steps: AWS CodePipeline
  20. 20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CI CD Services Source Build Test Production Third Party Tooling Software Release Steps: AWS CodeCommit AWS CodeBuild AWS CodeDeploy AWS CodePipeline
  21. 21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Docker Images
  22. 22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why Containers? Packaged Application Code and Runtime Dependencies Reproducible Immutable Portable
  23. 23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1c2acd7c 8ab2ba66 91bd52b7 d2cccfda Image Layers microservice:latest
  24. 24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1c2acd7c 8ab2ba66 91bd52b7 d2cccfda microservice:latestDockerfile FROM amazonlinux:2017.03 RUN yum install –y nginx COPY ./app /bin/app CMD [”/bin/app”]
  25. 25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  26. 26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Development CI UAT Production
  27. 27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Best Practices • Pin external dependencies to specific versions for reproducibility (e.g. no * or ^ in package.json) • Package only the runtime requirements for production • Minimize changes in each layer to maximize cachability • Maintain a .dockerignore file to exclude unneeded files from the image
  28. 28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building Docker Images
  29. 29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  30. 30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CodeBuild Build and test code with continuous scaling with pay-as-you-go pricing • Build and test projects across services and runtimes including Java, Ruby, Python, Android, Docker, etc. • Never pay for idle time • Fully extensible to other services through custom build environments
  31. 31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build Specification – Phases Phase Description Examples install Installation of packages into the environment Install testing frameworks e.g. RSpec, Mocha pre_build Commands to run before the build such as login steps or installation of dependencies Log in to Amazon ECR. run Ruby bundler or npm build Sequence to run the build such as compilation and/or running tests Run go build, sbt, Mocha, RSpec post_build Commands to run after a build on success or failure Build a JAR via Maven or push a Docker image to Amazon ECR
  32. 32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build Specification – Docker version: 0.2 phases: pre_build: commands: - $(aws ecr get-login --no-include-email) - TAG="$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | head -c 8)" - IMAGE_URI="${REPOSITORY_URI}:${TAG}" build: commands: - docker build --tag "$IMAGE_URI" . post_build: commands: - docker push "$IMAGE_URI" - printf '[{"name":"cicd-demo-container","imageUri":"%s"}]' "$IMAGE_URI" > images.json artifacts: files: images.json
  33. 33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Best Practices • Tag output artifacts to source control revisions (e.g. git SHA, semantic version) • Avoid using a “latest” or “production” tag • Optimize for build speed • Collocate build process with its artifact repository
  34. 34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deploying Docker Containers
  35. 35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  36. 36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  37. 37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  38. 38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Running Fargate Containers with ECS
  39. 39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Running Fargate Containers with ECS Use ECS APIs to launch Fargate Containers Stress-free migration – Run Fargate and EC2 launch type tasks in the same cluster Same Task Definition schema
  40. 40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Best Practices • Use Elastic Load Balancing health checks to prevent botched deploys • For higher confidence, integrate automated testing against a new environment or monitoring of a canary before cutover • Make sure your application can function against the same backend schema for adjacent releases
  41. 41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a Deployment Pipeline
  42. 42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deployment Pipeline The automated manifestation of the process for getting your software from version control and into the hands of your customers
  43. 43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Source Build Test Production
  44. 44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS CodePipeline Model deployment pipelines through a visual workflow interface which build, test, and deploy new revisions on code changes • Integrates with AWS services, open source and third party tools for building, testing, and deploying code • Extend deployment pipelines with custom logic through AWS Lambda functions or custom actions • Allows operators to block transitions to “stop the line” and manual approval steps
  45. 45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Action Stage Pipeline Transition
  46. 46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Developers AWS CodeCommit AWS CodePipeline AWS CodeBuild Amazon ECR Amazon ECS
  47. 47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Demo
  48. 48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  49. 49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. EKS Involves some more steps to leverage native tools. See blog post here: https://aws.amazon. com/blogs/devops/c ontinuous- deployment-to- kubernetes-using- aws-codepipeline- aws-codecommit- aws-codebuild- amazon-ecr-and- aws-lambda/
  50. 50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Resources A whole site on DevOps: https://aws.amazon.com/devops/ Getting Started with ECS Fargate: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_GetStarted.html Getting Started with Code Pipeline: https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-simple-codecommit.html Getting Started with Code Build: https://aws.amazon.com/codebuild/getting-started/ Getting Started with Code Deploy: https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-codedeploy.html Demo Code is available here: https://github.com/lenworthhenry/dc-summit-2018-node- ecs-fargate-demo.git
  51. 51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you!

×