CI/CD on AWS
By
Bhargav Amin
Oct 7
About me
• Tech Blogger (http://bhargavamin.com)
• 30,000+ views / 17,000+ visitors
• 27+ articles top ranked articles on Google Search
• 90+ tech articles
• AWS Certified SA
• Owner - DevOps and Security at Entytle Inc
• Manage Infrastructure including DevOps framework
• Travel vlogger
Agenda
 Introduction to DevOps
 Understanding DevOps practices
 Designing a CI/CD pipeline
 Creating a CI/CD pipeline on AWS
Feel free to ask question any time. 
DevOps the term
DevOps the term
• Making software development cycle more efficient
• Bridging gap between Development and Operations
A combination of :
• Cultural philosophies
• Best/well-known Practices
• Tools
Deliver application and services at higher velocity.
Practices
• Monolithic Architecture
• Service Oriented Architecture/Microservices Architecture
• Infra as code
• CI/CD
CI/CD
• Continuous Integration and Delivery/Deployment
• Continually build, test and deploy
• Find and address bugs quickly
• Work on multiple things parallel
• Ship code often in small chunks
CI/CD
Developers
commits changes
Changes are
built
Code is tested Changes deployed
New updates according to feedbacks
Areas that can be automated
• Version control
• Build process
• Testing
• Configuration management
• Infrastructure provisioning
• Deployment & release
AWS
CodeCommit
AWS
CodeDeploy
AWS
CodePipeline
AWS
CodeBuild
AWS
CloudFormation
AWS
OpsWorks
Continuous Integration
Developers
commits changes
Changes are
built
Code is tested Changes deployed
Build quickly and automatically from source
Continuous Integration (CI)
Continuous Integration
• A process which allows multiple
developers to work on same
code/repository simultaneously while
making sure that the integrity of code is
maintained.
The key goals
• Find and address bugs more quickly
• Improve software quality
• Overall reduce the time taken to
validate and release new software
updates.
Continuous Delivery
Developers
commits changes
Changes are
built
Code is tested Changes deployed
Automate Build and Test, deploy when required
Not allowed
beyond this limit!
Continuous Delivery (CD)
A software development process where code changes are
automatically…
 Built
 Tested
 Prepared for release
CD integrates with CI
 Helps deploy code changes
 It takes up delivery and deployment tasks once build is complete
Workflow process
 Fully automated
 Partially automated – Includes manual steps at certain stage
Continuous Deployment
It is totally different from Continuous Delivery.
Continuous delivery includes
• Decision making
• Authorization
Continuous deployment is about
• Deploy everything once the test are succeeded
• No approval, fully automated
Suitable for testing environments, not production
Continuous Deployment
Developers
commits changes
Changes are
built
Code is tested Changes deployed
Automate Build, Test & Deploy
Take it all the way
CI/CD on AWS
Developers
commits changes
Changes
are built
Code is deployed
and tested
Changes deployed to
production
environment
CodeBuildCodeCommit
CodeDeploy
Jenkins
Solano CI
Cloudformation
Opsworks
Elastic
Beanstalk
CodePipeline
Developers
commits changes
Changes
are built
Code is deployed
and tested
Changes deployed to
production
environment
CodeBuildCodeCommit
CodeDeploy
Jenkins
Solano CI
Cloudformation
Opsworks
Elastic
Beanstalk
CodePipeline
CodePipeline
Source Build Test Release
Developers Customer
Feedbacks
Deliver new features/updated
CodePipeline
• Setup and Automate release process
• Use tools/services as every stage
• View whole process at-a-glance
• Add. Features like:
• Custom action and manual approvals
Speed up delivery
while improving quality
Source Build Test Release
Developers Customer
Feedbacks/Requests
AWS
CodePipeline
Example
CodeCommit
• Equivalent to GitHub, Bitbucket and GitLab
• Fully managed version control service
• Support Git standard (same commands)
• No limit on repos and file size
• Dead cheap ($ 1 per active user)
CodeBuild
• Fully managed build service
• Run concurrent builds at same time/scalable
• Pre-configured environments
• Build from CodeCommit,Github,Bitbucket,S3
• Build using an EC2 instance or Docker image
• Build-as-a-code using buildspec.yaml
buildspec.yaml
AWS
CodeBuild
env:
variables:
JAVA_HOME: "/usr/lib/jvm/java-8-openjdk-amd64"
parameter-store:
LOGIN_PASSWORD: "dockerLoginPassword“
phases:
install:
commands:
- apt-get install -y maven
pre_build:
commands:
- docker login –u User –p $LOGIN_PASSWORD
build:
commands:
- echo Entered the build phase...
post_build:
commands:
- echo Entered the post_build phase...
artifacts:
files:
- target/messageUtil-1.0.jar
Actions to be perform
during build phase
CodeDeploy
• Automate deployments
• Provides highly available deployment options:
• Blue/Green deployment
• In-place deployment
• Has an option to Stop or Roll back updates
• Deployment-as-a code using appspec.yaml
appspec.yaml
AWS
CodeDeploy
version: 0.0
os: linux
files:
- source: Config/config.txt
destination: /webapps/Config
- source: source
destination: /webapps/myApp
hooks:
BeforeInstall:
- location: Scripts/UnzipResourceBundle.sh
AfterInstall:
- location: Scripts/RunResourceTests.sh
timeout: 180
ApplicationStart:
- location: Scripts/RunFunctionalTests.sh
timeout: 3600
ValidateService:
- location: Scripts/MonitorService.sh
timeout: 3600
runas: codedeployuser
Tasks to performed
during deployment
Application level CI/CD
CodeBuild CodeDeploy Group of Instances
Developers
Source
Repository
CodePipeline
Application level CI/CD
CodeBuild CodeDeploy Group of Instances
Developers
Source
Repository
1. Developers
commits code to
Git Repo
CodePipeline
Application level CI/CD
CodeBuild CodeDeploy Group of Instances
Developers
Source
Repository
1. Developers
commits code to
Git Repo
2. Polls Git repo
for changes
Store Artifacts on
Bucket
CodePipeline
Application level CI/CD
CodeBuild CodeDeploy Group of Instances
Developers
Source
Repository
1. Developers
commits code to
Git Repo
2. Polls Git repo
for changes
3. Picks up artifact from
CodeBuild, deploy them to
Instances
CodePipeline
Continuous Delivery
CodeBuild CodeDeploy Group of Instances
Developers
Source
Repository
1. Developers
commits code to
Git Repo
2. Polls Git repo
for changes
4. Picks up artifact from
CodeBuild, deploy them to
Instances
CodePipeline
3. Delivered, awaiting for
manual approval
Manual
Approval
Continuous Deployment
CodeBuild CodeDeploy Group of Instances
Developers
Source
Repository
1. Developers
commits code to
Git Repo
2. Polls Git repo
for changes
3. Picks up artifact from
CodeBuild, deploy them to
Instances
CodePipeline
NO APPROVAL, FULLY AUTOMATED
Repo link for the demo
Visit : https://bhargavamin.github.io/devops-techtalk
It also includes step-by-step guide to setup CI/CD pipeline for both the demo
Demo
Manage Infrastructure using CI/CD
Bucket
CodePipeline Create stack Delete stack
Approve
stack
Test environment
Production environment
Create
Change Set
Execute
Change SetApprove
stack
1. Updated CF
template
DevOps
2. Upload CF
changes to S3
bucket
3. The changes on S3
bucket triggers
Pipeline execution
4. CP will create a
Test stack
5. After testing,
manual approval
is required
6. Changes are
incorrect or has
issues, the stack will
be deleted
7. If changes are
approved, A
production stack
will be created.
8. Once approved,
The change set will
be deployed prod
env
Demo
A continuous process which will involve making significant
changes that will increase speed of software delivery
DevOps
Github/bhargavamin
Website/bhargavamin.com
LinkedIn/bhargavamin
Twitter/@bhargav7amin
Mail/bhargavamin@outlook.in
Let’s connect

CI/CD on AWS

  • 1.
  • 2.
    About me • TechBlogger (http://bhargavamin.com) • 30,000+ views / 17,000+ visitors • 27+ articles top ranked articles on Google Search • 90+ tech articles • AWS Certified SA • Owner - DevOps and Security at Entytle Inc • Manage Infrastructure including DevOps framework • Travel vlogger
  • 3.
    Agenda  Introduction toDevOps  Understanding DevOps practices  Designing a CI/CD pipeline  Creating a CI/CD pipeline on AWS Feel free to ask question any time. 
  • 4.
  • 5.
    DevOps the term •Making software development cycle more efficient • Bridging gap between Development and Operations A combination of : • Cultural philosophies • Best/well-known Practices • Tools Deliver application and services at higher velocity.
  • 6.
    Practices • Monolithic Architecture •Service Oriented Architecture/Microservices Architecture • Infra as code • CI/CD
  • 7.
    CI/CD • Continuous Integrationand Delivery/Deployment • Continually build, test and deploy • Find and address bugs quickly • Work on multiple things parallel • Ship code often in small chunks
  • 8.
    CI/CD Developers commits changes Changes are built Codeis tested Changes deployed New updates according to feedbacks
  • 9.
    Areas that canbe automated • Version control • Build process • Testing • Configuration management • Infrastructure provisioning • Deployment & release AWS CodeCommit AWS CodeDeploy AWS CodePipeline AWS CodeBuild AWS CloudFormation AWS OpsWorks
  • 10.
    Continuous Integration Developers commits changes Changesare built Code is tested Changes deployed Build quickly and automatically from source
  • 11.
    Continuous Integration (CI) ContinuousIntegration • A process which allows multiple developers to work on same code/repository simultaneously while making sure that the integrity of code is maintained. The key goals • Find and address bugs more quickly • Improve software quality • Overall reduce the time taken to validate and release new software updates.
  • 12.
    Continuous Delivery Developers commits changes Changesare built Code is tested Changes deployed Automate Build and Test, deploy when required Not allowed beyond this limit!
  • 13.
    Continuous Delivery (CD) Asoftware development process where code changes are automatically…  Built  Tested  Prepared for release CD integrates with CI  Helps deploy code changes  It takes up delivery and deployment tasks once build is complete Workflow process  Fully automated  Partially automated – Includes manual steps at certain stage
  • 14.
    Continuous Deployment It istotally different from Continuous Delivery. Continuous delivery includes • Decision making • Authorization Continuous deployment is about • Deploy everything once the test are succeeded • No approval, fully automated Suitable for testing environments, not production
  • 15.
    Continuous Deployment Developers commits changes Changesare built Code is tested Changes deployed Automate Build, Test & Deploy Take it all the way
  • 16.
    CI/CD on AWS Developers commitschanges Changes are built Code is deployed and tested Changes deployed to production environment CodeBuildCodeCommit CodeDeploy Jenkins Solano CI Cloudformation Opsworks Elastic Beanstalk
  • 17.
    CodePipeline Developers commits changes Changes are built Codeis deployed and tested Changes deployed to production environment CodeBuildCodeCommit CodeDeploy Jenkins Solano CI Cloudformation Opsworks Elastic Beanstalk CodePipeline
  • 18.
    CodePipeline Source Build TestRelease Developers Customer Feedbacks Deliver new features/updated
  • 19.
    CodePipeline • Setup andAutomate release process • Use tools/services as every stage • View whole process at-a-glance • Add. Features like: • Custom action and manual approvals Speed up delivery while improving quality Source Build Test Release Developers Customer Feedbacks/Requests
  • 20.
  • 21.
    CodeCommit • Equivalent toGitHub, Bitbucket and GitLab • Fully managed version control service • Support Git standard (same commands) • No limit on repos and file size • Dead cheap ($ 1 per active user)
  • 22.
    CodeBuild • Fully managedbuild service • Run concurrent builds at same time/scalable • Pre-configured environments • Build from CodeCommit,Github,Bitbucket,S3 • Build using an EC2 instance or Docker image • Build-as-a-code using buildspec.yaml
  • 23.
    buildspec.yaml AWS CodeBuild env: variables: JAVA_HOME: "/usr/lib/jvm/java-8-openjdk-amd64" parameter-store: LOGIN_PASSWORD: "dockerLoginPassword“ phases: install: commands: -apt-get install -y maven pre_build: commands: - docker login –u User –p $LOGIN_PASSWORD build: commands: - echo Entered the build phase... post_build: commands: - echo Entered the post_build phase... artifacts: files: - target/messageUtil-1.0.jar Actions to be perform during build phase
  • 24.
    CodeDeploy • Automate deployments •Provides highly available deployment options: • Blue/Green deployment • In-place deployment • Has an option to Stop or Roll back updates • Deployment-as-a code using appspec.yaml
  • 25.
    appspec.yaml AWS CodeDeploy version: 0.0 os: linux files: -source: Config/config.txt destination: /webapps/Config - source: source destination: /webapps/myApp hooks: BeforeInstall: - location: Scripts/UnzipResourceBundle.sh AfterInstall: - location: Scripts/RunResourceTests.sh timeout: 180 ApplicationStart: - location: Scripts/RunFunctionalTests.sh timeout: 3600 ValidateService: - location: Scripts/MonitorService.sh timeout: 3600 runas: codedeployuser Tasks to performed during deployment
  • 26.
    Application level CI/CD CodeBuildCodeDeploy Group of Instances Developers Source Repository CodePipeline
  • 27.
    Application level CI/CD CodeBuildCodeDeploy Group of Instances Developers Source Repository 1. Developers commits code to Git Repo CodePipeline
  • 28.
    Application level CI/CD CodeBuildCodeDeploy Group of Instances Developers Source Repository 1. Developers commits code to Git Repo 2. Polls Git repo for changes Store Artifacts on Bucket CodePipeline
  • 29.
    Application level CI/CD CodeBuildCodeDeploy Group of Instances Developers Source Repository 1. Developers commits code to Git Repo 2. Polls Git repo for changes 3. Picks up artifact from CodeBuild, deploy them to Instances CodePipeline
  • 30.
    Continuous Delivery CodeBuild CodeDeployGroup of Instances Developers Source Repository 1. Developers commits code to Git Repo 2. Polls Git repo for changes 4. Picks up artifact from CodeBuild, deploy them to Instances CodePipeline 3. Delivered, awaiting for manual approval Manual Approval
  • 31.
    Continuous Deployment CodeBuild CodeDeployGroup of Instances Developers Source Repository 1. Developers commits code to Git Repo 2. Polls Git repo for changes 3. Picks up artifact from CodeBuild, deploy them to Instances CodePipeline NO APPROVAL, FULLY AUTOMATED
  • 32.
    Repo link forthe demo Visit : https://bhargavamin.github.io/devops-techtalk It also includes step-by-step guide to setup CI/CD pipeline for both the demo
  • 33.
  • 34.
    Manage Infrastructure usingCI/CD Bucket CodePipeline Create stack Delete stack Approve stack Test environment Production environment Create Change Set Execute Change SetApprove stack 1. Updated CF template DevOps 2. Upload CF changes to S3 bucket 3. The changes on S3 bucket triggers Pipeline execution 4. CP will create a Test stack 5. After testing, manual approval is required 6. Changes are incorrect or has issues, the stack will be deleted 7. If changes are approved, A production stack will be created. 8. Once approved, The change set will be deployed prod env
  • 35.
  • 36.
    A continuous processwhich will involve making significant changes that will increase speed of software delivery DevOps
  • 37.

Editor's Notes

  • #3 Asdhaskdjasdasd
  • #7 MA- Take whole code deploy whole thing together, if one thing fails- deployment fails Micro- Divide a software/product, makes it easy to manage by smaller teams, ownership, focused approach, faster delivery/updates, no centralized change management IAC- Automate static stacks, actions as code, easily manage versions, quick to deploy with systematic approach
  • #9 For major stages for software delivery
  • #11 For major stages for software delivery
  • #13 For major stages for software delivery
  • #16 For major stages for software delivery
  • #25 With CodeDeploy, you can choose the specific settings for your blue/green deployments. For example, you can choose to manually provision the new instances or let CodeDeploy provision them for you by copying an existing Auto Scaling group. You can also choose when to reroute traffic to the new instances, the rate at which traffic is routed to them, and whether to terminate your old instances upon completion.
  • #26 Agent looks for a instruction file which resides in root of your source