CI/CD on AWS

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
CI/CD on AWS
Danilo Poccia
Principal Evangelist, Serverless
AWS
@danilop
Tonino Greco
Head of Infrastructure and DevOps
Dunelm
@toninog

ListenIterate
Experiment
Innovation
Flywheel
Experiments power the engine of rapid innovation

Pillars of releasing modern applications

Infrastructure
as code

Infrastructure as code goals
1. Make infrastructure changes repeatable and predictable
2. Release infrastructure changes using the same tools as code changes
3. Replicate production environment in a staging environment to enable
continuous testing

Release infrastructure-as-code with AWS CloudFormation
“Master”
branch
Prepare
template
Create & execute
change set
Create & execute
change set

Model function environments with AWS
Serverless Application Model (SAM)
• Open source framework for building serverless
applications on AWS
• Shorthand syntax to express functions, APIs,
databases, and event source mappings
• Transforms and expands SAM syntax into AWS
CloudFormation syntax on deployment
• Supports all AWS CloudFormation resource types
https://aws.amazon.com/serverless/sam/

Model container environments with AWS
Cloud Development Kit (CDK)
• Open source framework to define cloud
infrastructure in TypeScript, Java, C#, …
• Provides library of higher-level resource types
(“construct” classes) that have AWS best practices
built in by default, packaged as npm modules
• Provisions resources with CloudFormation
• Supports all CloudFormation resource types
AWS
CDK
https://awslabs.github.io/aws-cdk
D
eveloper
Preview

AWS Cloud Development Kit (CDK)
npm install -g aws-cdk
cdk init app --language typescript
cdk synth
cdk deploy
cdk diff
cdk destroy
CodePipeline
Use CloudFormation
deployment actions with
any synthesized CDK
application
Jenkins
Use CDK CLI
D
eveloper
Preview
TypeScript
C#
F#
Java
Python
…

CDK template
import ec2 = require('@aws-cdk/aws-ec2');
import ecs = require('@aws-cdk/aws-ecs');
import cdk = require('@aws-cdk/cdk');
class BonjourFargate extends cdk.Stack {
constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
super(parent, name, props);
const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
new ecs.LoadBalancedFargateService(
this, "FargateService", {
cluster,
image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
});
}
}
const app = new cdk.App();
new BonjourFargate(app, 'Bonjour');
app.run();
TypeScript

import ec2 = require('@aws-cdk/aws-ec2');
import ecs = require('@aws-cdk/aws-ecs');
class BonjourFargate extends cdk.Stack {
const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
new ecs.LoadBalancedFargateService(
this, "FargateService", {
cluster,
image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
});
}
}
new BonjourFargate(app, 'Bonjour');
app.run();
CDK template
TypeScript

CDK Lambda cron example
export class LambdaCronStack extends cdk.Stack {
constructor(app: cdk.App, id: string) {
super(app, id);
const lambdaFn = new lambda.Function(this, 'Singleton', {
code: new lambda.InlineCode(
fs.readFileSync('lambda-handler.py', { encoding: 'utf-8' })),
handler: 'index.main’,
timeout: 300,
runtime: lambda.Runtime.Python37,
});
const rule = new events.EventRule(this, 'Rule', {
scheduleExpression: 'cron(0 18 ? * MON-FRI *)’,
});
rule.addTarget(lambdaFn);
}
}
Lambda function
CloudWatch Events rule
CloudFormation Stack
Set the target
TypeScript

Model pipelines with AWS CDK
• Minimize copy-and-paste by using object-oriented language
• Define microservice pipeline “shape” in one class, then re-use it across
many pipelines
• CDK includes many high-level constructs for modeling a CodePipeline
pipeline, including automatically configuring IAM role policies

CDK pipelines: Construct
export class MyMicroservicePipeline extends cdk.Construct {
constructor(parent: cdk.Construct, name: string, props: MyMicroservicePipelineProps) {
super(parent, name);
const pipeline = new codepipeline.Pipeline(this, 'Pipeline', {
pipelineName: props.serviceName,
});
const githubAccessToken = new cdk.SecretParameter(this, 'GitHubToken',
{ ssmParameter: 'GitHubToken' });
new codepipeline.GitHubSourceAction(this, 'GitHubSource', {
stage: pipeline.addStage('Source'),
owner: 'myorg',
repo: props.serviceName,
oauthToken: githubAccessToken.value
});
…

CDK pipelines: Stack
import { MyMicroservicePipeline } from './pipeline';
class MyMicroservicePipelinesStack extends cdk.Stack {
new MyMicroservicePipeline(this, 'Pipeline1', { 'serviceName': 'Microservice1' });
}
}
new MyMicroservicePipelinesStack(app, 'MyMicroservicePipelines');
app.run();

Continuous
integration

Continuous integration goals
Source Build Test Production

Continuous integration goals
1. Automatically kick off a new release when new code is checked in
2. Build and test code in a consistent, repeatable environment
3. Continually have an artifact ready for deployment
4. Continually close feedback loop when build fails

AWS CodePipeline
• Continuous delivery service for fast and reliable
application updates
• Model and visualize your software release process
• Builds, tests, and deploys your code every time
there is a code change
• Integrates with third-party tools and AWS

AWS CodePipeline: Supported sources
Pick branch
AWS CodeCommit
GitHub
Pick object or folder
Amazon S3
Pick Docker tag
Amazon ECR
Automatically kick off release and pull latest source code

AWS CodePipeline: ECR source action
Source code:
“master” branch
ECR repository:
“release” tag

AWS CodePipeline: Supported triggers
Automatically kick off release
Amazon CloudWatch Events
• Scheduled (nightly release)
• AWS Health events (Fargate
platform retirement)
Available in CloudWatch Events
console, API, SDK, CLI, and AWS
CloudFormation
Webhooks
• DockerHub
• Quay
• Artifactory
Available in CodePipeline API,
SDK, CLI, and CloudFormation

AWS CodeBuild
• Fully managed build service that compiles source
code, runs tests, and produces software packages
• Scales continuously and processes multiple builds
concurrently
• No build servers to manage
• Pay by the minute, only for the compute
resources you use
• Monitor builds through CloudWatch Events

AWS CodeBuild
• Each build runs in a new Docker container for a
consistent, immutable environment
• Docker and AWS CLI are installed in every official
CodeBuild image
• Provide custom build environments suited to
your needs through the use of Docker images

AWS CodeBuild: Lambda buildspec
version: 0.2
phases:
build:
commands:
- npm ci
- npm test
- >
aws cloudformation package
--template-file template.yaml
--output-template packaged.yaml
--s3-bucket $BUCKET
artifacts:
type: zip
files:
- packaged.yaml

AWS CodeBuild: Lambda buildspec using SAM CLI
version: 0.2
phases:
install:
commands:
- pip install --upgrade awscli aws-sam-cli
build:
commands:
- sam build
- sam package --s3-bucket $BUCKET --output-template-file packaged.yaml
artifacts:
type: zip
files:
- packaged.yaml

AWS CodeBuild: Docker buildspec
version: 0.2
phases:
build:
commands:
- $(aws ecr get-login --no-include-email)
- docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
- docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $ECR_REPO:$IMAGE_TAG
- docker push $ECR_REPO:$IMAGE_TAG

Continuous
deployment

Continuous deployment goals
Source Build Test Production

Continuous deployment goals
1. Automatically deploy new changes to staging environments for testing
2. Deploy to production safely without impacting customers
3. Deliver to customers faster: Increase deployment frequency,
and reduce change lead time and change failure rate

AWS CodeDeploy
• Automates code deployments to any instance
and Lambda
• Handles the complexity of updating your
applications
• Avoid downtime during application deployment
• Roll back automatically if failure detected
• Deploy to Amazon EC2, Lambda, ECS, or on-
premises servers

CodeDeploy – Lambda canary deployment
API
Gateway
Lambda
function
weighted
alias “live”
v1 Lambda
function
code
100%

API
Gateway
Lambda
function
weighted
alias “live”
v1 code100%
Run PreTraffic hook against v2 code before it receives traffic
v2 code0%

API
Gateway
Lambda
function
weighted
alias “live”
v1 code90%
Wait for 10 minutes, roll back in case of alarm
v2 code10%

API
Gateway
Lambda
function
weighted
alias “live”
v1 code0%
Run PostTraffic hook and complete deployment
v2 code100%

CodeDeploy – Lambda deployments in SAM templates
Resources:
GetFunction:
Type: AWS::Serverless::Function
Properties:
AutoPublishAlias: live
DeploymentPreference:
Type: Canary10Percent10Minutes
Alarms:
- !Ref ErrorsAlarm
- !Ref LatencyAlarm
Hooks:
PreTraffic: !Ref PreTrafficHookFunction
PostTraffic: !Ref PostTrafficHookFunction
Canary10Percent30Minutes
Linear10PercentEvery10Minutes
Linear10PercentEvery1Minute
AllAtOnce

API Gateway canary stage
API
Gateway
Production
stage
v1 code
v2 code
99.5%
0.5%
Canary
stage

CodeDeploy-ECS blue-green deployments
• Provisions “green” tasks, then flips
traffic at the load balancer
• Validation “hooks” enable testing at
each stage of the deployment
• Fast rollback to “blue” tasks in seconds if
case of hook failure or CloudWatch
alarms
• Monitor deployment status and history
via console, API, Amazon SNS
notifications, and CloudWatch Events
• Use “CodeDeploy-ECS” deploy action in
CodePipeline or “aws ecs deploy”
command in Jenkins

CodeDeploy-ECS appspec
version: 1.0
Resources:
- TargetService:
Type: AWS::ECS::Service
Properties:
- TaskDefinition: "my_task_definition:8"
LoadBalancerInfos:
- ContainerName: "SampleApp"
ContainerPort: 80
Hooks:
- BeforeInstall: "LambdaFunctionToExecuteAnythingBeforeNewRevisionInstalltion"
- AfterInstall: "LambdaFunctionToExecuteAnythingAfterNewRevisionInstallation"
- AfterAllowTestTraffic: "LambdaFunctionToValidateAfterTestTrafficShift"
- BeforeAllowTraffic: "LambdaFunctionToValidateBeforeTrafficShift"
- AfterAllowTraffic: "LambdaFunctionToValidateAfterTrafficShift"

CodeDeploy-ECS blue-green deployment
100%
Prod
traffic

Target
group 2
100%
Prod
traffic
Test traffic listener
(port 9000)

Green tasks:
v2 code
100%
Prod
traffic
Provision green tasks

100%
Prod
traffic
Run hook against test endpoint before green tasks receive prod traffic
0%
Prod
traffic

Flip traffic to green tasks, rollback in case of alarm
0%
Prod
traffic
100%
Prod
traffic

100%
Prod
traffic
Drain blue tasks

AWS Deployment Framework (ADF)
https://github.com/awslabs/aws-deployment-framework

CHALLENGES
• Multiple independent tribes deploying to AWS
• Complex Serverless deployments
• Deploy rapid and often
Our Answer
• SAM CLI - Wrapped in Ansible
• Part of a Pipeline library – functions for the developer
• Jenkins Pipeline as code
• Extensible, reproducible and simple
• Slack integrations
Dunelm Ltd

Git Commit
Jenkins
Webhook
Automation Code Product Code
Code
Linting
Prepare
Worker
Node
Install Software
Security
Validation
Build / compile
Run
Tests
Deploy
Install Software for building
Vulnerability and Library scan
Compile the software
Run Unit and functional tests
Deploy to AWS

THE BENEFITS
ü Simple, reproducible and security conscious
ü Scalable for performance – each pipeline has it’s
own worker node (on Spot instances)
ü Highly customisable for each tribe – self serve
ü Centrally managed library of functions – easy to
adapt to change
ü Rapid onboarding of new developers – easy to use
ü Rapid development of CI/CD pipelines
ü Over 100 developers using the pipelines (across all
Dunelm technology tribes)
Dunelm Ltd

THE BENEFITS – THE STATS
• Daily there are
• >200 pipelines active
• each deploying > 15 times / day
• Having > 10 stages per pipeline
• Productivity gains
• > 95% of pipelines are successful
• Allow tribes to be > 30% more effective (based
on burnup charts and sprint reviews)
• Allows us to test and security audit automatically
Dunelm Ltd

ADDITIONAL BENEFITS
• Slack integration with Jenkins
• Status updates in Tribe channels
• Kick off adhoc builds from Slack
• Slack integration with AWS
• Able to delete stacks (AWS CloudFormation)
• Add parameters to AWS Systems Manager
• Create Amazon Route53 DNS entries
• AWS Lambda function to delete AWS CloudFormation
stacks once they are removed from BitBucket
Dunelm Ltd

CI/CD on AWS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to CI/CD on AWS

Similar to CI/CD on AWS (20)

More from Amazon Web Services

More from Amazon Web Services (20)

CI/CD on AWS