Learning Objectives: - Learn how to set-up a CI/CD pipeline with AWS Fargate or Amazon ECS - Understand how CodePipeline can help your team release high quality code, faster - See a working sample of a CI/CD pipeline with AWS Fargate and AWS CodePipeline

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Setup a CI/CD Pipeline for Deploying
Containers using AWS Developer Tools
Arun Gupta
Principal Technologist
argu@amazon.com
@arungupta
arun-gupta
Jeremy Cowan
Solution Architect
jicowan@amazon.com
@jicowan
jicowan

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
A look back at
development at
Amazon..
https://secure.flickr.com/photos/pixelthing/15806918992/

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Development Transformation at Amazon: 2001-2009
2001 2009
monolithic
application + teams
microservices + 2 pizza teams

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Things went much
better under this
model and teams were
releasing faster than
ever, but we felt that
we could still improve.

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
In 2009, we ran
a study to find
out where
inefficiencies
might still exist

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We were just waiting.
WaitWrite
Code WaitBuild
Code WaitDeploy
to Test
Deploy
to
Prod
Weeks
Mins Days Mins Days Mins Days Mins

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We were just waiting.
WaitWrite
Code WaitBuild
Code WaitDeploy
to Test
Deploy
to
Prod
Weeks
Mins Days Mins Days Mins Days Mins

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We built tools to
automate our software
release process
https://secure.flickr.com/photos/lindseygee/5894617854/

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Automated actions and
transitions; from check-in
to production
Development benefits:
• Faster
• Safer
• Simplification &
standardization
• Visualization of the
process
Pipelines

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
This has continued to work out really well:
In 2014:
• Thousands of service teams across Amazon
• Building microservices
• Practicing continuous delivery
• Many environments (staging, beta, production)
50 million deploys

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Microservices and Containers
• Do one thing, really well
• Any app, any language
• Test and deploy same artifact
• Self-contained services
• Isolated execution environment
• Faster startup
• Scaling and upgrading

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Container Services Landscape
MANAGEMENT
Deployment, Scheduling, Scaling &
Management of containerized
applications
HOSTING
Where the containers run
Amazon Elastic
Container Service
Amazon Elastic
Container Service for
Kubernetes
Amazon EC2 AWS Fargate
IMAGE REGISTRY
Container Image Repository
Amazon Elastic
Container Registry

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Helping Customers Scale Containers
450+%
growth
Hundreds of millions
of containers started each week
millions
of container instances

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Containers using Amazon Managed Services
Availability Zone #1 Availability Zone #2 Availability Zone #3

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS
AMI
Docker
agent
ECS
agent
ECSTaskECSTask
ECSTaskECSTask
EC2 Instance

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Running Fargate Containers with ECS
Use ECS APIs to launch Fargate Containers
Easy migration – Run Fargate and EC2 launch type
tasks in the same cluster
Same Task Definition schema

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Rightsize your microservices with
AWS Fargate

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Task CPU & Memory Configurations
Flexible configuration options –
50 CPU/memory configurations
CPU Memory
256 (.25 vCPU) 512MB*, 1GB, 2GB
512 (.5 vCPU) 1GB to 4GB (1GB increments)
1024 (1 vCPU) 2GB to 8GB (1GB increments)
2048 (2 vCPU) 4GB to 16GB (1GB increments)
4096 (4 vCPU) 8GB to 30GB (1GB increments)

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS Instance ECS Instance ECS Instance ECS InstanceECS Instance ECS Instance
EC2
FARGATE
Notifications
Amazon ECS CLUSTER
Availability Zone #1 Availability Zone #2 Availability Zone #3
Subnet 2
172.31.2.0/24
Subnet 1
172.31.1.0/24
Subnet 3
172.31.3.0/24
Web
Shopping
Cart

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Where do you
?

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Integration
tests with
other systems
• Load testing
• UI tests
• Penetration
testing
Five major phases of release and monitor
Source Build Test Deploy Monitor
• Check-in
source code
such as .java
files.
• Peer review
new code
• Compile code
• Unit tests
• Style checkers
• Code metrics
• Create
container
images
• Deployment to
production
environments
• Monitor code
in production
to quickly
detect unusual
activity or
errors

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Code Services
Software Release Steps
Source Build Test Deploy Monitor
AWS CodeBuild +
Third Party
AWS CodeCommit AWS CodeBuild AWS CodeDeploy
AWS CodePipeline
AWS
CodeStar
AWS X-Ray
Amazon
CloudWatch

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Demo

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What is AWS Service Catalog?
Definition of a Service Catalog A structured service used for
management, maintenance and deployment of services made available to
customers.
AWS Service Catalog allows you to centrally manage commonly deployed
AWS services and provisioned software products, and helps your
organization achieve consistent governance and compliance
requirements, while enabling users to quickly deploy only the approved
AWS services they need.

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key Benefits
Standardize
Enforce Consistency and
Compliance
Limit Access
Enforce Tagging, Security Groups
Developer Autonomy
One-Stop Shop
Automate Deployments
Agile Governance
Configure Consume

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Service Catalog – A Few Terms to Note
Constraint
restriction on the ways that specific
AWS resources can be deployed for a
product, e.g. template constraints to
allow only certain EC2 instance sizes
Product
an IT service (VPC, web server, n-tier
environment, database, etc.) that you
want to make available for
deployment on AWS
Provisioned Product
an AWS Service Catalog product is
launched via an AWS CloudFormation
process, and the collection of launched
services is called a Provisioned Product
Portfolio
a collection of products, together
with configuration information,
launch controls, and administrator-
controlled access

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Browse
Products
5
4
3
2
1
Portfolio
AWS
Service Catalog
End-users
Select version,
Provision
Product,
configure
parametersDeploy
Notifications
and outputs
Notifications and outputs
4
Scheduled
functions
AWS
Administrator
End User Interaction

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Service Catalog Configuration

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Core Infrastructure Template

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Service Template
AWS
CodeCommit
AWS
CodeBuild
Amazon ECR
Fargate
Source
Repository Build Deploy
or or
AWS
CodePipeline

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CodeBuild Buildspec
phases:
install:
commands:
- apt-get update && apt-get -y install default-jdk maven
pre_build:
commands:
- printenv
- java -version
- echo -n "$CODEBUILD_LOG_PATH" > /tmp/build_id.out
- printf "%s:%s" "$REPOSITORY_URI" "$(cat /tmp/build_id.out)" > /tmp/build_tag.out
- printf '{"tag":"%s"}' "$(cat /tmp/build_id.out)" > /tmp/build.json
- printf "$(cat /tmp/build.json)"
- $(aws ecr get-login --region $AWS_DEFAULT_REGION)
- mvn package
build:
commands:
- echo Build started on `date`
- echo -n "$CODEBUILD_LOG_PATH" > /tmp/build_id.out
- echo Building the Docker image...
- docker build -t $REPOSITORY_URI:$CODEBUILD_LOG_PATH -t $REPOSITORY_URI:latest .
post_build:
commands:
- docker push $REPOSITORY_URI:$CODEBUILD_LOG_PATH
- docker push $REPOSITORY_URI:latest
- echo Build completed on `date`
- printf '[{"name":"%s", "imageUri":"%s"}]' $CONTAINER_NAME $REPOSITORY_URI:$CODEBUILD_LOG_PATH > /tmp/imagedefinitions.json

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deployment Pipeline with Kubernetes

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!