Interstella 8888 is an intergalactic trading company that deals in rare resources, but their antiquated monolithic logistics systems are causing the business to lose money. Join this workshop to learn how to set up a CI/CD pipeline for containerized microservices. You'll get hands-on experience deploying Docker container images using Amazon ECS AWS CloudFormation, AWS CodeBuild, and AWS CodePipline, automating everything from code check-in to production. AWS credits are provided. Bring your laptop, and have an active AWS account.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
Interstella GTC: CI/CD for
Containers on AWS
C O N 3 1 9
H u b e r t C h e u n g , A W S S o l u t i o n s A r c h i t e c t
A n d y M u i , A W S S o l u t i o n s A r c h i t e c t
D a v i d K u o , A W S S o l u t i o n s A r c h i t e c t
N o v e m b e r 3 0 , 2 0 1 7

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
CI/CD for Containers on AWS
H u b e r t C h e u n g , A W S S o l u t i o n s A r c h i t e c t
A n d y M u i , A W S S o l u t i o n s A r c h i t e c t
D a v i d K u o , A W S S o l u t i o n s A r c h i t e c t
C O N 3 1 9

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ABOOT ME
Hubert Cheung hubertc@amazon.com
Solutions Architect
Canuck
@ AWS 5 Years
- AWS Support
- AWS Solutions Architecture

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WHAT TO EXPECT FROM THIS WORKSHOP
Release Processes
CI/CD on AWS
• Amazon EC2
• AWS CodeCommit
• AWS CodePipeline
• AWS CodeBuild
• Amazon ECS Deployments
Hands on lab
• Workshop Architecture/Goals

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RELEASE PROCESS PHASES
Source Build Test Production
• Integration
tests with
other systems
• Load testing
• UI tests
• Penetration
testing
• Check-in
source code
such as
Dockerfiles
• Peer review
new code
• Compile code
• Unit tests
• Style checkers
• Code metrics
• Create
container
images
• Deployment
to production
environments

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RELEASE PROCESS PHASES
Source Build Test Production
Continuous integration
Continuous delivery
Continuous deployment

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WHY SHOULD I CARE ABOUT CI/CD?
• Velocity
• Improved Productivity and Efficiency
• Reduced Risk
• Shorter Feedback Loop
• Automation!
Source: xkcd - https://xkcd.com/1629/

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CHALLENGES
• Automation!
• Metrics and Monitoring
• Legacy processes
• Legacy…anything

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
COMMON PATTERNS AND SOLUTIONS
Automate as much as possible
• Start small. Don’t try to automate everything
Microservices
Strict API Contracts
• Get expected outputs from teams consuming your
service
Testing
• Mandate arbitrary number of tests to start

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AUTOMATE AUTOMATE AUTOMATE!
Source Build Test Production
AWS CodeCommit

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AUTOMATE AUTOMATE AUTOMATE!
Source Build Test Production
AWS CodeCommit AWS CodeBuild

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AUTOMATE AUTOMATE AUTOMATE!
Source Build Test Production
AWS CodeCommit AWS CodeBuild Third Party
Tooling

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AUTOMATE AUTOMATE AUTOMATE!
Source Build Test Production
AWS CodeCommit AWS CodeBuild Third Party
Tooling
Amazon Elastic
Container Service

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AUTOMATE AUTOMATE AUTOMATE!
Source Build Test Production
AWS CodeCommit AWS CodeBuild Third Party
Tooling
Amazon Elastic
Container Service
AWS CodePipeline

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AUTOMATE AUTOMATE AUTOMATE!
Source Build Test Production
AWS CodeCommit AWS CodeBuild Third Party
Tooling
Amazon Elastic
Container
Service
AWS CodePipeline
AWS CodeStar

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AUTOMATE INFRASTRUCTURE AS CODE!
Source Build Test Production
AWS CodeCommit AWS CodeBuild Third Party
Tooling
AWS CodePipeline
AWS
CloudFormation

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CODECOMMIT
Fully managed Git service
Available in all public AWS Regions
Data encrypted at rest with a customer-specific
AWS KMS key
Integrated with IAM and AWS CloudTrail for
security and compliance
Extensible through Amazon CloudWatch Events

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CODEBUILD
Fully managed build service that compiles source
code, runs tests, and produces software packages
Scales continuously and processes multiple builds
concurrently
Custom build environments suited to your needs via
Docker images
Only pay by the minute for the compute resources
you use
AWS CodePipeline and Jenkins integration

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
HOW DOES AWS CODEBUILD WORK?
1. Downloads source code
2. Executes commands configured in the
buildspec in temporary compute
containers (created fresh on every build)
3. Streams the build output to the service
console and CloudWatch logs
4. Uploads the generated artifact to an
Amazon S3 bucket

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SAMPLE BUILDSPEC.YML
Variables to be used by
phases of build
Examples phases of a build:
• Install packages or run commands to
prepare your environment in ”install”.
• Run syntax checking, commands in
“pre_build”.
• Execute your build tool/command in
“build”
• Test your app further or ship a
container image to a repository in
post_build
Create and store
an artifact in S3
version: 0.1
environment_variables:
plaintext:
JAVA_HOME: "/usr/lib/jvm/java-8-openjdk-amd64"
phases:
install:
commands:
- apt-get update -y
- apt-get install -y maven
pre_build:
commands:
- echo Nothing to do in the pre_build phase...
build:
commands:
- echo Build started on `date`
- mvn install
post_build:
commands:
- echo Build completed on `date`
artifacts:
type: zip
files:
- target/messageUtil-1.0.jar
discard-paths: yes

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CODEPIPELINE
Fully managed continuous integration and delivery
service for fast and reliable application and infrastructure
updates
Builds, tests, and deploys your code and infrastructure
with every source change
Let’s you model, and visualize your software release
process
Integrates with multiple AWS services and 3rd party tools
Extensible and customizable through custom actions and
AWS Lambda functions

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CLOUDFORMATION
Create templates of your infrastructure
Version control /code review /update
templates like code
CloudFormation provisions AWS resources
based on dependency needs
Integrates with development, CI/CD,
management tools
No additional charge to use

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CLOUDFORMATION CHANGE SETS
Preview the set of actions that CloudFormation will take on
your behalf before you create or update stacks.
Change sets show you what resources will be created, updated
or replaced. This ensures that only expected operations are
executed.

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
HANDS-ON LAB

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WELCOME TO INTERSTELLA GTC
Interstella GTC is an intergalactic trading company that was
established to import and export rare resources in the universe.
Interstella was a thriving company but business has been
declining despite a booming economy. You and a few other
consultants have been brought on board to turn this around. Will
you be able to restore Interstella to its former glory?

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
YOUR MISSION
Interstella GTC already enlisted help to move their system into Docker and then
break up their services into microservices. Now that they have broken up their
application, they want to move faster to beat their competitors.
It’s up to you to figure out how to apply CI/CD principles to each one of their
processing centers of the 2 rare resources – Iridium and Magnesite ore.
Lab 0: Deploy fulfillment service manually
Lab 1: Break apart monolith repo and offload builds
Lab 2: Automate end to end deployment
Lab 3: Build in tests into deployment pipeline
Bonus Lab: Build governance into pipeline - black days

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WORKSHOP 1 RECAP
Public subnet – AZ #1 Public subnet – AZ #2
Amazon SNS
EC2 Instance
ECS Cluster
EC2 Instance
Amazon API
Gateway
ORDER
Amazon S3
bucket
Amazon
DynamoDB
1. Build a Docker image
for the monolith
from the provided
Dockerfile. Push
image to Amazon
ECR.
2. Create an Amazon
ECS task definition to
deploy the monolith
image and log to
CloudWatch Logs.
3. Deploy the monolith
using ECS.
M
FULFILL
Amazon
CloudWatch
Amazon
ECR
Amazon
ECS
LOGS

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
WORKSHOP 2 RECAP
Public subnet – AZ #1 Public subnet – AZ #2
Amazon SNS
EC2 Instance
ECS Cluster
EC2 Instance
Amazon
ECR
Amazon API
Gateway
ORDER
Amazon S3
bucket
Amazon
DynamoDB
Amazon
ECS
1. Build a Docker image for
a resource microservice
and store in ECR.
2. Create an an ECS task
definition to deploy the
microservice image.
3. Deploy the microservice
image using ECS to start
processing orders for that
resource.
4. Modify the monolith code
to remove functionality
of the microservice.
Rebuild the monolith
image and re-deploy.
M
FULFILL
Amazon
CloudWatch
m
FULFILL

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CI/CD WORKSHOP ENVIRONMENT
Public subnet – AZ #1 Public subnet – AZ #2
Amazon SNS
EC2 Instance
ECS Cluster
EC2 Instance
Amazon
ECR
Amazon API
Gateway
ORDER
Amazon S3
bucket
Amazon
DynamoDB
Amazon
ECS
Amazon
CloudWatch Application
Load Balancer
Orders come into
the logistics
platform as HTTP
POST messages via
Amazon Simple
Notification Service
(SNS)
Amazon S3 hosts
web content and
utilities – API key
generator, order
subscription, order
dashboard
The logistics platform
communicates with an
order fulfillment API
running on Amazon
API Gateway. Order
data is stored in
Amazon DynamoDB

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CI/CD WORKSHOP ENVIRONMENT
1. Microservices are
subscribed to ALB
behind path based
routes
2. Microservices process
and use glue code
through existing
monolith to fulfill
requests to API
Gateway
Public subnet – AZ #1 Public subnet – AZ #2
Amazon SNS
EC2 Instance
ECS Cluster
EC2 Instance
Amazon
ECR
Amazon API
Gateway
ORDER
Amazon S3
bucket
Amazon
DynamoDB
Amazon
ECS
M
FULFILL
Amazon
CloudWatch
Application
Load Balancer
m
FULFILL
/iridium/
/magnesite/
m
/fulfill/

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 0: DEPLOY MONOLITH
Public subnet – AZ #1 Public subnet – AZ #2
Amazon SNS
EC2 Instance
ECS Cluster
EC2 Instance
Amazon
ECR
Amazon API
Gateway
ORDER
Amazon S3
bucket
Amazon
DynamoDB
Amazon
ECS
1. Manually build and
deploy monolith onto
ECS. This is just to
understand what
you’re automating
M
Amazon
CloudWatch Application
Load Balancer

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 1: BREAK APART REPO AND OFFLOAD BUILDS
AWS
CodeCommit
AWS
CodeBuild
Amazon ECR
Amazon
CloudWatch

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 2: AUTOMATE END TO END DEPLOYMENT
AWS
CodeCommit
AWS
CodeBuild
Amazon ECR
Amazon
CloudWatch
AWS
CodePipeline
Amazon ECS
source deploy
build
pushwrite pull

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 2: AUTOMATE WITH CLOUDFORMATION
AWS
CodeCommit
AWS
CodeBuild
Amazon ECR
Amazon
CloudWatch
AWS
CodePipeline
Amazon ECS
source create change set
build
pushwrite pull
AWS
CloudFormation
deploy

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LAB 3: IMPLEMENT AUTOMATED TESTING
AWS
CodeCommit
AWS
CodeBuild
Amazon ECR
Amazon
CloudWatch
AWS
CodePipeline
Amazon ECS
source
create change set
build
pushwrite pull
AWS
CloudFormation
deploy
AWS
CodeBuild
test
write

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
BONUS LAB: IMPLEMENT CODE FREEZE
AWS
CodeCommit
Amazon
CloudWatch
AWS
CodePipeline
Amazon ECS
source
build, deploy, etc
check policy
AWS
CodeBuild
test
write
Lambda
function

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LOGISTICS
INSTRUCTIONS:
http://interstella.trade/workshop3/
Raise your hand if you have any questions. High five your neighbors and
have fun!
Please fill out feedback forms once you are done! Feel free to email us
with comments/questions/feedback: aws-interstella-team@amazon.com

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THANK YOU!