2. Aboot Me
Hubert Cheung hubertc@amazon.com
Solutions Architect
Canuck
@ AWS 4.5 Years
- AWS Support
- AWS Solutions Architecture
3. What to expect from this session
• We’ll show you how to:
• Architect your infrastructure using AWS CloudFormation
• Use AWS CloudFormation to set up AWS CodePipeline
pipelines
• Continuously deliver changes to stacks as you make
changes to your templates
• Demo
4. Let’s look at release
processes
https://www.flickr.com/photos/jurvetson/5201796697/
5. • Integration
tests with
other systems
• Load testing
• UI tests
• Penetration
testing
Release processes have four major phases
Source Build Test Production
• Check-in
source code
such as .java
files.
• Peer review
new code
• Compile code
• Unit tests
• Style checkers
• Code metrics
• Create
container
images
• Deployment
to production
environments
9. What do we need for infrastructure continuous
delivery?
• A way to treat infrastructure as code.
• Tools to manage the workflow that creates and updates
infrastructure resources.
• Tools to properly test and inspect your changes for
defects and potential issues
10. What do we need for infrastructure continuous
delivery?
Infrastructure as code
A practice in which infrastructure
is provisioned and managed using
code and software development
techniques, such as version
control and continuous
integration.
Workflow
Build, test, and deploy your code
every time there is a code
change, based on the release
process models you define,
enabling you to rapidly and
reliably deliver changes.
12. AWS CloudFormation
• Create templates of your infrastructure
• Version control /code review /update
templates like code
• CloudFormation provisions AWS resources
based on dependency needs
• Integrates with development, CI/CD,
management tools
• No additional charge to use
13. Author templates in JSON or YAML
Use change sets to preview your changes
Continuous delivery workflows for stacks
Support for AWS Serverless App Model
Enable cross-stack references with exports
Key new features
16. Preview the set of actions that CloudFormation will take on your
behalf before you create or update stacks.
CloudFormation Change Sets
Change sets show you what resources will be created, updated
or replaced. This ensures that only expected operations are
executed.
17. Cross Stack References (Exports)
Network Stack
Outputs:
VPC
Description: reference VPC
Value: !Ref VPC
Export:
Name: ProdVPC
App Stack
Resources:
myTargetGroup:
Type: AWS::ELBV2::TargetGroup
Properties:
VpcId:
Fn::ImportValue: ProdVPC
• Allows you to share information between independent stacks.
• Export a stack’s output values. Other stacks in the same account and region
can import the exported values.
19. Considerations for Exports and Nested Stacks
Nested Stacks Cross Stack References
Recommended
uses cases
Advantages
Considerations
• Template reuse
• Use multiple templates but
manage as single stack
• Sharing common resources
• Allows for independent stacks based
on resource lifecycle or ownership.
• Convenient management.
One stack manages all
resources and nested stacks.
• Creation order and
dependencies are managed
• Separation of concern
• Share databases and VPCs
• Lets you limit blast radius with
safeguards
• Updates and rollbacks have
a wide surface area
• Reusing templates that have
custom resource names
• Replacing updates requires
changes to the importing stacks to
execute.
• Does not manage creation order
21. Let’s examine a sample application
Deconstruct the application into the necessary AWS resources
Create CloudFormation templates based your management needs
Model your continuous delivery pipeline
Continuously deliver infrastrucure changes as you iterate on your architecture
Use CloudFormation to model, provision, and manage changes to your pipeline
22. Microservices Application Based on Amazon
ECS
Two interconnecting microservices deployed as ECS
services (website-service and product-service).
The application runs on a highly available ECS cluster
deployed across multiple Availability Zones with auto
scaling
Available at github.com/awslabs/ecs-refarch-cloudformation
23. Reference architecture
Public Subnet
Private Subnet
Availability Zone
Internet
Gateway
Public Subnet
Private Subnet
Availability Zone
Application
Load Balancer
NAT GatewayNAT Gateway
ECS Cluster
CloudWatch Logs (Container Logs)
ECS Host ECS Host ECS Host ECS HostAuto Scaling Group
github.com/awslabs/ecs-refarch-cloudformation
24. Decompose into AWS resource types
NAT
Gateway
Elastic IP
Default
Public Route
Public Subnet 1Private Subnet 1
Default
Private
Route
Private
Route
Table
NAT
Gateway
Elastic IP
Public Subnet 2Private Subnet 2
Default
Private
Route
Private
Route
Table
AvailabilityZone1AvailabilityZone2
VPC
Internet
Gateway
Public
Route Table
Load Balancer
Security Group
ECS Host
Security Group
Application
Load Balancer
Load Balancer
Listener
Load Balancer
Default TargetGroup
ECS Cluster
Auto Scaling
Group
Auto Scaling
Launch Configuration
ECS (IAM) Role
IAM
Instance Profile
ECS Service
ECS Task Definition
CloudWatch
Log Group
TargetGroup
Listener Rule
Service Role
ECS Service
ECS Task Definition
CloudWatch
Log Group
TargetGroup
Listener Rule
Service Role
Network Security
Load
Balancing
Front End
Service
ECS
Cluster
Back End
Service
25. Build CloudFormation templates based on this
logical grouping
Template Description
Network
VPC, Availability Zones, subnets, routing, NAT and
internet gateways
Security groups Security groups for the application
Load balancers ALBs that are deployed to the public subnets
ECS cluster ECS cluster deployed to private subnets
Back end service ECS service and task definition for the back end app
Front end service ECS service and task definition for the webpage
26. Set up your template to flow configuration to
each other
Network
Template
Security
Template
Load Balancing
Template
Front End svc
Template
ECS Cluster
Template
Back End Svc
Template
Outputs
Load Balancer
Listener
Load Balancer
DNS Name
ECS Cluster
Load Balancer
Security Group
ECS Host
Security Group
VPC
Public
Subnets
Private
Subnets
27. Network
Security
Load Balancing
ECS Cluster
Front End
Back End
with nested stacks
Use these templates to build your stacks
with cross-stack references
Network
Security
Load
Balancing
ECS Cluster
Front End
Back End
Parent Template Microservices
Stack
Nested
templates
Templates Individual Stacks
29. Applying continuous delivery for your
infrastructure
Continuous delivery service for fast and reliable
application and infrastructure updates
Builds, tests and deploys your code each time there is a
code change.
Built in actions for AWS CloudFormation
AWS
CodePipeline
30. How does this align with release phases?
Source Test Deploy
Source stage for
CloudFormation
templates can be
AWS CodeCommit,
S3, or GitHub
Use CloudFormation
change sets to ensure
to verify deployments
prior to execution
Create, update or
delete stacks, or
change sets.
31. Model your pipelines
Iterate more often on your application
and infrastructure code
Launch new versions in dev and
promote to prod
Manage your network resources
separately per its own cadence.
Maintain separate, mirror sandbox, and
production network environments.
Production
VPC, Security Groups,
Load Balancing
Sandbox
VPC, Security Groups,
Load Balancing
Production
ECS Cluster, Application
Front & Back Ends
Dev
ECS Cluster, Application
Front & Back Ends
Application PipelineNetwork Resources Pipeline
32. Create and manage your pipeline using
CloudFormation
Pipeline artifact store
S3 bucket
Pipeline notifications
SNS email notifications
Pipeline IAM roles
CloudFormation template to set up your pipeline
Could be provisioned
in a separate stack
with IAM resources –
with cross-stack refs
33. Create and manage your pipeline using
CloudFormation
Choose ‘deploy’ action with CloudFormation
as the provider
CloudFormation has enabled several action modes
– REPLACE_ON_FAILURE creates a new stack if one
doesn’t exist, updates it if it does, or replaces it if
its in a failed state
You can use template configuration files or
specify parameter overrides within the template
that defines your pipeline
Stage
Action
Action
config
Name of your CloudFormation template
34. Pipeline for network resources
Source repo
Networking resources for
sandbox/dev environments
Individual stacks. Ordered to
account for dependencies.
Change sets to preview changes to prod
Manual approval before you
changes are applied to prod
Apply changes to Prod
1
2
3
4
5
35. Pipeline for your application
Pipeline triggered as soon as new
versions are posted
Run your tests and clean up
your dev environment when
done, so you aren’t charged
for the instances you don’t
use.
Review to ensure resource modification
or replacement is what you expect
Continuously deliver changes to Prod
1
2
3
4
38. FIN, ACK
We’ve seen how to compose and continuously deliver your
infrastructure as code on our software release process:
• Different ways to decompose your infrastructure into
templates and stacks
• Create and provision your continuous delivery pipeline
for your infrastructure
• Deliver changes to your environments with speed and
quality.
39. re:Invent 2016 sessions on Continuous Delivery:
• DEV201 - DevOps on AWS: Accelerating Software Delivery with the AWS Developer
Tools
• CON302 - Development Workflow with Docker and Amazon ECS
• DEV403 - DevOps on AWS: Advanced Continuous Delivery Techniques
Resources to learn more:
• Continuous delivery: https://aws.amazon.com/devops/continuous-delivery/
• Continuous delivery for CloudFormation stacks -
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-
delivery-codepipeline.html
• CodePipeline - https://aws.amazon.com/documentation/codepipeline/
But wait, there’s more!
And together we will take next one hour to focus more on “how” and less on “what”.
We will show you how to architect your infrastructure using AWS CloudFormation. Then, we will see how to setup up a continuous delivery workflow using CloudFormation and AWS CodePipeline, and finally we will see how to use this workflow to update a CloudFormation stack and continuously deliver changes to this stack. Also, we will follow up with a demo.
But before doing all this, we will take a look at software release processes.
If you have any questions, we are happy to answer them after our session. After the session, Dominic and I will hang around a bit off stage. So, please come and find us. You can also visit our AWS management tools booth if you want to interact with us further.
https://www.flickr.com/photos/jurvetson/5201796697/
- Let’s take a step back and see all the phases involved in the software release process.
I want to take a moment to talk about different release processes.
Each team’s release process takes a different shape to accommodate the needs of each team.
Nearly all release processes can be simplified down to four stages – source, build, test and production. Each phase of the process provides increase confidence that the code being made available to customers will work in the way that was intended.
During the source phase, developers check changes into a source code repository. Many teams require peer feedback on code changes before shipping code into production. Some teams use code reviews to provide peer feedback on the quality of code change. Others use pair programming as a way to provide real time peer feedback.
During the Build phase an application’s source code is built and the quality of the code is tested on the build machine. The most common type of quality check are automated tests that do not require a server in order to execute and can be initiated from a test harness. Some teams extend their quality tests to include code metrics and style checks. There is an opportunity for automation any time a human is needed to make a decision on the code.
The goal of the test phase is to perform tests that cannot be done on during the build phase and require the software to be deployed to a production like stages. Often these tests include testing integration with other live systems, load testing, UI testing and penetration testing. At Amazon we have many different pre-production stages we deploy to. A common pattern is for engineers to deploy builds to a personal development stage where an engineer can poke and prod their software running in a mini prod like stage to check that their automated tests are working correctly. Teams deploy to pre-production stages where their application interacts with other systems to ensure that the newly changed software work in an integrated environment.
Finally code gets deployed to production. Different teams have different deployment strategies though we all share a goal of reducing risk when deploying new changes and minimizing the impact if a bad change does get out to production.
Each of these steps can be automated without the entire release process being automated. There are several levels of release automation that I’ll step through.
Continuous Integration
Continuous Integration is the practice of checking in your code to the continuously and verifying each change with an automated build and test process. Over the past 10 years Continuous Integration has gained popularity in the software community. In the past developers were working in isolation for an extended period of time and only attempting to merge their changes into the mainline of their code once their feature was completed. Batching up changes to merge back into the mainline made not only merging the business logic hard, but it also made merging the test logic difficult. Continuous Integration practices have made teams more productive and allowed them to develop new features faster. Continuous Integration requires teams to write automated tests which, as we learned, improve the quality of the software being released and reduce the time it takes to validate that the new version of the software is good.
There are different definitions of Continuous Integration, but the one we hear from our customers is that CI stops at the build stage, so I’m going to use that definition.
Continuous Delivery
Continuous Delivery extends Continuous Integration to include testing out to production-like stages and running verification testing against those deployments. Continuous Delivery may extend all the way to a production deployment, but they have some form of manual intervention between a code check-in and when that code is available for customers to use.
Continuous Delivery is a big step forward over Continuous Integration allowing teams to be gain a greater level of certainty that their software will work in production.
Continuous Deployment
Continuous Deployment extends continuous delivery and is the automated release of software to customers from check in through to production without human intervention. Many of the teams at Amazon have reached a state of continuous deployment. Continuous Deployment reduces the time for your customers to get value from the code your team has just written, with the team getting faster feedback on the changes you’ve made. This fast customer feedback loop allow you to iterate quickly, allowing you to deliver more valuable software to your customers, quicker.
Continuous Integration
Continuous Integration is the practice of checking in your code to the continuously and verifying each change with an automated build and test process. Over the past 10 years Continuous Integration has gained popularity in the software community. In the past developers were working in isolation for an extended period of time and only attempting to merge their changes into the mainline of their code once their feature was completed. Batching up changes to merge back into the mainline made not only merging the business logic hard, but it also made merging the test logic difficult. Continuous Integration practices have made teams more productive and allowed them to develop new features faster. Continuous Integration requires teams to write automated tests which, as we learned, improve the quality of the software being released and reduce the time it takes to validate that the new version of the software is good.
There are different definitions of Continuous Integration, but the one we hear from our customers is that CI stops at the build stage, so I’m going to use that definition.
Continuous Delivery
Continuous Delivery extends Continuous Integration to include testing out to production-like stages and running verification testing against those deployments. Continuous Delivery may extend all the way to a production deployment, but they have some form of manual intervention between a code check-in and when that code is available for customers to use.
Continuous Delivery is a big step forward over Continuous Integration allowing teams to be gain a greater level of certainty that their software will work in production.
Continuous Deployment
Continuous Deployment extends continuous delivery and is the automated release of software to customers from check in through to production without human intervention. Many of the teams at Amazon have reached a state of continuous deployment. Continuous Deployment reduces the time for your customers to get value from the code your team has just written, with the team getting faster feedback on the changes you’ve made. This fast customer feedback loop allow you to iterate quickly, allowing you to deliver more valuable software to your customers, quicker.
So what about tools that can help us implement continuous delivery for creating and updating infrastructure.
Let’s start at a high level and then we will filter down to specific AWS Services that you can make use of to implement continuous delivery for creating and updating infrastructure on AWS.
The first thing is that you have to start treating Infrastructure as code.
Then you need a tool or a service to manage the workflow that binds all the different phases that I talked about earlier.
Lastly, you should be in a position to test or preview your changes for any potential issues.
In few minutes Dominic will show you how you can use a feature to preview proposed changes before executing them.
Ok, let’s distill this further.
We have to use code and software development techniques to provision and manage infrastructure.
Once you have something in a codified format you enjoy a number of benefits:
you can version it,
you can share with your colleagues for review,
you can create and codify standards,
you can re-use it and use it to replicate environments rapidly.
And then we need a workflow. On every code commit your workflow should be able to build, test and deploy changes.
Build test and deploy Infrastructure changes.
You can use AWS CloudFormation - our Infrastructure as code service and AWS CodePipeline – our continuous delivery workflow service to achieve all this. You can use these two service together to continuously deliver fast and reliable infrastructure updates.
We’ve assumed that most of you are pretty familiar with CloudFormation, but lets start off with a quick overview of the basics
CloudFormation is an Infrastructure as code service by AWS. It helps you model and set up your AWS resources.
You use declarative ways to describe all the AWS resources that you need in a template and CloudFormation takes care of provisioning and configuring those resources for you. CloudFormation figures out all the dependencies and execution order, provisions and configures all your resources.
You can update your templates and version control them as you make incremental changes to your infrastructure.
CloudFormation integrates with popular CI/CD and management tools including AWS CodePipleine.
Here are some of the key features that we've added this year:
We’ve added the capability to author templates in YAML. YAML provides for concise, readable templates that you can comment. You can also use new shorthands for functions and a new Sub function to substitute variables in an string.
Change sets provides you with a preview of the actions CloudFormation will take on your behalf when you create or update a stack
Cross Stack references lets use output values from another stack that are given an export name
You can now build continuous delivery workflows for CloudFormation stacks using CodePipeline. AWS CodePipeline has built-in integration with AWS CloudFormation, so you can specify AWS CloudFormation-specific actions, such as creating, updating, or deleting a stack and Change Sets, within a pipeline
We launched newer abstractions for serverless architectures. CloudFormation now supports the AWS Serverless Application Model with special resource types that simplify expression of Lambda functions, APIs, mappings and IAM resources to create Serverless applications. It’s little bit our of scope for this session, but please meet us after the session if you need more details on this.
Based on customer feedback we launched support for YAML ver 1.1
YAML is more concise and readable with a lot less punctuation.
YAML allows you to add comment blocks to your templates.
YAML supports all the functionality that is available with JSON.
1. We have not only added the YAML support but we have enhanced some syntaxes to further improve the template authoring experience.
2. We have introduced short forms. Using these short forms you can express CloudFormation intrinsic functions (such as Fn::Join, Fn::FindInMap, Fn::GetAtt, Fn::GetAZs) in a more readable and concise manner in CloudFormation YAML templates.
3. Also, we have introduced a new intrinsic Function called Fn::Sub for conducting basic string interpolations within a CloudFormation template. This intrinsic function Fn::Sub substitutes variables in an input string with values that you specify.
4. Here is an arbitrary user data section in CloudFormation JSON template.
And here is how you can express the same thing in YAML using these new enhancements.
…and here’s the same user data section in YAML with a new Sub function that can substitute variables & in this case pseudo parameters in a string.
There’s another enhancement - you can use YAML tag directive and then the CloudFormation intrinsic function name as a short form.
1. Many customers asked for insight into the changes or preview the changes that CloudFormation is planning to perform when it creates or updates a stack based on whatever is present in the CloudFormation template and parameter values. Previewing the changes, can help verify if they are in line with the expectations.
2. Change sets lets you preview and approve the set of actions CloudFormation will take on your behalf when you create or update a stack.
You can view exactly what resources will get created, modified or replaced, so you can ensure only expected operations are executed.
3. What I am showing you here is a basic flow involving change sets during stack update.
Suppose you have your original stack, and you want to update it. Next step is to provide updated template and create change sets
Change Set is generated. It provides blow by blow information of what is going to be created , modified and deleted. You can review this and make sure that the changes are in line with your expectation, and if you’re happy you can go ahead and execute the changes sets to update the stack.
There was this need for making configuration values flow from one independent stack to another independent stack. You can tackle some the bits using customer resources but a native CloudFormation feature would just making things easier, standardized and improve overall experience with CloudFormation.
2. So we launched a feature called cross stack references :
Cross stack reference feature can help you do things such as share IAM roles, VPC information, and security groups across CloudFormation stacks in a standard way.
You can export values from one stack and use them in another
Let’s take a short example: In this case the Network stack is exporting its VPC with an export name of ProdVPC
The App stack is consuming this value using a new function Fn::ImportValue
Just considering this example, using this feature you can manage your network resources separately from the application resources that support your application.
Any stack within the account and region can consume the exported value
You can view your available exports using the console, API or CLI
I wanted to talk little bit about Nested stacks. Its not a new CloudFormation concepts at all . It has been there for years, but its worth recapping as this concept is later used later in this session.
The Nested stacks feature lets you create stacks using multiple templates.
You can describe nested templates within a parent template using the AWS::CloudFormation::Stack resource type and a pointer to the S3 URL where the template is located.
When you use CloudFormation to create a stack using the parent template, CloudFormation creates all the resources in the template including nested stacks with its resources that are described in the nested templates.
Nested stacks provides a way to break up and organize templates that are too large.
It allows you to separate commonly used infrastructure components into their own templates
Lets recap the use cases, advantages and considerations for Nested Stacks and Cross Stack References.
Its important to remember that these are not alternative approaches but rather tools that can be used together!
If you like to separate and re-use templates that contain descriptions of commonly used infrastructure resources, nested stacks gives you a good way to provision and manage them together as a single unit
When you use nested stacks CloudFormation manages all the dependencies and creation order including your nested templates to provisions all of your resources.
However, you do need to consider that all the resources in the stack are subject to updates and rollbacks….and if you are looking to reuse generic templates you’ll want to make sure you don’t have any custom names for resources.
Cross stack references is a great way to share common resources like networking or database resources that may need to used by several applications.
It allows you to manage stacks of resources independently. The folks that own network and security can manage those resources independent of the instances, containers or functions.
However, you do need to consider that CloudFormation will prevent you from deleting or replacing a resource that is being imported by another stack.
Nested stacks is a convenient way to deploy using a library of common/ shared templates. Cross stack references is a convenient way to share resources across stacks.
Again, its important to remember that these are not alternative approaches but rather tools that can be used together!
Consider a simple micro services based application.
Deconstruct the application into the necessary AWS resources
Create CloudFormation templates based your management needs – determine if you need to structure your resources in nested stacks for convenience or cross stack references so they may be managed independently
Use CloudFormation to model your continuous delivery pipeline as code and version control it like you would your application code
CodePipeline uses CloudFormation to continuously deliver changes you make to your infrastructure code
The sample application we’ve picked to examine is available at github at awslabs under ecs-refarch-cloudformation.
The application consists of two interconnecting microservices deployed as ECS services
The application runs on a highly available ECS cluster deployed across multiple availability zones with auto scaling
Lets look at the published reference architecture for this application.
A tiered VPC with public and private subnets, spanning an AWS region.
A highly available ECS cluster
NAT gateways (1 per Zone) to handle outbound traffic.
An Application Load Balancer (ALB) to the public subnets to handle inbound traffic.
ALB path-based routes for each ECS service to route the inbound traffic to the correct service.
Centralized container logging with Amazon CloudWatch Logs.
Lets look at the AWS resource types that are needed to support this application.
Decomposing the requisite elements by category we will need:
VPC
A pair of public and private subnets split across two zones
A pair of private route tables, default routes and associations that tie them together
A pair of NAT gateways and an EIP for each public subnet
A public route table, default route and an internet gateway
Create templates based on the logical grouping
The Network template outputs the VPC, and subnets that are consumed by all other templates
Security groups are managed and exported out of the security template.
The loadbalancer is launched in the public subnets and outputs the DNS name and Listener ARN
The ECS cluster exports the ARN of the ECS cluster
Different ways to compose infrastructure
Single stack supporting the application per region
One parent stack and several nested stacks leveraging standard templates
Multiple standalone stacks, loosely coupled together with cross stack references
Combination of nested stacks and cross stack references
For the rest of this demo lets examine using Cross Stack references within a continuous delivery workflow
Integrate your infrastructure with your Continuous Delivery framework. Lets review modeling a release pipeline for this infrastrucure.
To automate the rollout of infrastructure changes lets use CodePipeline to trigger deployments using CloudFormation
Lets look at incorporating continuous delivery for your infrastructure
AWS CodePipeline is a continuous delivery service for fast and reliable application and infrastructure updates.
CodePipeline builds, tests and deploys your code each time there is a code change, based on the release process you define.
We’ve added new built-in actions for CloudFormation that let you create, update or delete stacks and create and execute change sets
Take a common scenario for many customers. Network resources are managed by a separate team with separate policies and update needs than the resources that support applications.
Lets model an example for this pipeline with two separate VPCs
Use a CloudFormation template to setup and manage your pipeline
This particular example creates:
an S3 bucket as an artifact store for the pipeline
SNS Topic to subscribe to email notifications for approvals
The pipeline with its various stages
IAM roles that CloudFormation will use to provision resources and CodePipeline will need too call CloudFormation on your behalf. A best practice for this would be to model this in a separate