AWS CloudFormation Deep Dive and Recent Enhancements

P U B L I C S E C T O R
S U M M I T
WASHINGTON DC

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
AWS CloudFormation: Deep Dive
and Recent Enhancements
Luis Colon (@luiscolon1)
Senior Developer Advocate
AWS CloudFormation
3 0 1 5 9 2
Dan Blanco (@thedanblanco)
Developer Advocate
AWS CloudFormation

S U M M I T
Deep dive: agenda
• Modernizing and Extending AWS CloudFormation
• Updates to UX/Console
• Service Coverage
• Public Coverage Roadmap
• Enterprise Management
• Drift Detection
• Updates to AWS CloudFormation Stacksets
• Operational Safety: Managing Secrets
• Developer Productivity
• Authoring with Linter
• Template Schema Plugin
• Macros
• CDK

S U M M I T
Infrastructure as code with AWS CloudFormation
Code in YAML or
JSON directly or
use sample
templates
Upload local
files or from
an S3 bucket
Create stack
using console,
API or CLI
Stacks and
resources are
provisioned

S U M M I T
Modernizing and Extending

S U M M I T
Redesigned console

S U M M I T
Redesigned console: split pane

S U M M I T
Redesigned console: nested stack - parent only view

S U M M I T
New and updated resource types
Close to 400 resource types supported today
Added or updated support for over 70 resource types since January
Amazon API Gateway V2
Amazon CloudWatch Events
Amazon Elastic Cloud Compute
(Amazon EC2)
Amazon GuardDuty
Amazon MQ
Amazon Neptune
Amazon SageMaker
Amazon Simple Email Service
(Amazon SES)
Amazon AppStream
AWS AppSync
AWS Lambda Layers
AWS Service Discovery
AWS Greengrass
AWS AppMesh
AWS RoboMaker
Amazon Kinesis Analytics V2
AWS IoT 1-Click
Amazon DocumentDB

S U M M I T
Coming soon: public coverage roadmap
Get more feedback from the community on most impactful coverage
Focusing on existing resource types coverage

S U M M I T
Enterprise Management

S U M M I T
Drift detection
Allows you to detect if configuration changes were made to your stack resources outside of
AWS CloudFormation via the AWS Management Console, CLI, and SDKs

S U M M I T
Drift detection
Use the diff viewer in
the console to
pinpoint the changes

S U M M I T
Drift detection
Also available via CLI and API
Supports the most commonly
used resources
Automatic drift alerts via AWS
Config rule
Remediate by updating live
configuration values to match the
template values
Looking ahead: supporting more
resources, preventing false
positives, handling edge cases -
help us by providing feedback!

S U M M I T
AWS Private Link support
AWS PrivateLink is a purpose-built
technology designed to access
AWS services, while keeping all the
network traffic within the AWS
network
Use AWS CloudFormation APIs
inside of your Amazon Virtual
Private Cloud (Amazon VPC) and
route data between your Amazon
VPC and AWS CloudFormation
entirely within the AWS network.
No proxies, NATs, or Internet
Gateways required
Improve security posture. E.g.
sending a signal back to AWS
CloudFormation stack from within
a private Amazon VPC without
going across the public internet
Private Subnet
Amazon VPC
Private
Instance
172.16.0.0
172.16.3.0
172.16.2.0
S3
Endpoint
AWS CloudFormation
Endpoint
Amazon
S3
AWS CloudFormation
Wait Condition

S U M M I T
Updates to StackSets
Stacksets extends the
functionality of stacks by
enabling you to create,
update, or delete stacks
across multiple accounts and
regions with a single
operation
Limit increase: 1500 stack
instance operations running
on a region concurrently, per
admin account
Override parameters gives
more fine-grained control of
stack instance updates

S U M M I T
Operational safety: improved handling of secrets
Parameters:
InstanceType:
Type: 'AWS::SSM::Parameter::Value<String>'
Default: ssbEC2iDev
KeyName:
Type: 'AWS::EC2::KeyPair::KeyName'
Default: brinks
LatestAmiId:
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Resources:
PSBInstance:

S U M M I T
Operational safety: improved handling of secrets
Parameters:
InstanceType:
Type: 'AWS::SSM::Parameter::Value<String>'
Default: ssbEC2iDev
KeyName:
Default: brinks
LatestAmiId:
Resources:
PSBInstance:
Type: ‘AWS::EC2::Instance’
Properties:
ImageId: !Ref LatestAmiId
KeyName: !Ref KeyName
InstanceType: !Ref InstanceType

S U M M I T
Operational safety: new dynamic references
Parameters:
KeyName:
Default: brinks
LatestAmiId:
Resources:
PSBInstance:
Type: ‘AWS::EC2::Instance’
Properties:
ImageId: !Ref LatestAmiId
KeyName: !Ref KeyName
InstanceType: '{{resolve:ssm:ssbEC2iDev:1}}'

S U M M I T
Dynamic references: with secure strings
Resources:
MyRDSDB:
Type: "AWS::RDS::DBInstance"
Properties:
DBInstanceClass: db.t2.medium
AllocatedStorage: ’20’
Engine: mariadb
EngineVersion: ’10.2’
MasterUsername: appadmin
MasterUserPassword: ch4ng1ng-s3cr3t
Resources:
MyRDSDB:
Properties:
Engine: mariadb
MasterUserPassword: ‘{{resolve:ssm-secure:ssbRDSmEcntl:1}}'

S U M M I T
Dynamic references: with secrets manager
Resources:
MyRDSDB:
Properties:
Engine: mariadb
MasterUserPassword: ch4ng1ng-s3cr3t
Resources:
MyRDSDB:
Properties:
Engine: mariadb
MasterUsername: '{{resolve:secretsmanager:MyRDSSecret:SecretString:username}}'
MasterUserPassword: '{{resolve:secretsmanager:MyRDSSecret:SecretString:password}}'

S U M M I T
Developer Productivity

S U M M I T
Linter
https://github.com/aws-cloudformation/cfn-python-lint
• Plugins for Atom, VisualStudio
Code, Sublime, VIM
• Run headless in pipelines
• Process multiple files
• Handles Conditions/Fn::If
• SAM Local integration
• Available now on GitHub, over
100,000 downloads

S U M M I T
Template Schema Plugin
https://github.com/aws-cloudformation/aws-cloudformation-template-schem

S U M M I T
Taskcat
https://github.com/aws-quickstart/taskcat
• From the AWS QuickStart team
• Open Source
• Catches problems that aren’t
obvious in a single template or
stack
• Tests templates by creating stacks
in multiple AWS regions
simultaneously
• Generates a report with a pass/fail
grade for each region

S U M M I T
Pipeline
Build Test Promote
AWS Cloud
Region
Developers
Git Push
Templates Taskcat
• Cfn-lint
Source
Staging
Production
Testing Change set

S U M M I T
Macros
• Write short-hand, abbreviated instructions that expand automatically
before deployment
• Add utility functions, iteration loops, strings …
• Ensure resources are defined to comply to standards
• Easy to share code to reuse across stacks
• Key Benefit: once deployed, downstream users can be isolated from
macro program details
• Macros are AWS Lambda functions and can use all supported languages
• New: run macros without change sets

S U M M I T
Macros: examples
Iterator/Loop
• Make me X number of this resource
Execute Python
• Pass arbitrary code
Perform String Functions
• Upper, Lower …
Globals
• Add Global Variables
Defaults
• If resource X is declared, add default attributes

S U M M I T
Iterator: Code
import copy
def process_template(template):
new_template = copy.deepcopy(template)
status = 'success'
for name, resource in template['Resources'].items():
if 'Count' in resource:
count = new_template['Resources'][name].pop('Count')
multiplied = multiply(name, new_template['Resources'][name], count)
if not set(multiplied.keys()) & set(new_template['Resources'].keys()):
new_template['Resources'].update(multiplied)
else:
status = 'failed'
return status, template
return status, new_template
def multiply(resource_name, resource_structure, count):
resources = {}
for iteration in range(1, count):
resources[resource_name+str(iteration)] = resource_structure
return resources
def handler(event, context):
result = process_template(event['fragment'])
return {
'requestId': event['requestId'],
'status': result[0],
'fragment': result[1],
}

S U M M I T
Iterator: Deploy the macro
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
Macro:
Type: AWS::CloudFormation::Macro
Properties:
Name: Count
FunctionName: !GetAtt CountMacroFunction.Arn
CountMacroFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: src
Handler: index.handler
Runtime: python3.6
Timeout: 5

S U M M I T
Iterator: Using your macro
Transform:
- Count
Resources:
Bucket:
Type: AWS::S3::Bucket
Count: 3
Transform:
- Count
Sqs:
Type: AWS:::SQS::Queue
Count: 2
Resources:
Bucket1:
Bucket2:
Bucket3:
Resources:
Sqs1:
Sqs2:

S U M M I T
Macro: Execute Python
AWSTemplateFormatVersion: "2010-09-09"
Description: tests String macro functions
Parameters:
Tags:
Default:
"Env=Prod,Application=MyApp,BU=ModernisationTeam"
Type: "CommaDelimitedList"
Resources:
S3Bucket:
Type: "AWS::S3::Bucket"
Properties:
Tags: |
#!PyPlate
output = []
for tag in params['Tags']:
key, value = tag.split('=')
output.append({"Key": key, "Value": value})
Transform: [PyPlate]
macro_response = {
"requestId": event["requestId"],
"status": "success"
}
try:
params = {
"params": event["templateParameterValues"],
"template": event["fragment"],
"account_id": event["accountId"],
"region": event["region"]
}
response = event["fragment"]
macro_response["fragment"] =
obj_iterate(response, params)
except Exception as e:
traceback.print_exc()
macro_response["status"] = "failure"
macro_response["errorMessage"] = str(e)
return macro_response

S U M M I T
Macro: Add String Functions
Parameters:
InputString:
Default: "This is a test input string"
Type: String
Resources:
S3Bucket:
Properties:
Tags:
- Key: Upper
Value:
'Fn::Transform':
- Name: 'StringMacro'
Parameters:
InputString: !Ref InputString
Operation: Upper
Parameters:
InputString:
Default: "This is a test input string"
Type: String
Resources:
S3Bucket:
Properties:
Tags:
- Key: Upper
Value: “THIS IS A TEST INPUT STRING”

S U M M I T
Code: Add String Functionsdef handler(event, context):
response = {
"status": "success"
}
try:
operation = event["params"]["Operation"]
input = event["params"]["InputString"]
no_param_string_funcs = [
"Upper", "Lower", "Capitalize",
"Title", "SwapCase"]
if operation in no_param_string_funcs:
…
elif operation == "Strip":
…
elif operation == "Replace":
…
elif operation == "MaxLength":
…
except Exception:
traceback.print_exc()
response["status"] = "failure"
macro_response["errorMessage"] = str(e)
return response
Multiple Functions in a single macro:
• Upper
• Lower
• Capitalize
• Title
• SwapCase
• Strip
• Replace
• MaxLength

S U M M I T
Macro: Global Variables
Transform: Globals
Globals:
SomeText: some-text
ThingTag:
Key: Thing
Value: This is a thing
Resources:
Bucket:
Properties:
BucketName: "@SomeText"
Tags:
- "@ThingTag"
- Key: OtherThing
Value: Other thing value
Resources:
Bucket:
Properties:
BucketName: “some-text"
Tags:
- Key: Thing
- Key: OtherThing

S U M M I T
Code: Globals
Transform: Globals
Globals:
SomeText: some-text
ThingTag:
Key: Thing
Resources:
Bucket:
Properties:
BucketName: "@SomeText"
Tags:
- "@ThingTag"
- Key: OtherThing
class Repeater():
def __init__(self, template):
self.repeaters = template["Globals"]
del template["Globals"]
self.template = template
def process(self):
return self.__walk(self.template)
def __walk(self, fragment):
if isinstance(fragment, str) and any(fragment == "@{}".format(key) for key in
self.repeaters):
return self.repeaters[fragment[1:]]
elif isinstance(fragment, dict):
return {
key: self.__walk(value)
for key, value
in fragment.items()
}
elif isinstance(fragment, list):
return [
self.__walk(value)
for value in fragment
]
return fragment
return {
"status": "success",
"fragment": Repeater(event["fragment"]).process(),
}

S U M M I T
Macro: Generate Additional Resources
Transform: Defaults
Resources:
Bucket1:
Resources:
Bucket1:
Properties:
AccessControl: Private
Bucket1Policy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: Bucket1
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Deny
Principal: "*"
Action: "s3:Delete*"
Resource:
Fn::Sub:
"arn:aws:s3:::${Bucket1}/*"
Condition:
Bool:
aws:MultiFactorAuthPresent:
"false"
Whenever a bucket is defined…
• Add access control property
• Add bucket policy
• Generate additional resources, intrinsic
function calls, conditions, more
• Macro can allow user to override defaults

S U M M I T
Advanced: Setting up Defaults
DEFAULTS = json.load(open("defaults.json"))
def interpolate(name, string):
return string.replace("{$}", name)
def get_additional_resources(name, props):
additional_resources = {}
for key, value in props.items():
key = interpolate(name, key)
if isinstance(value, dict):
additional_resources[key] = get_additional_resources(name, value)
elif isinstance(value, list):
additional_resources[key] = [
get_additional_resources(name, v)
for v in value
]
elif isinstance(value, str):
additional_resources[key] = interpolate(name, value)
else:
additional_resources[key] = value
return additional_resources
def process_property(key, default, resource):
# Recursive
prop = resource[key]
if isinstance(prop, dict):
if "Defaults::Override" in prop:
resource[key] = prop["Defaults::Override"]
else:
resource[key] = default
elif isinstance(default, dict):
for k in default.keys():
if k in prop.keys():
process_property(k, default[k], prop)
else:
prop[k] = default[k]
else:
resource[key] = default
def process_resource(name, resource, additional_resources):
default = DEFAULTS[resource["Type"]]
if "Properties" not in resource:
resource["Properties"] = {}
# Handle properties
for key, prop in default["Properties"].items():
if key not in resource["Properties"]:
resource["Properties"][key] = prop
else:
process_property(key, prop, resource["Properties"])
# Add additional resources
additional_resources.update(get_additional_resources(name,
default.get("AdditionalResources", {})))
def process(template):
additional_resources = {}
for name, resource in template["Resources"].items():
if resource["Type"] in DEFAULTS:
process_resource(name, resource, additional_resources)
template["Resources"].update(additional_resources)
return template

S U M M I T
Advanced: Setting up Defaults
{
"AWS::S3::Bucket": {
"Properties": {
"AccessControl": "Private",
"VersioningConfiguration": {
"Status": "Enabled"
}
},
"AdditionalResources": {
"{$}Policy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "{$}"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Principal": "*",
"Action": "s3:Delete*",
"Resource": {
"Fn::Sub": "arn:aws:s3:::${{$}}/*"
},
"Condition": {
"Bool": {
"aws:MultiFactorAuthPresent": "false"
}
…
Specify additional resources in a side file

S U M M I T
AWS CDK (Cloud Development Kit) in Dev Preview
CDK Application
Stack(s)
Construct(s)
CDK
CLI
AWS CloudFormation
Templates“compiler”
“assembly
language”
“processor”
TypeScript/JavaScript, Java, .Net, Python in Dev Preview

S U M M I T
Summary
• Updates to UX/Console
• Service Coverage
• Public Coverage Roadmap
• Drift Detection
• Updates to Stacksets
• Operational Safety: Managing Secrets
• Authoring with Linter
• Template Schema Plugin
• Macros
• CDK

Thank you!
S U M M I T
Luis Colon (@luiscolon1)
Senior Developer Advocate
AWS CloudFormation
Dan Blanco (@thedanblanco)
Developer Advocate
AWS CloudFormation

AWS CloudFormation Deep Dive and Recent Enhancements

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to AWS CloudFormation Deep Dive and Recent Enhancements

Similar to AWS CloudFormation Deep Dive and Recent Enhancements (20)

More from Amazon Web Services

More from Amazon Web Services (20)

AWS CloudFormation Deep Dive and Recent Enhancements