More Related Content Similar to CloudFormation techniques from the Dutch trenches (DVC07) - AWS re:Invent 2018 (20) 1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CloudFormation techniques
from the Dutch trenches
Martijn van Dongen
Chief AWS Technology / AWS APN Ambassador
Binx.io (proudly part of Xebia)
D V C 0 7
2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
CloudFormation Custom Resources
Cfn-lint Custom Rules
Taskcat
3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build
if event['RequestType'] == 'Create' or event['RequestType'] == 'Update’:
client = boto3.client('iam’)
response = client.tag_role(
RoleName=event['ResourceProperties']['RoleName’],
Tags=event['ResourceProperties']['Tags’]
)
cfnresponse.send(event, context, cfnresponse.SUCCESS, {}, "NA")
elif event['RequestType'] == 'Delete’:
cfnresponse.send(event, context, cfnresponse.SUCCESS, {})
6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Add
IAMTaggingFunction:
Type: AWS::Serverless::Function
Properties:
Handler: lambda.lambda_handler
Timeout: 30
Role: !GetAtt 'IAMTaggingFunctionRole.Arn’
Runtime: python3.7
CodeUri: ./build/iamtagging.zip
IAMTaggingFunctionRole:
Type: AWS::IAM::Role
...
7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Package
$ tree
├── build
├── src
│ └── iamtagging
│ ├── cfnresponse.py
│ ├── lambda.py
│ └── requirements.txt
$ docker run
-v $(pwd)/src:/src
-v $(pwd)/build:/build
binxio/python-lambda-packager:3.7
8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Use
TeamRoleTags:
Type: Custom::IAMTagging
Properties:
ServiceToken:
!GetAtt IAMTaggingFunction.Arn
RoleName: !Ref TeamRedRole
Tags:
- Key: Project
Value: Alexa
- Key: CostCenter
Value: 382
9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy
$ aws cloudformation package
--template-file template.yml
--s3-bucket mys3bucket
--output-template-file packaged.yml
...
$ aws cloudformation deploy
--template-file packaged.yml
--capabilities CAPABILITY_IAM
--stack-name iamtag
10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1
2
3
11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
To Production
TeamRoleTags:
Type: AWS::IAM::Tags
Properties:
ServiceToken:
!GetAtt IAMTaggingFunction.Arn
RoleName: !Ref TeamRedRole
Tags:
- Key: Project
Value: Alexa
- Key: CostCenter
Value: 0382
13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Example Extension: Secret Provider
DBPassword:
Type: Custom::Secret
Properties:
Name: /demo/PGPASSWORD
KeyAlias: alias/aws/ssm
Alphabet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890
Length: 30
ReturnSecret: true
ServiceToken:
!Sub 'arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:secret-provider'
https://github.com/binxio/cfn-secret-provider
14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developer Workflow
Stack
TemplateDevelopers
eu-west-1
eu-west-3
Stack
Bucket
Ops
CI/CD
16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Custom Rule
from typing import List
from cfnlint import CloudFormationLintRule
from cfnlint import RuleMatch, Template
class S3BucketsNotEncrypted(CloudFormationLintRule):
"""Check if S3 Bucket is not encrypted"""
id = 'E9S3BucketEncryption’
shortdesc = 'S3 Buckets must always be encrypted’
description = 'S3 Buckets should always have BucketEncryption’
def match(self, cfn: Template) -> List[RuleMatch]:
<your custom rule here>
19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Custom Rule
def match(self, cfn: Template) -> List[RuleMatch]:
matches: List[RuleMatch] = []
recordsets = cfn.get_resources(['AWS::S3::Bucket’])
for name, recordset in recordsets.items():
path = ['Resources', name, 'Properties’]
full_path = ('/'.join(str(x) for x in path))
if isinstance(recordset, dict):
props = recordset.get('Properties’)
if 'BucketEncryption' not in props:
message = "Property BucketEncryption not set in {0}"
matches.append(RuleMatch(path, message.format(full_path)))
return matches
20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Custom Rule
$ cfn-lint -a ../cfn-compliancy-check/rules/s3 -t badtemplate.yml
E9S3BucketEncryption Property BucketEncryption not set in Resources/S3Bucket/Properties
badtemplate.yml:4:5
E3012 Property Resources/S3Bucket/Properties/BucketName should be of type String
badtemplate.yml:5:7
21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Taskcat
$ cat ci/config.yml
global:
owner: martijn@binx.io
qsname: dcv07
regions:
- eu-west-1
- eu-west-3
tests:
scenario-1:
template_file: stack.yml
parameter_input: parameters.json
$ taskcat -c ci/config.yml
... Deploying and generating reports ...
23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developer Workflow
Stack
TemplateDevelopers
eu-west-1
eu-west-3
Stack
Bucket
cfn-lint
DevOps
IAM Policies / cloud-custodian
25. 100+ set of custom cfn-lint rules, to
achieve a significant set of compliancy
coverage, across many AWS services
26. Every service or feature is supported in
CloudFormation, within 24h after the
launch
28. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Martijn van Dongen
martijn@binx.io
29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
