Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS CloudFormation Best Practices

16,875 views

Published on

Learn about recent product updates that can help users make the most of the service, such as AWS CloudFormation Designer and Change Sets.

Published in: Technology

AWS CloudFormation Best Practices

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Peter Dalbhanjan, Solutions Architect September 2016 Infrastructure as Code: Best Practices with AWS CloudFormation
  2. 2. Agenda • Introduction • Best practices • Key new features • YAML support • Cross-stack references • Q & A’s New New
  3. 3. AWS CloudFormation • Create templates that describe and model AWS infrastructure • CloudFormation then provisions AWS resources based on dependency needs • Version control/replicate/update the templates like application code • Integrates with development, CI/CD, management tools • No additional charge to use
  4. 4. CloudFormation concepts and technology JSON/YAML formatted file Parameter definition Resource creation Configuration actions Framework Stack creation Stack updates Error detection and rollback Configured AWS resources Comprehensive service support Service event aware Customizable Template CloudFormation Stack
  5. 5. Infrastructure as Code workflow code version control code review integrate deploy
  6. 6. Infrastructure as Code workflow code version control code review integrate deploy Text Editor Git/SVN/ Perforce Review Tools Syntax Validation Tools AWS Services
  7. 7. Infrastructure as Code workflow code version control code review integrate deploy “It’s all software” Text Editor Git/SVN/ Perforce Review Tools Syntax Validation Tools AWS Services
  8. 8. Update like software Blue-GreenIn-place Traffic• Faster • Cost-efficient • Simpler state and data migration • Working stack stays intact Templates Stacks
  9. 9. Template Anatomy - Resources { "Description" : "Create an EC2 instance.”, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : “my-key-pair”, "ImageId" : "ami-6869aa05”, “InstanceType” : “m3.medium” } } } }
  10. 10. Template Anatomy - Parameters { "Description" : "Create an EC2 instance.”, "Parameters": { "KeyName": { "Description" : "Name of an existing EC2 KeyPair to enable SSH access into the WordPress web server", "Type": "AWS::EC2::KeyPair::KeyName" }, "EC2InstanceType" : { "Description" : "EC2 instance type", "Type" : "String", "Default" : "t2.micro", "AllowedValues" : [ "t2.micro", "t2.small", "t2.medium" ], "ConstraintDescription" : "Must be t2.micro, t2.small, t2.medium" }, },
  11. 11. Template Anatomy - Outputs "Outputs" : { "WebsiteURL" : { "Description" : ”DNS name of the website", "Value" : { "Fn::GetAtt" : [ “LoadBalancer”, “DNSName” ] } } }
  12. 12. CloudFormation Best Practices
  13. 13. CloudFormation Designer • Visualize template resources • Modify template with drag-drop gestures • Customize sample templates
  14. 14. Avoid manual resource modifications • Avoid making quick-fixes out of band • Update your stacks with CloudFormation • Do not manually change resources • Consider using resource based permissions to limit ability to make changes directly
  15. 15. Preview updates with Change Sets
  16. 16. Learn the intrinsic functions
  17. 17. Fn::FindInMap "Mappings" : { "RegionMap" : { "us-east-1" : { "32" : "ami-6411e20d", "64" : "ami-7a11e213" }, "us-west-1" : { "32" : "ami-c9c7978c", "64" : "ami-cfc7978a" }, "eu-west-1" : { "32" : "ami-37c2f643", "64" : "ami-31c2f645" }, "ap-southeast-1" : { "32" : "ami-66f28c34", "64" : "ami-60f28c32" }, "ap-northeast-1" : { "32" : "ami-9c03a89d", "64" : "ami-a003a8a1" } } },
  18. 18. Fn::FindInMap "Resources" : { "myEC2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32"]}, "InstanceType" : "m1.small" } } }
  19. 19. Bootstrap your applications using EC2 UserData "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[ "#!/bin/bash -ex","n", "yum -y install gcc-c++ make","n", "yum -y install mysql-devel sqlite-devel","n", "yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","n", "gem install --no-ri --no-rdoc rails","n", "gem install --no-ri --no-rdoc mysql","n", "gem install --no-ri --no-rdoc sqlite3","n", "rails new myapp","n", "cd myapp","n", "rails server -d","n"]]}} } } Use EC2 UserData, which is available as a property of AWS::EC2::Instance resources
  20. 20. cfn-init cfn-hup AWS CloudFormation provides helper scripts for deployment within your EC2 instances Metadata Key — AWS::CloudFormation::Init Cfn-init reads this metadata key and installs the packages listed in this key (e.g., httpd, mysql, and php). Cfn-init also retrieves and expands files listed as sources. Amazon EC2 AWS CloudFormation cfn-signal cfn-get- metadata Bootstrap your applications using helper scripts
  21. 21. "Metadata": { "AWS::CloudFormation::Init" : { "config" : { "packages" : { }, "sources" : { }, "commands" : { }, "files" : { }, "services" : { }, "users" : { }, "groups" : { } } } Use AWS::CloudFormation::Init with cfn-init to help bootstrap instances: Bootstrapping example “WebAppHost" : { "Type" : "AWS::EC2::Instance", "Metadata" : { "AWS:CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "gcc" : [], "gcc-c++" : [], "make" : [], "automake" : [],
  22. 22. Prevent stack updates to protected resources using Stack policies Protect your resources using Stack policies { "Statement" : [ { "Effect" : "Allow", "Action" : "Update:*", "Principal": "*", "Resource" : "*" }, { "Effect" : "Deny", "Action" : "Update:*", "Principal": "*", "Resource" : "LogicalResourceId/ProductionDatabase" } ] }
  23. 23. Ownership based template design • Use Microservices approach to define templates • Limit one template to a single service • Use nested stacks and cross-stack reference to break up large templates • Organize templates according to team structure/job function/line of business
  24. 24. Ownership based template design
  25. 25. Ownership based template design
  26. 26. Ownership – nested stacks
  27. 27. Web-SG Ownership – cross-stack references App-SG App-SG DB-SG
  28. 28. Re-usable Templates – across AWS Regions • Consider environmental or regional differences • Amazon EC2 image Ids • VPC environment or “classic” environment • Available instance types • IAM policy principals • Endpoint names • Amazon Resource Names (arns)
  29. 29. Re-usable Templates – “Pseudo-Parameters” Use “pseudo-parameters” to retrieve environmental data • Account Id • Region • Stack Name and Id "LogsBucketPolicy": { "Type": "AWS::S3::BucketPolicy", "Properties": { "Bucket": {"Ref": "LogsBucket”}, "PolicyDocument": { "Version": "2008-10-17", "Statement": [{ "Sid": "ELBAccessLogs", "Effect": "Allow", "Resource": { "Fn::Join": [ "", [ “arn:aws:s3:::", { "Ref": "LogsBucket" }, "/", "Logs", "/AWSLogs/", { "Ref": "AWS::AccountId" }, "/*” ]] }, "Principal": …, "Action": [ "s3:PutObject" ] }] } } },
  30. 30. Re-usable Templates - Using mappings Use mappings to define variables • Single place for configuration • Re-usable within the template "LogsBucketPolicy": { "Type": "AWS::S3::BucketPolicy", "Properties": { "Bucket": {"Ref": "LogsBucket”}, "PolicyDocument": { "Version": "2008-10-17", "Statement": [{ "Sid": "ELBAccessLogs", "Effect": "Allow", "Resource": { "Fn::Join": [ "", [ { "Fn::FindInMap" : ["RegionalConfig", {"Ref" : "AWS::Region"}, "ArnPrefix”]}, "s3:::”, { "Ref": "LogsBucket" }, "/", "Logs", "/AWSLogs/”, { "Ref": "AWS::AccountId" }, "/*" ] ] }, } “Mappings” : { “RegionalConfig” : { “us-east-1” : { “AMI” : “ami-12345678”, ”ELBAccountId": "127311923021”, “ArnPrefix” : “arn:aws:” }, “us-west-1” : { “AMI” : “ami-98765432” ”ELBAccountId": “027434742980" “ArnPrefix” : “arn:aws:” }, : } }
  31. 31. Re-usable Templates – Using conditionals Use conditionals to customize resources and parameters "DBEC2SG": { "Type": "AWS::EC2::SecurityGroup", "Condition" : "Is-EC2-VPC", "Properties" : {…} }, "DBSG": { "Type": "AWS::RDS::DBSecurityGroup", "Condition" : "Is-EC2-Classic", "Properties": {…} }, "MySQLDatabase": { "Type": "AWS::RDS::DBInstance", "Properties": { : "VPCSecurityGroups": { "Fn::If" : [ "Is-EC2-VPC", [ { "Fn::GetAtt": [ "DBEC2SG", "GroupId" ] } ], { "Ref" : "AWS::NoValue"}]}, "DBSecurityGroups": { "Fn::If" : [ "Is-EC2-Classic", [ { "Ref": "DBSG" } ], { "Ref" : "AWS::NoValue"}]} } } } "Conditions" : { "Is-EC2-VPC” : { "Fn::Or" : [ {"Fn::Equals" : [ {"Ref” : "AWS::Region"}, "eu-central-1" ]}, {"Fn::Equals" : [ {"Ref" : "AWS::Region"}, "cn-north-1" ]}]}, "Is-EC2-Classic" : { "Fn::Not" : [ { "Condition" : "Is-EC2-VPC"}]} },
  32. 32. Best Practices Summary • CloudFormation Designer • Avoid manual resource modifications • Preview updates with Change Sets • Learn the intrinsic functions • Bootstrap your applications using UserData and helper scripts • Protect critical resources using stack policiess • Ownership based template design • Plan for multi-region • Use Pseudo-Parameters • Use Mappings • Use Conditionals
  33. 33. Key new features • YAML formatted templates • Overview of template structure / basics • New function formatting (!Ref / !GetAZs / !FindInMap) • New Intrinsic Function ( Fn::Sub ) •Cross Stack References • New function Fn::ImportValue • Allows use of outputs from unrelated stacks without custom resource New New
  34. 34. CloudFormation - YAML Why YAML? • Better authoring and readability of templates • Comments – Finally Yay!! • Simplification as templates get more and more complex New
  35. 35. Cloudformation - YAML • Structure is shown through indentation (one or more spaces). • Sequence items are denoted by a dash • Key value pairs within a map are separated by a colon. • Tips: Use a monospace font, don’t use Tab, save using UTF-8 Resources: VPC1: Type: "AWS::EC2::VPC" Properties: CidrBlock: !Ref VPC1Cidr Tags: - Key: "Name" Value: "TroubleShooting"
  36. 36. CloudFormation – YAML Template Structure All sections is the same as in a JSON template --- AWSTemplateFormatVersion: "version date" Description: String Metadata: template metadata Parameters: set of parameters Mappings: set of mappings Conditions: set of conditions Resources: set of resources Outputs: set of outputs
  37. 37. CloudFormation – YAML Function Declaration • Two ways to declare Intrinsic functions: Long and Short • Short Form: • !FindInMap [ MapName, TopLevelKey, SecondLevelKey ] • Long Form: • "Fn::FindInMap" : [ "MapName", "TopLevelKey", "SecondLevelKey"] • Tag = ! (Its not Negation operator) • Few things to note with Tags • You cannot use one tag immediately after another • !Base64 !Sub… • Instead, you can do this • "Fn::Base64": !Sub... • !Select [ !Ref Value, [1,2,3]]
  38. 38. CloudFormation – Intrinsic Functions Fn::Base64 Fn::And Short !Base64 valueToEncode Short !And [condition] Long "Fn::Base64": valueToEncode Long "Fn::And": [condition] Fn::Equals Fn::If Short !Equals [value_1, value_2] Short !If [condition_name, value_if_true, value_if_false] Long "Fn::Equals": [value_1, value_2] Long "Fn::If": [condition_name, value_if_true, value_if_false] Fn::Not Fn::Or Short !Not [condition] Short !Or [condition, ...] Long "Fn::Not": [condition] Long "Fn::Or": [condition, ...]
  39. 39. CloudFormation – Intrinsic Functions Cont. Fn::FindInMap Short !FindInMap [ MapName, TopLevelKey, SecondLevelKey ] Long "Fn::FindInMap" : [ "MapName", "TopLevelKey", "SecondLevelKey"] Fn::GetAtt Short A) !GetAtt logicalNameOfResource.attributeName B) !GetAtt - logicalID - attributeName C) !GetAtt [logicalID, attributeName] Long "Fn::GetAtt": [ logicalNameOfResource, attributeName ]
  40. 40. CloudFormation – Intrinsic Functions Cont. 2 Fn::Join Short A) !Join [ delimiter, [ comma-delimited list of values ] ] B) !Join - delimiter - - value1 - value2 Long "Fn::Join": [ delimiter, [ comma-delimited list of values ] ] Fn::GetAZs Short A) !GetAZs region (e:g !GetAZs "us-east-1") B) !GetAZs “” C) !GetAZs {Ref : "AWS::Region"} Long "Fn::GetAZs": region
  41. 41. CloudFormation – Intrinsic Functions Cont. 3 Fn::Select Short A) !Select [ index, listOfObjects ] B) !Select - index - - value1 - value2 Long "Fn::Select": [ index, listOfObjects ] Ref Fn::ImportValue Short !Ref logicalName Short !ImportValue sharedValueToImport Long “Ref”: logicalName Long "Fn::ImportValue": sharedValueToImport
  42. 42. CloudFormation – Fn::Sub • Substitute variables in an input string with values • Function accepts a string or a map as a parameter. • Usage • VarName: ${MyVariableValue} • Literal: ${!LiteralValue} • Use | if you are spanning multiple lines • Available in JSON as well New
  43. 43. CloudFormation – Fn::Sub Declaration Fn::Sub Option 1 Short: !Sub - String - VarName: VarValue Long: "Fn::Sub": - String - VarName: VarValue Option 2 When you’re only substituting parameters, logical IDs, or resource attributes do not specify a variable mapping Short: !Sub String Long: "Fn::Sub": String
  44. 44. CloudFormation – Fn::Sub Examples /tmp/create-wp-config: content: !Sub | #!/bin/bash -xe cp /var/www/html/wordpress/wp-config-sample.php /var/www/html/wordpress/wp-config.php sed -i "s/'database_name_here'/'${DBName}'/g" wp-config.php sed -i "s/'username_here'/'${DBUser}'/g" wp-config.php sed -i "s/'password_here'/'${DBPassword}'/g" wp-config.php mode: '000500' owner: root group: root configure_wordpress: commands: 01_set_mysql_root_password: command: !Sub | mysqladmin -u root password '${DBRootPassword}' test: !Sub | $(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? != 0 )) 02_create_database: command: !Sub | mysql -u root --password='${DBRootPassword}' < /tmp/setup.mysql test: !Sub | $(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? !=0)) 03_configure_wordpress: command: /tmp/create-wp-config cwd: /var/www/html/wordpress
  45. 45. CloudFormation – Cross Stack References • Sharing resources made easy • IAM roles, VPC, Security groups • Add an explicit “Export” declaration to stack output • Use the resource in another stack using a new intrinsic function, Fn::ImportValue` • Few guidelines: • Export names must be unique within an account and region • Cannot create references across regions • Cannot delete a stack that is referenced by another stack (Dependencies are communicated in errors) • Outputs cannot be modified or removed as long as it is referenced by a current stack New
  46. 46. CloudFormation – Fn::ImportValue The new intrinsic function for accessing exported outputs. JSON { "Fn::ImportValue" : sharedValueToImport } YAML "Fn::ImportValue": sharedValueToImport !ImportValue sharedValueToImport
  47. 47. CloudFormation – Cross Stack Examples Stack A Stack B "Outputs": { "WebServerSecurityGroup": { "Description": "TheIDofthesecuritygroup", "Value": {"Fn: : GetAtt": ["WebServerSecurityGroup", "GroupId"]}, "Export": { "Name": "AccountSecGroup"} } } "Resources" : { "WebServerInstance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "InstanceType" : "ts.micro", "ImageId" : "ami-a1b23456", "NetworkInterfaces" : [{ "GroupSet" : [{ "Fn::ImportValue" : "AccountSecGroup" ]} ]} } } }
  48. 48. Questions?
  49. 49. Thank you! Peter Dalbhanjan dalbhanj@amazon.com

×