Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS Infrastructure as Code - September 2016 Webinar Series

3,706 views

Published on

AWS CloudFormation lets you model, provision, and update a collection of AWS resources with JSON templates. You can manage your Infrastructure as Code and deploy stacks from a single Amazon EC2 instance to multi-tier applications. In this session, we will explore CloudFormation best practices in planning and provisioning your AWS infrastructure. We will cover recent product updates that will help users to make the most of this service and demonstrate new features. This session will benefit both new and experienced users of CloudFormation.

Learning Objectives:
• Learn best practices for managing your infrastructure as code using CloudFormation
• Discover new techniques for making the most of CloudFormation
• Hear about the latest product updates and new features released

Who Should Attend:
• Developers, DevOps, IT Operations, Systems Administrators, Solutions Architects

Published in: Technology

AWS Infrastructure as Code - September 2016 Webinar Series

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Chris Munns Business Development Manager – DevOps Amazon Web Services September 2016 Infrastructure as Code: Best Practices with AWS CloudFormation
  2. 2. CloudFormation concepts and technology JSON/YAML formatted file Parameter definition Resource creation Configuration actions Framework Stack creation Stack updates Error detection and rollback Configured AWS resources Comprehensive service support Service event aware Customizable Template CloudFormation Stack
  3. 3. Infrastructure as Code workflow code version control code review integrate deploy
  4. 4. Infrastructure as Code workflow code version control code review integrate deploy Text Editor Git/SVN/ Perforce Review Tools Syntax Validation Tools AWS Services
  5. 5. Infrastructure as Code workflow code version control code review integrate deploy “It’s all software” Text Editor Git/SVN/ Perforce Review Tools Syntax Validation Tools AWS Services
  6. 6. In-place Blue-Green Traffic • Quicker and more cost efficient • Updates or replaces resources • Simpler state and data migration • Working stack stays intact for easy roll backs • Potentially need to sync data • Shift traffic to new stack Templates Stacks Update like software Traffic
  7. 7. { "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if you create a stack from this template.", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" }, "Environment": { "Type" : "String", "Default" : ”Dev", "AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"], "Description" : "Environment that the instances will run in.” } }, "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-7f418316" }, "us-west-2" : { "AMI" : "ami-16fd7026" } } }, "Conditions" : { ”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]}, }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]}, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "PublicDNS" : { "Description" : "Public DNSName of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] } } } }
  8. 8. { "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if you create a stack from this template.", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" }, "Environment": { "Type" : "String", "Default" : ”Dev", "AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"], "Description" : "Environment that the instances will run in.” } }, "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-7f418316" }, "us-west-2" : { "AMI" : "ami-16fd7026" } } }, "Conditions" : { ”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]}, }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]}, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "PublicDNS" : { "Description" : "Public DNSName of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] } } } } HEADERS PARAMETERS MAPPINGS RESOURCES OUTPUTS CONDITIONALS
  9. 9. { "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if you create a stack from this template.", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" }, "Environment": { "Type" : "String", "Default" : ”Dev", "AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"], "Description" : "Environment that the instances will run in.” } }, "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-7f418316" }, "us-west-2" : { "AMI" : "ami-16fd7026" } } }, "Conditions" : { ”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]}, }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]}, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "PublicDNS" : { "Description" : "Public DNSName of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] } } } } HEADERS PARAMETERS MAPPINGS RESOURCES OUTPUTS CONDITIONALS Description of what your stack does, contains, etc Provision time values that add structured flexibility and customization Pre-defined conditional case statements Conditional values set via evaluations of passed references AWS resource definitions Resulting attributes of stack resource creation
  10. 10. { "Description" : "Create an EC2 instance.”, "Parameters": { "KeyName": { "Description" : "Name of an existing EC2 KeyPair to enable SSH access into the WordPress web server", "Type": "AWS::EC2::KeyPair::KeyName" }, "EC2InstanceType" : { "Description" : "EC2 instance type", "Type" : "String", "Default" : "t2.micro", "AllowedValues" : [ "t2.micro", "t2.small", "t2.medium" ], "ConstraintDescription" : "Must be t2.micro, t2.small, t2.medium" }, }, Template Anatomy - Parameters
  11. 11. { "Description" : "Create an EC2 instance.”, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : “my-key-pair”, "ImageId" : "ami-6869aa05”, “InstanceType” : “m3.medium” } } } } Template Anatomy - Resources
  12. 12. "Outputs" : { "WebsiteURL" : { "Description" : ”DNS name of the website", "Value" : { "Fn::GetAtt" : [ “LoadBalancer”, “DNSName” ] } } } Template Anatomy - Outputs
  13. 13. CloudFormation Best Practices
  14. 14.  Visualize template resources  Modify template with drag-drop gestures  Customize sample templates CloudFormation Designer
  15. 15. Avoid manual resource modifications  Avoid making quick-fixes out of band  Update your stacks with CloudFormation  Do not manually change resources  Consider using resource based permissions to limit ability to make changes directly
  16. 16. Preview updates with Change Sets
  17. 17. Learn the intrinsic functions
  18. 18. "Mappings" : { "RegionMap" : { "us-east-1" : { "32" : "ami-6411e20d", "64" : "ami-7a11e213" }, "us-west-1" : { "32" : "ami-c9c7978c", "64" : "ami-cfc7978a" }, "eu-west-1" : { "32" : "ami-37c2f643", "64" : "ami-31c2f645" }, "ap-southeast-1" : { "32" : "ami-66f28c34", "64" : "ami-60f28c32" }, "ap-northeast-1" : { "32" : "ami-9c03a89d", "64" : "ami-a003a8a1" } } }, Fn::FindInMap
  19. 19. Fn::FindInMap "Resources" : { "myEC2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32"]}, "InstanceType" : "m1.small" } } }
  20. 20. "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[ "#!/bin/bash -ex","n", "yum -y install gcc-c++ make","n", "yum -y install mysql-devel sqlite-devel","n", "yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","n", "gem install --no-ri --no-rdoc rails","n", "gem install --no-ri --no-rdoc mysql","n", "gem install --no-ri --no-rdoc sqlite3","n", "rails new myapp","n", "cd myapp","n", "rails server -d","n"]]}} } } Use EC2 UserData, which is available as a property of AWS::EC2::Instance resources Bootstrap your applications using EC2 UserData
  21. 21.  AWS CloudFormation provides helper scripts for deployment within your EC2 instances  Metadata Key — AWS::CloudFormation::Init  Cfn-init reads this metadata key and installs the packages listed in this key (e.g., httpd, mysql, and php). Cfn-init also retrieves and expands files listed as sources. Amazon EC2 AWS CloudFormation cfn-init cfn-hup cfn-signal cfn-get- metadata Bootstrap your applications using helper scripts
  22. 22. "Metadata": { "AWS::CloudFormation::Init" : { "config" : { "packages" : { }, "sources" : { }, "commands" : { }, "files" : { }, "services" : { }, "users" : { }, "groups" : { } } } “WebAppHost" : { "Type" : "AWS::EC2::Instance", "Metadata" : { "AWS:CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "gcc" : [], "gcc-c++" : [], "make" : [], "automake" : [], Use AWS::CloudFormation::Init with cfn-init to help bootstrap instances: Bootstrapping example
  23. 23. { "Statement" : [ { "Effect" : "Allow", "Action" : "Update:*", "Principal": "*", "Resource" : "*" }, { "Effect" : "Deny", "Action" : "Update:*", "Principal": "*", "Resource" : "LogicalResourceId/ProductionDatabase" } ] } Prevent stack updates to protected resources using Stack policies Protect your resources using Stack policies
  24. 24. Ownership based template design  Limit one template to a single service  Aim for reusability across environments (development, testing, production)  Use nested stacks and cross-stack reference to break up large templates  Organize templates according to team structure/job function/line of business
  25. 25. Ownership based template design
  26. 26. Template File Defining Stack The entire infrastructure can be represented in an AWS CloudFormation template. Many Stacks & Environments from One Template
  27. 27. Template File Defining Stack The entire infrastructure can be represented in an AWS CloudFormation template. Use the version control system of your choice to store and track changes to this template Many Stacks & Environments from One Template Git Perforce SVN …
  28. 28. Template File Defining Stack Git Perforce SVN … Dev Test Prod The entire infrastructure can be represented in an AWS CloudFormation template. Use the version control system of your choice to store and track changes to this template Build out multiple environments, such as for Development, Test, Production and even DR using the same template Many Stacks & Environments from One Template
  29. 29. Ownership based template design
  30. 30. Ownership – nested stacks
  31. 31. Web-SG Ownership – cross-stack references App-SG App-SG DB-SG
  32. 32. Re-usable Templates – across AWS Regions  Consider environmental or regional differences  Amazon EC2 image Ids  VPC environment or “classic” environment  Available instance types  IAM policy principals  Endpoint names  Amazon Resource Names (arns)
  33. 33. "LogsBucketPolicy": { "Type": "AWS::S3::BucketPolicy", "Properties": { "Bucket": {"Ref": "LogsBucket”}, "PolicyDocument": { "Version": "2008-10-17", "Statement": [{ "Sid": "ELBAccessLogs", "Effect": "Allow", "Resource": { "Fn::Join": [ "", [ “arn:aws:s3:::", { "Ref": "LogsBucket" }, "/", "Logs", "/AWSLogs/", { "Ref": "AWS::AccountId" }, "/*” ]] }, "Principal": …, "Action": [ "s3:PutObject" ] }] } } },  Use “pseudo-parameters” to retrieve environmental data  Account Id  Region  Stack Name and Id Re-usable Templates – “Pseudo-Parameters”
  34. 34. Re-usable Templates - Using mappings "LogsBucketPolicy": { "Type": "AWS::S3::BucketPolicy", "Properties": { "Bucket": {"Ref": "LogsBucket”}, "PolicyDocument": { "Version": "2008-10-17", "Statement": [{ "Sid": "ELBAccessLogs", "Effect": "Allow", "Resource": { "Fn::Join": [ "", [ { "Fn::FindInMap" : ["RegionalConfig", {"Ref" : "AWS::Region"}, "ArnPrefix”]}, "s3:::”, { "Ref": "LogsBucket" }, "/", "Logs", "/AWSLogs/”, { "Ref": "AWS::AccountId" }, "/*" ] ] }, } “Mappings” : { “RegionalConfig” : { “us-east-1” : { “AMI” : “ami-12345678”, ”ELBAccountId": "127311923021”, “ArnPrefix” : “arn:aws:” }, “us-west-1” : { “AMI” : “ami-98765432” ”ELBAccountId": “027434742980" “ArnPrefix” : “arn:aws:” }, : } }  Use mappings to define variables  Single place for configuration  Re-usable within the template
  35. 35. Use conditionals to customize resources and parameters "DBEC2SG": { "Type": "AWS::EC2::SecurityGroup", "Condition" : "Is-EC2-VPC", "Properties" : {…} }, "DBSG": { "Type": "AWS::RDS::DBSecurityGroup", "Condition" : "Is-EC2-Classic", "Properties": {…} }, "MySQLDatabase": { "Type": "AWS::RDS::DBInstance", "Properties": { : "VPCSecurityGroups": { "Fn::If" : [ "Is-EC2-VPC", [ { "Fn::GetAtt": [ "DBEC2SG", "GroupId" ] } ], { "Ref" : "AWS::NoValue"}]}, "DBSecurityGroups": { "Fn::If" : [ "Is-EC2-Classic", [ { "Ref": "DBSG" } ], { "Ref" : "AWS::NoValue"}]} } } } "Conditions" : { "Is-EC2-VPC” : { "Fn::Or" : [ {"Fn::Equals" : [ {"Ref” : "AWS::Region"}, "eu-central-1" ]}, {"Fn::Equals" : [ {"Ref" : "AWS::Region"}, "cn-north-1" ]}]}, "Is-EC2-Classic" : { "Fn::Not" : [ { "Condition" : "Is-EC2-VPC"}]} }, Re-usable Templates – Using conditionals
  36. 36. Best Practices Summary  CloudFormation Designer  Avoid manual resource modifications  Preview updates with Change Sets  Learn the intrinsic functions  Bootstrap your applications using UserData and helper scripts  Protect critical resources using stack policies  Ownership based template design  Plan for environment and multi-region  Use Pseudo-Parameters  Use Mappings  Use Conditionals
  37. 37.  YAML formatted templates  Overview of template structure / basics  New function formatting (!Ref / !GetAZs / !FindInMap)  New Intrinsic Function ( Fn::Sub )  Cross Stack References  New function Fn::ImportValue  Allows use of outputs from unrelated stacks with custom resource New Key new features New
  38. 38. CloudFormation - YAML Why YAML?  Better authoring and readability of templates  Native comment support  Simplification as templates get more and more complex New
  39. 39. Cloudformation - YAML Resources: VPC1: Type: "AWS::EC2::VPC" Properties: CidrBlock: !Ref VPC1Cidr Tags: - Key: "Name" Value: "TroubleShooting"  Structure is shown through indentation (one or more spaces).  Sequence items are denoted by a dash  Key value pairs within a map are separated by a colon.  Tips: Use a monospace font, don’t use Tab, save using UTF-8
  40. 40. All sections are the same as in a JSON template --- AWSTemplateFormatVersion: "version date" Description: String Metadata: template metadata Parameters: set of parameters Mappings: set of mappings Conditions: set of conditions Resources: set of resources Outputs: set of outputs CloudFormation – YAML Template Structure
  41. 41.  Two ways to declare Intrinsic functions: Long and Short  Short Form:  !FindInMap [ MapName, TopLevelKey, SecondLevelKey ]  Long Form:  "Fn::FindInMap" : [ "MapName", "TopLevelKey", "SecondLevelKey"]  Tag = ! (Its not Negation operator)  Few things to note with Tags  You cannot use one tag immediately after another  !Base64 !Sub…  Instead, you can do this  "Fn::Base64": !Sub...  !Select [ !Ref Value, [1,2,3]] CloudFormation – YAML Function Declaration
  42. 42. Fn::Base64 Fn::And Short !Base64 valueToEncode Short !And [condition] Long "Fn::Base64": valueToEncode Long "Fn::And": [condition] Fn::Equals Fn::If Short !Equals [value_1, value_2] Short !If [condition_name, value_if_true, value_if_false] Long "Fn::Equals": [value_1, value_2] Long "Fn::If": [condition_name, value_if_true, value_if_false] Fn::Not Fn::Or Short !Not [condition] Short !Or [condition, ...] Long "Fn::Not": [condition] Long "Fn::Or": [condition, ...] CloudFormation – Intrinsic Functions
  43. 43. NewCloudFormation – Fn::Sub  Substitute variables in an input string with values  Function accepts a string or a map as a parameter.  Usage  VarName: ${MyVariableValue}  Literal: ${!LiteralValue}  Use | if you are spanning multiple lines  Available in JSON as well
  44. 44. /tmp/create-wp-config: content: !Sub | #!/bin/bash -xe cp /var/www/html/wordpress/wp-config-sample.php /var/www/html/wordpress/wp-config.php sed -i "s/'database_name_here'/'${DBName}'/g" wp-config.php sed -i "s/'username_here'/'${DBUser}'/g" wp-config.php sed -i "s/'password_here'/'${DBPassword}'/g" wp-config.php mode: '000500' owner: root group: root configure_wordpress: commands: 01_set_mysql_root_password: command: !Sub | mysqladmin -u root password '${DBRootPassword}' test: !Sub | $(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? != 0 )) 02_create_database: command: !Sub | mysql -u root --password='${DBRootPassword}' < /tmp/setup.mysql test: !Sub | $(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? !=0)) 03_configure_wordpress: command: /tmp/create-wp-config cwd: /var/www/html/wordpress CloudFormation – Fn::Sub Examples
  45. 45. CloudFormation – Cross Stack References  Sharing resources made easy  IAM roles, VPC, Security groups  Add an explicit “Export” declaration to stack output  Use the resource in another stack using a new intrinsic function, Fn::ImportValue`  Few guidelines:  Export names must be unique within an account and region  Cannot create references across regions  Cannot delete a stack that is referenced by another stack (Dependencies are communicated in errors)  Outputs cannot be modified or removed as long as it is referenced by a current stack New
  46. 46. The new intrinsic function for accessing exported outputs. JSON { "Fn::ImportValue" : sharedValueToImport } YAML "Fn::ImportValue": sharedValueToImport !ImportValue sharedValueToImport CloudFormation – Fn::ImportValue
  47. 47. Stack A Stack B "Outputs": { "WebServerSecurityGroup": { "Description": "TheIDofthesecuritygroup", "Value": {"Fn: : GetAtt": ["WebServerSecurityGroup", "GroupId"]}, "Export": { "Name": "AccountSecGroup"} } } "Resources" : { "WebServerInstance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "InstanceType" : "ts.micro", "ImageId" : "ami-a1b23456", "NetworkInterfaces" : [{ "GroupSet" : [{ "Fn::ImportValue" : "AccountSecGroup" ]} ]} } } } CloudFormation – Cross Stack Examples
  48. 48. Questions?
  49. 49. Thank you!

×