AWS CloudFormation under the Hood (DMG303) | AWS re:Invent 2013

9,176 views

Published on

You already know that AWS CloudFormation is a powerful tool for provisioning and managing your AWS infrastructure, but did you know that it can also provision and manage resources outside of AWS? Did you know that CloudFormation can fully bootstrap your EC2 instances, securely download data from S3, and even supports Mustache templates? In this session you will go on a deep dive, touring of some of CloudFormation's most advanced features with a member of the team that built the service. Explore custom resources, cfn-init, S3 authentication, and Mustache templates in a series of technical demos with code samples available for download afterwards.

Published in: Technology, Business

AWS CloudFormation under the Hood (DMG303) | AWS re:Invent 2013

  1. 1. AWS CloudFormation Under the Hood Adam Thomas, Amazon Web Services DJ Edwards, Amazon Web Services November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. So, what is CloudFormation?
  3. 3. This talk will not answer that question • DMG201 - Zero to Sixty: AWS CloudFormation – Has already happened, but will be available online • Hands-on Labs – Working with CloudFormation – Launching and Managing a Web Application with CloudFormation – Creating an Amazon Virtual Private Cloud (VPC) with CloudFormation
  4. 4. This talk will answer these questions: • • • • What is a custom resource? What can they do for me? How do I write one for myself? What’s new in cfn-init?
  5. 5. Custom Resources
  6. 6. What can custom resources do? • • • • Add New Resources Interact with the CloudFormation Workflow Inject dynamic data into a stack Extend the capabilities of existing resources
  7. 7. What is a custom resource? • An SNS topic… • …hooked up to a service that can: – Respond to JSON messages from CloudFormation – Manage the lifecycle of resources
  8. 8. How are custom resources defined? "myCustomResource" : { "Type": "Custom::MyCustomResource", "Version" : "1.0", "Properties" : { "ServiceToken": "arn:aws:sns:us-east-1:84969EXAMPLE:CRTest", "CustomProperty" : "foo" } }
  9. 9. How are custom resources defined? "myCustomResource" : { "Type": “Custom::MyCustomResource", "Version" : "1.0", "Properties" : { "ServiceToken": "arn:aws:sns:us-east-1:84969EXAMPLE:CRTest", "CustomProperty" : "foo" } }
  10. 10. How are custom resources defined? "myCustomResource" : { "Type": “Custom::MyCustomResource", "Version" : "1.0", "Properties" : { "ServiceToken": "arn:aws:sns:us-east-1:84969EXAMPLE:CRTest", "CustomProperty" : "foo" } }
  11. 11. What can custom resources do? • • • • Add New Resources Interact with the CloudFormation Workflow Inject dynamic data into a stack Extend the capabilities of existing resources
  12. 12. Adding New Resources • Something that can be Created, Updated, and/or Deleted • Can be a software resource – Database schema, Docker container
  13. 13. Meet Steve • Steve loves RDBMS • The schema is very important to Steve – it defines his application • Running SQL scripts by hand is Steve’s worst nightmare
  14. 14. Steve’s requirements • The Template should define the schema explicitly • The schema should be updated by updating the stack • If the update fails, the schema should roll back
  15. 15. Steve’s solution • Steve is very familiar with Liquibase • Liquibase supports JSON formatting! • Steve writes a custom resource with inline JSON schema
  16. 16. DB Schema Template Snippet "appSchema" : { "Type" : "Custom::DatabaseSchema", "Properties" : { "databaseChangeLog" : [{ "changeSet" : { "id" : "1", "author" : “adam", "changes" : [{ "createTable" : { "tableName" : "person", "columns" : …
  17. 17. DB Schema Template Snippet "appSchema" : { "Type" : "Custom::DatabaseSchema", "Properties" : { "databaseChangeLog" : [{ "changeSet" : { "id" : "1", "author" : “adam", "changes" : [{ "createTable" : { "tableName" : "person", "columns" : …
  18. 18. DB Schema Demo
  19. 19. What can custom resources do? • • • • Add New Resources Interact with the CloudFormation Workflow Inject dynamic data into a stack Extend the capabilities of existing resources
  20. 20. Interacting with the CloudFormation Workflow • Use custom resources as a hook into create/update/delete workflows • Built-in example: WaitCondition • Can react to workflow, halt it, or fail it under certain conditions
  21. 21. Meet Frank • Frank analyzes data stored on EBS • Frank uses CloudFormation’s Snapshot on Delete feature to save his analysis results
  22. 22. Frank’s requirements • Frank wants a consistent EBS snapshot when the stack is deleted • Before CloudFormation attempts to detach his EBS volume, it should: – Cleanly shut down his analysis service – Unmount the volume
  23. 23. Why is this a challenge? • CloudFormation can detach volumes without any issues – if you never mount them • What CloudFormation does not do, it cannot undo • Custom resources let you model your steps within the workflow
  24. 24. Frank’s solution • 3 simple bash scripts • A “local” Custom Resource – runs directly on the instance • Create and Update mount the drive; Delete unmounts it.
  25. 25. Volume Mount Template Snippet “VolumeAttach" : { "Type" : "AWS::EC2::VolumeAttachment", "Properties" : … }, "VolumeMount" : { "Type" : "Custom::VolumeMount", "Properties" : { "Device" : “/dev/xvdh”, “MountPoint” : “/mnt/analysis” } }
  26. 26. Volume Mount Template Snippet “VolumeAttach" : { "Type" : "AWS::EC2::VolumeAttachment", "Properties" : … }, "VolumeMount" : { "Type" : "Custom::VolumeMount", "Properties" : { "Device" : “/dev/xvdh”, “MountPoint” : “/mnt/analysis” } }
  27. 27. Volume Mount Template Snippet “VolumeAttach" : { "Type" : "AWS::EC2::VolumeAttachment", "Properties" : … }, "VolumeMount" : { "Type" : "Custom::VolumeMount", "Properties" : { "Device" : “/dev/xvdh”, “MountPoint” : “/mnt/analysis” } }
  28. 28. Volume Mount Template Snippet “VolumeAttach" : { "Type" : "AWS::EC2::VolumeAttachment", "Properties" : … }, "VolumeMount" : { "Type" : "Custom::VolumeMount", "Properties" : { "Device" : “/dev/xvdh”, “MountPoint” : “/mnt/analysis” } }
  29. 29. Volume Mount Demo
  30. 30. What can custom resources do? • • • • Add New Resources Interact with the CloudFormation Workflow Inject dynamic data into a stack Extend the capabilities of existing resources
  31. 31. Injecting Dynamic Data into a Stack • Parameters are standard route into a stack – Allow free-form user input – Constrainable, but on a per-stack level • Mappings are traditionally used to map humanreadable input to static values – AMI IDs, instance type architectures, regional URLs
  32. 32. Injecting Data into a Stack • Custom resources allow for centralized selection logic • Lookups in: – – – – S3 DynamoDB/RDS APIs (EC2.DescribeImages, etc) Third Party datastore
  33. 33. Meet Bill • Bill is the head of operations at a large tech firm • Each of Bill’s 44 services must run on a fully validated and tested AMI • Bill keeps track of these AMIs in a sweet multitabbed Excel spreadsheet
  34. 34. Bill’s requirements • New AMIs should be rolled out centrally • Bill does not want to edit the Mappings section of 44 templates for every release • Bill wants to audit where AMIs are being used
  35. 35. Bill’s solution • A manifest of named, approved AMIs stored in a versioned S3 file • A simple python script that looks up the AMI ID by region and os, architecture, and version
  36. 36. AMI Lookup Template Snippet "AMILookup": { "Type": "Custom::AmiLookup", "Properties": { "os": "ubuntu", "version": “13.04", "arch": "64" } }, "WebServer": { "Type": "AWS::EC2::Instance", "Properties": { "ImageId" : { “Ref" : “AMILookup” } } }
  37. 37. AMI Lookup Template Snippet "AMILookup": { "Type": "Custom::AmiLookup", "Properties": { "os": "ubuntu", "version": "13.04", "arch": "64" } }, "WebServer": { "Type": "AWS::EC2::Instance", "Properties": { "ImageId" : { “Ref" : “AMILookup” } } }
  38. 38. AMI Lookup Template Snippet "AMILookup": { "Type": "Custom::AmiLookup", "Properties": { "os": "ubuntu", "version": "13.04", "arch": "64" } }, "WebServer": { "Type": "AWS::EC2::Instance", "Properties": { "ImageId" : { “Ref" : “AMILookup” } } }
  39. 39. AMI Lookup Demo
  40. 40. What can custom resources do? • • • • Add New Resources Interact with the CloudFormation Workflow Inject dynamic data into a stack Extend the capabilities of existing resources
  41. 41. Extending Resource Capabilities • CloudFormation is concerned only with Create, Update, and Delete • Some services, like AutoScaling, have lifecycles outside of these phases • No place in template to encapsulate longrunning, resource-based business logic
  42. 42. Meet Tom • Tom manages a fleet of virtual desktops in AWS • Tom uses AutoScaling for consistent fleet size • Tom’s users use VNC to connect to their virtual desktops
  43. 43. Tom’s requirements • Servers should be named using his clever, easy-toremember Simpsons scheme • Names should be recycled as machines are replaced
  44. 44. Tom’s solution • Python scripts respond to Auto Scaling notifications to manage Route53 records • Names are managed in a simple DynamoDB table
  45. 45. Auto Scaled DNS Snippet (1 of 2) "DNSProcessor" : { "Type": "Custom::DNSProcessor", "Properties": { "HostedZoneId" : { "Ref" : "HostedZone" }, "DNSPattern" : {"Fn::Join" : [".",[“{{simpsons_name}}", { "Ref" : "AWS::Region" }, “{{hosted_zone_name}}"]] } } },
  46. 46. Auto Scaled DNS Snippet (1 of 2) "DNSProcessor" : { "Type": "Custom::DNSProcessor", "Properties": { "HostedZoneId" : { "Ref" : "HostedZone" }, "DNSPattern" : {"Fn::Join" : [".",[“{{simpsons_name}}", { "Ref" : "AWS::Region" }, “{{hosted_zone_name}}"]] } } },
  47. 47. Auto Scaled DNS Snippet (2 of 2) "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "NotificationConfiguration" : { "TopicARN" : { "Fn::GetAtt" : ["DNSProcessor", “Topic"] }, "NotificationTypes" : [ "autoscaling:EC2_INSTANCE_LAUNCH","autoscaling:EC2_INSTANCE_TERMINATE"] }, "Tags" : [{ "Key" : "ProcessorId", "Value" : { "Ref" : "DNSProcessor" }, "PropagateAtLaunch" : false }] } }
  48. 48. Auto Scaled DNS Snippet (2 of 2) "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "NotificationConfiguration" : { "TopicARN" : { "Fn::GetAtt" : ["DNSProcessor", “Topic"] }, "NotificationTypes" : [ "autoscaling:EC2_INSTANCE_LAUNCH","autoscaling:EC2_INSTANCE_TERMINATE"] }, "Tags" : [{ "Key" : "ProcessorId", "Value" : { "Ref" : "DNSProcessor" }, "PropagateAtLaunch" : false }] } }
  49. 49. Auto Scaled DNS Demo
  50. 50. Building Your Own Custom Resource • Write code to respond to Create, Update, and Delete events • Route Custom Resource SNS Topic to an SQS Queue for maximum fault tolerance
  51. 51. Can you give me a diagram? CloudFormation Stack Workflow starts building Custom Resource CloudFormation sends CREATE notification to Custom Resource Custom Resource creates resource and returns JSON message CloudFormation processes JSON message and stores result Stack workflow continues Other resources access Custom Resource attributes via GetAtt and Ref
  52. 52. How about an architectural overview? SQS Queue Custom Resource Topic AWS CloudFormation Custom Resource Implementation Auto scaling Group Region
  53. 53. Can you add VPC? Custom Resource Topic SQS Queue AWS CloudFormation Custom Resource Implementation Region VPN Existing Service Corporate Data center
  54. 54. What makes for a good resource? • Good resources are: idempotent – One unique request, n times == one unique response • Immediately usable when complete • Can be deleted cleanly from any state • Represent one standalone piece of functionality – Embedded resources look convenient, but are hard to update – Elastic Load Balancers embed Policies, which can depend on each other, yet this is not modeled in the template
  55. 55. You keep telling me it’s simple… • It’s really simple if you use aws-cfn-resourcebridge • Cross-platform hook-based daemon • Simply supply scripts for Create, Update, and Delete • Open source (Apache 2.0) • Install or fork from https://github.com/aws/awscfn-resource-bridge
  56. 56. Example Code • All examples from this talk are available at https://github.com/awslabs/aws-cfn-customresource-examples • Stealing from others is the easiest way to get started – And the best way to use CloudFormation!
  57. 57. cfn-init
  58. 58. cfn-init • Simple library for “getting bits on the box” • Install packages, download files, start services • Works on Windows, Linux, and any platform with Python 2.6, 2.7
  59. 59. {{cfn-init}} • • • • Fn::Join can be hard to follow Many configuration files are largely boilerplate Files can process Mustache templates Simply add context
  60. 60. Wordpress config "/var/www/html/wordpress/wp-config.php" : { "content" : { "Fn::Join" : ["", [ "<?phpn", "define('DB_NAME', '", {"Ref" : "DBName"}, "');n", "define('DB_USER', '", {"Ref" : "DBUser"}, "');n", "define('DB_PASSWORD', '", {"Ref" : "DBPassword" }, "');n", "define('DB_HOST', '", {"Fn::GetAtt" : [“MyDB", "Endpoint.Address"]},"');n", "define('DB_CHARSET', 'utf8');n", "define('DB_COLLATE', '');n", "define('AUTH_KEY', 'f@A17vs{ mO0}:&I,6SB.QzV`E?!`/tN5:~GZX%=@ZA%!_T0-]9>g]4ll6~,6G|R');n",
  61. 61. Wordpress config "/var/www/html/wordpress/wp-config.php" : { "content" : { "Fn::Join" : ["", [ "<?phpn", "define('DB_NAME', '", {"Ref" : "DBName"}, "');n", "define('DB_USER', '", {"Ref" : "DBUser"}, "');n", "define('DB_PASSWORD', '", {"Ref" : "DBPassword" }, "');n", "define('DB_HOST', '", {"Fn::GetAtt" : [“MyDB", "Endpoint.Address"]},"');n", "define('DB_CHARSET', 'utf8');n", "define('DB_COLLATE', '');n", "define('AUTH_KEY', 'f@A17vs{ mO0}:&I,6SB.QzV`E?!`/tN5:~GZX%=@ZA%!_T0-]9>g]4ll6~,6G|R');n",
  62. 62. Wordpress config "define('SECURE_AUTH_KEY', 'gTFTI|~rYHY)|mlu:Cv7RN]GQ^3ngyUbw;L0o!12]0c-ispR<-yt3qj]xjquz^&9');n", "define('LOGGED_IN_KEY', 'Jd:HG9M)1p5t2<v~+R-vd{pQ*|*RB^&PUI{vIrydAEEiV!{HS{jN:nErCmLv`p}');n", "define('NONCE_KEY', '4aMj4KZV;,Gu7(B|qOCve[c5?*J5x1+x93i:Ey6hh/6jXh+V_{V4+hw!qE^d*U,-');n", "define('AUTH_SALT', '_Y_&8m)FH)Cns)8}Yb8b88KDSn:p1#p(qBa<~VW&Y1v}P.*9/8S8@P`{mkNxV lC');n", "define('SECURE_AUTH_SALT', '%nG3Ag41^Lew5c86,#zbN:yPFs.GA5a)z5*:Oce1>v6uF~D`,.o1pzS)F8[bM9i[');n", "define('LOGGED_IN_SALT', '~K<y+Ly+_Ww1~dtq>;rSQ^+{P5/k|=!]k%RXAFY@XMY6GSp+wJ5{(|rCzaWjZ%/');n", "define('NONCE_SALT', ',Bs_*Y9:b/1Z:apVLHtz35uim|okkA,b|Jt[&Nla=T{<l_#D?~6Tj-.2.]FonI~');n",
  63. 63. Wordpress config "define('WPLANG' , '');n", "define('WP_DEBUG' , false);n", "$table_prefix = 'wp_';n", "if ( !defined('ABSPATH') )n", " define('ABSPATH', dirname(__FILE__) . '/');n", "require_once(ABSPATH . 'wp-settings.php');n" ]] } }
  64. 64. {{cfn-init}} Wordpress Config “files” : { "/var/www/html/wordpress/wp-config.php" : { “source” : “https://github.com/FAKEPATH/wp-config.mustache”, “context” : { “DbEndpoint” : {“Fn::GetAtt” : [“MyDB”, “Endpoint.Address”]}, “DbName” : { “Ref” : “DbName” }, “DbUser” : { “Ref” : “DbUser” }, “DbPassword” : { “Ref” : “DbPassword” } } } }
  65. 65. {{cfn-init}} Wordpress Config “files” : { "/var/www/html/wordpress/wp-config.php" : { “source” : “https://github.com/FAKEPATH/wp-config.mustache”, “context” : { “DbEndpoint” : {“Fn::GetAtt” : [“MyDB”, “Endpoint.Address”]}, “DbName” : { “Ref” : “DbName” }, “DbUser” : { “Ref” : “DbUser” }, “DbPassword” : { “Ref” : “DbPassword” } } } }
  66. 66. {{cfn-init}} Wordpress Config “files” : { "/var/www/html/wordpress/wp-config.php" : { “source” : “https://github.com/FAKEPATH/wp-config.mustache”, “context” : { “DbEndpoint” : {“Fn::GetAtt” : [“MyDB”, “Endpoint.Address”]}, “DbName” : { “Ref” : “DbName” }, “DbUser” : { “Ref” : “DbUser” }, “DbPassword” : { “Ref” : “DbPassword” } } } }
  67. 67. Roleplaying • cfn-init can use roles to download from S3 • Secured files are not just for proprietary code – Non-AWS credentials – Private service endpoints – Dynamic code (enabling or disabling features)
  68. 68. Roleplaying Template Snippet “AWS::CloudFormation::Authentication” : { “roleCreds” : { “type” : “S3”, “roleName” : “MyS3Role” } } … “files” : { “/etc/secrets.txt” : { “source” : “https://s3.amazonaws.com/mybucket/secrets.txt”, “authentication” : “roleCreds” } }
  69. 69. Roleplaying Template Snippet “AWS::CloudFormation::Authentication” : { “roleCreds” : { “type” : “S3”, “roleName” : “MyS3Role” } } … “files” : { “/etc/secrets.txt” : { “source” : “https://s3.amazonaws.com/mybucket/secrets.txt”, “authentication” : “roleCreds” } }
  70. 70. Roleplaying Template Snippet “AWS::CloudFormation::Authentication” : { “roleCreds” : { “type” : “S3”, “roleName” : “MyS3Role” } } … “files” : { “/etc/secrets.txt” : { “source” : “https://s3.amazonaws.com/mybucket/secrets.txt”, “authentication” : “roleCreds” } }
  71. 71. cfn-hup • • • • Not new, but not often used in samples Installed in same package as cfn-init Available as Linux and Windows service Listens for changes to the stack and runs scripts when they occur – Usually just runs or re-runs cfn-init
  72. 72. Custom Resources vs. cfn-hup • Custom Resources require an SNS topic, and usually an SQS queue • cfn-hup cannot interact with CloudFormation workflow – Workflow will not wait for cfn-hup – cfn-hup cannot fail workflow – cfn-hup cannot inject data into stack
  73. 73. Summary • Custom Resources let you extend CloudFormation beyond the existing Resource Library • For more than just “things that can be created” • cfn-init lets you use Mustache and Roles to create simple, secure configuration
  74. 74. Corner us in the Developer Lounge Adam Thomas DJ Edwards
  75. 75. Please give us your feedback on this presentation DMG303 - AWS CloudFormation Under the Hood As a thank you, we will select prize winners daily for completed surveys!

×