Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Dive: Infrastructure as Code

6,636 views

Published on

Many AWS customers have adopted a DevOps model for faster and more reliable software delivery. Applying software engineering best practices such as revision control and continuous delivery to your infrastructure is essential for adopting DevOps. In this session, find out how CloudFormation and associated AWS tools allow you to leverage a DevOps model by treating infrastructure as code and applying software engineering best practices to your AWS infrastructure.

Published in: Technology

Deep Dive: Infrastructure as Code

  1. 1. ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Deep Dive: Infrastructure as Code Chetan Dandekar, Senior Product Manager, AWS
  2. 2. You are on-board … needs to experiment, innovate, reduce risk Business of services and applications Continuous Delivery culture, automation, measurement, sharing DevOps infrastructure-as-code Cloud
  3. 3. AWS CloudFormation
  4. 4. AWS CloudFormation • Create templates of the infrastructure and applications you want to run on AWS • Have the CloudFormation service automatically provision the required AWS resources and their relationships from the templates • Easily version control, replicate or update the infrastructure and applications using the templates • Integrates with other development, CI/CD, and management tools.
  5. 5. Basic workflow Design Create Infrastructure Templates Write Application Code Create Stacks Iterate
  6. 6. depends on Design - Imagine building a food ordering service Food Catalog Website Ordering Website Customer DB Service Inventory Service Recommendations Service Analytics Service Fulfillment Service Payment Service
  7. 7. Create template – For example, for the food catalog website security group Auto Scaling group EC2 instance Elastic Load Balancing Customer DB Service Inventory Service Recommendations Service ElastiCache memcached cluster Software pkgs, config, & dataCloudWatch alarms
  8. 8. Create template – Resources security group Auto Scaling group EC2 instance Elastic Load Balancing ElastiCache memcached cluster Software pkgs, config, & dataCloudWatch alarms "Resources" : { "SecurityGroup" : {}, "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "MinSize" : "1", "MaxSize" : "3", "LoadBalancerNames" : [ { "Ref" : "LoadBalancer" } ], ... } }, "LoadBalancer" : {}, "CacheCluster" : {}, "Alarm" : {} }, CloudFormation Template
  9. 9. Create template – Parameters Auto Scaling group EC2 instance Recommendations Service Inventory Service Customer DB Service Info to Customize Stack at Creation Examples: Instance Type, App Pkg Version "Parameters" : { "CustomerDBServiceEndPoint" : { "Description" : "URL of the Customer DB Service", "Type" : "String" }, "CustomerDBServiceKey" : { "Description" : "API key for the Customer DB Service", "Type" : "String", "NoEcho" : "true" }, "InstanceType" : { "Description" : "WebServer EC2 instance type", "Type" : "String", "Default" : "m3.medium", "AllowedValues" : ["m3.medium","m3.large","m3.xlarge"], "ConstraintDescription" : "Must be a valid instance type" CloudFormation Template
  10. 10. Create template – Outputs Elastic Load Balancing "Resources" : { "LoadBalancer" : {}, ... }, "Outputs" : { "WebsiteDNSName" : { "Description" : "The DNS name of the website", "Value" : { "Fn::GetAtt" : [ "LoadBalancer", "DNSName" ] } } } CloudFormation Template
  11. 11. Create template – Deploy and configure software Auto Scaling group EC2 instance Software pkgs, config, & data "AWS::CloudFormation::Init": { "webapp-config": { "packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {}, "commands" : {}, "services" : {} }, "chef-config" : {} } CloudFormation Template  Declarative  Debug-able  Updatable  Highly Secure  BIOT™ Bring In Other Tools
  12. 12. Create template – Language features
  13. 13. Create stack
  14. 14. Operate stack
  15. 15. Use a wide range of AWS services  Auto Scaling  Amazon CloudFront  AWS CloudTrail  Amazon CloudWatch  Amazon DynamoDB  Amazon EC2  AWS Elastic Beanstalk  Amazon ElastiCache  Elastic Load Balancing  Amazon Kinesis  IAM  AWS OpsWorks  Amazon RDS  Amazon Redshift  Amazon Route 53  Amazon S3  Amazon SimpleDB  Amazon SNS  Amazon SQS  Amazon VPC and more …
  16. 16. Basic workflow Design Create Infrastructure Templates Write Application Code Create Stacks Iterate
  17. 17. Infrastructure-as-code workflow Code templates Version control Code review Integrate “It’s all software”
  18. 18. “It’s all software” – organize like it’s software Frontend Services • Consumer Website, Seller Website, Mobile Backend Backend Services • Search, Payments, Reviews, Recommendations Shared Services • CRM DBs, Common Monitoring, Alarms, Subnets, Security Groups Base Network • VPCs, Internet Gateways, VPNs, NATs Identity • IAM Users, Groups, Roles
  19. 19. “It’s all software” – build and operate like it’s software Application software Source code Package Loader/Interpreter Desired application state in memory Infrastructure software JSON templates / JSON template generators JSON templates AWS CloudFormation Desired infrastructure in the cloud
  20. 20. Iterate on infrastructure
  21. 21. Update stack In-place Blue-Green Faster Cost-efficient Simpler state and data migration Working stack not touched
  22. 22. Extending AWS CloudFormation
  23. 23. Extend with Custom Resources security group Auto Scaling group EC2 instance Elastic Load Balancing ElastiCache memcached cluster Software pkgs, config, & dataCloudWatch alarms Web Analytics Service AWS CloudFormation Provision AWS Resources "Resources" : { "WebAnalyticsTrackingID" : { "Type" : "Custom::WebAnalyticsService::TrackingID", "Properties" : { "ServiceToken" : "arn:aws:sns:...", "Target" : {"Fn::GetAtt" : ["LoadBalancer", "DNSName"]}, "Plan" : "Gold" } }, ... “Success” + Metadata “Create, Update, Rollback, or Delete” + Metadata
  24. 24. Lambda-backed custom resources security group Auto Scaling group EC2 instance Elastic Load Balancing ElastiCache memcached cluster Software pkgs, config, & dataCloudWatch alarms Your AWS CloudFormation stack // Implement custom logic here Look up an AMI ID Your AWS Lambda functions Look up VPC ID and Subnet ID Reverse an IP address Lambda-powered custom resources
  25. 25. Application-deployment-as-code
  26. 26. Infrastructure Provisioning EC2 SQS, SNS, Amazon Kinesis, etc. Databases VPC IAM Application Deployment Download Packages, Install Software, Configure Apps, Bootstrap Apps, Update Software, Restart Apps, etc. CloudFormation • Templatize • Replicate • Automate
  27. 27. Application-deployment-as-code inside a CloudFormation template Amazon Machine Images CloudFormation::Init Chef, Puppet, CodeDeploy, … OpsWorks Chef
  28. 28. Metadata AWS::CloudFormation::Init AWS::CloudFormation::Init  Declarative  Reusable  Grouping & Ordering  Debug-able  Updatable  Highly Secure  BIOT™ (Bring In Other Tools) ow.ly/DiNCm
  29. 29. AWS::CloudFormation::Init "AWS::CloudFormation::Init": { "webapp-config": { "packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {}, "commands" : {}, "services" : {} Declarative
  30. 30. AWS::CloudFormation::Init Debug-able
  31. 31. AWS::CloudFormation::Init Supports updates "packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {}, "commands" : {}, "services" : {}
  32. 32. AWS::CloudFormation::Init "install_chef" : {}, "install_wordpress" : { "commands" : { "01_get_cookbook" : {}, ..., "05_configure_node_run_list" : { "command" : "knife node run_list add -z `knife node list -z` recipe[wordpress]", "cwd" : "/var/chef/chef-repo", "env" : { "HOME" : "/var/chef" } Flexibility to bring in other tools such as AWS CodeDeploy and Chef ow.ly/DiNkz
  33. 33. AWS::CloudFormation::Init "YourInstance": { "Metadata": { "AWS::CloudFormation::Authentication": { "S3AccessCreds": { "type": "S3", "roleName": { "Ref" : "InstanceRole"}, "buckets" : ["your-bucket"] } }, "AWS::CloudFormation::Init": {} Supports role-based auth Securely download Choose auth type. IAM Role is recommended ow.ly/DqkrB
  34. 34. Use AWS::CloudFormation::Init "UserData": { "# Get the latest CloudFormation helper scripts packagen", "yum update -y aws-cfn-bootstrapn", "# Trigger CloudFormation::Init configuration n", "/opt/aws/bin/cfn-init --stack ", {"Ref": "AWS::StackId"}, " --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "n", "# Signal completionn", "/opt/aws/bin/cfn-signal –e $? --stack ", {"Ref": "AWS::StackId"}, " --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "n"
  35. 35. Use CloudWatch Logs for debugging "install_logs": { "packages" : { ... "awslogs" ... }, "services" : { ... "awslogs" ... } "files": { "/tmp/cwlogs/cfn-logs.conf": {} file = /var/log/cfn-init.log log_stream_name = {instance_id}/cfn-init.log file = /var/log/cfn-hup.log log_stream_name = {instance_id}/cfn-hup.log ow.ly/E0zO3
  36. 36. Use CloudWatch Logs for debugging ow.ly/E0zO3
  37. 37. Bake AMIs for faster booting Bake AMIs for maintaining golden images Dev/test stacks Bake AMI Staging/prod stacks Tracking
  38. 38. Using CloudFormation and OpsWorks together
  39. 39. Infrastructure Provisioning EC2 SQS, SNS, Amazon Kinesis, etc. Databases VPC IAM Application Deployment Download Packages, Install Software, Configure Apps, Bootstrap Apps, Update Software, Restart Apps, etc. CloudFormation • Templatize • Replicate • Automate OpsWorks • Built-in Application Lifecycle • Interactive Application Console OpsWorks & CloudFormation “side-by-side”
  40. 40. OpsWorks • Built-in Application Lifecycle • Interactive Application Console Infrastructure Provisioning EC2 SQS, SNS, Amazon Kinesis, etc. Databases VPC IAM Application Deployment Download Packages, Install Software, Configure Apps, Bootstrap Apps, Update Software, Restart Apps, etc. CloudFormation • Templatize • Replicate • Automate OpsWorks “inside” CloudFormation
  41. 41. Infrastructure-as-code in a CI/CD pipeline
  42. 42. CloudFormation in a CI/CD pipeline AWS CloudFormationIssue Tracker App Developers DevOps Engineers, Infrastructure Developers, Systems Engineers Dev Env Code Repo App Pkgs, CloudFormation Templates, Etc. CI Server Test Staging ProdCode Review "Infra-as-Code" App Code & Templates
  43. 43. Templatize existing resources
  44. 44. CloudFormer: Templatize existing resources 1. Launch a CloudFormer application stack 2. Walkthrough the CloudFormer UI & select resources to templatize 4. Customize Example: parameterize resource properties 5. Create a new stack
  45. 45. Practitioners of infrastructure-as-code • Developers/DevOps teams value CloudFormation for its ability to treat infrastructure as code, allowing them to apply software engineering principles, such as SOA, revision control, code reviews, integration testing to infrastructure. • IT Admins and MSPs value CloudFormation as a platform to enable standardization, managed consumption, and role-specialization. • ISVs value CloudFormation for its ability to support scaling out of multi-tenant SaaS products by quickly replicating or updating stacks. ISVs also value CloudFormation as a way to package and deploy their software in their customer accounts on AWS.
  46. 46. SAN FRANCISCO
  47. 47. SAN FRANCISCO ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

×