DevOps on AWS: Deep Dive on Infrastructure as Code

3,479 views

Published on

While many organizations have started to automate their software development processes, many still engineer their infrastructure largely by hand. Treating your infrastructure just like any other piece of code creates a “programmable infrastructure” that allows you to take full advantage of the scalability and reliability of the AWS cloud. In this session, you will learn how you can provision, configure, and manage your infrastructure using code and treat it just like your software code. We will discuss the AWS services that enable these practices (AWS CloudFormation, AWS OpsWorks, and AWS CodeDeploy) and that allow you to control everything from Amazon VPCs and AWS Identity and Access Management to the configuration of individual applications on a single host. We’ll cover best practices for ongoing management, how to best update resources, and which tools are best suited for AWS resource management and host-based configuration management.

Published in: Engineering

DevOps on AWS: Deep Dive on Infrastructure as Code

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Alex Corley – Solutions Architect Public Sector - State and Local Government June 2016 Infrastructure as Code Best Practices on AWS
  2. 2. Learning objectives • Understand Infrastructure as Code • Understand the AWS services that help you manage your infrastructure as code • Discover best practices for managing your AWS infrastructure, host configuration, and applications
  3. 3. Background Moving to the cloud and AWS allows you to provision and manage infrastructure in new ways: • Infrastructure can be provisioned in seconds • Scale can be achieved without complicated capacity planning • APIs let you interact with infrastructure using languages typically used in applications
  4. 4. What is Infrastructure as Code? A practice in which traditional infrastructure management techniques are supplemented by or replaced with code- based tools and software development techniques
  5. 5. Infrastructure as Code workflow Code Version Control Code Review Integrate Deploy
  6. 6. Infrastructure as Code workflow Code Version Control Code Review Integrate Deploy Text Editor Git/SVN/ Perforce Review Tools Syntax Validation Tools AWS Services
  7. 7. Infrastructure as Code workflow “It’s all software” Code Version Control Code Review Integrate Deploy Text Editor Git/SVN/ Perforce Review Tools Syntax Validation Tools AWS Services
  8. 8. Application Configuration AWS Resources Infrastructure as Code workflow Operating System and Host Configuration
  9. 9. AWS Resources Operating System and Host Configuration Application Configuration
  10. 10. AWS Resources Operating System and Host Configuration Application Configuration Infrastructure Resource Management
  11. 11. AWS Resources Operating System and Host Configuration Application Configuration Infrastructure Resource Management Host Configuration Management
  12. 12. AWS Resources Operating System and Host Configuration Application Configuration Infrastructure Resource Management Host Configuration Management Application Deployment
  13. 13. AWS Resources Operating System and Host Configuration Application Configuration AWS CloudFormation AWS OpsWorks AWS CodeDeploy
  14. 14. AWS Resources Operating System and Host Configuration Application Configuration AWS CloudFormation AWS OpsWorks AWS CodeDeploy Amazon Virtual Private Cloud (Amazon VPC) Amazon Elastic Compute Cloud (Amazon EC2) AWS Identity and Access Management (IAM) Amazon Relational Database Service (Amazon RDS) Amazon Simple Storage Service (Amazon S3) AWS CodePipeline … Microsoft Windows Registry Linux networking OpenSSH LDAP Active Directory domain registration Centralized logging System metrics Deployment agents Host monitoring … Application dependencies Application configuration Service registration Management scripts Database credentials …
  15. 15. allOfThis == $Code
  16. 16. AWS CloudFormation • Create templates that describe and model AWS infrastructure • CloudFormation then provisions AWS resources based on dependency needs • Perform version control on, replicate, and update the templates like app code • Integrates with development, CI/CD, management tools • No additional charge to use
  17. 17. Benefits Templated resource provisioning Infrastructure as code Declarative and flexible Easy to use
  18. 18. CloudFormation concepts and technology JSON formatted file Parameter definition Resource creation Configuration actions Framework Stack creation Stack updates Error detection and rollback Configured AWS resources Comprehensive service support Service event aware Customizable Template CloudFormation Stack
  19. 19. Anatomy of a CloudFormation template: JSON Plain text Perfect for version control Can be validated
  20. 20. Anatomy of a CloudFormation template: JSON { "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if you create a stack from this template.", "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" }, "Environment": { "Type" : "String", "Default" : ”Dev", "AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"], "Description" : "Environment that the instances will run in.” } }, "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-7f418316" }, "us-west-2" : { "AMI" : "ami-16fd7026" } } }, "Conditions" : { ”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]}, }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]}, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, "Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "PublicDNS" : { "Description" : "Public DNSName of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] } } } }
  21. 21. Anatomy of a CloudFormation template: JSON Parameters "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" }, "Environment": { "Type" : "String", "Default" : ”Dev", "AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"], "Description" : "Environment that the instances will run in.” } }, Mappings "Mappings" : { "RegionMap" : { "us-east-1" : { "AMI" : "ami-7f418316" }, "us-west-2" : { "AMI" : "ami-16fd7026" } } }, Conditionals "Conditions" : { ”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]}, }, Resources "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]}, "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : "80" } } } }, Outputs Outputs" : { "InstanceId" : { "Description" : "InstanceId of the newly created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } }, "PublicDNS" : { "Description" : "Public DNSName of the newly created EC2 instance", "Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] } } } } Headers { "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if you create a stack from this template.",
  22. 22. Description of what your stack does, contains, and so on Provision time values that add structured flexibility and customization Predefined conditional case statements Conditional values set through evaluations of passed references AWS resource definitions Resulting attributes of stack resource creation Headers Parameters Mappings Conditionals Resources Outputs Template components
  23. 23. Template example Templates (in action): "ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", { "Ref" : "AWS::Region" }, {"Fn::FindInMap": ["AWSInstanceType2Virt", { "Ref" : "myInstanceType" }, "Virt"]} ]},
  24. 24. Template example Templates (in action): "ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", { "Ref" : "AWS::Region" }, {"Fn::FindInMap": ["AWSInstanceType2Virt", { "Ref" : "myInstanceType" }, "Virt"]} ]}, “AWSRegionVirt2AMI” map
  25. 25. Template example Templates (in action): "ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", { "Ref" : "AWS::Region" }, “AWSRegionVirt2AMI” map {"Fn::FindInMap": ["AWSInstanceType2Virt", { "Ref" : "myInstanceType" }, "Virt"]} ]}, “AWSInstanceType2Virt” map
  26. 26. Template example Templates (in action): "ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", { "Ref" : "AWS::Region" }, “AWSRegionVirt2AMI” map {"Fn::FindInMap": ["AWSInstanceType2Virt", “AWSInstanceType2Virt” map { "Ref" : "myInstanceType" }, "Virt"]} ]}, “myInstanceType” parameter
  27. 27. Template example Templates (in action): "ImageId" : { "Fn::FindInMap" : [ "AWSRegionVirt2AMI", “AWSRegionVirt2AMI” map {"Fn::FindInMap": ["AWSInstanceType2Virt", “AWSInstanceType2Virt” map { "Ref" : "myInstanceType" }, "Virt"]} ]}, “myInstanceType” parameter { "Ref" : "AWS::Region" }, AWS::Region pseudo parameter
  28. 28. Template example "myInstanceType" : { "Type" : "String", "Default" : "t2.large", "AllowedValues" : ["t2.micro", "t2.small", "t2.medium", "t2.large"], "Description" : "Instance type for instances created, must be in the t2 family." } "AWSInstanceType2Virt": { "t2.micro": {"Virt": "HVM"}, "t2.small": {"Virt": "HVM"}, "t2.medium": {"Virt": "HVM"}, "t2.large": {"Virt": "HVM"}, } "AWSRegionVirt2AMI": { "us-east-1": { "PVM": "ami-50842d38", "HVM": "ami-08842d60" }, "us-west-2": { "PVM": "ami-af86c69f", "HVM": "ami-8786c6b7" }, "us-west-1": { "PVM": "ami-c7a8a182", "HVM": "ami-cfa8a18a" } } Parameters: Mappings: Mappings:
  29. 29. Bootstrapping applications and handling updates "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyName" }, "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]}, "UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[ "#!/bin/bash -ex","n", "yum -y install gcc-c++ make","n", "yum -y install mysql-devel sqlite-devel","n", "yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","n", "gem install --no-ri --no-rdoc rails","n", "gem install --no-ri --no-rdoc mysql","n", "gem install --no-ri --no-rdoc sqlite3","n", "rails new myapp","n", "cd myapp","n", "rails server -d","n"]]}} } } Option 1: Use Amazon EC2 UserData, which is available as a property of AWS::EC2::Instance resources
  30. 30. cfn-init cfn-hup Bootstrapping applications and handling updates Option 2: CloudFormation provides helper scripts for deployment within your EC2 instances Metadata key— AWS::CloudFormation::Init The cfn-init helper script reads this metadata key and installs the packages listed in this key (for example, httpd, mysql, and php); cfn- init also retrieves and expands files listed as sources EC2 CloudFormation cfn-signal cfn-get- metadata
  31. 31. Bootstrapping example "Metadata": { "AWS::CloudFormation::Init" : { "config" : { "packages" : { }, "sources" : { }, "commands" : { }, "files" : { }, "services" : { }, "users" : { }, "groups" : { } } } Use AWS::CloudFormation::Init with cfn-init to help bootstrap instances:
  32. 32. Bootstrapping example “WebAppHost" : { "Type" : "AWS::EC2::Instance", "Metadata" : { "AWS:CloudFormation::Init" : { "config" : { "packages" : { "yum" : { "gcc" : [], "gcc-c++" : [], "make" : [], "automake" : [], Install packages with the native package management tool:
  33. 33. Manage a wide range of AWS services and resources • Amazon EC2 • Amazon EC2 Container Service • Amazon EC2 Container Registry • Amazon EC2 Simple Systems Manager • AWS Lambda (including event sources) • AWS Elastic Beanstalk • Auto Scaling (including Spot fleet) • Amazon VPC and Managed NAT Gateway • Elastic Load Balancing • Amazon Route 53 • Amazon CloudFront • AWS WAF • Amazon S3 • Amazon RDS • Amazon Redshift • Amazon DynamoDB • Amazon ElastiCache • Amazon RDS (including Amazon Aurora) • Amazon Elastic MapReduce • Amazon Elasticsearch Service • AWS Data Pipeline • AWS Identity and Access Management (including managed policies) • AWS Directory Service (Amazon Simple AD) / Microsoft Active Directory • Amazon Kinesis • Amazon SNS • Amazon SQS • AWS CloudTrail • Amazon CloudWatch • AWS Config • AWS Key Management Service • AWS OpsWorks • AWS CodeDeploy • AWS CodePipeline • Amazon WorkSpaces • Amazon GameLift AWS resource support is always growing. See the most up-to-date list here.
  34. 34. Template file defining stack • The entire infrastructure can be represented in a CloudFormation template Many stacks and environments from one template
  35. 35. Template file defining stack • The entire infrastructure can be represented in a CloudFormation template • Use the version control system of your choice to store and track changes to this template Git Perforce SVN … Many stacks and environments from one template
  36. 36. Template file defining stack • The entire infrastructure can be represented in a CloudFormation template • Use the version control system of your choice to store and track changes to this template • Build out multiple environments, such as for development, test, production, and even disaster recovery, using the same template Git Perforce SVN … Dev Test Prod Many stacks and environments from one template
  37. 37. Infrastructure as Code with CloudFormation Versioning You track changes within your code Do it with your infrastructure: • What is changing? • Who made that change? • When was it made? • Why was it made?(Is it tied to a ticket or bug or project system?)
  38. 38. Testing your CloudFormation templates Testing your template: • Validate by using API or AWS Command Line Interface (CLI) • $ aws cloudformation validate-template—confirm CloudFormation syntax • Use something like JSONLint (http://jsonlint.com/) to find JSON issues like missing commas or brackets • Throw this into your testing and/or continuous integration pipelines
  39. 39. Visualizing your CloudFormation templates • AWS CloudFormation Designer • Visualize template resources • Modify template with drag and drop gestures • Customize sample templates
  40. 40. Deploying your CloudFormation templates Deploy and update by using console, API, or CLI OR • aws cloudformation create-stack --stack-name myteststack --template-body file:////home//local//test//sampletemplate.json -- parameters ParameterKey=string,ParameterValue=string
  41. 41. But what do we do once your resources are provisioned and running?
  42. 42. Your infrastructure needs ongoing management • Updates or patches? • New software? • New configurations? • New code deployments? • Pool-specific changes? • Environment-specific changes? • Run commands across all hosts? • Be on top of all running resources?
  43. 43. Ongoing management requires proper tooling Some common challenges: • Changing a vhost configuration on every web server across multiple environments (development, staging, production) • Installing a package on certain hosts to test out newer versions • Changing the LDAP configuration on every running Amazon EC2 Linux host when the hosts exist across 25 different CloudFormation templates
  44. 44. We need a tool to interact with each host that we manage and that makes it easier to configure these hosts
  45. 45. AWS OpsWorks • Configuration management service for automating operational tasks using Chef • Model, control, and automate applications of nearly any scale and complexity • Manage Linux and Microsoft Windows environments • Supports both AWS and on- premises servers • Launched in 2013
  46. 46. AWS OpsWorks concepts A stack represents the cloud infrastructure and applications that you want to manage together A layer defines how to set up and configure a set of instances and related resources You decide how to scale: manually, with 24/7 instances, or automatically, with load-based or time-based instances Then deploy your app to specific instances and customize the deployment with Chef recipes
  47. 47. AWS OpsWorks concepts: instance lifecycle Set up Configure Deploy Undeploy Shut down An agent on each instance understands a set of commands that are triggered by OpsWorks. The agent when triggered runs Chef.
  48. 48. OpsWorks agent communication 1. The EC2 instance connects with the OpsWorks service to send keepalive/ heartbeat and receive lifecycle events 2. OpsWorks sends a lifecycle event with a pointer to the configuration JSON (metadata, recipes) in an S3 bucket 3. The agent downloads configuration JSON 4. The agent pulls cookbooks and other build assets from your repository 5. The agent executes the recipe 6. The agent uploads the Chef log 7. The agent reports Chef run status EC2 instance OpsWorks service “Deploy App” Your repository, for example GitHub       
  49. 49. How OpsWorks bootstraps EC2 instances The EC2 instance is started by using an IAM role • UserData passed with instance private key, OpsWorks public key • The instance downloads and installs the OpsWorks agent The agent connects to the instance service, gets run info • Authenticates the instance using the instance’s IAM role • Picks up configuration JSON from the OpsWorks instance queue • Decrypts and verifies the message, runs Chef recipes • Uploads Chef log, returns Chef run status The agent then polls the instance service for more messages
  50. 50. AWS OpsWorks + Chef OpsWorks uses Chef to configure the software on the instance OpsWorks provides many Chef Server functions to users • Associates cookbooks with instances • Dynamic metadata describes each registered node in the infrastructure Supports "push" command and control client runs Supports community cookbooks
  51. 51. Working with Chef and OpsWorks Similar to CloudFormation templates and application code: • Mixture of JSON and a Ruby DSL • Tools exist to do linting and syntax checking • Versioning • Built in cookbook versioning • Some manual/processes scripted abilities • But still can use source control for versioning • Use with continuous integration systems like CloudFormation templates and the rest of your code
  52. 52. Working with Chef and OpsWorks Basics: • Nodes • Roles • Cookbooks • Recipes • Attributes • Data bags • Environments
  53. 53. Host configuration management with Chef package "ntp" do action :install end service "ntpd" do supports :status => true, :restart => true, :reload => true action [ :enable, :start ] end cookbook_file "/etc/ntp.conf" do source "ntp.conf" owner "root" group "root" mode 00644 # Restart ntp.conf if /etc/ntp.conf changes notifies :restart, resources(:service => "ntpd") End group "ganglia" do gid 499 end user "ganglia" do home "/var/lib/ganglia" shell "/sbin/nologin" uid 499 gid "ganglia" end directory "/etc/ganglia" do action :create end Examples:
  54. 54. Host configuration management with Chef template "/etc/ganglia/gmond.conf" do source "gmond.conf.erb" owner "root" group "root" mode 00644 notifies :restart, resources(:service => "gmond") variables( :gmetad_name => node[:ganglia][:gmetad_name], :cluster_name => node[:ganglia][:cluster_name] ) end cron "all-gmetrics" do command "for FILE in `ls /opt/bin/gmetric-*`; do command $FILE; done >/dev/null 2>&1" end Examples:
  55. 55. Custom JSON { "opsworks": { "data_bags": { "myapp": { "mysql": { "username": "default-user", "password": "default-pass" } } } } } Host configuration management with Chef Recipe mything = data_bag_item("myapp", "mysql") Chef::Log.info("username: #{mything['username']}")
  56. 56. AWS OpsWorks
  57. 57. Deploying applications
  58. 58. Automates code deployments to any instance Handles the complexity of updating your applications Use it to avoid downtime during application deployment Deploy to Amazon EC2 or on-premise servers, in any language and on any operating system Integrates with third-party tools and AWS services AWS CodeDeploy
  59. 59. AWS CodeDeploy concepts Application Revision #1 Revision #2 Revision #3 What to deploy? Revision #1 How to deploy? Instance Instance Instance Deployment group Auto Scaling group Where to deploy?
  60. 60. How it works: package app with appspec.yml version: 0.0 os: linux files: - source: / destination: /var/www/html permissions: - object: /var/www/html pattern: “*.html” owner: root group: root mode: 755 hooks: ApplicationStop: - location: scripts/deregister_from_elb.sh BeforeInstall: - location: scripts/install_dependencies.sh ApplicationStart: - location: scripts/start_httpd.sh ValidateService: - location: scripts/test_site.sh - location: scripts/register_with_elb.sh
  61. 61. How it works: package app with appspec.yml version: 0.0 os: linux files: - source: / destination: /var/www/html • Send application files to one directory and configuration files to another • Set specific permissions on specific directories and files • Remove or add instance to Elastic Load Balancing • Install dependency packages • Start Apache • Confirm successful deploy • More! permissions: - object: /var/www/html pattern: “*.html” owner: root group: root mode: 755 hooks: ApplicationStop: - location: scripts/deregister_from_elb.sh BeforeInstall: - location: scripts/install_dependencies.sh ApplicationStart: - location: scripts/start_httpd.sh ValidateService: - location: scripts/test_site.sh - location: scripts/register_with_elb.sh
  62. 62. How it works: Specify targets Group instances by: • Auto Scaling group • Amazon EC2 tag • On-premises tag Development deployment group AgentAgent Agent Production deployment group AgentAgent Agent AgentAgent Agent
  63. 63. How it works: Deploy • AWS CLI and SDKs • AWS Management Console • AWS CodePipeline and CI/CD partners • Amazon S3, GitHub aws deploy create-deployment --application-name MyApp --deployment-group-name TargetGroup --s3-location bucket=MyBucket,key=MyApp.zip
  64. 64. v2 v1 v1 v1 v1 v1 v1 v1 v2 v2 v1 v1 v1 v1 v1 v1 v2 v2 v2 v2 v1 v1 v1 v1 v2 v2 v2 v2 v2 v2 v2 v2 One at a time Minimum healthy hosts = 99% [Custom] Minimum healthy hosts = 75% Half at a time Minimum healthy hosts = 50% All at once Minimum healthy hosts = 0 Choose your deployment configuration
  65. 65. Summary
  66. 66. Summary • Create, update, and manage AWS resources and their configuration and properties with CloudFormation • You can configure OpsWorks and CodeDeploy by using CloudFormation • Use OpsWorks for ongoing tweaks to software and configuration of host-based applications and the operating system • You can configure and deploy CodeDeploy’s agent with OpsWorks • Use CodeDeploy to deploy your applications and their configurations
  67. 67. Best practices • Your CloudFormation templates and Chef cookbooks should go in separate repositories • Include the appspec.yml file and related scripts in your application’s code repositories • Every commit should cause an execution of your continuous delivery pipeline to lint, validate, and/or test • Use each related service’s CLI, console, and APIs to update or deploy as necessary
  68. 68. AWS Resources Operating System and Host Configuration Application Configuration AWS CloudFormation AWS OpsWorks AWS CodeDeploy Amazon Virtual Private Cloud (Amazon VPC) Amazon Elastic Compute Cloud (Amazon EC2) AWS Identity and Access Management (IAM) Amazon Relational Database Service (Amazon RDS) Amazon Simple Storage Service (Amazon S3) AWS CodePipeline … Microsoft Windows Registry Linux networking OpenSSH LDAP Active Directory domain registration Centralized logging System metrics Deployment agents Host monitoring … Application dependencies Application configuration Service registration Management scripts Database credentials …
  69. 69. allOfThis == $Code
  70. 70. Learn more • AWS CloudFormation • https://aws.amazon.com/cloudformation/ • https://aws.amazon.com/documentation/cloudformation/ • https://aws.amazon.com/cloudformation/aws-cloudformation-templates/ • AWS OpsWorks • https://aws.amazon.com/opsworks/ • https://aws.amazon.com/documentation/opsworks/ • https://github.com/aws/opsworks-cookbooks • AWS CodeDeploy • https://aws.amazon.com/codedeploy/ • https://aws.amazon.com/documentation/codedeploy/ • https://github.com/awslabs/aws-codedeploy-samples
  71. 71. Thank you!

×