Manage AWS infrastructure as code using Terraform
•Download as PPTX, PDF•
15 likes•9,176 views
Update (June 2017): There is more recent version of similar talk by me -https://www.slideshare.net/AntonBabenko/continuously-delivering-infrastructure-using-terraform-and-packer-training-material State of AWS infrastructure management tools. Getting started with Terraform. Feature comparison with AWS Cloudformation. Multiple demos of how Terraform can supplement 100% of AWS features. Continuous Integration & Continuous Deployment greenfield demo. All code demonstrated during this presentation is open-source.
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
More from Anton Babenko
More from Anton Babenko (20)
Recently uploaded
Recently uploaded (20)
Manage AWS infrastructure as code using Terraform
- 1. MANAGE AWS INFRASTRUCTURE AS CODE USING TERRAFORM
- 2. Hello! I AM ANTON BABENKO I enjoy AWS, DevOps, solutions architecture & web-development github.com/antonbabenko linkedin.com/in/antonbabenko
- 4. 0. AGENDA
- 5. 0. AGENDA 1.State of things 2.Terraform 101 ●Getting started with Terraform 3.Terraform 201 ●Advanced concepts in Terraform ● Demos 4.CI/CD with Terraform ●Demo
- 6. 1. STATE OF THINGS AWS + Infrastructure as code
- 7. AVAILABLE TOOLS ○AWS CloudFormation ○Puppet, Chef, Ansible, Salt… ○AWS API, libraries (Boto, Fog) ○Terraform by HashiCorp
- 8. AVAILABLE TOOLS ○AWS CloudFormation ●http://www.slideshare.net/AntonBabenko/mana ging-aws-infrastructure-using-cloudformation ○Puppet, Chef, Ansible, Salt… ○AWS API, libraries (Boto, Fog) ○Terraform by HashiCorp
- 10. ““HashiCorp is Atlassian for DevOps.” Someone at DevOps conference
- 11. TERRAFORM Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. www.terraform.io
- 12. Latest version: 0.6.8 (released 2.12.2015) Open-source, written in Golang. Very active development: ○CHANGELOG.md (ca. 1 release per month) ○GitHub Issues (ca. 5-15 issues resolving daily) ○Growing community (IRC, Mailing list, Stack Overflow) TERRAFORM FACTS
- 14. CloudFormation Terraform Configuration format JSON HCL/JSON State management No Yes Execution control No Yes! Logical comparisons Yes Limited Supports iterations No Yes Manage already created resources No Yes (hard) Providers supported Only AWS 20+ (incl. AWS, GCE, Azure)
- 15. CloudFormation Terraform AWS resource types 121 103 Resource properties and operations completeness 90% Work in progress Handle failures * Optional rollback Fix it & retry Contribute? No Yes! GH issue #28 AWS SPECIFICS
- 17. TERRAFORM GRAPH
- 19. TERRAFORM COMMANDS $ terraform usage: terraform [--version] [--help] <command> [<args>] Available commands are: apply Builds or changes infrastructure destroy Destroy Terraform-managed infrastructure get Download and install modules for the configuration graph Create a visual graph of Terraform resources init Initializes Terraform configuration from a module output Read an output from a state file plan Generate and show an execution plan refresh Update local state file against real resources remote Configure remote state storage show Inspect Terraform state or plan taint Manually mark a resource for recreation version Prints the Terraform version
- 20. TERRAFORM COMMANDS $ terraform usage: terraform [--version] [--help] <command> [<args>] Available commands are: apply Builds or changes infrastructure destroy Destroy Terraform-managed infrastructure get Download and install modules for the configuration graph Create a visual graph of Terraform resources init Initializes Terraform configuration from a module output Read an output from a state file plan Generate and show an execution plan refresh Update local state file against real resources remote Configure remote state storage show Inspect Terraform state or plan taint Manually mark a resource for recreation version Prints the Terraform version
- 21. TERRAFORM REMOTE Configures remote state storage with Terraform *.tf AWS infrastructure terraform.tfstate S3, Atlas, Consul, etcd, HTTP
- 22. TERRAFORM PLAN Generates an execution plan for Terraform *.tf AWS infrastructure terraform.tfstate
- 23. TERRAFORM APPLY Builds or changes infrastructure according to Terraform configuration files *.tf AWS infrastructure terraform.tfstate
- 24. TERRAFORM REFRESH Update the state file of your infrastructure with metadata that matches the physical resources they are tracking *.tf AWS infrastructure terraform.tfstate
- 25. TERRAFORM DESTROY Destroy Terraform-managed infrastructure *.tf AWS infrastructure terraform.tfstate
- 26. TERRAFORM TAINT Manually mark a resource as tainted, forcing a destroy and recreate on the next plan/apply *.tf AWS infrastructure terraform.tfstate
- 27. TERRAFORM GRAPH Draw nice visual dependency graph of Terraform resources according to configuration files $ terraform graph -draw-cycles | dot -Tpng -o graph.png
- 28. TERRAFORM etc $ terraform --help
- 30. TERRAFORM - WARM-UP Keep Terraform shared state files on Amazon S3 and enable bucket versioning: aws s3api create-bucket --bucket my-terraform-states --acl authenticated-read --create-bucket-configuration LocationConstraint=eu-west-1 aws s3api put-bucket-versioning --bucket my-terraform-states --versioning-configuration Status=Enabled
- 31. TERRAFORM WARM-UP QUESTIONS? ○ How many environments? ○ How many AWS regions? ○ How many DevOps will be involved? ● It is not so important.
- 33. TERRAFORM - DEMO 1 There was nothing in AWS account, so let’s create new VPC, subnets and deploy heavy web-app Complete code and slides: http://github.com/antonbabenko/terraform-aws-devops
- 35. PROJECTS /HEAVY
- 36. TERRAFORM WAYS TO STRUCTURE CONFIGS ○ One-in-all: ● Good for partial and disposable setups ○ Separate by environments: ● One project = one environment ● Each environment may contain different modules ● Read more ○ Layered projects (shared infrastructure): ● Separate responsibilities (eg, “read-only” shared infrastructure for app developers) ● Easy to extend layers independently (using modules) ● Small = fast
- 37. TERRAFORM HOW TO STRUCTURE CONFIGS ○ Keep 1 Terraform state for each combination of project per environment (in 1 AWS region) ● eg, one-in-all = 1 Terraform state per environment ○ More environments = more combinations ○ Global AWS resources (eg, S3 buckets, EIP, IAM, Route53 zones, CodeDeploy applications): ● Keep them in Terraform states without separation by environments ○ Use environment in resource tags ○ Use modules
- 38. TERRAFORM - MODULES “Modules in Terraform are self-contained packages of Terraform configurations that are managed as a group.” Support versioning: Community modules - https://github.com/terraform-community-modules/ module "vpc" { source = "github.com/terraform-community-modules/tf_aws_vpc_only?ref=v1.0.0" cidr = "${var.vpc_cidr}" }
- 39. Let’s import very important S3 bucket into Terraform configs, so that we can manage that resource using Terraform. Explanation: Import of already created resources to Terraform state is not supported by Terraform natively, but it is possible. TERRAFORM - DEMO 2 New resource!
- 40. Our application ELB should contain custom security policy with specific set of SSL ciphers. Explanation: “SSL ciphers” is not implemented as aws_elb resource type property. TERRAFORM - DEMO 3 Updated!
- 41. Our Heavy application team needs Redshift cluster available, so that developers can query it. Explanation: Redshift is not among supported resource types by Terraform, but it is supported by AWS CloudFormation. TERRAFORM - DEMO 4
- 42. Terraform can create and manage AWS infrastructure which is: ○ New (has no resources) ○ Contains already existing resources Terraform can: ○ Supplement resource types properties currently not supported natively ○ Supplement resource types currently not supported natively. TERRAFORM - DEMO SUMMARY
- 43. 4. TERRAFORM Demo: Continuous Integration & Continuous Deployment (beta)
- 44. ○ Using feature branches ○ Lock master branch ○ New push into feature branch: ● terraform production init + plan ○ Feature merged into master branch: ● terraform production init + plan + apply ○ Too risky? Combine: ● terraform plan -out=plan_${GIT_SHA}.out ● terraform apply plan_${GIT_SHA}.out ○ Terraform plugin for Jenkins, if you ask TERRAFORM - CI/CD
- 45. Responsibly deploy applications and make changes to infrastructure with Atlas by HashiCorp atlas.hashicorp.com
- 46. SUMMARY Terraform is cool, isn’t it ?
- 47. I REALLY LIKE QUESTIONS
- 48. THANK YOU! All code from this talk: https://github.com/antonbabenko/terraform-aws-devops
Editor's Notes
- Organizer of AWS user group norway AWS certified solution architect and sysops Doing web-development, devops for the last 10+ years. Doing AWS for the last 5 years. open-source, team leadership windsurfing, sailing, paragliding
- Who is using AWS API directly or using libraries (like Troposphere written in Python) ?
- Who is using AWS API directly or using libraries (like Troposphere written in Python) ?
- State management - TF has local tfstate file describing metadata of created resources Execution control = well controlled. Plan => output file or limit by targets => apply with confidence. CF can only validate syntax. Logical comparisons = more, less, equal value. In TF you can use “count=0” or “count=1” resource parameter instead of boolean true/false to control resource creation. Manage already created resources like EIP, S3 buckets, VPC is not possible without deleting them first.
- Some resource properties (for example, ec2 keypair) can be created using AWS API, but not available in CloudFormation. Terraform uses AWS API, so you can get/update missing properties in many cases. update_rollback_failed = contact customer service --- Handle failures => Partial State and Error Handling If an error happens at any stage in the lifecycle of a resource, Terraform stores a partial state of the resource. This behavior is critical for Terraform to ensure that you don't end up with any zombie resources: resources that were created by Terraform but no longer managed by Terraform due to a loss of state.
- Atlas, Consul, etcd, S3 or HTTP Terraform will automatically update remote state file once where are any changes in it. There are also ways to pull and push to remote state file.
- Refresh state locally and generate execution plan based on tf configs
- Apply the changes required to reach the desired state of the configuration. Or the pre-determined set of actions generated by a terraform plan execution plan.
- Refresh state locally and generate execution plan based on tf configs
- Destroy resources managed by Terraform. Can be previewed by running “$ terraform plan -destroy”. Also can be limited by using “-target” and it will delete resources which were dependent on targets.
- This will not modify your infrastructure. This command changes your state to mark a resource as tainted so that during the next plan or apply, that resource will be destroyed and recreated.
- This will not modify your infrastructure. This command changes your state to mark a resource as tainted so that during the next plan or apply, that resource will be destroyed and recreated.