Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Terraform 101

227 views

Published on

This presentation is for you if your just getting started with terraform or using it on a daily basis.

Published in: Technology
  • Be the first to comment

Terraform 101

  1. 1. FullStack Developers Israel TERRAFORM TRUE INFRASTRUCTURE AS CODE WITH
  2. 2. WHO WE ARE ? ▸ Tikal helps ISV’s in Israel & abroad in their technological challenges. ▸ Our Engineers are Fullstack Developers with expertise in Android, DevOps, Java, JS, Python, ML ▸ We are passionate about technology and specialise in OpenSource technologies. ▸ Our Tech and Group leaders help establish & enhance existing software teams with innovative & creative thinking. https://www.meetup.com/full-stack-developer-il/
  3. 3. FullStack Developers Israel SELF INTRODUCTION ▸ My open thinking and open techniques ideology is driven by Open Source technologies and the collaborative manner defining my M.O. ▸ My solution driven approach is strongly based on hands-on and deep understanding of Operating Systems, Applications stacks and Software languages, Networking, Cloud in general and today more an more Cloud Native solutions. HAGGAI PHILIP ZAGURY - DEVOPS ARCHITECT AND GROUP TECH LEAD
  4. 4. FullStack Developers Israel DevOps
  5. 5. FullStack Developers Israel DISTRIBUTED SYSTEMS || 12 FACTOR APPS ▸ Micorservices ! ▸ Each micro service has it’s own Data Srouce ▸ Queues ▸ Load Balancers ▸ Auto Scaling ▸ Cloud / Cloud Native / Multi-Cloud TERRAFORM | GETTING STARTED
  6. 6. FullStack Developers Israel INFRASTRUCTURE & AUTOMATION EVOLUTION ▸ Automation is the #1 enables for: ▸ Scalability - modularity ▸ Reproducibility - re-spin env’s on the fly ▸ Reliability - continuous practices TERRAFORM | GETTING STARTED
  7. 7. FullStack Developers Israel TERRAFORM TRUE INFRASTRUCTURE AS CODE WITH BECAUSE K8S IS THE LEAST OF OUR WORRIES …
  8. 8. FullStack Developers Israel TERRAFORM | GETTING STARTED INFRASTRUCTURE AS CODE Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine- readable definition files, rather than physical hardware configuration or interactive configuration tools.[1] The IT infrastructure meant by this comprises both physical equipment such as bare-metal servers as well as virtual machines and associated configuration resources. The definitions may be in a version control system. It can use either scripts or declarative definitions, rather than manual processes, but the term is more often used to promote declarative approaches.
  9. 9. FullStack Developers Israel TERRAFORM | GETTING STARTED IAC - A SOFTWARE ENGINEERING APPROACH TO OPERATIONS ▸ Automation, Automation, Automation … ▸ Quality management (“test driven infrastructure”) ▸ D.R.Y -> Don’t Repeat Yourself ▸ Collaborate, Share, Communicate ▸ Messure / Audit / Asses
  10. 10. FullStack Developers Israel TERRAFORM | GETTING STARTED POLYGLOT PROVISIONING … ▸ Azure Resource Manager ▸ Google Cloud Deployment Manager  ▸ AWS CloudFormation ▸ Ansible -> CloudFormation ▸ Chef -> Heat templates ▸ …
  11. 11. FullStack Developers Israel TERRAFORM | GETTING STARTED A TOOL FOR INFRASTRUCTURE PROVISIONING
  12. 12. FullStack Developers Israel TERRAFORM | GETTING STARTED PROVIDERS - MANY PROVIDERS
  13. 13. FullStack Developers Israel TERRAFORM | GETTING STARTED PROVIDERS - AN EXAMPLE 1 # Configure the GitHub Provider 2 provider "github" { 3 token = "MyMicrosoftToken" 4 organization = "tikal.io" 5 } 6 7 # Add a user to the organization 8 resource "github_membership" "user" { 9 username = “terry.forman" 10 role = "member" 11 }
  14. 14. FullStack Developers Israel TERRAFORM | GETTING STARTED PROVIDER - ACCESS TOKEN ‣ Create an access token in Github ‣ Configure the Github Provider ‣ Start managing groups, users, tokens etc …
  15. 15. FullStack Developers Israel TERRAFORM | GETTING STARTED RESOURCES 1 # Configure the PagerDuty provider 2 provider "pagerduty" { 3 token = “MyPgToken" 4 } 5 6 # Create a PagerDuty team 7 resource "pagerduty_team" "engineering" { 8 name = "Engineering" 9 description = "All engineering" 10 } 11 12 # Create a PagerDuty user and add it to a team 13 resource "pagerduty_user" “tikal_io” { 14 name = “Haggai Philip Zagury" 15 email = “hagzag@tikalk.com” 16 teams = ["${pagerduty_team.engineering.id}"] 17 }
  16. 16. FullStack Developers Israel TERRAFORM | GETTING STARTED VARIABLES - DECLARE 1 variable "profile" { 2 type = "string" 3 description = “see ~/.aws/credentials file for details" 4 default = "my_company" 5 } 6 7 variable "region" { 8 type = "string" 9 description = "your default aws region" 10 default = "eu-west-1" 11 } 12 13 variable "general_tags" { 14 type = "map" 15 16 default = { 17 Name = "example.com" 18 Environment = "dev" 19 KubernetesCluster = "dev.example.com" 20 } 21 } Best Practice to declare types Best Practice to be descriptive
  17. 17. FullStack Developers Israel TERRAFORM | GETTING STARTED VARIABLES - USE Use ${var.var_name} 1 resource "aws_route53_zone" "dev" { 2 name = "${var.env}.${var.domain_name}" 3 4 tags { 5 Environment = "dev" 6 } 7 }
  18. 18. FullStack Developers Israel TERRAFORM | GETTING STARTED OUTPUTS terraform output “output_key”1 output "availability_zones" { 2 value = "${var.azs}" 3 } More on this later on when we discuss modules and states ! TERRAFORM 1 TERRAFORM 2 availability_zones
  19. 19. FullStack Developers Israel TERRAFORM | GETTING STARTED TERRAFORM TEMPLATE (HCL) ‣ Instantiate / Activate Providers ‣ Define resources in terraform templates ‣ Basically any file in a given directory ending with .tf is treated as a resource template definition 1 # Configure the PagerDuty provider 2 provider "pagerduty" { 3 token = “MyPgToken" 4 } 5 6 # Create a PagerDuty team 7 resource "pagerduty_team" "engineering" { 8 name = "Engineering" 9 description = "All engineering" 10 }
  20. 20. FullStack Developers Israel TERRAFORM | GETTING STARTED *.tf ‣ 1 FAT file with everything … ‣ main.tf for resources ‣ variables.tf for variables ‣ outputs.tf for outputs ‣ * provider.tf for providers … ‣ What ever suits your use case
  21. 21. FullStack Developers Israel TERRAFORM | GETTING STARTED HCL SYNTAX - HIGHLIGHTS ‣ Single line comments start with # or // ‣ Values are assigned with the syntax key = value (whitespace doesn't matter). The value can be any primitive: a string, number, boolean, object, or list. ‣ Multi-line strings start with <<EOF at the end of a line, and end with EOF on its own line. ‣ See full @ https://github.com/hashicorp/hcl#syntax <<FOO Any text Would do … FOO
  22. 22. FullStack Developers Israel TERRAFORM | GETTING STARTED Lifecycle Init Destroy Apply Plan
  23. 23. FullStack Developers Israel TERRAFORM | GETTING STARTED OUR SCENARIO 1 # Configure the GitHub Provider 2 provider "github" { 3 token = "${var.github_token}" 4 organization = "${var.github_org}" 5 } 6 7 # Add a user to the organization 8 resource "github_membership" "user" { 9 username = "shellegtk" 10 role = "member" 11 }
  24. 24. FullStack Developers Israel TERRAFORM | GETTING STARTED OUR SCENARIO 1 variable "github_token" { 2 default = “not_my_real_token…” 3 } 4 5 variable "github_org" { 6 default = "tikalio" 7 }
  25. 25. FullStack Developers Israel TERRAFORM | GETTING STARTED INIT ‣ Get provider libraries ‣ Get modules ‣ Store in ${CWD}/.terraform
  26. 26. FullStack Developers Israel TERRAFORM | GETTING STARTED INIT The .git of your terraform code Provider code Terraform will not execute until it has all it’s dependencies locally
  27. 27. FullStack Developers Israel TERRAFORM | GETTING STARTED PLAN ‣ Preflight -> Show the “Todo List” before you apply it. ‣ Identify potential pitfalls / imports …
  28. 28. FullStack Developers Israel TERRAFORM | GETTING STARTED PLAN terraform plan -out=tfplan -input=false Creates a plan locally for later execution
  29. 29. FullStack Developers Israel TERRAFORM | GETTING STARTED APPLY ‣ Provision -> Execute the “Todo List”.
  30. 30. FullStack Developers Israel TERRAFORM | GETTING STARTED APPLY Exacutes the plan we saved locallyterraform apply -input=false tfplan
  31. 31. FullStack Developers Israel TERRAFORM | GETTING STARTED DESTROY ‣ Don’t like what you see - Rollback or Delete …
  32. 32. FullStack Developers Israel TERRAFORM | GETTING STARTED Lifecycle Init Destroy Apply Plan
  33. 33. Tikal KnowledgeTikal Knowledge STATE { A.K.A TFSTATE } TERRAFORM’S MAGIC…
  34. 34. FullStack Developers Israel TERRAFORM | GETTING STARTED IN OUR SCENARIO - BETWEEN “PLAN” & “DESTROY” 1 { 2 "version": 3, 3 "terraform_version": "0.11.7", 4 "serial": 3, 5 "lineage": "d6a3fc33-4869-c90f-6adb-c21947133250", 6 "modules": [ 7 { 8 "path": [ 9 "root" 10 ], 11 "outputs": {}, 12 "resources": { 13 "github_membership.user": { 14 "type": "github_membership", 15 "depends_on": [], 16 "primary": { 17 "id": "tikalio:shellegtk", 18 "attributes": { 19 "id": "tikalio:shellegtk", 20 "role": "member", 21 "username": "shellegtk" 22 }, 23 "meta": {}, 24 "tainted": false 25 }, Our Infrastructure has a state and a version ! ‣ By default terraform state is stored locally under terraform.tfstate file and can be shared via git { but there are better ways … }
  35. 35. FullStack Developers Israel TERRAFORM | GETTING STARTED SHARING STATE IN GIT - COULD WORK ‣ 1-3 developers with tight communications -> git might do the trick ‣ 2 or more it depends … terraform.tfstate terraform apply
 git add terraform.tfstate
 git commit “update state”
 git push origin master terraform plan
 # oh wait …
 git pull origin master
 terraform plan 
 # phew that was close …
  36. 36. FullStack Developers Israel TERRAFORM | GETTING STARTED SHARING STATE IN S3, CONSUL, ATLAS - BETTER ‣ Instead of using git … ‣ terraform init -backend- config=backend.tfvars terraform.tfstate terraform apply
 # state is uploaded to the bucket as part of the execution terraform plan
 terraform refreshes the 
 state from the remote 
 store prior to execution
 terraform.tfstate 1 bucket = "example-tfstate" 2 dynamodb_table = "TerraformStatelock" 3 key = "example.tfstate" 4 profile = “example" 5 region = "eu-west-1"
  37. 37. FullStack Developers Israel TERRAFORM | GETTING STARTED SHARING STATE IN S3, CONSUL, ATLAS - SAME PROBLEM ‣ Instead of using git … ‣ terraform init -backend- config=backend.tfvars terraform.tfstate terraform.tfstate 1 bucket = "example-tfstate" 2 dynamodb_table = "TerraformStatelock" 3 key = "example.tfstate" 4 profile = “example" 5 region = "eu-west-1" terraform apply
 git add terraform.tfstate
 git commit “update state”
 git push origin master terraform plan
 # oh wait …
 git pull origin master
 terraform plan 
 # phew that was close …
  38. 38. FullStack Developers Israel TERRAFORM | GETTING STARTED STATE LOCK ‣ terraform plan [better] terraform.tfstate In memory terraform.tfstate 🌀bob 👉 terraform plan Acquiring state lock. This may take a few moments... Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage.
 🌀bob 👉 terraform apply
 Acquiring state lock. This may take a few moments... 🌀ann 👉 terraform plan
 Acquiring state lock. This may take a few moments...
 This plan is locked by user <bob ….>
 User A cannot get / update the state until 
 User B had released the lock in the Dynamodb table
  39. 39. FullStack Developers Israel TERRAFORM | GETTING STARTED STATE + LOCK - CONSUL ‣ terraform plan [better] terraform.tfstate In memory terraform.tfstate 🌀bob 👉 terraform plan Acquiring state lock. This may take a few moments... Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage.
 🌀bob 👉 terraform apply
 Acquiring state lock. This may take a few moments... 🌀ann 👉 terraform plan
 Acquiring state lock. This may take a few moments...
 This plan is locked by user xxxx ….
 User A cannot get / update the state until 
 User B had released the lock in the Dynamodb table
  40. 40. FullStack Developers Israel TERRAFORM | GETTING STARTED REFERENCING STATES data.terraform_remote_state.mainstate.domain_name What is the default domain_name ? ask the state !
  41. 41. FullStack Developers Israel TERRAFORM | GETTING STARTED STATE AS DATA SOURCE 1 data "terraform_remote_state" "mainstate" { 2 backend = "s3" 3 4 config { 5 key = “main/main.tfstate” 6 bucket = "${var.bucket}" 7 dynamodb_table = "${var.dynamodb_table}" 8 profile = "${var.profile}" 9 region = "${var.region}" 10 } 11 }
  42. 42. FullStack Developers Israel TERRAFORM | GETTING STARTED DATA SOURCES ARE LIKE A “REGIONAL” MAP data.terraform_remote_state.mainstate.domain_name CURRENT “MODULE” REPRESENTATION REMOTE OUTPUT
  43. 43. FullStack Developers Israel TERRAFORM | GETTING STARTED STATE AS DATA SOURCE 1 resource "aws_route53_zone" "dev" { 2 name = “${data.terraform_remote_state.mainstate.domain_name}" 3 4 tags { 5 Environment = "dev" 6 } 7 } This would work only if the remote state outputs this info …. Enter - Terraform Modules
  44. 44. Tikal KnowledgeTikal Knowledge MODULES TERRAFORM
  45. 45. FullStack Developers Israel TERRAFORM | GETTING STARTED DON’T REPEAT YOURSELF ▸ Reusable Terraform manifests ▸ Configure & Customise features
  46. 46. FullStack Developers Israel TERRAFORM | GETTING STARTED YOU ALREADY KNOW MODULES … ▸ *.tf files ▸ Outputs ▸ Variables …
  47. 47. FullStack Developers Israel TERRAFORM | GETTING STARTED USE CASE ▸ Create an ec2 instance ▸ In many regions (AMI-id differs …) ▸ Use a custom image (custom built AMI), ▸ Community managed image eu-west-1 us-west-1 us-east-1
  48. 48. FullStack Developers Israel TERRAFORM | GETTING STARTED main.tf 1 resource "aws_instance" "example" { 2 ami = “ami-xxxxxxx" 3 4 instance_type = "t2.micro" 5 6 tags { 7 Name = “generic-ec2-instance“ 8 } 9 } This will very in each region …
  49. 49. FullStack Developers Israel TERRAFORM | GETTING STARTED A terraform “module” is basically a set of reusable terror files ! This will very in each region … data "aws_ami" "centos" { owners = ["679593333241"] most_recent = true filter { name = "name" values = ["CentOS Linux 7 x86_64 HVM EBS *"] } filter { name = "architecture" values = ["x86_64"] } filter { name = "root-device-type" values = ["ebs"]
  50. 50. FullStack Developers Israel TERRAFORM | GETTING STARTED A terraform “module” is basically a set of reusable terror files ! This will very in each region … data "aws_ami" "ubuntu" { most_recent = true filter { name = "name" values = ["ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server } filter { name = "virtualization-type" values = ["hvm"] } owners = ["099720109477"] # Canonical }
  51. 51. FullStack Developers Israel TERRAFORM | GETTING STARTED OUTPUT THE RETURN VALUES FOR THE NEXT MODULE … { THE ROOT MODULE } output "regional_centos_ami_id" { value = "${data.aws_ami.centos.id}" } output "regional_ubuntu_ami_id" { value = "${data.aws_ami.ubuntu.id}" }
  52. 52. FullStack Developers Israel TERRAFORM | GETTING STARTED USING A MODULE module "locate_ami" { source = "./modules/locate_ami/" } module “aws_instance" { source = "terraform-aws-modules/ec2-instance/aws" name = “my-cool-tool“ instance_count = 1 associate_public_ip_address = true ami = "${module.locate_ami.regional_ubuntu_ami_id}" instance_type = "t2.medium" … tags = { Terraform = "true" Environment = "dev" Module output becomes Another modules input
  53. 53. FullStack Developers Israel TERRAFORM | GETTING STARTED MODULE CAN HAVE VARIABLES … module “aws_instance" { source = "terraform-aws-modules/ec2-instance/aws" name = “my-cool-tool“ instance_count = 2 … } } Module variable variable block expects a value / sets default one variable "instance_count" { description = "Number of instances to launch" default = 1 }
  54. 54. FullStack Developers Israel TERRAFORM | GETTING STARTED MODULE CAN HAVE VARIABLES … module “aws_instance" { source = "terraform-aws-modules/ec2-instance/aws" name = “my-cool-tool“ instance_count = 2 … } } Module variable variable block expects a value / sets default one variable "instance_count" { description = "Number of instances to launch" default = 1 } The same as writing your root “module"
  55. 55. FullStack Developers Israel TERRAFORM | GETTING STARTED SO TERRAFORM IS A BUNCH OF MODULES . |-- common | |-- backend.tf | |-- files | |-- global_variables.tf | `-- provider.tf |-- mainvpc | |-- README.md | |-- backend.tf | |-- backend.tfvars | |-- default_sgs.tf | |-- files | |-- global_variables.tf -> ../common/global_variables.tf | |-- main.tf | |-- outputs.tf | |-- templates | `-- variables.tf | A module which sets up your s3 bucket, access and state store etc a module which sets up the main VPC
 security groups, vpn, nat gateways etc
  56. 56. FullStack Developers Israel TERRAFORM | GETTING STARTED SO TERRAFORM IS A BUNCH OF MODULES . |-- common | |-- backend.tf | |-- files | |-- global_variables.tf | `-- provider.tf |-- mainvpc | |-- README.md | |-- backend.tf | |-- backend.tfvars | |-- default_sgs.tf | |-- files | |-- global_variables.tf -> ../common/global_variables.tf | |-- main.tf | |-- outputs.tf | |-- templates | `-- variables.tf | IMO - use symbolic links to reduce duplicate variable decelerations.
  57. 57. FullStack Developers Israel TERRAFORM | GETTING STARTED SO TERRAFORM IS A BUNCH OF MODULES . |-- common | |-- backend.tf | |-- files | |-- global_variables.tf | `-- provider.tf |-- mainvpc | |-- README.md | |-- backend.tf | |-- backend.tfvars | |-- default_sgs.tf | |-- files | |-- global_variables.tf -> ../common/global_variables.tf | |-- main.tf | |-- outputs.tf | |-- templates | `-- variables.tf | Use outputs.tf the help understand the Interface between modules
  58. 58. FullStack Developers Israel TERRAFORM | GETTING STARTED SO TERRAFORM IS A BUNCH OF MODULES -- mainvpc-k8s | |-- README.md | |-- backend.tf | |-- backend.tfvars | |-- global_variables.tf -> ../common/global_variables.tf | |-- main.tf | |-- outputs.tf | `-- variables.tf |-- mainvpc-peers | |-- dev.tf | `-- global_variables.tf -> ../common/global_variables.tf `-- modules |-- backend |-- kops_reqs |-- kubernetes |-- locate_ami `-- rabbitmq-cluster A module which sets up Kubernetes Connect vpc peers between main / others Modules directory
  59. 59. FullStack Developers Israel TERRAFORM | GETTING STARTED TERRAFORM REGISTRY ▸ A long list of high quality / battle-tested modules such as: ▸ vpc ▸ security groups ▸ ….
  60. 60. Tikal KnowledgeTikal Knowledge 0.12 TERRAFORM
  61. 61. FullStack Developers Israel TERRAFORM | GETTING STARTED TERRAFORM 0.12 IMPROVEMENTS … • First-class expression syntax: express references and expressions directly rather than using string interpolation syntax.
 • Generalized type system: use lists and maps more freely, and use resources as object values. • Iteration constructs: transform and filter one collection into another collection, and generate nested configuration blocks from collections. • Structural rendering of plans: plan output now looks more like configuration making it easier to understand. • Context-rich error messages: error messages now include a highlighted snippet of configuration and often suggest exactly what needs to be changed to resolve them. https://www.hashicorp.com/blog/announcing-terraform-0-12 https://medium.com/@sudhakar.singh/how-terraform-0-12-can-simplify-writing-out-your-infrastructure-d325ba221340
  62. 62. FullStack Developers Israel TERRAFORM 0.12-CHECK-UPGRADE ▸ terraform 0.12checklist Looks good! We did not detect any problems that ought to be addressed before upgrading to Terraform v0.12. This tool is not perfect though, so please check the v0.12 upgrade guide for additional guidance, and for next steps: https://www.terraform.io/upgrade-guides/0-12.html
  63. 63. FullStack Developers Israel TFENV - VIRTUAL ENV FOR TERRAFORM … ▸ tfenv install 0.11.14 (0.11 latest) ▸ tfenv install 0.12.2 (0.12 latest) https://github.com/tfutils/tfenv tfenv use 0.11.14 [INFO] Switching to v0.11.14 [INFO] Switching completed tfenv use 0.12.2 [INFO] Switching to v0.12.2 [INFO] Switching completed
  64. 64. Tikal Knowledge TERRAFORM | GETTING STARTED TERRAGRUNT ▸ A terraform wrapper ▸ Forces module usage
  65. 65. Tikal Knowledge TERRAFORM | GETTING STARTED MODULES.TF
  66. 66. Tikal Knowledge TERRAFORM | GETTING STARTED RUN ATLANTIS ▸ Terraform Pull Request Automation ▸ Every pull request becomes a web hook check ▸ Run terraform plan from the Github UI
  67. 67. FullStack Developers Israel TERRAFORM | GETTING STARTED A CI/CD CYCLE FOR TERRAFORM BASED
  68. 68. FullStack Developers Israel HANDS ON
  69. 69. FullStack Developers Israel ▸ git clone https://github.com/tikalk/terraform-101.git TERRAFORM | GETTING STARTED GET CODE
  70. 70. FullStack Developers Israel TERRAFORM | GETTING STARTED SETUP YOUR ENVIRONMENT ▸ Get a Github Personal access token ▸ setup and environnent variable named 
 TF_VAR_github_token=“<your token>”
  71. 71. FullStack Developers Israel TERRAFORM | GETTING STARTED SETUP VARS (VARIABLES.TF) ▸ set the github_orignzation var ▸ yes you can also
 TF_VAR_github_orignzation =“<your org>”
  72. 72. FullStack Developers Israel TERRAFORM | GETTING STARTED SETUP YOUR GITHUB USER IN MAIN.TF ▸ Update main.tf with your GitHub ID ▸ (or convert it to a var of type list …)
  73. 73. FullStack Developers Israel TERRAFORM | GETTING STARTED Lifecycle Init Destroy Apply Plan
  74. 74. FullStack Developers Israel We Love Tech Thank you for tuning in

×