Terraform 101

FullStack Developers Israel
TERRAFORM
TRUE INFRASTRUCTURE AS CODE WITH

WHO WE ARE ?
▸ Tikal helps ISV’s in Israel & abroad in their technological
challenges.
▸ Our Engineers are Fullstack Developers with expertise in
Android, DevOps, Java, JS, Python, ML
▸ We are passionate about technology and specialise in
OpenSource technologies.
▸ Our Tech and Group leaders help establish & enhance
existing software teams with innovative & creative thinking.
https://www.meetup.com/full-stack-developer-il/

SELF INTRODUCTION
▸ My open thinking and open techniques
ideology is driven by Open Source
technologies and the collaborative manner
defining my M.O.
▸ My solution driven approach is strongly based on hands-on
and deep understanding of Operating Systems,
Applications stacks and Software languages, Networking,
Cloud in general and today more an more Cloud Native
solutions.
HAGGAI PHILIP ZAGURY - DEVOPS ARCHITECT AND GROUP TECH LEAD

DevOps

DISTRIBUTED SYSTEMS || 12 FACTOR APPS
▸ Micorservices !
▸ Each micro service has it’s own Data Srouce
▸ Queues
▸ Load Balancers
▸ Auto Scaling
▸ Cloud / Cloud Native / Multi-Cloud
TERRAFORM | GETTING STARTED

INFRASTRUCTURE & AUTOMATION EVOLUTION
▸ Automation is the #1 enables for:
▸ Scalability - modularity
▸ Reproducibility - re-spin env’s on the fly
▸ Reliability - continuous practices

TERRAFORM
TRUE INFRASTRUCTURE AS CODE WITH
BECAUSE K8S IS THE LEAST
OF OUR WORRIES …

INFRASTRUCTURE AS CODE
Infrastructure as code (IaC) is the process of managing and
provisioning computer data centers through machine-
readable definition files, rather than physical hardware
configuration or interactive configuration tools.[1] The IT
infrastructure meant by this comprises both physical
equipment such as bare-metal servers as well as virtual
machines and associated configuration resources. The
definitions may be in a version control system. It can use
either scripts or declarative definitions, rather than manual
processes, but the term is more often used to promote
declarative approaches.

IAC - A SOFTWARE ENGINEERING APPROACH TO OPERATIONS
▸ Automation, Automation, Automation …
▸ Quality management (“test driven infrastructure”)
▸ D.R.Y -> Don’t Repeat Yourself
▸ Collaborate, Share, Communicate
▸ Messure / Audit / Asses

POLYGLOT PROVISIONING …
▸ Azure Resource Manager
▸ Google Cloud Deployment Manager
▸ AWS CloudFormation
▸ Ansible -> CloudFormation
▸ Chef -> Heat templates
▸ …

A TOOL FOR INFRASTRUCTURE PROVISIONING

PROVIDERS - MANY PROVIDERS

PROVIDERS - AN EXAMPLE
1 # Configure the GitHub Provider
2 provider "github" {
3 token = "MyMicrosoftToken"
4 organization = "tikal.io"
5 }
6
7 # Add a user to the organization
8 resource "github_membership" "user" {
9 username = “terry.forman"
10 role = "member"
11 }

PROVIDER - ACCESS TOKEN
‣ Create an access token in
Github
‣ Configure the Github Provider
‣ Start managing groups, users,
tokens etc …

RESOURCES
1 # Configure the PagerDuty provider
2 provider "pagerduty" {
3 token = “MyPgToken"
4 }
5
6 # Create a PagerDuty team
7 resource "pagerduty_team" "engineering" {
8 name = "Engineering"
9 description = "All engineering"
10 }
11
12 # Create a PagerDuty user and add it to a team
13 resource "pagerduty_user" “tikal_io” {
14 name = “Haggai Philip Zagury"
15 email = “hagzag@tikalk.com”
16 teams = ["${pagerduty_team.engineering.id}"]
17 }

VARIABLES - DECLARE
1 variable "profile" {
2 type = "string"
3 description = “see ~/.aws/credentials file for details"
4 default = "my_company"
5 }
6
7 variable "region" {
8 type = "string"
9 description = "your default aws region"
10 default = "eu-west-1"
11 }
12
13 variable "general_tags" {
14 type = "map"
15
16 default = {
17 Name = "example.com"
18 Environment = "dev"
19 KubernetesCluster = "dev.example.com"
20 }
21 }
Best Practice to declare types
Best Practice to be descriptive

VARIABLES - USE
Use ${var.var_name}
1 resource "aws_route53_zone" "dev" {
2 name = "${var.env}.${var.domain_name}"
3
4 tags {
6 }
7 }

OUTPUTS
terraform output “output_key”1 output "availability_zones" {
2 value = "${var.azs}"
3 }
More on this later on when we discuss modules and states !
TERRAFORM 1 TERRAFORM 2
availability_zones

TERRAFORM TEMPLATE (HCL)
‣ Instantiate / Activate Providers
‣ Define resources in terraform
templates
‣ Basically any file in a given
directory ending with .tf is treated
as a resource template definition
1 # Configure the PagerDuty provider
2 provider "pagerduty" {
3 token = “MyPgToken"
4 }
5
6 # Create a PagerDuty team
7 resource "pagerduty_team" "engineering" {
8 name = "Engineering"
9 description = "All engineering"
10 }

*.tf
‣ 1 FAT file with everything …
‣ main.tf for resources
‣ variables.tf for variables
‣ outputs.tf for outputs
‣ * provider.tf for providers …
‣ What ever suits your use case

HCL SYNTAX - HIGHLIGHTS
‣ Single line comments start with # or //
‣ Values are assigned with the syntax key = value (whitespace doesn't matter).
The value can be any primitive: a string, number, boolean, object, or list.
‣ Multi-line strings start with <<EOF at the end of a line, and end with EOF on its
own line.
‣ See full @ https://github.com/hashicorp/hcl#syntax
<<FOO
Any text
Would do …
FOO

Lifecycle
Init
Destroy Apply
Plan

OUR SCENARIO
1 # Configure the GitHub Provider
2 provider "github" {
3 token = "${var.github_token}"
4 organization = "${var.github_org}"
5 }
6
7 # Add a user to the organization
8 resource "github_membership" "user" {
9 username = "shellegtk"
10 role = "member"
11 }

OUR SCENARIO
1 variable "github_token" {
2 default = “not_my_real_token…”
3 }
4
5 variable "github_org" {
6 default = "tikalio"
7 }

INIT
‣ Get provider libraries
‣ Get modules
‣ Store in ${CWD}/.terraform

INIT
The .git of your terraform code
Provider code
Terraform will not execute until it has all it’s dependencies locally

PLAN
‣ Preflight -> Show the
“Todo List” before you
apply it.
‣ Identify potential pitfalls /
imports …

PLAN
terraform plan -out=tfplan -input=false Creates a plan locally for later execution

APPLY
‣ Provision -> Execute the
“Todo List”.

APPLY
Exacutes the plan we saved locallyterraform apply -input=false tfplan

DESTROY
‣ Don’t like what you see -
Rollback or Delete …

Tikal KnowledgeTikal Knowledge
STATE { A.K.A TFSTATE }
TERRAFORM’S MAGIC…

IN OUR SCENARIO - BETWEEN “PLAN” & “DESTROY”
1 {
2 "version": 3,
3 "terraform_version": "0.11.7",
4 "serial": 3,
5 "lineage": "d6a3fc33-4869-c90f-6adb-c21947133250",
6 "modules": [
7 {
8 "path": [
9 "root"
10 ],
11 "outputs": {},
12 "resources": {
13 "github_membership.user": {
14 "type": "github_membership",
15 "depends_on": [],
16 "primary": {
17 "id": "tikalio:shellegtk",
18 "attributes": {
19 "id": "tikalio:shellegtk",
20 "role": "member",
21 "username": "shellegtk"
22 },
23 "meta": {},
24 "tainted": false
25 },
Our Infrastructure has a state and a version !
‣ By default terraform state is
stored locally under
terraform.tfstate file and can be
shared via git { but there are
better ways … }

SHARING STATE IN GIT - COULD WORK
‣ 1-3 developers with tight
communications -> git might do the
trick
‣ 2 or more it depends …
terraform.tfstate
terraform apply 
git add terraform.tfstate 
git commit “update state” 
git push origin master
terraform plan 
# oh wait … 
git pull origin master 
terraform plan  
# phew that was close …

SHARING STATE IN S3, CONSUL, ATLAS - BETTER
‣ Instead of using git …
‣ terraform init -backend-
config=backend.tfvars
terraform.tfstate
terraform apply 
# state is uploaded to the bucket as part of the execution
terraform plan 
terraform refreshes the  
state from the remote  
store prior to execution 
terraform.tfstate
1 bucket = "example-tfstate"
2 dynamodb_table = "TerraformStatelock"
3 key = "example.tfstate"
4 profile = “example"
5 region = "eu-west-1"

SHARING STATE IN S3, CONSUL, ATLAS - SAME PROBLEM
‣ Instead of using git …
‣ terraform init -backend-
config=backend.tfvars
terraform.tfstate
terraform.tfstate
1 bucket = "example-tfstate"
2 dynamodb_table = "TerraformStatelock"
3 key = "example.tfstate"
4 profile = “example"
5 region = "eu-west-1"
terraform apply 
git add terraform.tfstate 
git commit “update state” 
git push origin master
terraform plan 
# oh wait … 
git pull origin master 
terraform plan  
# phew that was close …

STATE LOCK
‣ terraform plan [better]
terraform.tfstate
In memory terraform.tfstate
🌀bob 👉 terraform plan
Acquiring state lock. This may take a few moments...
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan,
but will not be persisted to local or remote state
storage. 
🌀bob 👉 terraform apply 
🌀ann 👉 terraform plan 
Acquiring state lock. This may take a few moments... 
This plan is locked by user <bob ….> 
User A cannot get / update the state until  
User B had released the lock in the Dynamodb table

STATE + LOCK - CONSUL
‣ terraform plan [better]
terraform.tfstate
In memory terraform.tfstate
🌀bob 👉 terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan,
but will not be persisted to local or remote state
storage. 
🌀bob 👉 terraform apply 
🌀ann 👉 terraform plan 
Acquiring state lock. This may take a few moments... 
This plan is locked by user xxxx …. 
User A cannot get / update the state until  
User B had released the lock in the Dynamodb table

REFERENCING STATES
data.terraform_remote_state.mainstate.domain_name
What is the default domain_name ?
ask the state !

STATE AS DATA SOURCE
1 data "terraform_remote_state" "mainstate" {
2 backend = "s3"
3
4 config {
5 key = “main/main.tfstate”
6 bucket = "${var.bucket}"
7 dynamodb_table = "${var.dynamodb_table}"
8 profile = "${var.profile}"
9 region = "${var.region}"
10 }
11 }

DATA SOURCES ARE LIKE A “REGIONAL” MAP
data.terraform_remote_state.mainstate.domain_name
CURRENT “MODULE” REPRESENTATION REMOTE OUTPUT

STATE AS DATA SOURCE
1 resource "aws_route53_zone" "dev" {
2 name = “${data.terraform_remote_state.mainstate.domain_name}"
3
4 tags {
6 }
7 }
This would work only if the remote state outputs this info ….
Enter - Terraform Modules

MODULES
TERRAFORM

DON’T REPEAT YOURSELF
▸ Reusable Terraform manifests
▸ Configure & Customise features

YOU ALREADY KNOW MODULES …
▸ *.tf files
▸ Outputs
▸ Variables …

USE CASE
▸ Create an ec2 instance
▸ In many regions (AMI-id differs …)
▸ Use a custom image (custom built
AMI),
▸ Community managed image
eu-west-1 us-west-1
us-east-1

main.tf
1 resource "aws_instance" "example" {
2 ami = “ami-xxxxxxx"
3
4 instance_type = "t2.micro"
5
6 tags {
7 Name = “generic-ec2-instance“
8 }
9 }
This will very in each region …

A terraform “module” is basically a set of reusable terror files !
data "aws_ami" "centos" {
owners = ["679593333241"]
most_recent = true
filter {
name = "name"
values = ["CentOS Linux 7 x86_64 HVM EBS *"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
filter {
name = "root-device-type"
values = ["ebs"]

A terraform “module” is basically a set of reusable terror files !
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"] # Canonical
}

OUTPUT THE RETURN VALUES FOR THE NEXT MODULE … { THE ROOT MODULE }
output "regional_centos_ami_id" {
value = "${data.aws_ami.centos.id}"
}
output "regional_ubuntu_ami_id" {
value = "${data.aws_ami.ubuntu.id}"
}

USING A MODULE
module "locate_ami" {
source = "./modules/locate_ami/"
}
module “aws_instance" {
source = "terraform-aws-modules/ec2-instance/aws"
name = “my-cool-tool“
instance_count = 1
associate_public_ip_address = true
ami = "${module.locate_ami.regional_ubuntu_ami_id}"
instance_type = "t2.medium"
…
tags = {
Terraform = "true"
Environment = "dev"
Module output becomes
Another modules input

MODULE CAN HAVE VARIABLES …
instance_count = 2
…
}
}
Module variable
variable block expects a value /
sets default one
variable "instance_count" {
description = "Number of instances to launch"
default = 1
}

MODULE CAN HAVE VARIABLES …
instance_count = 2
…
}
}
Module variable
variable block expects a value /
sets default one
variable "instance_count" {
description = "Number of instances to launch"
default = 1
}
The same as writing your root “module"

TERRAFORM REGISTRY
▸ A long list of high quality / battle-tested
modules such as:
▸ vpc
▸ security groups
▸ ….

0.12
TERRAFORM

TERRAFORM 0.12 IMPROVEMENTS …
• First-class expression syntax: express references and expressions directly rather than using string
interpolation syntax. 
• Generalized type system: use lists and maps more freely, and use resources as object values.
• Iteration constructs: transform and filter one collection into another collection, and generate nested
configuration blocks from collections.
• Structural rendering of plans: plan output now looks more like configuration making it easier to understand.
• Context-rich error messages: error messages now include a highlighted snippet of configuration and often
suggest exactly what needs to be changed to resolve them.
https://www.hashicorp.com/blog/announcing-terraform-0-12
https://medium.com/@sudhakar.singh/how-terraform-0-12-can-simplify-writing-out-your-infrastructure-d325ba221340

TERRAFORM 0.12-CHECK-UPGRADE
▸ terraform 0.12checklist
Looks good! We did not detect any problems that ought to be
addressed before upgrading to Terraform v0.12.
This tool is not perfect though, so please check the v0.12 upgrade
guide for additional guidance, and for next steps:
https://www.terraform.io/upgrade-guides/0-12.html

TFENV - VIRTUAL ENV FOR TERRAFORM …
▸ tfenv install 0.11.14 (0.11 latest)
▸ tfenv install 0.12.2 (0.12 latest)
https://github.com/tfutils/tfenv
tfenv use 0.11.14
[INFO] Switching to v0.11.14
[INFO] Switching completed
tfenv use 0.12.2
[INFO] Switching to v0.12.2
[INFO] Switching completed

Tikal Knowledge
TERRAGRUNT
▸ A terraform wrapper
▸ Forces module usage

Tikal Knowledge
MODULES.TF

Tikal Knowledge
RUN ATLANTIS
▸ Terraform Pull Request Automation
▸ Every pull request becomes a web hook
check
▸ Run terraform plan from the Github UI

A CI/CD CYCLE FOR TERRAFORM BASED

HANDS ON

▸ git clone https://github.com/tikalk/terraform-101.git
GET CODE

SETUP YOUR ENVIRONMENT
▸ Get a Github Personal access token
▸ setup and environnent variable named  
TF_VAR_github_token=“<your token>”

SETUP VARS (VARIABLES.TF)
▸ set the github_orignzation var
▸ yes you can also 
TF_VAR_github_orignzation =“<your org>”

SETUP YOUR GITHUB USER IN MAIN.TF
▸ Update main.tf with your GitHub ID
▸ (or convert it to a var of type list …)

We Love
Tech
Thank you for tuning in

Terraform 101

More Related Content

What's hot

Similar to Terraform 101

More from Haggai Philip Zagury

Recently uploaded

Terraform 101