TERRAFORM 0.13:
RISE OF THE MODULES_
Marko Bevc
BUILDING
BLOCKS_
“Abstraction of common blocks of
configuration into a reusable
infrastructure elements.”
— @sheriffjackson, 2018 blog
DRY code – Don’t Repeat Yourself
Everything is a module!
ABOUT
ME_ ●
Senior IT Consultant at The Scale Factory (DevOps consultancy,
AWS advanced consulting partner and K8s service provider)
●
IT system engineering and design background with extensive
Linux and virtualization experience
●
Certifications and competencies: AWS, CKA, RHEL, Hashi stack
●
Open source contributor and supporter
●
Fan of automation/simplifying things, hiking, cycling and travelling
TOPICS
COVERED_
●
Terraform time line
●
Module improvements
●
Provider ecosystem
●
Other enhancements and breaking
changes
●
How to get started and a demo
●
Conclusions and path forward
TERRAFORM
TIME LINE_
• Pre-v0.10 era (provisioners, state ENV, remote
state and locking)
• Period v0.10 – v0.11 (workspaces, core/provider
split, TF registry (private), module providers,
streamlined APPLY)
• Big changes with v0.12 (HCL2: 1st
class
expressions, gen.types, iterations/dyn.blocks,
structural plans and better error messages)
• Module and provider improvements in v0.13
HISTORY
2014 (initial v0.1.0) 2020 (v0.13) v0.14..1.0*2017 (v0.9-0.11) 2019 (v0.12)
MODULE
EXPANSION_
●
Improved module usability and functionality
●
Create multiple instances of a module from a single
module block:
– count (tuple/list)
– for_each (map)
●
Simplified configuration logic and better automation
●
Can create some indirect module-level dependencies
●
Cannot use provider blocks per module instance! *
(i.e. AWS regions)
locals {
resources = {
prod = "prod-eks"
qa = "qa-eks"
dev = "dev-eks"
}
}
module "eks" {
source = "terraform-aws-modules/eks/aws"
for_each = local.resources
cluster_name = each.value
cluster_version = "1.17"
subnets = ["subnet-abcde012", "subnet-bcde012a", "subnet-fghi345a"]
vpc_id = "vpc-1234556abcdef"
worker_groups = [
{
name = each.key
instance_type = "m4.large"
asg_max_size = 3
}
]
}
eks-clusters.tf
MODULE
DEPENDENCIES_
• Dependencies types:
– implicit
– explicit
• Previously module instances were NOT nodes in dependency graph
(just separate namespaces, dependencies via I/O values)
• depends_on meta-argument ensures order of module resource
changes – create after any dependent resource changes have been
applied
• Far more coarse declaration of dependency than encouraged – last
resort! (*auto deps: easier to maintain and maximise concurrency)
resource "aws_iam_policy_attachment" "example" {
name = "example"
roles = [aws_iam_role.example.name]
policy_arn = aws_iam_policy.example.arn
}
module "uses-role" {
# ...
depends_on = [aws_iam_policy_attachment.example]
}
PROVIDER
ECOSYSTEM_
• Improvements to Terraform Provider Registry
• Automatic install of 3rd
party providers
• Provider source block with namespace separation, defaults to
HashiCorp’s providers: [registry.terraform.io/][hashicorp/]type
• One provider per module, but nesting not recommended!
• Simplifying the installation of community providers:
– Official HashiCorp supported providers
– Partner-supported providers (needs required_providers now)
– Locally installed providers (3rd
party provider plugins):
./plugins/example.com/myorg/customplugin/0.1/linux_amd64/
OTHER
ENHANCEMENTS_
●
Custom variable validation rules (experimental since
v0.12, zero or more rules)
●
Terraform Cloud streamlined authentication process
(terraform login connects CLI to TF Cloud)
●
Providers mirror (terraform providers mirror)
●
Terraform CLI supports TLS 1.3
●
Breaking changes:
– Source for non-default providers
– Locking was improved and changes to the
TableStore schema now require a primary key
named LockID of type String
– macOS builds of Terraform CLI are no longer
compatible with macOS 10.10 Yosemite
– Terraform CLI now requires FreeBSD 11.2 or later.
variable "image_id" {
type = string
description = "The id of the machine image (AMI) to use for the server."
validation {
# regex(...) fails if it cannot find a match
condition = can(regex("^ami-", var.image_id))
error_message = "The image_id value must be a valid AMI id, starting
with "ami-"."
}
}
# NOT needed anymore
terraform {
experiments = [variable_validation]
}
Variable custom validation rules
HOW TO
GET STARTED_
●
Get binary from: releases.hashicorp.com
●
Ensure no pending configuration changes
●
Versioning or backup of code
●
Data resource reads can no longer be disabled - deps
●
Destroy-time provisioners cannot refer to other
resources – deprecated
●
New hierarchical namespace - explicit source for any
not HashiCorp-maintained providers
●
New directory structure for manually-installed
providers
●
terraform 0.13upgrade (gradual one release jumps)
●
Follow CHANGELOG for more details
TIME FOR
DEMO!_
CONCLUSIONS_
& TAKEAWAYS
●
Module improvements: expansion (count, for_each) and
dependencies
●
Provider source and hierarchical namespace – wider ecosystem
and required source, terraform mirror to avoid re-download
●
Custom validation rules for input variables
●
Streamlined Terraform Cloud auth CLI process
●
Improved state locking and schema changes
●
v0.14 (path to v1.0): possible features(CLI/Cloud parity, sensitive
values, import, module testing)
●
Resources:
– https://www.scalefactory.com/blog/2020/06/25/what-we-are-looking-forward-to-in-terraform-
0.13/
– https://www.hashicorp.com/blog/announcing-the-terraform-0-13-beta/
– https://www.hashicorp.com/blog/custom-variable-validation-in-terraform-0-13/
– https://www.terraform.io/docs/configuration/syntax.html
– https://www.hashicorp.com/blog/hashicorp-terraform-modules-as-building-blocks-for/
– https://www.hashicorp.com/resources/the-path-to-terraform-1-0/
– https://www.hashicorp.com/blog/announcing-terraform-0-12/
– https://en.wikipedia.org/wiki/Don%27t_repeat_yourself
FURTHER
READING_
KEEP IN
TOUCH_
https://www.scalefactory.com/
@_MarkoB
@mbevc1
@mbevc1
https://www.linkedin.com/in/marko-bevc/
https://www.scalefactory.com/Web:
Twitter:
GitHub:
GitLab:
LinkedIn:

Terraform 0.13: Rise of the modules

  • 2.
    TERRAFORM 0.13: RISE OFTHE MODULES_ Marko Bevc
  • 3.
    BUILDING BLOCKS_ “Abstraction of commonblocks of configuration into a reusable infrastructure elements.” — @sheriffjackson, 2018 blog DRY code – Don’t Repeat Yourself Everything is a module!
  • 4.
    ABOUT ME_ ● Senior ITConsultant at The Scale Factory (DevOps consultancy, AWS advanced consulting partner and K8s service provider) ● IT system engineering and design background with extensive Linux and virtualization experience ● Certifications and competencies: AWS, CKA, RHEL, Hashi stack ● Open source contributor and supporter ● Fan of automation/simplifying things, hiking, cycling and travelling
  • 5.
    TOPICS COVERED_ ● Terraform time line ● Moduleimprovements ● Provider ecosystem ● Other enhancements and breaking changes ● How to get started and a demo ● Conclusions and path forward
  • 6.
    TERRAFORM TIME LINE_ • Pre-v0.10era (provisioners, state ENV, remote state and locking) • Period v0.10 – v0.11 (workspaces, core/provider split, TF registry (private), module providers, streamlined APPLY) • Big changes with v0.12 (HCL2: 1st class expressions, gen.types, iterations/dyn.blocks, structural plans and better error messages) • Module and provider improvements in v0.13 HISTORY 2014 (initial v0.1.0) 2020 (v0.13) v0.14..1.0*2017 (v0.9-0.11) 2019 (v0.12)
  • 7.
    MODULE EXPANSION_ ● Improved module usabilityand functionality ● Create multiple instances of a module from a single module block: – count (tuple/list) – for_each (map) ● Simplified configuration logic and better automation ● Can create some indirect module-level dependencies ● Cannot use provider blocks per module instance! * (i.e. AWS regions)
  • 8.
    locals { resources ={ prod = "prod-eks" qa = "qa-eks" dev = "dev-eks" } } module "eks" { source = "terraform-aws-modules/eks/aws" for_each = local.resources cluster_name = each.value cluster_version = "1.17" subnets = ["subnet-abcde012", "subnet-bcde012a", "subnet-fghi345a"] vpc_id = "vpc-1234556abcdef" worker_groups = [ { name = each.key instance_type = "m4.large" asg_max_size = 3 } ] } eks-clusters.tf
  • 9.
    MODULE DEPENDENCIES_ • Dependencies types: –implicit – explicit • Previously module instances were NOT nodes in dependency graph (just separate namespaces, dependencies via I/O values) • depends_on meta-argument ensures order of module resource changes – create after any dependent resource changes have been applied • Far more coarse declaration of dependency than encouraged – last resort! (*auto deps: easier to maintain and maximise concurrency)
  • 10.
    resource "aws_iam_policy_attachment" "example"{ name = "example" roles = [aws_iam_role.example.name] policy_arn = aws_iam_policy.example.arn } module "uses-role" { # ... depends_on = [aws_iam_policy_attachment.example] }
  • 11.
    PROVIDER ECOSYSTEM_ • Improvements toTerraform Provider Registry • Automatic install of 3rd party providers • Provider source block with namespace separation, defaults to HashiCorp’s providers: [registry.terraform.io/][hashicorp/]type • One provider per module, but nesting not recommended! • Simplifying the installation of community providers: – Official HashiCorp supported providers – Partner-supported providers (needs required_providers now) – Locally installed providers (3rd party provider plugins): ./plugins/example.com/myorg/customplugin/0.1/linux_amd64/
  • 12.
    OTHER ENHANCEMENTS_ ● Custom variable validationrules (experimental since v0.12, zero or more rules) ● Terraform Cloud streamlined authentication process (terraform login connects CLI to TF Cloud) ● Providers mirror (terraform providers mirror) ● Terraform CLI supports TLS 1.3 ● Breaking changes: – Source for non-default providers – Locking was improved and changes to the TableStore schema now require a primary key named LockID of type String – macOS builds of Terraform CLI are no longer compatible with macOS 10.10 Yosemite – Terraform CLI now requires FreeBSD 11.2 or later.
  • 13.
    variable "image_id" { type= string description = "The id of the machine image (AMI) to use for the server." validation { # regex(...) fails if it cannot find a match condition = can(regex("^ami-", var.image_id)) error_message = "The image_id value must be a valid AMI id, starting with "ami-"." } } # NOT needed anymore terraform { experiments = [variable_validation] } Variable custom validation rules
  • 14.
    HOW TO GET STARTED_ ● Getbinary from: releases.hashicorp.com ● Ensure no pending configuration changes ● Versioning or backup of code ● Data resource reads can no longer be disabled - deps ● Destroy-time provisioners cannot refer to other resources – deprecated ● New hierarchical namespace - explicit source for any not HashiCorp-maintained providers ● New directory structure for manually-installed providers ● terraform 0.13upgrade (gradual one release jumps) ● Follow CHANGELOG for more details
  • 15.
  • 16.
    CONCLUSIONS_ & TAKEAWAYS ● Module improvements:expansion (count, for_each) and dependencies ● Provider source and hierarchical namespace – wider ecosystem and required source, terraform mirror to avoid re-download ● Custom validation rules for input variables ● Streamlined Terraform Cloud auth CLI process ● Improved state locking and schema changes ● v0.14 (path to v1.0): possible features(CLI/Cloud parity, sensitive values, import, module testing)
  • 17.
    ● Resources: – https://www.scalefactory.com/blog/2020/06/25/what-we-are-looking-forward-to-in-terraform- 0.13/ – https://www.hashicorp.com/blog/announcing-the-terraform-0-13-beta/ –https://www.hashicorp.com/blog/custom-variable-validation-in-terraform-0-13/ – https://www.terraform.io/docs/configuration/syntax.html – https://www.hashicorp.com/blog/hashicorp-terraform-modules-as-building-blocks-for/ – https://www.hashicorp.com/resources/the-path-to-terraform-1-0/ – https://www.hashicorp.com/blog/announcing-terraform-0-12/ – https://en.wikipedia.org/wiki/Don%27t_repeat_yourself FURTHER READING_
  • 18.