This document discusses best practices for developing Terraform modules at scale. It covers key topics like defining module structure, using modules, managing module versions and upgrades, discoverability, and release processes. The goal is to help make modules reusable, versioned, and easily consumed as infrastructure codebases grow in size and complexity.

2. Developing Terraform Modules at Scale
Tom Straub
(He/Him)
Delivery Manager of Infrastructure
Implementation Services

3. Rick
Systems Architect
Core Services Team
Morty
Application Developer
Billing Team

4. I wrote some awesome Terraform
Me too!

5. About Me
Delivery Manager of Infrastructure
Implementation Services at HashiCorp
Tom Straub
Terraform Enterprise
Terraform since: 0.6.0
Nickname: Terraform Tom
@straubt1
@terraform-tom

6. main.tf
variables.tf
outputs.tf
What is a Module
Spoiler alert: It’s just Terraform

7. main.tf
module "object_storage" {
source = "..."
bucket_name = "my_bucket"
}
output "bucket_id" {
value = module.object_storage .bucket_id
}
Module Reference
main.tf
variables.tf
outputs.tf

8. main.tf
module "object_storage" {
source = "..."
version = "=1.0.0"
bucket_name = "my_bucket"
}
output "bucket_id" {
value = module.object_storage .bucket_id
}
Module Reference

9. Producer Consumer
Producer vs. Consumer
main.tf
variables.tf
outputs.tf

10. How Many Resources?

11. How Many Resources?
Variables
Outputs
1:1

12. How Many Resources?
Variables
Outputs
Variables
Outputs
1:1 1:∞

13. How Many Resources?
Variables
Outputs
Variables
Outputs
1:1 1:∞
Goldilocks
Principle

14. Blast Radius
$ terraform destroy
Key Considerations

15. Blast Radius
$ terraform destroy
Key Considerations
Rate of Change
$ terraform apply

16. Blast Radius
$ terraform destroy
Dependencies
data "aws_vpc" "selected" {}
Key Considerations
Rate of Change
$ terraform apply

17. Define Module Structure
terraform-cloud-mymodule
variables.tf
outputs.tf
main.tf

18. Define Module Structure
terraform-google-mymodule
variables.tf
outputs.tf
README.md
CHANGELOG.md
main.tf

19. Define Module Structure
terraform-google-mymodule
variables.tf
outputs.tf
README.md
CHANGELOG.md
examples/
main.tf

20. Define Module Structure
terraform-google-mymodule
variables.tf
outputs.tf
README.md
CHANGELOG.md
examples/
tests/
main.tf

21. Example
Module
Data lookup within module
Example Module Code
variable "vpc_name" {}
data "aws_vpc" "main" {
tags {
Name = var.vpc_name
}
}
resource "aws_security_group" "allow_tls" {
vpc_id = data.aws_vpc.main.id
name = "allow_tls"
...
}

22. Example
Module
Module exposes a variable
Example Module Code
variable "vpc_id" {}
resource "aws_security_group" "allow_tls" {
name = "allow_tls"
vpc_id = var.vpc_id
...
}
Dependency
Injection
Principle

23. Using
Module
Data lookup of Resource
Root Module Code
data "aws_vpc" "main" {
tags {
Name = "myTeamVPCName"
}
}
module "example" {
source = "./modules/example"
vpc_id = data.aws_vpc.main.id
}

24. Using
Module
Resource Create
Root Module Code
resource "aws_vpc" "main" {
cidr_block = "10.0.0.0/16"
}
module "example" {
source = "./modules/example"
vpc_id = aws_vpc.main.id
}

25. Using
Module
Variable pass-through
Root Module Code
variable "vpc_id" {}
module "example" {
source = "./modules/example"
vpc_id = var.vpc_id
}

26. Module Depth Example
Module

27. Module Depth Example
Module

28. Module Depth Example
Module

29. Module Depth
Root Module Code
module "consul_cluster" {
source = ""
cluster_name = "my-cluster"
}
Consul Cluster
module "consul_vpc" {
source = ""
name = var.cluster_name
}
module "consul_servers" {
source = ""
name = var.cluster_name
}
Consul VPC
module "consul_sg" {
source = ""
name = var.name
}
Consul Servers
resource
"aws_autoscaling_group"
"consul" {
name = var.name
}

30. Module Depth
Example
Module

31. Module Depth Consul Cluster
module "consul_vpc" {
source = ""
name = var.cluster_name
}
module "consul_sg" {
source = ""
name = var.cluster_name
vpc_id = module.consul_vpc.vpc_id
}
module "consul_servers" {
source = ""
name = var.cluster_name
vpc_id = module.consul_vpc.vpc_id
}

32. Version Your Modules
▪ PATCH version when you make
backwards compatible bug fixes
Example: Fix a typo
▪ MINOR version when you add
functionality in a backwards
compatible
Example: Add a new feature
▪ MAJOR version when you make
incompatible API changes
Example: New behavior
v1.0.0
Major Version Minor Version Patch Version

33. Module Upgrade Scenarios
v1.0.0 v1.0.1
v1.1.0
v2.0.0

34. Module Version Constraints
v1.0.0
v1.1.0
v2.0.0
v1.0.1
v1.1.1
v1.1.2
versions
version = "= 1.0.0"
version = "~> 1.0.0"
version = "~> 1.1.0"
version = ">= 2.0.0"
module "object_storage"
{
source = "PMR"
version = "???"
}
main.tf

35. Depreciate Module Versions
v1.0.0
v1.1.0
v2.0.0
v1.0.1
v1.1.1
v1.1.2
versions

36. Module Discoverability
Where to find existing, new, updated modules
Module Release Pipeline
Automation, gates, releases are critical
Module When Ready
Do not over complicate early on
Additional
Thoughts on
Scale
So many things
So little time

37. Thank You
37
@straubt1
@terraform-tom