Tips & tools on testing your terraform code and infrastructure

1. Testing & Deploying
Terraform
Tips & Tools
Farid Nouri Neshat
farid@olindata.com

2. In the begging
resource "aws_instance" "web" {
ami = "ami-00162fa"
instance_type = "t2.micro"
}

3. Just plan!
$ terraform plan
# aws_instance.web must be replaced
-/+ resource "aws_instance" "web" {
~ ami = "ami-00162fa" -> "ami-00162fb" # forces replacement
}
Plan: 1 to add, 0 to change, 1 to destroy.

4. But even successful plans fail!
$ terraform apply
aws_instance.web: Destroying... [id=i-067578c3e140d38e5]
aws_instance.web: Destruction complete after 30s
aws_instance.web: Creating...
Error: Error launching source instance: InvalidAMIID.Malformed:
Invalid id: "ami-00162fb"
on main.tf line 1, in resource "aws_instance" "web":
1: resource "aws_instance" "web" {

5. Terraform failure = Half formed infrastructure!
Taken from clipart.email

6. Let’s Start With Basics
Taken from clipart.email

7. Monitor your infrastructure
● Monitoring checks
● Log filters
● Alarms
● Dashboards with fancy graphs!

8. ● Monitoring

9. Use open source modules

10. Use open source modules
● Reduce development time
● Reduce code
● Reduce testing!

11. ● Monitoring
● Open Source Modules

13. Test environment
module "watcher_test" {
source = "../watcher"
name = "watcher-test"
s3_bucket = "od-watcher-test"
maximum = 1
}
module "watcher_prod" {
source = "../watcher"
name = "watcher-prod"
s3_bucket = "od-watcher-prod"
maximum = 10
}

14. ● Monitoring
● Open Source Modules
● Test Environment

15. Avoid using a single large terraform stack
● Reduce blast radius
● If possible break it up to independent stacks
● Use remote state data source provider for dependencies
● Smaller plan output
● Less changes
● Faster to plan and apply

16. ● Monitoring
● Open Source Modules
● Test Environment
● Multiple stacks

17. Lint your code
$ tflint --deep
1 issue(s) found:
Error: "ami-00162fa" is invalid AMI ID. (aws_instance_invalid_ami)
on a.tf line 3:
3: ami = "ami-00162fa"

18. ● Monitoring
● Open Source Modules
● Test Environment
● Multiple stacks
● Linting

19. Run Terraform in your CI tool
● Automation Reduces Mistakes
● You can review the code before it gets applied!
● Less latency so more efficient
● Will not get interrupted by leaving for lunch!
● Developers can deploy without admin access

20. Taken from runatlantis.io

21. ● Monitoring
● Open Source Modules
● Test Environment
● Multiple stacks
● Linting
● CI/CD

22. Apply policies
● terraform_validate
● conftest
● sentinel
● terraform-compliance

23. Sentinel Example: Enforce Tags
import "tfplan"
main = rule {
all tfplan.resources.aws_instance as _, instances {
all instances as _, r {
(length(r.applied.tags) else 0) > 0
}
}
}

24. Write tests for your modules
Tool Language
Terratest Go
Kitchen-Terraform + Inspec Ruby
tftest Python

25. Automated Tests
1. Run module with some input
2. Verify infrastructure state
3. Destroy

26. Terratest
terraformOptions := &terraform.Options {
// The path to where your Terraform code is located
TerraformDir: "../examples/terraform-basic-example",
}
// At the end of the test, run `terraform destroy` to clean up any resources
that were created
defer terraform.Destroy(t, terraformOptions)
// This will run `terraform init` and `terraform apply` and fail the test if
there are any errors
terraform.InitAndApply(t, terraformOptions)
// Validate your code works as expected
validateServerIsWorking(t, terraformOptions)

27. Tips
● Test various configurations that your module will be used
● Be aware of unique constraints
● Parallelism can speed things up
● Don’t pack too many things in the module

28. Testing Pyramid
E2E
Integration
Unit Tests
Linting

29. More?
Automated Testing
for Terraform,
Docker, Packer,
Kubernetes, and
More
By Yevgeniy Brikman

30. Questions?