Slides from my meetup talks at various AWS and DevOps meetups. Follow me: https://twitter.com/antonbabenko https://github.com/antonbabenko https://linkedin.com/in/antonbabenko

1. Terraform AWS modules and some best-
practices
Anton Babenko
@antonbabenko
September 2019

2. Anton Babenko
AWS Community Hero / Certified Terraform fanatic since 2015
Organiser of HashiCorp UG, AWS UG, DevOps Norway, DevOpsDays Oslo
I 💚 open-source:
terraform-community-modules + terraform-aws-modules
antonbabenko/pre-commit-terraform — clean code and documentation
antonbabenko/tfvars-annotations — update terraform.tfvars using annotations
antonbabenko/modules.tf-lambda — generate Terraform code from visual diagrams
antonbabenko/terragrunt-reference-architecture — Terragrunt reference architecture
www.terraform-best-practices.com
medium.com/@anton.babenko
@antonbabenko — Twitter, GitHub, Linkedin

3. What do I do?
All-things Terraform + AWS + DevOps
Consulting
Workshops
Trainings
Mentorship
My interview: https://medium.com/@anton.babenko/my-terraform-aws-journey-hashitimes-interview-73d1b542fcc0
My email: anton@antonbabenko.com
LinkedIn: https://www.linkedin.com/in/antonbabenko
@antonbabenko

4. Collection of open-source Terraform AWS modules supported by the community.
More than 5 mil. downloads since September 2017.
(VPC, Autoscaling, RDS, Security Groups, ELB, ALB, Redshift, SNS, SQS, IAM, EKS, ECS…)
github.com/terraform-aws-modules
registry.terraform.io/modules/terraform-aws-modules
@antonbabenko

5. Cloudcraft.co — the best way to draw AWS diagrams
@antonbabenko

6. cloudcraft.co features
• Manage components in browser (EC2 instances, autoscaling groups, RDS, etc)
• Connect components
• Import live AWS infrastructure
• Calculate the budget
• Share link to a blueprint
• Export as image
• Embed drawing to wiki, Confluence, etc
@antonbabenko

7. Infrastructure as code makes DevOps possible
Key benefits:
• Treat infrastructure like application code
• Always know what changed
• Validate infrastructure before deployment
https://dzone.com/articles/infrastructure-as-code-the-benefits @antonbabenko

8. – What is infrastructure?
“Infrastructure is all what is needed to support the computing
requirements of an application.”
@antonbabenko

9. Evolution — from manual to automated
Maturity and complexity grows:
Manual at first
Ad-hoc scripts
Virtualisation
IaaS, SaaS, PaaS, *aaS
Datacenter as computer
@antonbabenko

10. Configuration
Management Tools
Google Cloud
Deployment Manager
Azure Resource
Manager
@antonbabenko

11. +morethan250providers
@antonbabenko
Configuration
Management Tools

12. Why Terraform and not AWS CloudFormation,
Azure ARM, Google Cloud Deployment Manager?
Terraform supports 250+ providers, has easier syntax (HCL), has native support for
modules and remote states, has teamwork related features, is an open-source project
Provides a high-level abstraction of infrastructure, unifies the view of resources
Supports the modern datacenter (IaaS, PaaS, SaaS)
Separates planning from execution (dry-run)
Provides a workflow which is technology agnostic
Manages anything with an API
@antonbabenko

13. Terraform 0.11
@antonbabenko

14. Terraform 0.12
@antonbabenko

15. @antonbabenko

16. @antonbabenko

17. @antonbabenko

18. Terraform — universal tool for everything with an API
Google G Suite
Dropbox files and access
New Relic metrics
Datadog users and metrics
Jira issues
Minecraft, or even order Domino’s pizza
All Terraform providers — https://www.terraform.io/docs/providers/index.html
@antonbabenko

19. Let’s begin — everything fits in main.tf
@antonbabenko

20. @antonbabenko

21. Project grows — main.tf:
20+ resources and data sources
@antonbabenko

22. Why?
@antonbabenko

23. Resources, regions, providers, …
@antonbabenko

24. @antonbabenko

25. @antonbabenko

26. @antonbabenko

27. @antonbabenko

28. @antonbabenko

29. main.tf:
10-20 Kb
300+ LOC
@antonbabenko

30. Emerging issues
Code size is increasing
Dependencies between resources become complicated
@antonbabenko

31. Solution — Terraform modules
@antonbabenko

32. – What is Terraform module?
“Modules in Terraform are self-contained packages
of Terraform configurations that are managed as a group.”
@antonbabenko

33. Resource modules
Create resources in a very flexible configuration
Open-source
@antonbabenko

34. Resource modules
@antonbabenko

35. Resource modules
@antonbabenko

36. Resource modules
@antonbabenko

37. Resource modules
@antonbabenko

38. Would you use Terraform module to manage AWS
EC2 security group?
@antonbabenko

39. @antonbabenko

40. Would you use Terraform module to manage AWS EC2
security group?
In 0.11 — Yes, of course, sure and certainly!
In 0.12 — Yes!
@antonbabenko

41. Infrastructure modules
Also known as "curated modules" and "company-wide modules"
Consist of resource modules
Enforce tags and company standards
In 0.11 — use preprocessors, jsonnet, cookiecutter
In 0.12 — may implement complex logic (conditions, loops, nested
blocks)
@antonbabenko

42. Infrastructure modules
@antonbabenko

43. Infrastructure modules
@antonbabenko

44. Infrastructure modules
@antonbabenko

45. @antonbabenko

46. Types of Terraform modules
Resource modules (github.com/terraform-aws-modules , for eg)
Infrastructure modules
@antonbabenko

47. - [ ] How to write modules?
- [ ] How to call modules?
- [ ] How to work with the code?
@antonbabenko

48. Tip №0
Check Terraform Registry before writing resource modules
@antonbabenko

49. Hide implementation details
@antonbabenko

50. Size
@antonbabenko

51. Size
https://github.com/mbtproject/mbt
@antonbabenko

52. Things to avoid in modules
@antonbabenko

53. Exception: logical providers (template, random, local, http, external)
Providers in modules — evil
@antonbabenko

54. @antonbabenko

55. Provisioner — evil
Avoid provisioner in all resources
@antonbabenko

56. Provisioner — evil
Avoid provisioner in all resources
@antonbabenko

57. Provisioner — evil
Avoid provisioner even in EC2 resources
@antonbabenko

58. Provisioner — evil
Avoid provisioner even in EC2 resources
@antonbabenko

59. @antonbabenko

60. @antonbabenko

61. null_resource provisioner — good
@antonbabenko

62. Traits of good Terraform modules
Documentation and examples
Feature rich
Sane defaults
Clean code
Tests
Read more: http://bit.ly/common-traits-in-terraform-modules
@antonbabenko

63. - [x] How to write modules?
- [x] Do not write, if you can
- [x] Avoid providers in modules, provisioners
- [ ] How to call modules?
- [ ] How to work with the code?
@antonbabenko

64. How to structure Terraform
configurations? How to call them?
@antonbabenko

65. Call Terraform modules
Use Terraform modules, because amount of resources and code is
increasing
How to organize Terraform configurations and invoke them?
How to orchestrate modules?
@antonbabenko

66. All-in-one
Good:
Declare variables and outputs in
fewer places
Bad:
Large blast radius
Everything is blocked at once
Impossible to specify
dependencies between modules
(depends_on)
@antonbabenko

67. 1-in-1
Good:
Smaller blast radius
Possible to join invocation
Easier and faster to work with
Bad:
Declare variables and outputs in
more places
@antonbabenko

68. Which way do you group your code?
All-in-one or 1-in-1?
@antonbabenko

69. All-in-one 1-in-1
or
@antonbabenko

70. Correct
MFA (Most Frequent Answer):
Somewhere in between
@antonbabenko

71. All-in-one
Undefined project scope
Fast prototyping and initial
development phase
Small number of resources &
developers
Tightly connected resources
1-in-1
@antonbabenko
Defined project scope
Different types of developers
involved *
Code reuse is encouraged
(across organization and
environments)
Use Terragrunt

72. What about Terraform workspaces?
@antonbabenko

73. Problems with Terraform workspaces
Terraform Workspaces aren’t infrastructure-as-code friendly. You
can’t answer straight from the code:
"How many workspaces do you have?"
"What infrastructure has been deployed in workspaceX?"
"What is the difference between workspaceX and workspaceY?"
Introducing complexity almost in all cases.
@antonbabenko

74. Solution — use re-usable modules
instead of workspaces
@antonbabenko

75. What kind of orchestration method do you use?
-target
Makefile
…
@antonbabenko

76. Orchestration in Terraform
@antonbabenko

77. No really, do not try this at home!
@antonbabenko

78. Orchestration = Terragrunt
https://github.com/gruntwork-io/terragrunt/
@antonbabenko

79. Orchestration = Terragrunt
@antonbabenko

80. Orchestration = Terragrunt
@antonbabenko

81. Orchestration = Terragrunt
@antonbabenko

82. Orchestration = Terragrunt
@antonbabenko

83. tfvars can’t contain dynamic values :(
Orchestration = Terragrunt
@antonbabenko
subnet_id = "???"

84. Orchestration = Terragrunt
tfvars can’t contain dynamic values, so I
have fixed it :)
@antonbabenko
subnet_id = "???"

85. before_hook + shell script Go binary
https://github.com/antonbabenko/tfvars-annotations — Update values in
terraform.tfvars using annotations (WIP)
or take a look at modules.tf
@antonbabenko

86. Orchestration = Terragrunt
@antonbabenko
subnet_id = "" # @tfvars:terragrunt_output.vpc.public_subnets

87. Orchestration = Terragrunt
@antonbabenko
subnet_id = ["subnet-1a2b3c4d"] # @tfvars:terragrunt_output.vpc.public_subnets

88. - [x] How to write modules?
- [x] How to call modules?
- [x] 1-in-1 works beter over time
- [x] Orchestration = Terragrunt
- [x] Dynamic values in tfvars
- [ ] How to work with the code?
@antonbabenko

89. Integration
@antonbabenko

90. Integration
@antonbabenko

91. Auto-integration
@antonbabenko

92. Edge cases
Different AWS regions (version of S3 signature, EC2 ClassicLink,
IPv6)
Date of creation of AWS account
Limits on resources in AWS
Services and features availability
@antonbabenko

93. Avoid in Terraform
Not secret arguments should not be specified as command line
arguments => put them in tfvars
Reduce usage of "-target" and "-parallelism"
"Terraform workspaces" evil in=> separate by directories
Dependency hell in modules
@antonbabenko

94. - [x] How to write modules?
- [x] How to call modules?
- [x] How to work with the code?
- [ ] What about Terraform 0.12?
@antonbabenko

95. Terraform 0.12
…and why it is so important?
@antonbabenko

96. Terraform 0.12
HCL2 — simplified syntax
Loops ("for")
Dynamic blocks ("for_each")
Correct conditional operators (… ? … : …)
Extended types of variables
Templates in values
Links between resources are supported (depends_on everywhere)
Read more — https://www.hashicorp.com/blog/announcing-terraform-0-1-2-beta
@antonbabenko

97. Different types of Terraform users
@antonbabenko

98. Types of Terraform users
Terraform developers
Terraform users (everyone else)
@antonbabenko

99. Terraform developers
Write and support Terraform modules
Implement company’s standards (security, encryption, integrations)
Maintain reference architectures
@antonbabenko

100. Terraform users (everyone)
Use Terraform modules by specifying correct values
Domain experts
May not have "Terraform" in LinkedIn profile
@antonbabenko

101. Terraform 0.12 for developers
DevOps&Terraform developers
Allow to implement flexible/dynamic/reusable Terraform modules
@antonbabenko

102. Terraform 0.12 for users
Terraform users
Like HCL2 lightweight syntax more
@antonbabenko

103. F.A.Q.
@antonbabenko

104. How to handle secrets in Terraform?
• Can you accept secrets to be saved in state file in plaintext? Probably not.
• AWS IAM password & access secret keys — use PGP as keybase.io
• AWS RDS — set dummy password and change after DB is created
• AWS RDS — use iam_database_authentication_enabled = true
• EC2 instance user-data + AWS KMS
• EC2 instance user-data + AWS System Manager’s Parameter Store
• AWS Secrets Manager
• https://github.com/opencredo/terrahelp
• https://secrethub.io/ - A simple tool to keep secrets out of .tfvars and .tfstate files
• Other options:
• Secure remote state location (S3 bucket policy, KMS key)
• Terraform Saas (Cloud or Enterprise)
@antonbabenko

105. What are the tools/solutions out there?
• Terraform Registry (https://registry.terraform.io/) — collection of public Terraform
modules for common infrastructure configurations for any provider.
• Terraform linter to detect errors that can not be detected by `terraform plan` —
https://github.com/wata727/tflint
• Terraform version manager — https://github.com/kamatama41/tfenv
• Terraform modules manager — https://github.com/devopsmakers/xterrafile
• A web dashboard to inspect Terraform States — https://github.com/camptocamp/
terraboard
• Jsonnet — The data templating language — http://jsonnet.org
• pre-commit-terraform — pre-commit git hooks to take care of Terraform
configurations —https://github.com/antonbabenko/pre-commit-terraform/
@antonbabenko

106. Atlantis — Start working on Terraform as a team
A unified workflow for collaborating on Terraform through GitHub, GitLab and Bitbucket
https://www.runatlantis.io
@antonbabenko

107. What about testing?
@antonbabenko

108. Testing
@antonbabenko
InSpec
AWSSpec
Terratest — https://github.com/gruntwork-io/terratest
kitchen-terraform — https://github.com/newcontext-oss/kitchen-
terraform

109. What about FREE Sentinel compliance checks?
@antonbabenko

110. Compliance
@antonbabenko
Open Policy Agent — https://www.openpolicyagent.org/docs/latest/
terraform/
BDD style validation/compliancy check — https://github.com/
eerkunt/terraform-compliance

111. Some of Terraform Best Practices
https://www.terraform.io/
http://learn.hashicorp.com
https://www.terraform-best-practices.com/
+ ask me for more
@antonbabenko

112. Bonus

117. ✓ cloudcraft.co — design, plan and visualize
✓ terraform-aws-modules — building blocks of AWS infrastructure
✓ Terraform — infrastructure as code

118. Infrastructure as code generator — from visual diagrams to Terraform
https://github.com/antonbabenko/modules.tf-lambda
Demo video: https://www.youtube.com/watch?v=F1Ax1zfZbiY

119. 1. Go to cloudcraft.co
2. Sign up, sign in (free account)
3. Draw your AWS infrastructure
4. Click "Export"
5. Click "Terraform code export"
Try it yourself!

120. modules.tf — generated code
✓ Potentially ready-to-use Terraform configurations
✓ Suits best for bootstrapping
✓ Enforces Terraform best-practices
✓ Batteries included (terraform-aws-modules, terragrunt, tfvars-
annotations, pre-commit)
✓ 100% free and open-source (https://github.com/antonbabenko/
modules.tf-lambda)
✓ Released under MIT license

121. Thanks!
Questions?
github.com/antonbabenko
twitter.com/antonbabenko