Ridwan Fadjar Septian
DKATALIS
Development Toolbelt for
Building Terraform Module
We are working from anywhere, but our main
hubs are in Jakarta and Singapore.
Our remote workers are also located in
various countries such as Malaysia,
Thailand, Vietnam, Russia, Hungary,
Ukraine, and India.
A team of tech and digital product enthusiasts,
focusing on building scalable digital solutions.
OUR PRODUCT
Jago
Enhance the lives of
millions with a life-centric
financial solution app.
9 million users
PX
Support and simplify
work experiences with an
interactive &
human-centric HR app
OUR ROLES
The mastermind behind our digital
products. They shape product
vision, craft product requirements,
make prioritizations based on
customer needs, and ensure
everyone is on the same page for a
seamless development.
The brilliant analytical minds that
unleash the power of machine
learning to drive our business
forward, consisting of Data
Scientist and Machine Learning
Engineer.
The creative force sculpting our
stunning website and app
interfaces. They're on a mission to
make customer's journey delightful
by balancing aesthetics,
innovation, and usability.
Our digital builders, crafting the
backbone of our tech environment from
websites to apps. The Principal
Engineer, Software Engineer, SRE,
Cloud Infra Engineer, Data Engineer,
and Security Engineer ensure
everything runs smoothly, scales
seamlessly, and is built to last.
A. What is Terraform Module?
Explaining the Terraform Module concept
5
1. Modules are containers for multiple resources that are used
together
2. A module consists of a collection of .tf and/or .tf.json files kept
together in a directory.
3. Modules are the main way to package and reuse resource
configurations with Terraform.
Source: https://developer.hashicorp.com/terraform/language/modules
B. Terraform Module Common Structure
Basic structure of Terraform Module
README.md
LICENSE
Modules
Examples
Quickstart, dependencies, how to use, how to contribute, etc.
License chosed by the developer
Standalone module or collection of submodules
Examples for the TF module
6
B. Terraform Module Common Structure (2)
Common structure for module and submodule
main.tf
outputs.tf
README.md
variables.tf
Main source code for the TF module. Usually contain resource definitions for desired
modules
Contain output for each variable you defined under the TF module
Generated by TFDocs. Usually contain the information of the submodule
Set of variables required by TF module
versions.tf Contain dependency and providers for your TF module
7
B. Terraform Module Common Structure (3)
Common structure for module and submodule
Source: https://github.com/ridwanbejo/terraform-cassandra-admin
8
B. Terraform Module Common Structure (4)
Common structure for module and submodule
Source: https://github.com/ridwanbejo/terraform-cassandra-admin
9
B. Terraform Module Common Structure (5)
Common structure for module and submodule
Source: https://github.com/ridwanbejo/terraform-cassandra-admin
10
C. Tools that can help you out
I use these tools to build and test my Terraform Modules
Text Editor
Git
Terraform CLI
Terraform
Validate
Terraform
Format
TFLint
TFSec
Pre-commit
Github Action
Docker
Docker
Compose
TFDoc
11
C. Tools that can help you out (1) - Text Editor
I use Visual Studio Code and Hashicorp Terraform Extension for it
12
C. Tools that can help you out (1) - Text Editor
I use Visual Studio Code and Hashicorp Terraform Extension for it
13
C. Tools that can help you out (2) - Git (and Github)
Basic commands
$ git checkout -b feature/new-branch-name
$ git branch
$ git add .
$ git commit -m “feature || new change from me”
$ git push origin feature/new-branch-name
$ git log
$ git reset –hard HEAD~1
14
C. Tools that can help you out (2) - Git (and Github)
Github Repo
15
C. Tools that can help you out (2) - Git (and Github)
Github Issue Tracker
16
C. Tools that can help you out (2) - Git (and Github)
Github Pull Request
17
C. Tools that can help you out (2) - Git (and Github)
Github Tags
18
C. Tools that can help you out (2) - Git (and Github)
Github Releases. And many others
19
C. Tools that can help you out (3) - Terraform CLI
Terraform CLI at a glance
20
C. Tools that can help you out (3) - Terraform CLI
Terraform CLI at a glance
21
C. Tools that can help you out (3) - Terraform CLI
How to install?
$ brew tap hashicorp/tap
$ brew install hashicorp/tap/terraform
Mac OS
Ubuntu / Debian
Windows
Other installation https://developer.hashicorp.com/terraform/install
$ wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o
/usr/share/keyrings/hashicorp-archive-keyring.gpg
$ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com
$(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
$ sudo apt update && sudo apt install terraform
Windows Installer
22
C. Tools that can help you out (4) - Terraform Validate
Output when your codebase is failed to be validated
23
C. Tools that can help you out (4) - Terraform Validate
When your codebase successfully validated
24
C. Tools that can help you out (5) - Terraform Format
Terraform Format when not detecting any formatting errors
25
C. Tools that can help you out (5) - Terraform Format
Terraform Format when found errors
26
C. Tools that can help you out (5) - Terraform Format
Terraform Format when found errors
27
C. Tools that can help you out (5) - Terraform Format
Terraform Format when found errors
28
C. Tools that can help you out (5) - Terraform Format
Terraform Format when found errors
29
C. Tools that can help you out (6) - TFSec
If you work with AWS, GCP, Azure, Kubernetes and others. You can use TFSec if the cloud is supported by TFSec
Checks
Sources:
- https://github.com/aquasecurity/tfsec
- https://aquasecurity.github.io/tfsec/latest/guides/trivy/
30
C. Tools that can help you out (6) - TFSec
If you work with AWS, GCP, Azure, Kubernetes and others. You can use TFSec if the cloud is supported by TFSec
Checks
Sources:
- https://github.com/aquasecurity/tfsec
- https://aquasecurity.github.io/tfsec/latest/guides/trivy/
31
C. Tools that can help you out (6) - TFSec
If you work with AWS, GCP, Azure, Kubernetes and others. You can use TFSec if the cloud is supported by TFSec
Checks
Sources:
- https://github.com/aquasecurity/tfsec
- https://aquasecurity.github.io/tfsec/latest/guides/trivy/
32
C. Tools that can help you out (7) - Terraform Docs
Generate docs for your Terraform code automatically
Source: https://terraform-docs.io/user-guide/introduction/
33
C. Tools that can help you out (7) - Terraform Docs
Generate docs for your Terraform code automatically
Source: https://terraform-docs.io/user-guide/introduction/
34
C. Tools that can help you out (8) - Pre-commit
You can think pre-commit is local CI pipeline which triggered when attempting to commit your changes
35
Source: https://pre-commit.com/
C. Tools that can help you out (8) - Pre-commit
You can think pre-commit is local CI pipeline which triggered when attempting to commit your changes
36
Source: https://pre-commit.com/
C. Tools that can help you out (9) - Docker & Docker-Compose
Use Docker and Docker-Compose to prepare your sandbox for local development
37
C. Tools that can help you out (9) - Docker & Docker-Compose
Use Docker and Docker-Compose to prepare your sandbox for local development
38
C. Tools that can help you out (9) - Docker & Docker-Compose
Use Docker and Docker-Compose to prepare your sandbox for local development
39
C. Tools that can help you out (9) - Docker & Docker-Compose
Use Docker and Docker-Compose to prepare your sandbox for local development
40
C. Tools that can help you out (10) - Github Action
CI Pipeline for your Terraform
41
C. Tools that can help you out (10) - Github Action
CI Pipeline for your Terraform
42
C. Tools that can help you out (10) - Github Action
CI Pipeline for your Terraform
43
C. Tools that can help you out (10) - Github Action
CI Pipeline for your Terraform
44
C. Tools that can help you out (10) - Github Action
CI Pipeline for your Terraform
45
C. Tools that can help you out (10) - Github Action
CI Pipeline for your Terraform
46
C. Tools that can help you out (10) - Github Action
CI Pipeline for your Terraform
47
C. Tools that can help you out (11) - TFLint
This tool have ruleset for terraform, aws, azure and gcp. It has more ruleset. I’m still exploring this tool too.
source: https://github.com/terraform-linters/tflint
48
C. Tools that can help you out (11) - TFLint
This tool have ruleset for terraform, aws, azure and gcp. It has more ruleset. I’m still exploring this tool too.
source: https://github.com/terraform-linters/tflint
49
C. Tools that can help you out (12) - Terraform Test
Test feature that seems like unit test in certain programming languages. I’m still exploring this tool too.
Source: https://developer.hashicorp.com/terraform/language/v1.7.x/tests#syntax
50
D. Testing the TF Module locally
Test the TF Cassandra module locally - setup providers.tf with localhost Cassandra
51
D. Testing the TF Module locally (2)
Test the TF Cassandra module locally - terraform.tfvars
52
D. Testing the TF Module locally (3)
Test the TF Cassandra module locally - terraform plan
53
D. Testing the TF Module locally (3)
Test the TF Cassandra module locally - terraform apply -auto-approve
54
D. Testing the TF Module locally (3)
Test the TF Cassandra module locally - terraform apply result
55
D. Testing the TF Module locally (4)
Test the TF Cassandra module locally - terraform destroy
56
D. Testing the TF Module locally (5)
Test the TF Cassandra module locally - terraform destroy result
57
E. Publish the TF Module to Terraform Registry
Create new tags and draft new release
58
E. Publish the TF Module to Terraform Registry (2)
New release ready, go to Terraform Registry page
59
E. Publish the TF Module to Terraform Registry (3)
Login to Terraform Registry with your Github Account
60
E. Publish the TF Module to Terraform Registry (4)
If login succes, go to Profile page
61
E. Publish the TF Module to Terraform Registry (5)
Now you are in profile page. Let’s publish the TF module
62
E. Publish the TF Module to Terraform Registry (6)
Click Publish > Module menu
63
E. Publish the TF Module to Terraform Registry (7)
Choose repo and the targeted release. If you already published it. You will see this picture below
64
E. Publish the TF Module to Terraform Registry (8)
You can see the detail of Terraform Module in this kind of page
65
F. Summary
Recap for the tools I mentioned before
Development
Code Quality
Security
Automation
Terraform CLI, Text Editor, Git, Docker, Docker-Compose, TFDocs
Terraform Validate, Terraform Format, TFLint
TFSec
Pre-commit, Github Action
66
Thank you!
Q&A

My Hashitalk Indonesia April 2024 Presentation

  • 1.
    Ridwan Fadjar Septian DKATALIS DevelopmentToolbelt for Building Terraform Module
  • 2.
    We are workingfrom anywhere, but our main hubs are in Jakarta and Singapore. Our remote workers are also located in various countries such as Malaysia, Thailand, Vietnam, Russia, Hungary, Ukraine, and India. A team of tech and digital product enthusiasts, focusing on building scalable digital solutions.
  • 3.
    OUR PRODUCT Jago Enhance thelives of millions with a life-centric financial solution app. 9 million users PX Support and simplify work experiences with an interactive & human-centric HR app
  • 4.
    OUR ROLES The mastermindbehind our digital products. They shape product vision, craft product requirements, make prioritizations based on customer needs, and ensure everyone is on the same page for a seamless development. The brilliant analytical minds that unleash the power of machine learning to drive our business forward, consisting of Data Scientist and Machine Learning Engineer. The creative force sculpting our stunning website and app interfaces. They're on a mission to make customer's journey delightful by balancing aesthetics, innovation, and usability. Our digital builders, crafting the backbone of our tech environment from websites to apps. The Principal Engineer, Software Engineer, SRE, Cloud Infra Engineer, Data Engineer, and Security Engineer ensure everything runs smoothly, scales seamlessly, and is built to last.
  • 5.
    A. What isTerraform Module? Explaining the Terraform Module concept 5 1. Modules are containers for multiple resources that are used together 2. A module consists of a collection of .tf and/or .tf.json files kept together in a directory. 3. Modules are the main way to package and reuse resource configurations with Terraform. Source: https://developer.hashicorp.com/terraform/language/modules
  • 6.
    B. Terraform ModuleCommon Structure Basic structure of Terraform Module README.md LICENSE Modules Examples Quickstart, dependencies, how to use, how to contribute, etc. License chosed by the developer Standalone module or collection of submodules Examples for the TF module 6
  • 7.
    B. Terraform ModuleCommon Structure (2) Common structure for module and submodule main.tf outputs.tf README.md variables.tf Main source code for the TF module. Usually contain resource definitions for desired modules Contain output for each variable you defined under the TF module Generated by TFDocs. Usually contain the information of the submodule Set of variables required by TF module versions.tf Contain dependency and providers for your TF module 7
  • 8.
    B. Terraform ModuleCommon Structure (3) Common structure for module and submodule Source: https://github.com/ridwanbejo/terraform-cassandra-admin 8
  • 9.
    B. Terraform ModuleCommon Structure (4) Common structure for module and submodule Source: https://github.com/ridwanbejo/terraform-cassandra-admin 9
  • 10.
    B. Terraform ModuleCommon Structure (5) Common structure for module and submodule Source: https://github.com/ridwanbejo/terraform-cassandra-admin 10
  • 11.
    C. Tools thatcan help you out I use these tools to build and test my Terraform Modules Text Editor Git Terraform CLI Terraform Validate Terraform Format TFLint TFSec Pre-commit Github Action Docker Docker Compose TFDoc 11
  • 12.
    C. Tools thatcan help you out (1) - Text Editor I use Visual Studio Code and Hashicorp Terraform Extension for it 12
  • 13.
    C. Tools thatcan help you out (1) - Text Editor I use Visual Studio Code and Hashicorp Terraform Extension for it 13
  • 14.
    C. Tools thatcan help you out (2) - Git (and Github) Basic commands $ git checkout -b feature/new-branch-name $ git branch $ git add . $ git commit -m “feature || new change from me” $ git push origin feature/new-branch-name $ git log $ git reset –hard HEAD~1 14
  • 15.
    C. Tools thatcan help you out (2) - Git (and Github) Github Repo 15
  • 16.
    C. Tools thatcan help you out (2) - Git (and Github) Github Issue Tracker 16
  • 17.
    C. Tools thatcan help you out (2) - Git (and Github) Github Pull Request 17
  • 18.
    C. Tools thatcan help you out (2) - Git (and Github) Github Tags 18
  • 19.
    C. Tools thatcan help you out (2) - Git (and Github) Github Releases. And many others 19
  • 20.
    C. Tools thatcan help you out (3) - Terraform CLI Terraform CLI at a glance 20
  • 21.
    C. Tools thatcan help you out (3) - Terraform CLI Terraform CLI at a glance 21
  • 22.
    C. Tools thatcan help you out (3) - Terraform CLI How to install? $ brew tap hashicorp/tap $ brew install hashicorp/tap/terraform Mac OS Ubuntu / Debian Windows Other installation https://developer.hashicorp.com/terraform/install $ wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg $ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list $ sudo apt update && sudo apt install terraform Windows Installer 22
  • 23.
    C. Tools thatcan help you out (4) - Terraform Validate Output when your codebase is failed to be validated 23
  • 24.
    C. Tools thatcan help you out (4) - Terraform Validate When your codebase successfully validated 24
  • 25.
    C. Tools thatcan help you out (5) - Terraform Format Terraform Format when not detecting any formatting errors 25
  • 26.
    C. Tools thatcan help you out (5) - Terraform Format Terraform Format when found errors 26
  • 27.
    C. Tools thatcan help you out (5) - Terraform Format Terraform Format when found errors 27
  • 28.
    C. Tools thatcan help you out (5) - Terraform Format Terraform Format when found errors 28
  • 29.
    C. Tools thatcan help you out (5) - Terraform Format Terraform Format when found errors 29
  • 30.
    C. Tools thatcan help you out (6) - TFSec If you work with AWS, GCP, Azure, Kubernetes and others. You can use TFSec if the cloud is supported by TFSec Checks Sources: - https://github.com/aquasecurity/tfsec - https://aquasecurity.github.io/tfsec/latest/guides/trivy/ 30
  • 31.
    C. Tools thatcan help you out (6) - TFSec If you work with AWS, GCP, Azure, Kubernetes and others. You can use TFSec if the cloud is supported by TFSec Checks Sources: - https://github.com/aquasecurity/tfsec - https://aquasecurity.github.io/tfsec/latest/guides/trivy/ 31
  • 32.
    C. Tools thatcan help you out (6) - TFSec If you work with AWS, GCP, Azure, Kubernetes and others. You can use TFSec if the cloud is supported by TFSec Checks Sources: - https://github.com/aquasecurity/tfsec - https://aquasecurity.github.io/tfsec/latest/guides/trivy/ 32
  • 33.
    C. Tools thatcan help you out (7) - Terraform Docs Generate docs for your Terraform code automatically Source: https://terraform-docs.io/user-guide/introduction/ 33
  • 34.
    C. Tools thatcan help you out (7) - Terraform Docs Generate docs for your Terraform code automatically Source: https://terraform-docs.io/user-guide/introduction/ 34
  • 35.
    C. Tools thatcan help you out (8) - Pre-commit You can think pre-commit is local CI pipeline which triggered when attempting to commit your changes 35 Source: https://pre-commit.com/
  • 36.
    C. Tools thatcan help you out (8) - Pre-commit You can think pre-commit is local CI pipeline which triggered when attempting to commit your changes 36 Source: https://pre-commit.com/
  • 37.
    C. Tools thatcan help you out (9) - Docker & Docker-Compose Use Docker and Docker-Compose to prepare your sandbox for local development 37
  • 38.
    C. Tools thatcan help you out (9) - Docker & Docker-Compose Use Docker and Docker-Compose to prepare your sandbox for local development 38
  • 39.
    C. Tools thatcan help you out (9) - Docker & Docker-Compose Use Docker and Docker-Compose to prepare your sandbox for local development 39
  • 40.
    C. Tools thatcan help you out (9) - Docker & Docker-Compose Use Docker and Docker-Compose to prepare your sandbox for local development 40
  • 41.
    C. Tools thatcan help you out (10) - Github Action CI Pipeline for your Terraform 41
  • 42.
    C. Tools thatcan help you out (10) - Github Action CI Pipeline for your Terraform 42
  • 43.
    C. Tools thatcan help you out (10) - Github Action CI Pipeline for your Terraform 43
  • 44.
    C. Tools thatcan help you out (10) - Github Action CI Pipeline for your Terraform 44
  • 45.
    C. Tools thatcan help you out (10) - Github Action CI Pipeline for your Terraform 45
  • 46.
    C. Tools thatcan help you out (10) - Github Action CI Pipeline for your Terraform 46
  • 47.
    C. Tools thatcan help you out (10) - Github Action CI Pipeline for your Terraform 47
  • 48.
    C. Tools thatcan help you out (11) - TFLint This tool have ruleset for terraform, aws, azure and gcp. It has more ruleset. I’m still exploring this tool too. source: https://github.com/terraform-linters/tflint 48
  • 49.
    C. Tools thatcan help you out (11) - TFLint This tool have ruleset for terraform, aws, azure and gcp. It has more ruleset. I’m still exploring this tool too. source: https://github.com/terraform-linters/tflint 49
  • 50.
    C. Tools thatcan help you out (12) - Terraform Test Test feature that seems like unit test in certain programming languages. I’m still exploring this tool too. Source: https://developer.hashicorp.com/terraform/language/v1.7.x/tests#syntax 50
  • 51.
    D. Testing theTF Module locally Test the TF Cassandra module locally - setup providers.tf with localhost Cassandra 51
  • 52.
    D. Testing theTF Module locally (2) Test the TF Cassandra module locally - terraform.tfvars 52
  • 53.
    D. Testing theTF Module locally (3) Test the TF Cassandra module locally - terraform plan 53
  • 54.
    D. Testing theTF Module locally (3) Test the TF Cassandra module locally - terraform apply -auto-approve 54
  • 55.
    D. Testing theTF Module locally (3) Test the TF Cassandra module locally - terraform apply result 55
  • 56.
    D. Testing theTF Module locally (4) Test the TF Cassandra module locally - terraform destroy 56
  • 57.
    D. Testing theTF Module locally (5) Test the TF Cassandra module locally - terraform destroy result 57
  • 58.
    E. Publish theTF Module to Terraform Registry Create new tags and draft new release 58
  • 59.
    E. Publish theTF Module to Terraform Registry (2) New release ready, go to Terraform Registry page 59
  • 60.
    E. Publish theTF Module to Terraform Registry (3) Login to Terraform Registry with your Github Account 60
  • 61.
    E. Publish theTF Module to Terraform Registry (4) If login succes, go to Profile page 61
  • 62.
    E. Publish theTF Module to Terraform Registry (5) Now you are in profile page. Let’s publish the TF module 62
  • 63.
    E. Publish theTF Module to Terraform Registry (6) Click Publish > Module menu 63
  • 64.
    E. Publish theTF Module to Terraform Registry (7) Choose repo and the targeted release. If you already published it. You will see this picture below 64
  • 65.
    E. Publish theTF Module to Terraform Registry (8) You can see the detail of Terraform Module in this kind of page 65
  • 66.
    F. Summary Recap forthe tools I mentioned before Development Code Quality Security Automation Terraform CLI, Text Editor, Git, Docker, Docker-Compose, TFDocs Terraform Validate, Terraform Format, TFLint TFSec Pre-commit, Github Action 66
  • 67.