SlideShare a Scribd company logo
1 of 150
Reusable, composable, battle-tested
TERRAFORM
MODULES
gruntwork.io
Your project is code complete
and it’s time to deploy it!
I know, I’ll use AWS!
You login to the AWS console
(Spend hours reading docs)
OK, I have a server running!
What else do I need?
Well, you probably want more than
one server for high availability
And a load balancer to distribute
traffic across those servers
And EBS Volumes and RDS databases
to store all of your data
You’ll need an S3 bucket for files
CloudWatch for monitoring
Don’t forget the VPC, subnets, route
tables, NAT gateways
Route 53 for DNS
ACM for SSL/TLS certs and KMS to
encrypt / decrypt secrets
And you need all of that in separate
environments for stage and prod
stage prod
Plus DevOps tooling to manage it all
stage prod
And a CI server to test it all
stage prod
Plus alerts and on-call rotation to
notify you when it all breaks
stage prod
stage prod
And also…
And you have to maintain it all.
Forever.
AWS: 1,000 new releases in 2016
Terraform: release every ~2 weeks
Security vulnerabilities: daily
There’s a better way to deploy
and manage infrastructure:
TERRAFORM
MODULES
TERRAFORM
MODULES
Reusable, composable, battle-tested
infrastructure code
In this talk, I’ll show you how
Terraform Modules work
stage prod
And how they will allow you to do all
of this…
> terraform init <…>
> terraform apply
In just a few simple commands
I’m
Yevgeniy
Brikman
ybrikman.com
Author
Co-founder of
Gruntwork
gruntwork.io
Your entire AWS
infrastructure.
Defined as code.
In about a day.
gruntwork.io
1. What’s a Module
2. How to use a Module
3. How Modules work
4. The future of Modules
Outline
1. What’s a Module
2. How to use a Module
3. How Modules work
4. The future of Modules
Outline
The two primary types of
infrastructure providers:
Infrastructure as a Service (IaaS): e.g.,
AWS, Azure, Google Cloud
CPU Memory Disk Drive Network Server DB
They offer many primitives and it’s up
to you to put them together
CPU Memory Disk Drive Network Server DB
Platform as a Service (PaaS): e.g.,
Heroku, Docker Cloud, Engine Yard
Rails MySQL GitHub
CPU Memory Disk Drive Network Server DB
They hide all the lower-level details so
you can focus on your apps
Rails MySQL GitHub
CPU Memory Disk Drive Network Server DB
> cd my-nodejs-app
> heroku create my-nodejs-app
> git push heroku master
Getting started with a PaaS is
easy!
Heroku limitations
1. Can only use supported runtimes & versions (e.g., python-3.6.2 or python-2.7.13)
2. Can only use supported system software & libraries
3. Can only run web services (data stores and other services available only via paid add-ons)
4. Apps can’t access the shell
5. Devs can’t access servers via SSH
6. Local disk is read-only
7. Load balancing is HTTP/HTTPS only
8. Requests are limited to 30 seconds
9. Limited to one AWS region
10. App must boot in 60 seconds or less
11. Apps can be at most 100MB
12. Build must take less than 15 min
13. Logs are limited to 1500 lines unless you use supported (paid) add-ons
14. Manual scaling only
15. Pricing gets very steep as you scale up
16. Support only available on PST time zone
17. Limited control over security settings
However, customizing, debugging,
and scaling is not.
For most software companies, IaaS is
the only way to grow
CPU Memory Disk Drive Network Server DB
stage prod
But that requires dealing with this…
We are developers.
We know how to fix this.
Use code!
Terraform is a tool for defining and
managing infrastructure as code
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "example" {
ami = "ami-408c7f28"
instance_type = "t2.micro"
}
Example: Terraform code to
deploy a server in AWS
> terraform apply
aws_instance.example: Creating...
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>”
aws_instance.example: Creation complete
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Run terraform apply to deploy
the server
Terraform supports modules
They are like reusable blueprints
for your infrastructure
resource "aws_autoscaling_group" "example" {
name = "${var.name}-service"
min_size = "${var.num_instances}"
max_size = "${var.num_instances}"
}
resource "aws_launch_configuration" "example" {
image_id = "${var.image_id}"
instance_type = "${var.instance_type}"
root_block_device {
volume_type = "gp2"
volume_size = 200
}
}
Example: create a module to
deploy a microservice
module "service_foo" {
source = "/modules/microservice"
image_id = "ami-123asd1"
num_instances = 3
}
Now you can use the module to
deploy one microservice
module "service_foo" {
source = "/modules/microservice"
image_id = "ami-123asd1"
num_instances = 3
}
module "service_bar" {
source = "/modules/microservice"
image_id = "ami-f2bb05ln"
num_instances = 6
}
module "service_baz" {
source = "/modules/microservice"
image_id = "ami-ny6v24xa"
num_instances = 3
}
Or multiple microservices
Modules allow you to use your
favorite IaaS provider…
CPU Memory Disk Drive Network Server DB
With easy-to-use, high-level
abstractions, like a PaaS
CPU Memory Disk Drive Network Server DB
Rails MySQL GitHub
But since you have all the code, you
still have full control!
CPU Memory Disk Drive Network Server DB
Rails MySQL GitHub
Best of all: code can be shared
and re-used!
The Terraform Module Registry
A collection of reusable, verified,
supported Modules
Example: Vault Module for AWS
Example: Consul for Azure
Example: Nomad for GCP
1. What’s a Module
2. How to use a Module
3. How Modules work
4. The future of Modules
Outline
Imagine you wanted to deploy Vault
The old way:
1. Open up the Vault docs
2. Deploy a few servers
3. Install Vault
4. Install supervisord
5. Configure mlock
6. Generate self-signed TLS cert
7. Create Vault config file
8. Create an S3 bucket as storage backend
9. Figure out IAM policies for the S3 bucket
10. Tinker with security group rules
11. Figure out IP addresses to bind to and advertise
12. Fight with Vault for hours because it won’t accept your TLS cert
13. Regenerate cert with RSA encryption
14. Update OS certificate store to accept self-signed certs
15. Realize you need to deploy a Consul cluster for high availability
16. Open up the Consul docs…
The new way:
> terraform init hashicorp/vault/aws
> terraform apply
And now you have this deployed
As simple as a PaaS!
> tree
.
├── README.md
├── main.tf
├── outputs.tf
├── packer
├── user-data
└── variables.tf
But you also have all the code.
Feel free to edit it!
module "vault_cluster" {
source = "hashicorp/vault/aws"
cluster_name = "example-vault-cluster"
cluster_size = 3
vpc_id = "${data.aws_vpc.default.id}"
subnet_ids = "${data.aws_subnets.default.ids}"
}
module "consul_cluster" {
source = "hashicorp/consul/aws"
cluster_name = "example-consul-cluster"
cluster_size = 3
vpc_id = "${data.aws_vpc.default.id}"
subnet_ids = "${data.aws_subnets.default.ids}"
}Example: modify main.tf
> terraform apply
Run apply when you’re done!
1. What’s a Module
2. How to use a Module
3. How Modules work
4. The future of Modules
Outline
def add(x, y):
return x + y
Most programming languages
support functions
def add(x, y):
return x + y
The function has a name
def add(x, y):
return x + y
It can take in inputs
def add(x, y):
return x + y
And it can return outputs
def add(x, y):
return x + y
add(3, 5)
add(10, 35)
add(-45, 6)
Key idea: code reuse
def add(x, y):
return x + y
assert add(3, 5) == 8
assert add(10, 35) == 45
assert add(-45, 6) == -39
Key idea: testing
def add(x, y):
return x + y
def sub(x, y):
return x - y
sub(add(5, 3), add(4, 7))
Key idea: composition
def run_classifier(data_set):
X_pca = PCA(n_components=2).fit_transform(X_train)
clusters = clf.fit_predict(X_train)
fig, ax = plt.subplots(1, 2, figsize=(8, 4))
fig.subplots_adjust(top=0.85)
predicted = svc_model.predict(X_test)
images_and_predictions = list(zip(images_test, predicted))
ax[0].scatter(X_pca[:, 0], X_pca[:, 1], c=clusters)
ax[0].set_title('Predicted Training Labels')
ax[1].scatter(X_pca[:, 0], X_pca[:, 1], c=y_train)
ax[1].set_title('Actual Training Labels')
Key idea: abstraction
def run_classifier(data_set):
X_pca = PCA(n_components=2).fit_transform(X_train)
clusters = clf.fit_predict(X_train)
fig, ax = plt.subplots(1, 2, figsize=(8, 4))
fig.subplots_adjust(top=0.85)
predicted = svc_model.predict(X_test)
images_and_predictions = list(zip(images_test, predicted))
ax[0].scatter(X_pca[:, 0], X_pca[:, 1], c=clusters)
ax[0].set_title('Predicted Training Labels')
ax[1].scatter(X_pca[:, 0], X_pca[:, 1], c=y_train)
ax[1].set_title('Actual Training Labels')You want to hide a large
“volume”…
def run_classifier(data_set):
X_pca = PCA(n_components=2).fit_transform(X_train)
clusters = clf.fit_predict(X_train)
fig, ax = plt.subplots(1, 2, figsize=(8, 4))
fig.subplots_adjust(top=0.85)
predicted = svc_model.predict(X_test)
images_and_predictions = list(zip(images_test, predicted))
ax[0].scatter(X_pca[:, 0], X_pca[:, 1], c=clusters)
ax[0].set_title('Predicted Training Labels')
ax[1].scatter(X_pca[:, 0], X_pca[:, 1], c=y_train)
ax[1].set_title('Actual Training Labels')
Behind a small “surface area”
Modules are Terraform’s
equivalent of functions
A simple module:
> tree minimal-module
.
├── main.tf
├── outputs.tf
├── variables.tf
└── README.md
It’s just Terraform code in a folder!
variable "name" {
description = "The name of the EC2 instance"
}
variable "image_id" {
description = "The ID of the AMI to run"
}
variable "port" {
description = "The port to listen on for HTTP requests"
}
The inputs are in variables.tf
output "url" {
value = "http://${aws_instance.example.ip}:${var.port}"
}
The outputs are in outputs.tf
resource "aws_autoscaling_group" "example" {
name = "${var.name}-service"
min_size = "${var.num_instances}"
max_size = "${var.num_instances}"
}
resource "aws_launch_configuration" "example" {
image_id = "${var.image_id}"
instance_type = "${var.instance_type}"
root_block_device {
volume_type = "gp2"
volume_size = 200
}
}The resources are in main.tf
# Foo Module for AWS
This is a Terraform Module to deploy
[Foo](http://www.example.com) on AWS, including:
* foo
* bar
* baz
Documentation is in README.md
A more complicated module:
> tree complete-module
.
├── main.tf
├── outputs.tf
├── variables.tf
├── README.MD
├── modules
├── examples
└── test
We add three new folders:
modules, examples, test
> tree complete-module/modules
modules/
├── submodule-bar
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
└── submodule-foo
├── main.tf
├── outputs.tf
└── variables.tf
The modules folder contains
standalone “submodules”
> tree complete-module/modules
modules/
├── submodule-bar
│ ├── install-vault.sh
│ └── run-vault.sh
└── submodule-foo
└── main.go
Some of the submodules may not
even be Terraform code
For example, one submodule can be
used to install Vault in an AMI
Another to create self-signed TLS
certificates
Another to deploy the AMI across an
Auto Scaling Group (ASG)
Another to create an S3 bucket and
IAM policies as a storage backend
Another to configure the Security
Group settings
And one more to deploy a load
balancer (ELB)
You can use all the submodules
Or pick the ones you want and swap
in your own for the rest
> tree complete-module/examples
examples/
├── example-foo
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
└── example-bar
├── main.tf
├── outputs.tf
└── variables.tf
The examples folder shows how to
use the submodules
Note: the code in the root is
usually a “canonical” example
> tree complete-module
.
├── main.tf
├── outputs.tf
├── variables.tf
├── README.MD
├── modules
├── examples
└── test
It’s typically an opinionated way to
use all the submodules together
> tree complete-module/examples
examples/
├── example-foo
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
└── example-bar
├── main.tf
├── outputs.tf
└── variables.tf
The code in examples shows other
possible permutations
E.g., How to use just one or two of the
submodules together
Or how to combine with other
modules (e.g., Vault + Consul)
This is like function composition!
> tree complete-module/test
test/
├── example_foo_test.go
└── example_bar_test.go
The test folder contains
automated tests
> tree complete-module/test
test/
├── example_foo_test.go
└── example_bar_test.go
The tests are typically “integration
tests”
func vaultTest(t *testing.T, options *terratest.Options) {
tlsCert := generateSelfSignedTlsCert(t)
defer cleanupSelfSignedTlsCert(t, tlsCert)
amiId := buildVaultAmi(t)
defer cleanupAmi(t, amiId)
terratest.Apply(options)
defer terratest.Destroy(options)
assertCanInitializeAndUnsealVault(t, options)
}
Example test case for Vault
func vaultTest(t *testing.T, options *terratest.Options) {
tlsCert := generateSelfSignedTlsCert(t)
defer cleanupSelfSignedTlsCert(t, tlsCert)
amiId := buildVaultAmi(t)
defer cleanupAmi(t, amiId)
terratest.Apply(options)
defer terratest.Destroy(options)
assertCanInitializeAndUnsealVault(t, options)
}
Create test-time resources
func vaultTest(t *testing.T, options *terratest.Options) {
tlsCert := generateSelfSignedTlsCert(t)
defer cleanupSelfSignedTlsCert(t, tlsCert)
amiId := buildVaultAmi(t)
defer cleanupAmi(t, amiId)
terratest.Apply(options)
defer terratest.Destroy(options)
assertCanInitializeAndUnsealVault(t, options)
}
Run terraform apply
func vaultTest(t *testing.T, options *terratest.Options) {
tlsCert := generateSelfSignedTlsCert(t)
defer cleanupSelfSignedTlsCert(t, tlsCert)
amiId := buildVaultAmi(t)
defer cleanupAmi(t, amiId)
terratest.Apply(options)
defer terratest.Destroy(options)
assertCanInitializeAndUnsealVault(t, options)
}
Run terraform destroy at the end
func vaultTest(t *testing.T, options *terratest.Options) {
tlsCert := generateSelfSignedTlsCert(t)
defer cleanupSelfSignedTlsCert(t, tlsCert)
amiId := buildVaultAmi(t)
defer cleanupAmi(t, amiId)
terratest.Apply(options)
defer terratest.Destroy(options)
assertCanInitializeAndUnsealVault(t, options)
}
Check the Vault cluster works!
Using a module:
module "service_foo" {
source = "./minimal-module"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
For simple Modules and learning,
deploy the root
module "submodule_foo" {
source = "./complete-module/modules/submodule-foo"
param_foo = "foo"
param_bar = 8080
}
module "submodule_bar" {
source = "./complete-module/modules/submodule-bar"
param_foo = "abcdef"
param_bar = 9091
}
For more complicated use-cases,
use the submodules
module "service_foo" {
source = "./minimal-module"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
Abstraction: simple Module API
for complicated infrastructure
module "service_foo" {
source = "./minimal-module"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
module "service_bar" {
source = "./minimal-module"
name = "Bar"
image_id = "ami-abcd1234"
port = 9091
}
Re-use: create a Module once,
deploy it many times
Versioning:
module "service_foo" {
source = "./foo"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
You can set source to point to
modules at local file paths
module "service_foo" {
source = "hashicorp/vault/aws"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
Alternatively, you can use
Terraform registry URLs
module "service_foo" {
source = "git::git@github.com:foo/bar.git"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
Or arbitrary Git URLs
module "service_foo" {
source = "git::git@github.com:foo/bar.git?ref=v1.0.0"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
You can even link to a specific Git
tag (recommended!)
module "service_foo" {
source = "git::git@github.com:foo/bar.git?ref=v1.0.0"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
Modules use semantic versioning
module "service_foo" {
source = "git::git@github.com:foo/bar.git?ref=v2.0.0"
name = "Foo"
image_id = "ami-123asd1"
port = 8080
}
So upgrading infrastructure is just
a version number bump
Promote immutable, versioned
infrastructure across environments
qa
stage
prod
1. What’s a Module
2. How to use a Module
3. How Modules work
4. The future of Modules
Outline
“You are not special.
Your infrastructure is
not a beautiful and
unique snowflake. You
have the same tech debt
as everyone else.”
— your sysadmin,
probably
stage prod
You need this
stage prod
And so does everyone else
Stop reinventing the wheel
Start building on top of
battle-tested code
Start building on top of
commercially-supported code
Start building on top of
code
Advantages of code
1. Reuse
2. Compose
3. Configure
4. Customize
5. Debug
6. Test
7. Version
8. Document
At Gruntwork, we’ve
been building
Modules for years
gruntwork.io
Many companies, all
running on the same
infrastructure code
gruntwork.io
stage prod
Modules allow us to turn this…
> terraform init <…>
> terraform apply
… into this
With some help from this
Questions?
modules@gruntwork.io

More Related Content

What's hot

Terraform modules restructured
Terraform modules restructuredTerraform modules restructured
Terraform modules restructuredAmi Mahloof
 
Introduction To Terraform
Introduction To TerraformIntroduction To Terraform
Introduction To TerraformSasitha Iresh
 
Microsoft Azure IaaS and Terraform
Microsoft Azure IaaS and TerraformMicrosoft Azure IaaS and Terraform
Microsoft Azure IaaS and TerraformAlex Mags
 
Terraform: An Overview & Introduction
Terraform: An Overview & IntroductionTerraform: An Overview & Introduction
Terraform: An Overview & IntroductionLee Trout
 
Infrastructure-as-Code (IaC) Using Terraform (Intermediate Edition)
Infrastructure-as-Code (IaC) Using Terraform (Intermediate Edition)Infrastructure-as-Code (IaC) Using Terraform (Intermediate Edition)
Infrastructure-as-Code (IaC) Using Terraform (Intermediate Edition)Adin Ermie
 
A Hands-on Introduction on Terraform Best Concepts and Best Practices
A Hands-on Introduction on Terraform Best Concepts and Best Practices A Hands-on Introduction on Terraform Best Concepts and Best Practices
A Hands-on Introduction on Terraform Best Concepts and Best Practices Nebulaworks
 
Terraform -- Infrastructure as Code
Terraform -- Infrastructure as CodeTerraform -- Infrastructure as Code
Terraform -- Infrastructure as CodeMartin Schütte
 
Building infrastructure as code using Terraform - DevOps Krakow
Building infrastructure as code using Terraform - DevOps KrakowBuilding infrastructure as code using Terraform - DevOps Krakow
Building infrastructure as code using Terraform - DevOps KrakowAnton Babenko
 
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...Yevgeniy Brikman
 
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Amazon Web Services
 
Terraform in deployment pipeline
Terraform in deployment pipelineTerraform in deployment pipeline
Terraform in deployment pipelineAnton Babenko
 
Getting Started with Infrastructure as Code
Getting Started with Infrastructure as CodeGetting Started with Infrastructure as Code
Getting Started with Infrastructure as CodeWinWire Technologies Inc
 

What's hot (20)

Terraform modules restructured
Terraform modules restructuredTerraform modules restructured
Terraform modules restructured
 
Introduction To Terraform
Introduction To TerraformIntroduction To Terraform
Introduction To Terraform
 
Microsoft Azure IaaS and Terraform
Microsoft Azure IaaS and TerraformMicrosoft Azure IaaS and Terraform
Microsoft Azure IaaS and Terraform
 
Final terraform
Final terraformFinal terraform
Final terraform
 
Terraform: An Overview & Introduction
Terraform: An Overview & IntroductionTerraform: An Overview & Introduction
Terraform: An Overview & Introduction
 
Infrastructure-as-Code (IaC) Using Terraform (Intermediate Edition)
Infrastructure-as-Code (IaC) Using Terraform (Intermediate Edition)Infrastructure-as-Code (IaC) Using Terraform (Intermediate Edition)
Infrastructure-as-Code (IaC) Using Terraform (Intermediate Edition)
 
Terraform
TerraformTerraform
Terraform
 
A Hands-on Introduction on Terraform Best Concepts and Best Practices
A Hands-on Introduction on Terraform Best Concepts and Best Practices A Hands-on Introduction on Terraform Best Concepts and Best Practices
A Hands-on Introduction on Terraform Best Concepts and Best Practices
 
Terraform -- Infrastructure as Code
Terraform -- Infrastructure as CodeTerraform -- Infrastructure as Code
Terraform -- Infrastructure as Code
 
Terraform 101
Terraform 101Terraform 101
Terraform 101
 
Terraform
TerraformTerraform
Terraform
 
Building infrastructure as code using Terraform - DevOps Krakow
Building infrastructure as code using Terraform - DevOps KrakowBuilding infrastructure as code using Terraform - DevOps Krakow
Building infrastructure as code using Terraform - DevOps Krakow
 
Introduction to IAC and Terraform
Introduction to IAC and Terraform Introduction to IAC and Terraform
Introduction to IAC and Terraform
 
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
 
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...
 
Terraform
TerraformTerraform
Terraform
 
Terraform in deployment pipeline
Terraform in deployment pipelineTerraform in deployment pipeline
Terraform in deployment pipeline
 
Terraform on Azure
Terraform on AzureTerraform on Azure
Terraform on Azure
 
Getting Started with Infrastructure as Code
Getting Started with Infrastructure as CodeGetting Started with Infrastructure as Code
Getting Started with Infrastructure as Code
 
Terraform on Azure
Terraform on AzureTerraform on Azure
Terraform on Azure
 

Similar to Reusable, composable, battle-tested Terraform modules

Infrastructure as code, using Terraform
Infrastructure as code, using TerraformInfrastructure as code, using Terraform
Infrastructure as code, using TerraformHarkamal Singh
 
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...Codemotion
 
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...Codemotion
 
Distributed Traceability in AWS - Life of a Transaction
Distributed Traceability in AWS - Life of a TransactionDistributed Traceability in AWS - Life of a Transaction
Distributed Traceability in AWS - Life of a TransactionAmazon Web Services
 
Новый InterSystems: open-source, митапы, хакатоны
Новый InterSystems: open-source, митапы, хакатоныНовый InterSystems: open-source, митапы, хакатоны
Новый InterSystems: open-source, митапы, хакатоныTimur Safin
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStackPuppet
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackke4qqq
 
Puppetpreso
PuppetpresoPuppetpreso
Puppetpresoke4qqq
 
Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStackke4qqq
 
Cloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the CloudCloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the Cloudpetriojala123
 
Lab Manual Combaring Redis with Relational
Lab Manual Combaring Redis with RelationalLab Manual Combaring Redis with Relational
Lab Manual Combaring Redis with RelationalAmazon Web Services
 
DevOps Enabling Your Team
DevOps Enabling Your TeamDevOps Enabling Your Team
DevOps Enabling Your TeamGR8Conf
 
[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유
[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유
[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유Soowan Lee
 
Just one-shade-of-openstack
Just one-shade-of-openstackJust one-shade-of-openstack
Just one-shade-of-openstackRoberto Polli
 
Hands-on Lab - Combaring Redis with Relational
Hands-on Lab - Combaring Redis with RelationalHands-on Lab - Combaring Redis with Relational
Hands-on Lab - Combaring Redis with RelationalAmazon Web Services
 
Weave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 RecapWeave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 RecapPatrick Chanezon
 
Bare Metal to OpenStack with Razor and Chef
Bare Metal to OpenStack with Razor and ChefBare Metal to OpenStack with Razor and Chef
Bare Metal to OpenStack with Razor and ChefMatt Ray
 
Hands-on Lab: Amazon ElastiCache
Hands-on Lab: Amazon ElastiCacheHands-on Lab: Amazon ElastiCache
Hands-on Lab: Amazon ElastiCacheAmazon Web Services
 

Similar to Reusable, composable, battle-tested Terraform modules (20)

One-Man Ops
One-Man OpsOne-Man Ops
One-Man Ops
 
Infrastructure as code, using Terraform
Infrastructure as code, using TerraformInfrastructure as code, using Terraform
Infrastructure as code, using Terraform
 
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
 
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
Jörg Schad - Hybrid Cloud (Kubernetes, Spark, HDFS, …)-as-a-Service - Codemot...
 
Distributed Traceability in AWS - Life of a Transaction
Distributed Traceability in AWS - Life of a TransactionDistributed Traceability in AWS - Life of a Transaction
Distributed Traceability in AWS - Life of a Transaction
 
Новый InterSystems: open-source, митапы, хакатоны
Новый InterSystems: open-source, митапы, хакатоныНовый InterSystems: open-source, митапы, хакатоны
Новый InterSystems: open-source, митапы, хакатоны
 
Puppet and Apache CloudStack
Puppet and Apache CloudStackPuppet and Apache CloudStack
Puppet and Apache CloudStack
 
Infrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStackInfrastructure as code with Puppet and Apache CloudStack
Infrastructure as code with Puppet and Apache CloudStack
 
Puppetpreso
PuppetpresoPuppetpreso
Puppetpreso
 
Puppet and CloudStack
Puppet and CloudStackPuppet and CloudStack
Puppet and CloudStack
 
TIAD : Automating the modern datacenter
TIAD : Automating the modern datacenterTIAD : Automating the modern datacenter
TIAD : Automating the modern datacenter
 
Cloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the CloudCloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the Cloud
 
Lab Manual Combaring Redis with Relational
Lab Manual Combaring Redis with RelationalLab Manual Combaring Redis with Relational
Lab Manual Combaring Redis with Relational
 
DevOps Enabling Your Team
DevOps Enabling Your TeamDevOps Enabling Your Team
DevOps Enabling Your Team
 
[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유
[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유
[AWSKRUG 아키텍처 모임] 세일즈부스트 인프라스트럭처 사례 공유
 
Just one-shade-of-openstack
Just one-shade-of-openstackJust one-shade-of-openstack
Just one-shade-of-openstack
 
Hands-on Lab - Combaring Redis with Relational
Hands-on Lab - Combaring Redis with RelationalHands-on Lab - Combaring Redis with Relational
Hands-on Lab - Combaring Redis with Relational
 
Weave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 RecapWeave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 Recap
 
Bare Metal to OpenStack with Razor and Chef
Bare Metal to OpenStack with Razor and ChefBare Metal to OpenStack with Razor and Chef
Bare Metal to OpenStack with Razor and Chef
 
Hands-on Lab: Amazon ElastiCache
Hands-on Lab: Amazon ElastiCacheHands-on Lab: Amazon ElastiCache
Hands-on Lab: Amazon ElastiCache
 

More from Yevgeniy Brikman

Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsYevgeniy Brikman
 
Gruntwork Executive Summary
Gruntwork Executive SummaryGruntwork Executive Summary
Gruntwork Executive SummaryYevgeniy Brikman
 
The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...Yevgeniy Brikman
 
An intro to Docker, Terraform, and Amazon ECS
An intro to Docker, Terraform, and Amazon ECSAn intro to Docker, Terraform, and Amazon ECS
An intro to Docker, Terraform, and Amazon ECSYevgeniy Brikman
 
Infrastructure as code: running microservices on AWS using Docker, Terraform,...
Infrastructure as code: running microservices on AWS using Docker, Terraform,...Infrastructure as code: running microservices on AWS using Docker, Terraform,...
Infrastructure as code: running microservices on AWS using Docker, Terraform,...Yevgeniy Brikman
 
Startup Ideas and Validation
Startup Ideas and ValidationStartup Ideas and Validation
Startup Ideas and ValidationYevgeniy Brikman
 
A Guide to Hiring for your Startup
A Guide to Hiring for your StartupA Guide to Hiring for your Startup
A Guide to Hiring for your StartupYevgeniy Brikman
 
Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)Yevgeniy Brikman
 
Composable and streamable Play apps
Composable and streamable Play appsComposable and streamable Play apps
Composable and streamable Play appsYevgeniy Brikman
 
Play Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and ScalaPlay Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and ScalaYevgeniy Brikman
 
The Play Framework at LinkedIn
The Play Framework at LinkedInThe Play Framework at LinkedIn
The Play Framework at LinkedInYevgeniy Brikman
 
Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...Yevgeniy Brikman
 

More from Yevgeniy Brikman (20)

Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
 
Gruntwork Executive Summary
Gruntwork Executive SummaryGruntwork Executive Summary
Gruntwork Executive Summary
 
The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...
 
An intro to Docker, Terraform, and Amazon ECS
An intro to Docker, Terraform, and Amazon ECSAn intro to Docker, Terraform, and Amazon ECS
An intro to Docker, Terraform, and Amazon ECS
 
Infrastructure as code: running microservices on AWS using Docker, Terraform,...
Infrastructure as code: running microservices on AWS using Docker, Terraform,...Infrastructure as code: running microservices on AWS using Docker, Terraform,...
Infrastructure as code: running microservices on AWS using Docker, Terraform,...
 
Agility Requires Safety
Agility Requires SafetyAgility Requires Safety
Agility Requires Safety
 
Startup Ideas and Validation
Startup Ideas and ValidationStartup Ideas and Validation
Startup Ideas and Validation
 
A Guide to Hiring for your Startup
A Guide to Hiring for your StartupA Guide to Hiring for your Startup
A Guide to Hiring for your Startup
 
Startup DNA: Speed Wins
Startup DNA: Speed WinsStartup DNA: Speed Wins
Startup DNA: Speed Wins
 
Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)
 
Node.js vs Play Framework
Node.js vs Play FrameworkNode.js vs Play Framework
Node.js vs Play Framework
 
Rapid prototyping
Rapid prototypingRapid prototyping
Rapid prototyping
 
Composable and streamable Play apps
Composable and streamable Play appsComposable and streamable Play apps
Composable and streamable Play apps
 
Play Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and ScalaPlay Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and Scala
 
The Play Framework at LinkedIn
The Play Framework at LinkedInThe Play Framework at LinkedIn
The Play Framework at LinkedIn
 
Kings of Code Hack Battle
Kings of Code Hack BattleKings of Code Hack Battle
Kings of Code Hack Battle
 
Hackdays and [in]cubator
Hackdays and [in]cubatorHackdays and [in]cubator
Hackdays and [in]cubator
 
Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...
 
Dust.js
Dust.jsDust.js
Dust.js
 
LinkedIn Overview
LinkedIn OverviewLinkedIn Overview
LinkedIn Overview
 

Recently uploaded

What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????blackmambaettijean
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 

Recently uploaded (20)

What is Artificial Intelligence?????????
What is Artificial Intelligence?????????What is Artificial Intelligence?????????
What is Artificial Intelligence?????????
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 

Reusable, composable, battle-tested Terraform modules

  • 2. Your project is code complete and it’s time to deploy it!
  • 3. I know, I’ll use AWS!
  • 4. You login to the AWS console
  • 5.
  • 7. OK, I have a server running!
  • 8. What else do I need?
  • 9. Well, you probably want more than one server for high availability
  • 10. And a load balancer to distribute traffic across those servers
  • 11. And EBS Volumes and RDS databases to store all of your data
  • 12. You’ll need an S3 bucket for files
  • 14. Don’t forget the VPC, subnets, route tables, NAT gateways
  • 16. ACM for SSL/TLS certs and KMS to encrypt / decrypt secrets
  • 17. And you need all of that in separate environments for stage and prod stage prod
  • 18. Plus DevOps tooling to manage it all stage prod
  • 19. And a CI server to test it all stage prod
  • 20. Plus alerts and on-call rotation to notify you when it all breaks stage prod
  • 22.
  • 23. And you have to maintain it all. Forever.
  • 24. AWS: 1,000 new releases in 2016
  • 27.
  • 28. There’s a better way to deploy and manage infrastructure:
  • 31. In this talk, I’ll show you how Terraform Modules work
  • 32. stage prod And how they will allow you to do all of this…
  • 33. > terraform init <…> > terraform apply In just a few simple commands
  • 37. Your entire AWS infrastructure. Defined as code. In about a day. gruntwork.io
  • 38. 1. What’s a Module 2. How to use a Module 3. How Modules work 4. The future of Modules Outline
  • 39. 1. What’s a Module 2. How to use a Module 3. How Modules work 4. The future of Modules Outline
  • 40. The two primary types of infrastructure providers:
  • 41. Infrastructure as a Service (IaaS): e.g., AWS, Azure, Google Cloud CPU Memory Disk Drive Network Server DB
  • 42. They offer many primitives and it’s up to you to put them together CPU Memory Disk Drive Network Server DB
  • 43. Platform as a Service (PaaS): e.g., Heroku, Docker Cloud, Engine Yard Rails MySQL GitHub CPU Memory Disk Drive Network Server DB
  • 44. They hide all the lower-level details so you can focus on your apps Rails MySQL GitHub CPU Memory Disk Drive Network Server DB
  • 45. > cd my-nodejs-app > heroku create my-nodejs-app > git push heroku master Getting started with a PaaS is easy!
  • 46. Heroku limitations 1. Can only use supported runtimes & versions (e.g., python-3.6.2 or python-2.7.13) 2. Can only use supported system software & libraries 3. Can only run web services (data stores and other services available only via paid add-ons) 4. Apps can’t access the shell 5. Devs can’t access servers via SSH 6. Local disk is read-only 7. Load balancing is HTTP/HTTPS only 8. Requests are limited to 30 seconds 9. Limited to one AWS region 10. App must boot in 60 seconds or less 11. Apps can be at most 100MB 12. Build must take less than 15 min 13. Logs are limited to 1500 lines unless you use supported (paid) add-ons 14. Manual scaling only 15. Pricing gets very steep as you scale up 16. Support only available on PST time zone 17. Limited control over security settings However, customizing, debugging, and scaling is not.
  • 47. For most software companies, IaaS is the only way to grow CPU Memory Disk Drive Network Server DB
  • 48. stage prod But that requires dealing with this…
  • 49. We are developers. We know how to fix this.
  • 51. Terraform is a tool for defining and managing infrastructure as code
  • 52. provider "aws" { region = "us-east-1" } resource "aws_instance" "example" { ami = "ami-408c7f28" instance_type = "t2.micro" } Example: Terraform code to deploy a server in AWS
  • 53. > terraform apply aws_instance.example: Creating... ami: "" => "ami-408c7f28" instance_type: "" => "t2.micro" key_name: "" => "<computed>" private_ip: "" => "<computed>" public_ip: "" => "<computed>” aws_instance.example: Creation complete Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Run terraform apply to deploy the server
  • 55. They are like reusable blueprints for your infrastructure
  • 56. resource "aws_autoscaling_group" "example" { name = "${var.name}-service" min_size = "${var.num_instances}" max_size = "${var.num_instances}" } resource "aws_launch_configuration" "example" { image_id = "${var.image_id}" instance_type = "${var.instance_type}" root_block_device { volume_type = "gp2" volume_size = 200 } } Example: create a module to deploy a microservice
  • 57. module "service_foo" { source = "/modules/microservice" image_id = "ami-123asd1" num_instances = 3 } Now you can use the module to deploy one microservice
  • 58. module "service_foo" { source = "/modules/microservice" image_id = "ami-123asd1" num_instances = 3 } module "service_bar" { source = "/modules/microservice" image_id = "ami-f2bb05ln" num_instances = 6 } module "service_baz" { source = "/modules/microservice" image_id = "ami-ny6v24xa" num_instances = 3 } Or multiple microservices
  • 59. Modules allow you to use your favorite IaaS provider… CPU Memory Disk Drive Network Server DB
  • 60. With easy-to-use, high-level abstractions, like a PaaS CPU Memory Disk Drive Network Server DB Rails MySQL GitHub
  • 61. But since you have all the code, you still have full control! CPU Memory Disk Drive Network Server DB Rails MySQL GitHub
  • 62. Best of all: code can be shared and re-used!
  • 64. A collection of reusable, verified, supported Modules
  • 68. 1. What’s a Module 2. How to use a Module 3. How Modules work 4. The future of Modules Outline
  • 69. Imagine you wanted to deploy Vault
  • 71. 1. Open up the Vault docs 2. Deploy a few servers 3. Install Vault 4. Install supervisord 5. Configure mlock 6. Generate self-signed TLS cert 7. Create Vault config file 8. Create an S3 bucket as storage backend 9. Figure out IAM policies for the S3 bucket 10. Tinker with security group rules 11. Figure out IP addresses to bind to and advertise 12. Fight with Vault for hours because it won’t accept your TLS cert 13. Regenerate cert with RSA encryption 14. Update OS certificate store to accept self-signed certs 15. Realize you need to deploy a Consul cluster for high availability 16. Open up the Consul docs…
  • 73. > terraform init hashicorp/vault/aws > terraform apply
  • 74. And now you have this deployed
  • 75. As simple as a PaaS!
  • 76. > tree . ├── README.md ├── main.tf ├── outputs.tf ├── packer ├── user-data └── variables.tf But you also have all the code. Feel free to edit it!
  • 77. module "vault_cluster" { source = "hashicorp/vault/aws" cluster_name = "example-vault-cluster" cluster_size = 3 vpc_id = "${data.aws_vpc.default.id}" subnet_ids = "${data.aws_subnets.default.ids}" } module "consul_cluster" { source = "hashicorp/consul/aws" cluster_name = "example-consul-cluster" cluster_size = 3 vpc_id = "${data.aws_vpc.default.id}" subnet_ids = "${data.aws_subnets.default.ids}" }Example: modify main.tf
  • 78. > terraform apply Run apply when you’re done!
  • 79. 1. What’s a Module 2. How to use a Module 3. How Modules work 4. The future of Modules Outline
  • 80. def add(x, y): return x + y Most programming languages support functions
  • 81. def add(x, y): return x + y The function has a name
  • 82. def add(x, y): return x + y It can take in inputs
  • 83. def add(x, y): return x + y And it can return outputs
  • 84. def add(x, y): return x + y add(3, 5) add(10, 35) add(-45, 6) Key idea: code reuse
  • 85. def add(x, y): return x + y assert add(3, 5) == 8 assert add(10, 35) == 45 assert add(-45, 6) == -39 Key idea: testing
  • 86. def add(x, y): return x + y def sub(x, y): return x - y sub(add(5, 3), add(4, 7)) Key idea: composition
  • 87. def run_classifier(data_set): X_pca = PCA(n_components=2).fit_transform(X_train) clusters = clf.fit_predict(X_train) fig, ax = plt.subplots(1, 2, figsize=(8, 4)) fig.subplots_adjust(top=0.85) predicted = svc_model.predict(X_test) images_and_predictions = list(zip(images_test, predicted)) ax[0].scatter(X_pca[:, 0], X_pca[:, 1], c=clusters) ax[0].set_title('Predicted Training Labels') ax[1].scatter(X_pca[:, 0], X_pca[:, 1], c=y_train) ax[1].set_title('Actual Training Labels') Key idea: abstraction
  • 88. def run_classifier(data_set): X_pca = PCA(n_components=2).fit_transform(X_train) clusters = clf.fit_predict(X_train) fig, ax = plt.subplots(1, 2, figsize=(8, 4)) fig.subplots_adjust(top=0.85) predicted = svc_model.predict(X_test) images_and_predictions = list(zip(images_test, predicted)) ax[0].scatter(X_pca[:, 0], X_pca[:, 1], c=clusters) ax[0].set_title('Predicted Training Labels') ax[1].scatter(X_pca[:, 0], X_pca[:, 1], c=y_train) ax[1].set_title('Actual Training Labels')You want to hide a large “volume”…
  • 89. def run_classifier(data_set): X_pca = PCA(n_components=2).fit_transform(X_train) clusters = clf.fit_predict(X_train) fig, ax = plt.subplots(1, 2, figsize=(8, 4)) fig.subplots_adjust(top=0.85) predicted = svc_model.predict(X_test) images_and_predictions = list(zip(images_test, predicted)) ax[0].scatter(X_pca[:, 0], X_pca[:, 1], c=clusters) ax[0].set_title('Predicted Training Labels') ax[1].scatter(X_pca[:, 0], X_pca[:, 1], c=y_train) ax[1].set_title('Actual Training Labels') Behind a small “surface area”
  • 92. > tree minimal-module . ├── main.tf ├── outputs.tf ├── variables.tf └── README.md It’s just Terraform code in a folder!
  • 93. variable "name" { description = "The name of the EC2 instance" } variable "image_id" { description = "The ID of the AMI to run" } variable "port" { description = "The port to listen on for HTTP requests" } The inputs are in variables.tf
  • 94. output "url" { value = "http://${aws_instance.example.ip}:${var.port}" } The outputs are in outputs.tf
  • 95. resource "aws_autoscaling_group" "example" { name = "${var.name}-service" min_size = "${var.num_instances}" max_size = "${var.num_instances}" } resource "aws_launch_configuration" "example" { image_id = "${var.image_id}" instance_type = "${var.instance_type}" root_block_device { volume_type = "gp2" volume_size = 200 } }The resources are in main.tf
  • 96. # Foo Module for AWS This is a Terraform Module to deploy [Foo](http://www.example.com) on AWS, including: * foo * bar * baz Documentation is in README.md
  • 98. > tree complete-module . ├── main.tf ├── outputs.tf ├── variables.tf ├── README.MD ├── modules ├── examples └── test We add three new folders: modules, examples, test
  • 99. > tree complete-module/modules modules/ ├── submodule-bar │ ├── main.tf │ ├── outputs.tf │ └── variables.tf └── submodule-foo ├── main.tf ├── outputs.tf └── variables.tf The modules folder contains standalone “submodules”
  • 100. > tree complete-module/modules modules/ ├── submodule-bar │ ├── install-vault.sh │ └── run-vault.sh └── submodule-foo └── main.go Some of the submodules may not even be Terraform code
  • 101. For example, one submodule can be used to install Vault in an AMI
  • 102. Another to create self-signed TLS certificates
  • 103. Another to deploy the AMI across an Auto Scaling Group (ASG)
  • 104. Another to create an S3 bucket and IAM policies as a storage backend
  • 105. Another to configure the Security Group settings
  • 106. And one more to deploy a load balancer (ELB)
  • 107. You can use all the submodules
  • 108. Or pick the ones you want and swap in your own for the rest
  • 109. > tree complete-module/examples examples/ ├── example-foo │ ├── main.tf │ ├── outputs.tf │ └── variables.tf └── example-bar ├── main.tf ├── outputs.tf └── variables.tf The examples folder shows how to use the submodules
  • 110. Note: the code in the root is usually a “canonical” example > tree complete-module . ├── main.tf ├── outputs.tf ├── variables.tf ├── README.MD ├── modules ├── examples └── test
  • 111. It’s typically an opinionated way to use all the submodules together
  • 112. > tree complete-module/examples examples/ ├── example-foo │ ├── main.tf │ ├── outputs.tf │ └── variables.tf └── example-bar ├── main.tf ├── outputs.tf └── variables.tf The code in examples shows other possible permutations
  • 113. E.g., How to use just one or two of the submodules together
  • 114. Or how to combine with other modules (e.g., Vault + Consul)
  • 115. This is like function composition!
  • 116. > tree complete-module/test test/ ├── example_foo_test.go └── example_bar_test.go The test folder contains automated tests
  • 117. > tree complete-module/test test/ ├── example_foo_test.go └── example_bar_test.go The tests are typically “integration tests”
  • 118. func vaultTest(t *testing.T, options *terratest.Options) { tlsCert := generateSelfSignedTlsCert(t) defer cleanupSelfSignedTlsCert(t, tlsCert) amiId := buildVaultAmi(t) defer cleanupAmi(t, amiId) terratest.Apply(options) defer terratest.Destroy(options) assertCanInitializeAndUnsealVault(t, options) } Example test case for Vault
  • 119. func vaultTest(t *testing.T, options *terratest.Options) { tlsCert := generateSelfSignedTlsCert(t) defer cleanupSelfSignedTlsCert(t, tlsCert) amiId := buildVaultAmi(t) defer cleanupAmi(t, amiId) terratest.Apply(options) defer terratest.Destroy(options) assertCanInitializeAndUnsealVault(t, options) } Create test-time resources
  • 120. func vaultTest(t *testing.T, options *terratest.Options) { tlsCert := generateSelfSignedTlsCert(t) defer cleanupSelfSignedTlsCert(t, tlsCert) amiId := buildVaultAmi(t) defer cleanupAmi(t, amiId) terratest.Apply(options) defer terratest.Destroy(options) assertCanInitializeAndUnsealVault(t, options) } Run terraform apply
  • 121. func vaultTest(t *testing.T, options *terratest.Options) { tlsCert := generateSelfSignedTlsCert(t) defer cleanupSelfSignedTlsCert(t, tlsCert) amiId := buildVaultAmi(t) defer cleanupAmi(t, amiId) terratest.Apply(options) defer terratest.Destroy(options) assertCanInitializeAndUnsealVault(t, options) } Run terraform destroy at the end
  • 122. func vaultTest(t *testing.T, options *terratest.Options) { tlsCert := generateSelfSignedTlsCert(t) defer cleanupSelfSignedTlsCert(t, tlsCert) amiId := buildVaultAmi(t) defer cleanupAmi(t, amiId) terratest.Apply(options) defer terratest.Destroy(options) assertCanInitializeAndUnsealVault(t, options) } Check the Vault cluster works!
  • 124. module "service_foo" { source = "./minimal-module" name = "Foo" image_id = "ami-123asd1" port = 8080 } For simple Modules and learning, deploy the root
  • 125. module "submodule_foo" { source = "./complete-module/modules/submodule-foo" param_foo = "foo" param_bar = 8080 } module "submodule_bar" { source = "./complete-module/modules/submodule-bar" param_foo = "abcdef" param_bar = 9091 } For more complicated use-cases, use the submodules
  • 126. module "service_foo" { source = "./minimal-module" name = "Foo" image_id = "ami-123asd1" port = 8080 } Abstraction: simple Module API for complicated infrastructure
  • 127. module "service_foo" { source = "./minimal-module" name = "Foo" image_id = "ami-123asd1" port = 8080 } module "service_bar" { source = "./minimal-module" name = "Bar" image_id = "ami-abcd1234" port = 9091 } Re-use: create a Module once, deploy it many times
  • 129. module "service_foo" { source = "./foo" name = "Foo" image_id = "ami-123asd1" port = 8080 } You can set source to point to modules at local file paths
  • 130. module "service_foo" { source = "hashicorp/vault/aws" name = "Foo" image_id = "ami-123asd1" port = 8080 } Alternatively, you can use Terraform registry URLs
  • 131. module "service_foo" { source = "git::git@github.com:foo/bar.git" name = "Foo" image_id = "ami-123asd1" port = 8080 } Or arbitrary Git URLs
  • 132. module "service_foo" { source = "git::git@github.com:foo/bar.git?ref=v1.0.0" name = "Foo" image_id = "ami-123asd1" port = 8080 } You can even link to a specific Git tag (recommended!)
  • 133. module "service_foo" { source = "git::git@github.com:foo/bar.git?ref=v1.0.0" name = "Foo" image_id = "ami-123asd1" port = 8080 } Modules use semantic versioning
  • 134. module "service_foo" { source = "git::git@github.com:foo/bar.git?ref=v2.0.0" name = "Foo" image_id = "ami-123asd1" port = 8080 } So upgrading infrastructure is just a version number bump
  • 135. Promote immutable, versioned infrastructure across environments qa stage prod
  • 136. 1. What’s a Module 2. How to use a Module 3. How Modules work 4. The future of Modules Outline
  • 137. “You are not special. Your infrastructure is not a beautiful and unique snowflake. You have the same tech debt as everyone else.” — your sysadmin, probably
  • 139. stage prod And so does everyone else
  • 141. Start building on top of battle-tested code
  • 142. Start building on top of commercially-supported code
  • 143. Start building on top of code
  • 144. Advantages of code 1. Reuse 2. Compose 3. Configure 4. Customize 5. Debug 6. Test 7. Version 8. Document
  • 145. At Gruntwork, we’ve been building Modules for years gruntwork.io
  • 146. Many companies, all running on the same infrastructure code gruntwork.io
  • 147. stage prod Modules allow us to turn this…
  • 148. > terraform init <…> > terraform apply … into this
  • 149. With some help from this