An intro to Docker, Terraform, and Amazon ECS

Yevgeniy Brikman
Yevgeniy BrikmanCo-founder at Gruntwork
A quick intro to
Docker,
Terraform, and
Amazon ECS
TERRAFORM
Amazon ECS
In this talk, we’ll show how to
deploy two apps:
A Rails Frontend and a
Sinatra Backend
Slides and code from this talk:
ybrikman.com/speaking
require 'sinatra'
get "/" do
"Hello, World!"
end
The sinatra backend just returns
“Hello, World”.
class ApplicationController < ActionController::Base
def index
url = URI.parse(backend_addr)
req = Net::HTTP::Get.new(url.to_s)
res = Net::HTTP.start(url.host, url.port) {|http|
http.request(req)
}
@text = res.body
end
end
The rails frontend calls the sinatra
backend…
<h1>Rails Frontend</h1>
<p>
Response from the backend: <strong><%= @text %></strong>
</p>
And renders the response as
HTML.
We’ll package the two apps as
Docker containers…
Amazon ECS
Deploy those Docker containers
using Amazon ECS…
TERRAFORM
And define our infrastructure-as-
code using Terraform.
I’m
Yevgeniy
Brikman
ybrikman.com
Co-founder of
Gruntwork
gruntwork.io
gruntwork.io
We offer DevOps
as a Service
gruntwork.io
And DevOps
as a Library
PAST LIVES
Author of
Hello,
Startup
hello-startup.net
And
Terraform:
Up & Running
terraformupandrunning.com
1. Docker
2. Terraform
3. ECS
4. Recap
Outline
1. Docker
2. Terraform
3. ECS
4. Recap
Outline
Docker allows you to build and
run code in containers
Containers are like lightweight
Virtual Machines (VMs)
Like an isolated process that
happens to be an entire OS
> docker run –it ubuntu bash
root@12345:/# echo "I'm in $(cat /etc/issue)”
I'm in Ubuntu 14.04.4 LTS
Running an Ubuntu image in a
Docker container
> time docker run ubuntu echo "Hello, World"
Hello, World
real 0m0.183s
user 0m0.009s
sys 0m0.014s
Containers boot quickly, with
minimal CPU/memory overhead
You can define a Docker image
as code in a Dockerfile
FROM gliderlabs/alpine:3.3
RUN apk --no-cache add ruby ruby-dev
RUN gem install sinatra --no-ri --no-rdoc
RUN mkdir -p /usr/src/app
COPY . /usr/src/app
WORKDIR /usr/src/app
EXPOSE 4567
CMD ["ruby", "app.rb"]
Here is the Dockerfile for the
Sinatra backend
FROM gliderlabs/alpine:3.3
RUN apk --no-cache add ruby ruby-dev
RUN gem install sinatra --no-ri --no-rdoc
RUN mkdir -p /usr/src/app
COPY . /usr/src/app
WORKDIR /usr/src/app
EXPOSE 4567
CMD ["ruby", "app.rb"]
It specifies dependencies, code,
config, and how to run the app
> docker build -t gruntwork/sinatra-backend .
Step 0 : FROM gliderlabs/alpine:3.3
---> 0a7e169bce21
(...)
Step 8 : CMD ruby app.rb
---> 2e243eba30ed
Successfully built 2e243eba30ed
Build the Docker image
> docker run -it -p 4567:4567 gruntwork/sinatra-backend
INFO WEBrick 1.3.1
INFO ruby 2.2.4 (2015-12-16) [x86_64-linux-musl]
== Sinatra (v1.4.7) has taken the stage on 4567 for
development with backup from WEBrick
INFO WEBrick::HTTPServer#start: pid=1 port=4567
Run the Docker image
> docker push gruntwork/sinatra-backend
The push refers to a repository [docker.io/gruntwork/sinatra-
backend] (len: 1)
2e243eba30ed: Image successfully pushed
7e2e0c53e246: Image successfully pushed
919d9a73b500: Image successfully pushed
(...)
v1: digest: sha256:09f48ed773966ec7fe4558 size: 14319
You can share your images by
pushing them to Docker Hub
Now you can reuse the same
image in dev, stg, prod, etc
> docker pull rails:4.2.6
And you can reuse images created
by others.
FROM rails:4.2.6
RUN mkdir -p /usr/src/app
COPY . /usr/src/app
WORKDIR /usr/src/app
RUN bundle install
EXPOSE 3000
CMD ["rails", "start"]
The rails-frontend is built on top of
the official rails Docker image
rails_frontend:
image: gruntwork/rails-frontend
ports:
- "3000:3000"
links:
- sinatra_backend
sinatra_backend:
image: gruntwork/sinatra-backend
ports:
- "4567:4567"
Define your entire dev stack as
code with docker-compose
rails_frontend:
image: gruntwork/rails-frontend
ports:
- "3000:3000"
links:
- sinatra_backend
sinatra_backend:
image: gruntwork/sinatra-backend
ports:
- "4567:4567"
Docker links provide a simple
service discovery mechanism
> docker-compose up
Starting infrastructureascodetalk_sinatra_backend_1
Recreating infrastructureascodetalk_rails_frontend_1
sinatra_backend_1 | INFO WEBrick 1.3.1
sinatra_backend_1 | INFO ruby 2.2.4 (2015-12-16)
sinatra_backend_1 | Sinatra has taken the stage on 4567
rails_frontend_1 | INFO WEBrick 1.3.1
rails_frontend_1 | INFO ruby 2.3.0 (2015-12-25)
rails_frontend_1 | INFO WEBrick::HTTPServer#start: port=3000
Run your entire dev stack with one
command
1. Docker
2. Terraform
3. ECS
4. Recap
Outline
Terraform is a tool for
provisioning infrastructure
Terraform supports many
providers (cloud agnostic)
And many resources for each
provider
You define infrastructure as code
in Terraform templates
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "example" {
ami = "ami-408c7f28"
instance_type = "t2.micro"
}
This template creates a single EC2
instance in AWS
> terraform plan
+ aws_instance.example
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>"
Plan: 1 to add, 0 to change, 0 to destroy.
Use the plan command to see
what you’re about to deploy
> terraform apply
aws_instance.example: Creating...
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>”
aws_instance.example: Creation complete
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Use the apply command to apply
the changes
Now our EC2 instance is running!
resource "aws_instance" "example" {
ami = "ami-408c7f28"
instance_type = "t2.micro"
tags {
Name = "terraform-example"
}
}
Let’s give the EC2 instance a tag
with a readable name
> terraform plan
~ aws_instance.example
tags.#: "0" => "1"
tags.Name: "" => "terraform-example"
Plan: 0 to add, 1 to change, 0 to destroy.
Use the plan command again to
verify your changes
> terraform apply
aws_instance.example: Refreshing state...
aws_instance.example: Modifying...
tags.#: "0" => "1"
tags.Name: "" => "terraform-example"
aws_instance.example: Modifications complete
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Use the apply command again to
deploy those changes
Now our EC2 instance has a tag!
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http”
}
}
Let’s add an Elastic Load Balancer
(ELB).
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http”
}
}
Terraform supports variables,
such as var.instance_port
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http"
}
}
As well as dependencies like
aws_instance.example.id
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http"
}
}
It builds a dependency graph and
applies it in parallel.
After running apply, we have an ELB!
> terraform destroy
aws_instance.example: Refreshing state... (ID: i-f3d58c70)
aws_elb.example: Refreshing state... (ID: example)
aws_elb.example: Destroying...
aws_elb.example: Destruction complete
aws_instance.example: Destroying...
aws_instance.example: Destruction complete
Apply complete! Resources: 0 added, 0 changed, 2 destroyed.
Use the destroy command to
delete all your resources
For more info, check out The Comprehensive Guide
to Terraform
1. Docker
2. Terraform
3. ECS
4. Recap
Outline
EC2 Container Service (ECS) is a
way to run Docker on AWS
ECS Overview
EC2 Instance
ECS Cluster
ECS Scheduler
ECS Agent
ECS Tasks
ECS Task Definition
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Service Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
ECS Cluster: several servers
managed by ECS
EC2 Instance
ECS Cluster
Typically, the servers are in an
Auto Scaling Group
Auto Scaling Group
EC2 Instance
Each server must run the ECS
Agent
ECS Agent
EC2 Instance
ECS Cluster
ECS Task: Docker container(s)
to run, resources they need
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
ECS Agent
EC2 Instance
ECS Task Definition
ECS Cluster
ECS Service: long-running ECS
Task & ELB settings
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
EC2 Instance
ECS Task Definition ECS Service Definition
ECS Cluster
ECS Scheduler: Deploys Tasks
across the ECS Cluster
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition ECS Scheduler
ECS Cluster
You can associate an ALB or
ELB with each ECS service
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition
ECS Cluster
This allows you to distribute
load across your ECS Tasks
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition
ECS Cluster
You can also use it as a simple
form of service discovery
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition
ECS Cluster
Let’s deploy our apps on ECS
using Terraform
Define the ECS Cluster as an
Auto Scaling Group (ASG)
EC2 Instance
ECS Cluster
resource "aws_ecs_cluster" "example_cluster" {
name = "example-cluster"
}
resource "aws_autoscaling_group" "ecs_cluster_instances" {
name = "ecs-cluster-instances"
min_size = 5
max_size = 5
launch_configuration =
"${aws_launch_configuration.ecs_instance.name}"
}
Ensure each server in the ASG
runs the ECS Agent
ECS Agent
EC2 Instance
ECS Cluster
# The launch config defines what runs on each EC2 instance
resource "aws_launch_configuration" "ecs_instance" {
name_prefix = "ecs-instance-"
instance_type = "t2.micro"
# This is an Amazon ECS AMI, which has an ECS Agent
# installed that lets it talk to the ECS cluster
image_id = "ami-a98cb2c3”
}
The launch config runs AWS ECS
Linux on each server in the ASG
Define an ECS Task for each
microservice
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
ECS Agent
EC2 Instance
ECS Task Definition
ECS Cluster
resource "aws_ecs_task_definition" "rails_frontend" {
family = "rails-frontend"
container_definitions = <<EOF
[{
"name": "rails-frontend",
"image": "gruntwork/rails-frontend:v1",
"cpu": 1024,
"memory": 768,
"essential": true,
"portMappings": [{"containerPort": 3000, "hostPort": 3000}]
}]
EOF
}
Rails frontend ECS Task
resource "aws_ecs_task_definition" "sinatra_backend" {
family = "sinatra-backend"
container_definitions = <<EOF
[{
"name": "sinatra-backend",
"image": "gruntwork/sinatra-backend:v1",
"cpu": 1024,
"memory": 768,
"essential": true,
"portMappings": [{"containerPort": 4567, "hostPort": 4567}]
}]
EOF
}
Sinatra Backend ECS Task
Define an ECS Service for each
ECS Task
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
EC2 Instance
ECS Task Definition ECS Service Definition
ECS Cluster
resource "aws_ecs_service" "rails_frontend" {
family = "rails-frontend"
cluster = "${aws_ecs_cluster.example_cluster.id}"
task_definition =
"${aws_ecs_task_definition.rails-fronted.arn}"
desired_count = 2
}
Rails Frontend ECS Service
resource "aws_ecs_service" "sinatra_backend" {
family = "sinatra-backend"
cluster = "${aws_ecs_cluster.example_cluster.id}"
task_definition =
"${aws_ecs_task_definition.sinatra_backend.arn}"
desired_count = 2
}
Sinatra Backend ECS Service
Associate an ELB with each
ECS Service
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition
ECS Cluster
resource "aws_elb" "rails_frontend" {
name = "rails-frontend"
listener {
lb_port = 80
lb_protocol = "http"
instance_port = 3000
instance_protocol = "http"
}
}
Rails Frontend ELB
resource "aws_ecs_service" "rails_frontend" {
(...)
load_balancer {
elb_name = "${aws_elb.rails_frontend.id}"
container_name = "rails-frontend"
container_port = 3000
}
}
Associate the ELB with the Rails
Frontend ECS Service
resource "aws_elb" "sinatra_backend" {
name = "sinatra-backend"
listener {
lb_port = 4567
lb_protocol = "http"
instance_port = 4567
instance_protocol = "http"
}
}
Sinatra Backend ELB
resource "aws_ecs_service" "sinatra_backend" {
(...)
load_balancer {
elb_name = "${aws_elb.sinatra_backend.id}"
container_name = "sinatra-backend"
container_port = 4567
}
}
Associate the ELB with the Sinatra
Backend ECS Service
Set up service discovery
between the ECS Services
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition
ECS Cluster
resource "aws_ecs_task_definition" "rails_frontend" {
family = "rails-frontend"
container_definitions = <<EOF
[{
...
"environment": [{
"name": "SINATRA_BACKEND_PORT",
"value": "tcp://${aws_elb.sinatra_backend.dns_name}:4567"
}]
}]
EOF
}
Pass the Sinatra Bckend ELB URL
as env var to Rails Frontend
It’s time to deploy!
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition ECS Scheduler
ECS Cluster
> terraform apply
aws_ecs_cluster.example_cluster: Creating...
name: "" => "example-cluster"
aws_ecs_task_definition.sinatra_backend: Creating...
...
Apply complete! Resources: 17 added, 0 changed, 0 destroyed.
Use the apply command to deploy
the ECS Cluster & Tasks
See the cluster in the ECS console
Track events for each Service
As well as basic metrics
Test the rails-frontend
1. Docker
2. Terraform
3. ECS
4. Recap
Outline
Slides and code from this talk:
ybrikman.com/speaking
For more
info, see
Hello,
Startup
hello-startup.net
And
Terraform:
Up & Running
terraformupandrunning.com
gruntwork.io
For DevOps help, see
Gruntwork
Questions?
1 of 98

Recommended

What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn... by
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...
What Is Kubernetes | Kubernetes Introduction | Kubernetes Tutorial For Beginn...Edureka!
2K views30 slides
Kubernetes Basics by
Kubernetes BasicsKubernetes Basics
Kubernetes BasicsRishabh Kumar
2.6K views17 slides
Kubernetes on AWS with Amazon EKS - MAD301 - New York AWS Summit by
Kubernetes on AWS with Amazon EKS - MAD301 - New York AWS SummitKubernetes on AWS with Amazon EKS - MAD301 - New York AWS Summit
Kubernetes on AWS with Amazon EKS - MAD301 - New York AWS SummitAmazon Web Services
4K views93 slides
DevOps with Kubernetes by
DevOps with KubernetesDevOps with Kubernetes
DevOps with KubernetesEastBanc Tachnologies
6.9K views45 slides
Introduction to Docker Compose by
Introduction to Docker ComposeIntroduction to Docker Compose
Introduction to Docker ComposeAjeet Singh Raina
16.4K views29 slides
Kubernetes Basics by
Kubernetes BasicsKubernetes Basics
Kubernetes BasicsEueung Mulyana
8K views40 slides

More Related Content

What's hot

Amazon EKS Deep Dive by
Amazon EKS Deep DiveAmazon EKS Deep Dive
Amazon EKS Deep DiveAndrzej Komarnicki
978 views42 slides
Introduction to docker by
Introduction to dockerIntroduction to docker
Introduction to dockerInstruqt
660 views43 slides
Docker Introduction by
Docker IntroductionDocker Introduction
Docker IntroductionPeng Xiao
1.5K views48 slides
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker... by
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...Simplilearn
5.2K views61 slides
Présentation docker et kubernetes by
Présentation docker et kubernetesPrésentation docker et kubernetes
Présentation docker et kubernetesKiwi Backup
1.3K views29 slides
Comprehensive Terraform Training by
Comprehensive Terraform TrainingComprehensive Terraform Training
Comprehensive Terraform TrainingYevgeniy Brikman
23K views148 slides

What's hot(20)

Introduction to docker by Instruqt
Introduction to dockerIntroduction to docker
Introduction to docker
Instruqt660 views
Docker Introduction by Peng Xiao
Docker IntroductionDocker Introduction
Docker Introduction
Peng Xiao1.5K views
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker... by Simplilearn
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
Simplilearn5.2K views
Présentation docker et kubernetes by Kiwi Backup
Présentation docker et kubernetesPrésentation docker et kubernetes
Présentation docker et kubernetes
Kiwi Backup1.3K views
Docker introduction by Phuc Nguyen
Docker introductionDocker introduction
Docker introduction
Phuc Nguyen1.3K views
Kubernetes: A Short Introduction (2019) by Megan O'Keefe
Kubernetes: A Short Introduction (2019)Kubernetes: A Short Introduction (2019)
Kubernetes: A Short Introduction (2019)
Megan O'Keefe7.9K views
Docker introduction (1) by Gourav Varma
Docker introduction (1)Docker introduction (1)
Docker introduction (1)
Gourav Varma352 views
Hands-On Introduction to Kubernetes at LISA17 by Ryan Jarvinen
Hands-On Introduction to Kubernetes at LISA17Hands-On Introduction to Kubernetes at LISA17
Hands-On Introduction to Kubernetes at LISA17
Ryan Jarvinen1.1K views
Kubernetes for Beginners: An Introductory Guide by Bytemark
Kubernetes for Beginners: An Introductory GuideKubernetes for Beginners: An Introductory Guide
Kubernetes for Beginners: An Introductory Guide
Bytemark10.7K views
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu... by Edureka!
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
Edureka!2K views
Kubernetes Networking | Kubernetes Services, Pods & Ingress Networks | Kubern... by Edureka!
Kubernetes Networking | Kubernetes Services, Pods & Ingress Networks | Kubern...Kubernetes Networking | Kubernetes Services, Pods & Ingress Networks | Kubern...
Kubernetes Networking | Kubernetes Services, Pods & Ingress Networks | Kubern...
Edureka!939 views
Introduction to kubernetes by Gabriel Carro
Introduction to kubernetesIntroduction to kubernetes
Introduction to kubernetes
Gabriel Carro230 views

Similar to An intro to Docker, Terraform, and Amazon ECS

Fullstack conf 2017 - Basic dev pipeline end-to-end by
Fullstack conf 2017 - Basic dev pipeline end-to-endFullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-endEzequiel Maraschio
116 views76 slides
Cutting through the fog of cloud by
Cutting through the fog of cloudCutting through the fog of cloud
Cutting through the fog of cloudKyle Rames
1.5K views94 slides
Amazon Web Services and Docker: from developing to production by
Amazon Web Services and Docker: from developing to productionAmazon Web Services and Docker: from developing to production
Amazon Web Services and Docker: from developing to productionPaolo latella
1.5K views23 slides
Workshop Consul .- Service Discovery & Failure Detection by
Workshop Consul .- Service Discovery & Failure DetectionWorkshop Consul .- Service Discovery & Failure Detection
Workshop Consul .- Service Discovery & Failure DetectionVincent Composieux
4.3K views29 slides
Simple docker hosting in FIWARE Lab by
Simple docker hosting in FIWARE LabSimple docker hosting in FIWARE Lab
Simple docker hosting in FIWARE LabFernando Lopez Aguilar
919 views51 slides
Artem Zhurbila - docker clusters (solit 2015) by
Artem Zhurbila - docker clusters (solit 2015)Artem Zhurbila - docker clusters (solit 2015)
Artem Zhurbila - docker clusters (solit 2015)Artem Zhurbila
3.5K views47 slides

Similar to An intro to Docker, Terraform, and Amazon ECS(20)

Fullstack conf 2017 - Basic dev pipeline end-to-end by Ezequiel Maraschio
Fullstack conf 2017 - Basic dev pipeline end-to-endFullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-end
Ezequiel Maraschio116 views
Cutting through the fog of cloud by Kyle Rames
Cutting through the fog of cloudCutting through the fog of cloud
Cutting through the fog of cloud
Kyle Rames1.5K views
Amazon Web Services and Docker: from developing to production by Paolo latella
Amazon Web Services and Docker: from developing to productionAmazon Web Services and Docker: from developing to production
Amazon Web Services and Docker: from developing to production
Paolo latella1.5K views
Workshop Consul .- Service Discovery & Failure Detection by Vincent Composieux
Workshop Consul .- Service Discovery & Failure DetectionWorkshop Consul .- Service Discovery & Failure Detection
Workshop Consul .- Service Discovery & Failure Detection
Vincent Composieux4.3K views
Artem Zhurbila - docker clusters (solit 2015) by Artem Zhurbila
Artem Zhurbila - docker clusters (solit 2015)Artem Zhurbila - docker clusters (solit 2015)
Artem Zhurbila - docker clusters (solit 2015)
Artem Zhurbila3.5K views
Docker Introductory workshop by Runcy Oommen
Docker Introductory workshopDocker Introductory workshop
Docker Introductory workshop
Runcy Oommen536 views
Gianluca Arbezzano Wordpress: gestione delle installazioni e scalabilità con ... by Codemotion
Gianluca Arbezzano Wordpress: gestione delle installazioni e scalabilità con ...Gianluca Arbezzano Wordpress: gestione delle installazioni e scalabilità con ...
Gianluca Arbezzano Wordpress: gestione delle installazioni e scalabilità con ...
Codemotion951 views
Dockerized .Net Core based app services in azure K8s by Ranjeet Bhargava
Dockerized .Net Core based app services in azure K8s Dockerized .Net Core based app services in azure K8s
Dockerized .Net Core based app services in azure K8s
Ranjeet Bhargava246 views
Docker Security workshop slides by Docker, Inc.
Docker Security workshop slidesDocker Security workshop slides
Docker Security workshop slides
Docker, Inc.5.3K views
Nats meetup oct 2016 docker 112 by Nirmal Mehta
Nats meetup oct 2016 docker 112Nats meetup oct 2016 docker 112
Nats meetup oct 2016 docker 112
Nirmal Mehta558 views
JDO 2019: Tips and Tricks from Docker Captain - Łukasz Lach by PROIDEA
JDO 2019: Tips and Tricks from Docker Captain - Łukasz LachJDO 2019: Tips and Tricks from Docker Captain - Łukasz Lach
JDO 2019: Tips and Tricks from Docker Captain - Łukasz Lach
PROIDEA74 views
Microservices blue-green-deployment-with-docker by Kidong Lee
Microservices blue-green-deployment-with-dockerMicroservices blue-green-deployment-with-docker
Microservices blue-green-deployment-with-docker
Kidong Lee698 views
Apache MXNet Distributed Training Explained In Depth by Viacheslav Kovalevsky... by Big Data Spain
Apache MXNet Distributed Training Explained In Depth by Viacheslav Kovalevsky...Apache MXNet Distributed Training Explained In Depth by Viacheslav Kovalevsky...
Apache MXNet Distributed Training Explained In Depth by Viacheslav Kovalevsky...
Big Data Spain1.4K views
Docker, Kubernetes, and Google Cloud by Samuel Chow
Docker, Kubernetes, and Google CloudDocker, Kubernetes, and Google Cloud
Docker, Kubernetes, and Google Cloud
Samuel Chow1.1K views
Infrastructureascode slideshare-160331143725 by MortazaJohari
Infrastructureascode slideshare-160331143725Infrastructureascode slideshare-160331143725
Infrastructureascode slideshare-160331143725
MortazaJohari25 views

More from Yevgeniy Brikman

Cloud adoption fails - 5 ways deployments go wrong and 5 solutions by
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsYevgeniy Brikman
4.4K views150 slides
How to test infrastructure code: automated testing for Terraform, Kubernetes,... by
How to test infrastructure code: automated testing for Terraform, Kubernetes,...How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...Yevgeniy Brikman
33.8K views200 slides
Lessons learned from writing over 300,000 lines of infrastructure code by
Lessons learned from writing over 300,000 lines of infrastructure codeLessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure codeYevgeniy Brikman
177.3K views139 slides
Gruntwork Executive Summary by
Gruntwork Executive SummaryGruntwork Executive Summary
Gruntwork Executive SummaryYevgeniy Brikman
10.4K views38 slides
Reusable, composable, battle-tested Terraform modules by
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesYevgeniy Brikman
28.4K views150 slides
The Truth About Startups: What I wish someone had told me about entrepreneurs... by
The Truth About Startups: What I wish someone had told me about entrepreneurs...The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...Yevgeniy Brikman
10.8K views191 slides

More from Yevgeniy Brikman(20)

Cloud adoption fails - 5 ways deployments go wrong and 5 solutions by Yevgeniy Brikman
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Yevgeniy Brikman4.4K views
How to test infrastructure code: automated testing for Terraform, Kubernetes,... by Yevgeniy Brikman
How to test infrastructure code: automated testing for Terraform, Kubernetes,...How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
Yevgeniy Brikman33.8K views
Lessons learned from writing over 300,000 lines of infrastructure code by Yevgeniy Brikman
Lessons learned from writing over 300,000 lines of infrastructure codeLessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure code
Yevgeniy Brikman177.3K views
Reusable, composable, battle-tested Terraform modules by Yevgeniy Brikman
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modules
Yevgeniy Brikman28.4K views
The Truth About Startups: What I wish someone had told me about entrepreneurs... by Yevgeniy Brikman
The Truth About Startups: What I wish someone had told me about entrepreneurs...The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...
Yevgeniy Brikman10.8K views
Infrastructure as code: running microservices on AWS using Docker, Terraform,... by Yevgeniy Brikman
Infrastructure as code: running microservices on AWS using Docker, Terraform,...Infrastructure as code: running microservices on AWS using Docker, Terraform,...
Infrastructure as code: running microservices on AWS using Docker, Terraform,...
Yevgeniy Brikman177.5K views
Startup Ideas and Validation by Yevgeniy Brikman
Startup Ideas and ValidationStartup Ideas and Validation
Startup Ideas and Validation
Yevgeniy Brikman341.3K views
A Guide to Hiring for your Startup by Yevgeniy Brikman
A Guide to Hiring for your StartupA Guide to Hiring for your Startup
A Guide to Hiring for your Startup
Yevgeniy Brikman17.3K views
Node.js vs Play Framework (with Japanese subtitles) by Yevgeniy Brikman
Node.js vs Play Framework (with Japanese subtitles)Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)
Yevgeniy Brikman42.5K views
Composable and streamable Play apps by Yevgeniy Brikman
Composable and streamable Play appsComposable and streamable Play apps
Composable and streamable Play apps
Yevgeniy Brikman122.3K views
Play Framework: async I/O with Java and Scala by Yevgeniy Brikman
Play Framework: async I/O with Java and ScalaPlay Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and Scala
Yevgeniy Brikman108.6K views
The Play Framework at LinkedIn by Yevgeniy Brikman
The Play Framework at LinkedInThe Play Framework at LinkedIn
The Play Framework at LinkedIn
Yevgeniy Brikman282.2K views
Startup DNA: the formula behind successful startups in Silicon Valley (update... by Yevgeniy Brikman
Startup DNA: the formula behind successful startups in Silicon Valley (update...Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...
Yevgeniy Brikman170.3K views

Recently uploaded

STPI OctaNE CoE Brochure.pdf by
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdfmadhurjyapb
14 views1 slide
6g - REPORT.pdf by
6g - REPORT.pdf6g - REPORT.pdf
6g - REPORT.pdfLiveplex
10 views23 slides
SAP Automation Using Bar Code and FIORI.pdf by
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdfVirendra Rai, PMP
23 views38 slides
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N... by
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...James Anderson
92 views32 slides
"Running students' code in isolation. The hard way", Yurii Holiuk by
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk Fwdays
17 views34 slides
Uni Systems for Power Platform.pptx by
Uni Systems for Power Platform.pptxUni Systems for Power Platform.pptx
Uni Systems for Power Platform.pptxUni Systems S.M.S.A.
56 views21 slides

Recently uploaded(20)

STPI OctaNE CoE Brochure.pdf by madhurjyapb
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdf
madhurjyapb14 views
6g - REPORT.pdf by Liveplex
6g - REPORT.pdf6g - REPORT.pdf
6g - REPORT.pdf
Liveplex10 views
SAP Automation Using Bar Code and FIORI.pdf by Virendra Rai, PMP
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdf
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N... by James Anderson
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
James Anderson92 views
"Running students' code in isolation. The hard way", Yurii Holiuk by Fwdays
"Running students' code in isolation. The hard way", Yurii Holiuk "Running students' code in isolation. The hard way", Yurii Holiuk
"Running students' code in isolation. The hard way", Yurii Holiuk
Fwdays17 views
Serverless computing with Google Cloud (2023-24) by wesley chun
Serverless computing with Google Cloud (2023-24)Serverless computing with Google Cloud (2023-24)
Serverless computing with Google Cloud (2023-24)
wesley chun11 views
STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf by Dr. Jimmy Schwarzkopf
STKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdfSTKI Israeli Market Study 2023   corrected forecast 2023_24 v3.pdf
STKI Israeli Market Study 2023 corrected forecast 2023_24 v3.pdf
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ... by Jasper Oosterveld
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
HTTP headers that make your website go faster - devs.gent November 2023 by Thijs Feryn
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023
Thijs Feryn22 views
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院 by IttrainingIttraining
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
Five Things You SHOULD Know About Postman by Postman
Five Things You SHOULD Know About PostmanFive Things You SHOULD Know About Postman
Five Things You SHOULD Know About Postman
Postman36 views
Future of AR - Facebook Presentation by ssuserb54b561
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook Presentation
ssuserb54b56115 views
Piloting & Scaling Successfully With Microsoft Viva by Richard Harbridge
Piloting & Scaling Successfully With Microsoft VivaPiloting & Scaling Successfully With Microsoft Viva
Piloting & Scaling Successfully With Microsoft Viva
Business Analyst Series 2023 - Week 3 Session 5 by DianaGray10
Business Analyst Series 2023 -  Week 3 Session 5Business Analyst Series 2023 -  Week 3 Session 5
Business Analyst Series 2023 - Week 3 Session 5
DianaGray10300 views
Data Integrity for Banking and Financial Services by Precisely
Data Integrity for Banking and Financial ServicesData Integrity for Banking and Financial Services
Data Integrity for Banking and Financial Services
Precisely25 views

An intro to Docker, Terraform, and Amazon ECS

  • 1. A quick intro to Docker, Terraform, and Amazon ECS TERRAFORM Amazon ECS
  • 2. In this talk, we’ll show how to deploy two apps:
  • 3. A Rails Frontend and a Sinatra Backend
  • 4. Slides and code from this talk: ybrikman.com/speaking
  • 5. require 'sinatra' get "/" do "Hello, World!" end The sinatra backend just returns “Hello, World”.
  • 6. class ApplicationController < ActionController::Base def index url = URI.parse(backend_addr) req = Net::HTTP::Get.new(url.to_s) res = Net::HTTP.start(url.host, url.port) {|http| http.request(req) } @text = res.body end end The rails frontend calls the sinatra backend…
  • 7. <h1>Rails Frontend</h1> <p> Response from the backend: <strong><%= @text %></strong> </p> And renders the response as HTML.
  • 8. We’ll package the two apps as Docker containers…
  • 9. Amazon ECS Deploy those Docker containers using Amazon ECS…
  • 10. TERRAFORM And define our infrastructure-as- code using Terraform.
  • 18. 1. Docker 2. Terraform 3. ECS 4. Recap Outline
  • 19. 1. Docker 2. Terraform 3. ECS 4. Recap Outline
  • 20. Docker allows you to build and run code in containers
  • 21. Containers are like lightweight Virtual Machines (VMs)
  • 22. Like an isolated process that happens to be an entire OS
  • 23. > docker run –it ubuntu bash root@12345:/# echo "I'm in $(cat /etc/issue)” I'm in Ubuntu 14.04.4 LTS Running an Ubuntu image in a Docker container
  • 24. > time docker run ubuntu echo "Hello, World" Hello, World real 0m0.183s user 0m0.009s sys 0m0.014s Containers boot quickly, with minimal CPU/memory overhead
  • 25. You can define a Docker image as code in a Dockerfile
  • 26. FROM gliderlabs/alpine:3.3 RUN apk --no-cache add ruby ruby-dev RUN gem install sinatra --no-ri --no-rdoc RUN mkdir -p /usr/src/app COPY . /usr/src/app WORKDIR /usr/src/app EXPOSE 4567 CMD ["ruby", "app.rb"] Here is the Dockerfile for the Sinatra backend
  • 27. FROM gliderlabs/alpine:3.3 RUN apk --no-cache add ruby ruby-dev RUN gem install sinatra --no-ri --no-rdoc RUN mkdir -p /usr/src/app COPY . /usr/src/app WORKDIR /usr/src/app EXPOSE 4567 CMD ["ruby", "app.rb"] It specifies dependencies, code, config, and how to run the app
  • 28. > docker build -t gruntwork/sinatra-backend . Step 0 : FROM gliderlabs/alpine:3.3 ---> 0a7e169bce21 (...) Step 8 : CMD ruby app.rb ---> 2e243eba30ed Successfully built 2e243eba30ed Build the Docker image
  • 29. > docker run -it -p 4567:4567 gruntwork/sinatra-backend INFO WEBrick 1.3.1 INFO ruby 2.2.4 (2015-12-16) [x86_64-linux-musl] == Sinatra (v1.4.7) has taken the stage on 4567 for development with backup from WEBrick INFO WEBrick::HTTPServer#start: pid=1 port=4567 Run the Docker image
  • 30. > docker push gruntwork/sinatra-backend The push refers to a repository [docker.io/gruntwork/sinatra- backend] (len: 1) 2e243eba30ed: Image successfully pushed 7e2e0c53e246: Image successfully pushed 919d9a73b500: Image successfully pushed (...) v1: digest: sha256:09f48ed773966ec7fe4558 size: 14319 You can share your images by pushing them to Docker Hub
  • 31. Now you can reuse the same image in dev, stg, prod, etc
  • 32. > docker pull rails:4.2.6 And you can reuse images created by others.
  • 33. FROM rails:4.2.6 RUN mkdir -p /usr/src/app COPY . /usr/src/app WORKDIR /usr/src/app RUN bundle install EXPOSE 3000 CMD ["rails", "start"] The rails-frontend is built on top of the official rails Docker image
  • 34. rails_frontend: image: gruntwork/rails-frontend ports: - "3000:3000" links: - sinatra_backend sinatra_backend: image: gruntwork/sinatra-backend ports: - "4567:4567" Define your entire dev stack as code with docker-compose
  • 35. rails_frontend: image: gruntwork/rails-frontend ports: - "3000:3000" links: - sinatra_backend sinatra_backend: image: gruntwork/sinatra-backend ports: - "4567:4567" Docker links provide a simple service discovery mechanism
  • 36. > docker-compose up Starting infrastructureascodetalk_sinatra_backend_1 Recreating infrastructureascodetalk_rails_frontend_1 sinatra_backend_1 | INFO WEBrick 1.3.1 sinatra_backend_1 | INFO ruby 2.2.4 (2015-12-16) sinatra_backend_1 | Sinatra has taken the stage on 4567 rails_frontend_1 | INFO WEBrick 1.3.1 rails_frontend_1 | INFO ruby 2.3.0 (2015-12-25) rails_frontend_1 | INFO WEBrick::HTTPServer#start: port=3000 Run your entire dev stack with one command
  • 37. 1. Docker 2. Terraform 3. ECS 4. Recap Outline
  • 38. Terraform is a tool for provisioning infrastructure
  • 40. And many resources for each provider
  • 41. You define infrastructure as code in Terraform templates
  • 42. provider "aws" { region = "us-east-1" } resource "aws_instance" "example" { ami = "ami-408c7f28" instance_type = "t2.micro" } This template creates a single EC2 instance in AWS
  • 43. > terraform plan + aws_instance.example ami: "" => "ami-408c7f28" instance_type: "" => "t2.micro" key_name: "" => "<computed>" private_ip: "" => "<computed>" public_ip: "" => "<computed>" Plan: 1 to add, 0 to change, 0 to destroy. Use the plan command to see what you’re about to deploy
  • 44. > terraform apply aws_instance.example: Creating... ami: "" => "ami-408c7f28" instance_type: "" => "t2.micro" key_name: "" => "<computed>" private_ip: "" => "<computed>" public_ip: "" => "<computed>” aws_instance.example: Creation complete Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Use the apply command to apply the changes
  • 45. Now our EC2 instance is running!
  • 46. resource "aws_instance" "example" { ami = "ami-408c7f28" instance_type = "t2.micro" tags { Name = "terraform-example" } } Let’s give the EC2 instance a tag with a readable name
  • 47. > terraform plan ~ aws_instance.example tags.#: "0" => "1" tags.Name: "" => "terraform-example" Plan: 0 to add, 1 to change, 0 to destroy. Use the plan command again to verify your changes
  • 48. > terraform apply aws_instance.example: Refreshing state... aws_instance.example: Modifying... tags.#: "0" => "1" tags.Name: "" => "terraform-example" aws_instance.example: Modifications complete Apply complete! Resources: 0 added, 1 changed, 0 destroyed. Use the apply command again to deploy those changes
  • 49. Now our EC2 instance has a tag!
  • 50. resource "aws_elb" "example" { name = "example" availability_zones = ["us-east-1a", "us-east-1b"] instances = ["${aws_instance.example.id}"] listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.instance_port}" instance_protocol = "http” } } Let’s add an Elastic Load Balancer (ELB).
  • 51. resource "aws_elb" "example" { name = "example" availability_zones = ["us-east-1a", "us-east-1b"] instances = ["${aws_instance.example.id}"] listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.instance_port}" instance_protocol = "http” } } Terraform supports variables, such as var.instance_port
  • 52. resource "aws_elb" "example" { name = "example" availability_zones = ["us-east-1a", "us-east-1b"] instances = ["${aws_instance.example.id}"] listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.instance_port}" instance_protocol = "http" } } As well as dependencies like aws_instance.example.id
  • 53. resource "aws_elb" "example" { name = "example" availability_zones = ["us-east-1a", "us-east-1b"] instances = ["${aws_instance.example.id}"] listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.instance_port}" instance_protocol = "http" } } It builds a dependency graph and applies it in parallel.
  • 54. After running apply, we have an ELB!
  • 55. > terraform destroy aws_instance.example: Refreshing state... (ID: i-f3d58c70) aws_elb.example: Refreshing state... (ID: example) aws_elb.example: Destroying... aws_elb.example: Destruction complete aws_instance.example: Destroying... aws_instance.example: Destruction complete Apply complete! Resources: 0 added, 0 changed, 2 destroyed. Use the destroy command to delete all your resources
  • 56. For more info, check out The Comprehensive Guide to Terraform
  • 57. 1. Docker 2. Terraform 3. ECS 4. Recap Outline
  • 58. EC2 Container Service (ECS) is a way to run Docker on AWS
  • 59. ECS Overview EC2 Instance ECS Cluster ECS Scheduler ECS Agent ECS Tasks ECS Task Definition { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Service Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, }
  • 60. ECS Cluster: several servers managed by ECS EC2 Instance ECS Cluster
  • 61. Typically, the servers are in an Auto Scaling Group Auto Scaling Group EC2 Instance
  • 62. Each server must run the ECS Agent ECS Agent EC2 Instance ECS Cluster
  • 63. ECS Task: Docker container(s) to run, resources they need { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } ECS Agent EC2 Instance ECS Task Definition ECS Cluster
  • 64. ECS Service: long-running ECS Task & ELB settings { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent EC2 Instance ECS Task Definition ECS Service Definition ECS Cluster
  • 65. ECS Scheduler: Deploys Tasks across the ECS Cluster { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent ECS Tasks EC2 Instance ECS Task Definition ECS Service Definition ECS Scheduler ECS Cluster
  • 66. You can associate an ALB or ELB with each ECS service { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent ECS Tasks EC2 Instance ECS Task Definition ECS Service Definition ECS Cluster
  • 67. This allows you to distribute load across your ECS Tasks { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent ECS Tasks EC2 Instance ECS Task Definition ECS Service Definition ECS Cluster
  • 68. You can also use it as a simple form of service discovery { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent ECS Tasks EC2 Instance ECS Task Definition ECS Service Definition ECS Cluster
  • 69. Let’s deploy our apps on ECS using Terraform
  • 70. Define the ECS Cluster as an Auto Scaling Group (ASG) EC2 Instance ECS Cluster
  • 71. resource "aws_ecs_cluster" "example_cluster" { name = "example-cluster" } resource "aws_autoscaling_group" "ecs_cluster_instances" { name = "ecs-cluster-instances" min_size = 5 max_size = 5 launch_configuration = "${aws_launch_configuration.ecs_instance.name}" }
  • 72. Ensure each server in the ASG runs the ECS Agent ECS Agent EC2 Instance ECS Cluster
  • 73. # The launch config defines what runs on each EC2 instance resource "aws_launch_configuration" "ecs_instance" { name_prefix = "ecs-instance-" instance_type = "t2.micro" # This is an Amazon ECS AMI, which has an ECS Agent # installed that lets it talk to the ECS cluster image_id = "ami-a98cb2c3” } The launch config runs AWS ECS Linux on each server in the ASG
  • 74. Define an ECS Task for each microservice { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } ECS Agent EC2 Instance ECS Task Definition ECS Cluster
  • 75. resource "aws_ecs_task_definition" "rails_frontend" { family = "rails-frontend" container_definitions = <<EOF [{ "name": "rails-frontend", "image": "gruntwork/rails-frontend:v1", "cpu": 1024, "memory": 768, "essential": true, "portMappings": [{"containerPort": 3000, "hostPort": 3000}] }] EOF } Rails frontend ECS Task
  • 76. resource "aws_ecs_task_definition" "sinatra_backend" { family = "sinatra-backend" container_definitions = <<EOF [{ "name": "sinatra-backend", "image": "gruntwork/sinatra-backend:v1", "cpu": 1024, "memory": 768, "essential": true, "portMappings": [{"containerPort": 4567, "hostPort": 4567}] }] EOF } Sinatra Backend ECS Task
  • 77. Define an ECS Service for each ECS Task { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent EC2 Instance ECS Task Definition ECS Service Definition ECS Cluster
  • 78. resource "aws_ecs_service" "rails_frontend" { family = "rails-frontend" cluster = "${aws_ecs_cluster.example_cluster.id}" task_definition = "${aws_ecs_task_definition.rails-fronted.arn}" desired_count = 2 } Rails Frontend ECS Service
  • 79. resource "aws_ecs_service" "sinatra_backend" { family = "sinatra-backend" cluster = "${aws_ecs_cluster.example_cluster.id}" task_definition = "${aws_ecs_task_definition.sinatra_backend.arn}" desired_count = 2 } Sinatra Backend ECS Service
  • 80. Associate an ELB with each ECS Service { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent ECS Tasks EC2 Instance ECS Task Definition ECS Service Definition ECS Cluster
  • 81. resource "aws_elb" "rails_frontend" { name = "rails-frontend" listener { lb_port = 80 lb_protocol = "http" instance_port = 3000 instance_protocol = "http" } } Rails Frontend ELB
  • 82. resource "aws_ecs_service" "rails_frontend" { (...) load_balancer { elb_name = "${aws_elb.rails_frontend.id}" container_name = "rails-frontend" container_port = 3000 } } Associate the ELB with the Rails Frontend ECS Service
  • 83. resource "aws_elb" "sinatra_backend" { name = "sinatra-backend" listener { lb_port = 4567 lb_protocol = "http" instance_port = 4567 instance_protocol = "http" } } Sinatra Backend ELB
  • 84. resource "aws_ecs_service" "sinatra_backend" { (...) load_balancer { elb_name = "${aws_elb.sinatra_backend.id}" container_name = "sinatra-backend" container_port = 4567 } } Associate the ELB with the Sinatra Backend ECS Service
  • 85. Set up service discovery between the ECS Services { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent ECS Tasks EC2 Instance ECS Task Definition ECS Service Definition ECS Cluster
  • 86. resource "aws_ecs_task_definition" "rails_frontend" { family = "rails-frontend" container_definitions = <<EOF [{ ... "environment": [{ "name": "SINATRA_BACKEND_PORT", "value": "tcp://${aws_elb.sinatra_backend.dns_name}:4567" }] }] EOF } Pass the Sinatra Bckend ELB URL as env var to Rails Frontend
  • 87. It’s time to deploy! { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Agent ECS Tasks EC2 Instance ECS Task Definition ECS Service Definition ECS Scheduler ECS Cluster
  • 88. > terraform apply aws_ecs_cluster.example_cluster: Creating... name: "" => "example-cluster" aws_ecs_task_definition.sinatra_backend: Creating... ... Apply complete! Resources: 17 added, 0 changed, 0 destroyed. Use the apply command to deploy the ECS Cluster & Tasks
  • 89. See the cluster in the ECS console
  • 90. Track events for each Service
  • 91. As well as basic metrics
  • 93. 1. Docker 2. Terraform 3. ECS 4. Recap Outline
  • 94. Slides and code from this talk: ybrikman.com/speaking