SlideShare a Scribd company logo
ETCD CLUSTERS ON AWS
Terraform + CoreOS for
1
TEXT
ABOUT ME
▸ Software engineer, Dev-Ops by chance
▸ Currently at reBuy.de, helping with migration to AWS
▸ Previously - 4 years at Amazon (AWS)
2
THE PREMISE
WHAT IS ETCD?
▸ Distributed key-value store
▸ Based on Raft consensus algorithm
▸ Similar to Consul and ZooKeeper
▸ Used for storing state of distributed applications
(Kubernetes, Fleet, CoreUpdate)
▸ Should be treated like a database
▸ Comes bundled with CoreOS
3
THE ENVIRONMENT
TYPICAL ETCD DEPLOYMENT
▸ Odd number of instances
▸ Evenly distributed across AZs
▸ Low-latency connectivity between nodes
▸ Persistent storage (EBS)
▸ A way to determine the list of nodes
4
THE PROCESS
BOOTSTRAPPING ETCD
▸ Nodes need prior knowledge about all other nodes
▸ The bootstrap phase is a one-off scenario
▸ Has support for discovering nodes (DNS SRV records)
▸ Can use discovery for clients
5
THE PROCESS
…ON AWS
▸ Prepare CoreOS configuration (cloud-config)
▸ Launch node instances
▸ Create discovery DNS records
▸ Profit!
6
TERRAFORM + CoreOS
FINDING THE NODES
▸ Through DNS SRV records
▸ Route53 private DNS inside VPC
▸ Nodes get a stable hostname

(not ip-172-31-2-219.eu-west-1.compute.internal)
7
TERRAFORM + CoreOS
resource "aws_route53_record" "etcd_srv_discover" {



name = "_etcd-server._tcp"

type = "SRV"

records = ["${formatlist("0 0 2380 %s", aws_route53_record.etc_a_nodes.*.fqdn)}"]

ttl = “300"
zone_id = "${aws_route53_zone.etcd_zone.id}"

}


resource "aws_route53_record" "etc_a_nodes" {



count = "${var.node_count}"

type = "A" name = "node-${count.index}"

records = ["${aws_instance.etcd_node.*.private_ip[count.index]}"]

ttl = 300

zone_id = "${aws_route53_zone.etcd_zone.id}"

}
STABLE HOST NAMES
8
$ dig _etcd-server._tcp.cluster.etcd SRV
_etcd-server._tcp.cluster.etcd. 183 IN SRV 0 0 2380 node-0.cluster.etcd.
_etcd-server._tcp.cluster.etcd. 183 IN SRV 0 0 2380 node-1.cluster.etcd.
_etcd-server._tcp.cluster.etcd. 183 IN SRV 0 0 2380 node-2.cluster.etcd.
TERRAFORM + COREOS
CONFIGURING CoreOS
▸ Uses own version of cloud-init (subset of cloud-config)
▸ Config as EC2 user-data
▸ Template data-source for user-data
▸ Has to include hostname and DNS domain for discovery
9
TERRAFORM + CoreOS
CoreOS CONFIG AS USERDATA
#cloud-config

hostname: ${node_name}

coreos:

update:

reboot-strategy: "etcd-lock"

etcd2:

name: ${node_name}

discovery-srv: ${cluster_domain}

listen-peer-urls: "http://$private_ipv4:2380"

listen-client-urls: "http://0.0.0.0:2379"

initial-advertise-peer-urls: "http://${node_name}:2380"

advertise-client-urls: "http://${node_name}:2379"

units:

- name: "etcd2.service"

enable: false

command: start
10
data "template_file" "userdata" {



count = "${var.node_count}"

template = "${file("${path.root}/../resources/userdata.yaml")}"

vars {

node_name = "node-${count.index}.${var.cluster_domain}"

cluster_domain = "${var.cluster_domain}"

}

}
TERRAFORM + CoreOS
LAUNCH NODES
11
resource "aws_instance" "etcd_node" {



count = "${var.node_count}"

ami = "${data.aws_ami.coreos_ami.id}"

instance_type = "t2.medium"

subnet_id = "${aws_subnet.az_subnet.*.id[count.index]}"

key_name = "${aws_key_pair.ssh-key.id}"

user_data = "${data.template_file.userdata.*.rendered[count.index]}"

}
$ terraform apply
core@node-1 ~ $ etcdctl cluster-health
member 5bea3befcd2b527d is healthy: got healthy result from http://node-2.cluster.etcd:2379
member bfc4d7d3459cc4cb is healthy: got healthy result from http://node-1.cluster.etcd:2379
member d1b3f464b49063ac is healthy: got healthy result from http://node-0.cluster.etcd:2379
cluster is healthy
DEMO ?!
honourable crowd
TERRAFORM + CoreOS 12
TERRAFORM + CoreOS
THAT'S IT!
Take-aways:
▸ etcd operations are deliberately “manual”
▸ etcd requires a source-of-truth for member list (Terraform)
▸ auto-scaling possible, but discouraged
▸ Route53 useful for service discovery
13
TERRAFORM + CoreOS
QUESTIONS?
Terraform module at:
https://github.com/alexsomesan/tf-simple-etcd
Get in touch!
alex.somesan@gmail.com
@ASomesan
14

More Related Content

What's hot

37562259 top-consuming-process
37562259 top-consuming-process37562259 top-consuming-process
37562259 top-consuming-process
skumner
 
Node.js - A Quick Tour II
Node.js - A Quick Tour IINode.js - A Quick Tour II
Node.js - A Quick Tour II
Felix Geisendörfer
 
Dirty - How simple is your database?
Dirty - How simple is your database?Dirty - How simple is your database?
Dirty - How simple is your database?
Felix Geisendörfer
 
Node.js - A practical introduction (v2)
Node.js  - A practical introduction (v2)Node.js  - A practical introduction (v2)
Node.js - A practical introduction (v2)
Felix Geisendörfer
 
Hadley verse
Hadley verseHadley verse
Hadley verse
Ajay Ohri
 
Everything as Code with Terraform
Everything as Code with TerraformEverything as Code with Terraform
Everything as Code with Terraform
Mitchell Pronschinske
 
Alluxio in MOMO
Alluxio in MOMOAlluxio in MOMO
Alluxio in MOMO
Alluxio, Inc.
 
The Practice of Alluxio in Near Real-Time Data Platform at VIPShop [Chinese]
The Practice of Alluxio in Near Real-Time Data Platform at VIPShop [Chinese]The Practice of Alluxio in Near Real-Time Data Platform at VIPShop [Chinese]
The Practice of Alluxio in Near Real-Time Data Platform at VIPShop [Chinese]
Alluxio, Inc.
 
Solr on Docker - the Good, the Bad and the Ugly
Solr on Docker - the Good, the Bad and the UglySolr on Docker - the Good, the Bad and the Ugly
Solr on Docker - the Good, the Bad and the Ugly
Sematext Group, Inc.
 
CoreOS + Kubernetes @ All Things Open 2015
CoreOS + Kubernetes @ All Things Open 2015CoreOS + Kubernetes @ All Things Open 2015
CoreOS + Kubernetes @ All Things Open 2015
Brandon Philips
 
Automated Hadoop Cluster Construction on EC2
Automated Hadoop Cluster Construction on EC2Automated Hadoop Cluster Construction on EC2
Automated Hadoop Cluster Construction on EC2
Mark Kerzner
 
Nodejs - A quick tour (v6)
Nodejs - A quick tour (v6)Nodejs - A quick tour (v6)
Nodejs - A quick tour (v6)
Felix Geisendörfer
 
Tuning tips for Apache Spark Jobs
Tuning tips for Apache Spark JobsTuning tips for Apache Spark Jobs
Tuning tips for Apache Spark Jobs
Samir Bessalah
 
Mysqlnd uh
Mysqlnd uhMysqlnd uh
Mysqlnd uh
natmchugh
 
Everyone Loves a Sausage
Everyone Loves a SausageEveryone Loves a Sausage
Everyone Loves a Sausage
Nick Jones
 
Warp 10 Platform Presentation - Criteo Beer & Tech 2016-02-03
Warp 10 Platform Presentation - Criteo Beer & Tech 2016-02-03Warp 10 Platform Presentation - Criteo Beer & Tech 2016-02-03
Warp 10 Platform Presentation - Criteo Beer & Tech 2016-02-03
Mathias Herberts
 
Terraform 9
Terraform 9Terraform 9
Terraform 9
Jerry Singh
 
RedisConf17- durable_rules
RedisConf17- durable_rulesRedisConf17- durable_rules
RedisConf17- durable_rules
Redis Labs
 
Environment for training models
Environment for training modelsEnvironment for training models
Environment for training models
FlyElephant
 
Asphalt8.ifgiovanni
Asphalt8.ifgiovanniAsphalt8.ifgiovanni
Asphalt8.ifgiovanni
201419942010
 

What's hot (20)

37562259 top-consuming-process
37562259 top-consuming-process37562259 top-consuming-process
37562259 top-consuming-process
 
Node.js - A Quick Tour II
Node.js - A Quick Tour IINode.js - A Quick Tour II
Node.js - A Quick Tour II
 
Dirty - How simple is your database?
Dirty - How simple is your database?Dirty - How simple is your database?
Dirty - How simple is your database?
 
Node.js - A practical introduction (v2)
Node.js  - A practical introduction (v2)Node.js  - A practical introduction (v2)
Node.js - A practical introduction (v2)
 
Hadley verse
Hadley verseHadley verse
Hadley verse
 
Everything as Code with Terraform
Everything as Code with TerraformEverything as Code with Terraform
Everything as Code with Terraform
 
Alluxio in MOMO
Alluxio in MOMOAlluxio in MOMO
Alluxio in MOMO
 
The Practice of Alluxio in Near Real-Time Data Platform at VIPShop [Chinese]
The Practice of Alluxio in Near Real-Time Data Platform at VIPShop [Chinese]The Practice of Alluxio in Near Real-Time Data Platform at VIPShop [Chinese]
The Practice of Alluxio in Near Real-Time Data Platform at VIPShop [Chinese]
 
Solr on Docker - the Good, the Bad and the Ugly
Solr on Docker - the Good, the Bad and the UglySolr on Docker - the Good, the Bad and the Ugly
Solr on Docker - the Good, the Bad and the Ugly
 
CoreOS + Kubernetes @ All Things Open 2015
CoreOS + Kubernetes @ All Things Open 2015CoreOS + Kubernetes @ All Things Open 2015
CoreOS + Kubernetes @ All Things Open 2015
 
Automated Hadoop Cluster Construction on EC2
Automated Hadoop Cluster Construction on EC2Automated Hadoop Cluster Construction on EC2
Automated Hadoop Cluster Construction on EC2
 
Nodejs - A quick tour (v6)
Nodejs - A quick tour (v6)Nodejs - A quick tour (v6)
Nodejs - A quick tour (v6)
 
Tuning tips for Apache Spark Jobs
Tuning tips for Apache Spark JobsTuning tips for Apache Spark Jobs
Tuning tips for Apache Spark Jobs
 
Mysqlnd uh
Mysqlnd uhMysqlnd uh
Mysqlnd uh
 
Everyone Loves a Sausage
Everyone Loves a SausageEveryone Loves a Sausage
Everyone Loves a Sausage
 
Warp 10 Platform Presentation - Criteo Beer & Tech 2016-02-03
Warp 10 Platform Presentation - Criteo Beer & Tech 2016-02-03Warp 10 Platform Presentation - Criteo Beer & Tech 2016-02-03
Warp 10 Platform Presentation - Criteo Beer & Tech 2016-02-03
 
Terraform 9
Terraform 9Terraform 9
Terraform 9
 
RedisConf17- durable_rules
RedisConf17- durable_rulesRedisConf17- durable_rules
RedisConf17- durable_rules
 
Environment for training models
Environment for training modelsEnvironment for training models
Environment for training models
 
Asphalt8.ifgiovanni
Asphalt8.ifgiovanniAsphalt8.ifgiovanni
Asphalt8.ifgiovanni
 

Viewers also liked

Overview on IFX Standards
Overview on IFX StandardsOverview on IFX Standards
Overview on IFX Standards
Pinaki Guha
 
Interactive Financial Exchange (IFX)
Interactive Financial Exchange (IFX)Interactive Financial Exchange (IFX)
Interactive Financial Exchange (IFX)
Pratheeban Rajendran
 
Terraform
TerraformTerraform
Terraform
Otto Jongerius
 
Rapid Infrastructure Provisioning
Rapid Infrastructure ProvisioningRapid Infrastructure Provisioning
Rapid Infrastructure Provisioning
Uchit Vyas ☁
 
Terraform and cloud.ca
Terraform and cloud.caTerraform and cloud.ca
Terraform and cloud.ca
CloudOps2005
 
Infrastructure as Code: Introduction to Terraform
Infrastructure as Code: Introduction to TerraformInfrastructure as Code: Introduction to Terraform
Infrastructure as Code: Introduction to Terraform
Alexander Popov
 
Terraform
TerraformTerraform
Terraform
Adam Vincze
 
Terraform
TerraformTerraform
Terraform at Scale
Terraform at ScaleTerraform at Scale
Terraform at Scale
Calvin French-Owen
 
DevOps - Infrastructure as Code by Andre Marcelo-Tanner
DevOps - Infrastructure as Code by Andre Marcelo-TannerDevOps - Infrastructure as Code by Andre Marcelo-Tanner
DevOps - Infrastructure as Code by Andre Marcelo-Tanner
DEVCON
 
Terraform: Cloud Configuration Management (WTC/IPC'16)
Terraform: Cloud Configuration Management (WTC/IPC'16)Terraform: Cloud Configuration Management (WTC/IPC'16)
Terraform: Cloud Configuration Management (WTC/IPC'16)
Martin Schütte
 
Terraform Introduction
Terraform IntroductionTerraform Introduction
Terraform Introduction
soniasnowfrog
 
2016 - IGNITE - Terraform to go from Zero to Prod in less than 1 month and TH...
2016 - IGNITE - Terraform to go from Zero to Prod in less than 1 month and TH...2016 - IGNITE - Terraform to go from Zero to Prod in less than 1 month and TH...
2016 - IGNITE - Terraform to go from Zero to Prod in less than 1 month and TH...
devopsdaysaustin
 
Refactoring terraform
Refactoring terraformRefactoring terraform
Refactoring terraform
Nell Shamrell-Harrington
 
Rediscovering Developer Opportunities in the Philippines by Fred Tshidimba
Rediscovering Developer Opportunities in the Philippines by Fred TshidimbaRediscovering Developer Opportunities in the Philippines by Fred Tshidimba
Rediscovering Developer Opportunities in the Philippines by Fred Tshidimba
DEVCON
 
TerraformでECS+ECRする話
TerraformでECS+ECRする話TerraformでECS+ECRする話
TerraformでECS+ECRする話
Satoshi Hirayama
 
Jsonnet, terraform & packer
Jsonnet, terraform & packerJsonnet, terraform & packer
Jsonnet, terraform & packer
David Cunningham
 
Infrastructure as code with Terraform
Infrastructure as code with TerraformInfrastructure as code with Terraform
Infrastructure as code with Terraform
Sam Bashton
 
London Hug 19/5 - Terraform in Production
London Hug 19/5 - Terraform in ProductionLondon Hug 19/5 - Terraform in Production
London Hug 19/5 - Terraform in Production
London HashiCorp User Group
 
Automation with Packer and TerraForm
Automation with Packer and TerraFormAutomation with Packer and TerraForm
Automation with Packer and TerraForm
Wesley Charles Blake
 

Viewers also liked (20)

Overview on IFX Standards
Overview on IFX StandardsOverview on IFX Standards
Overview on IFX Standards
 
Interactive Financial Exchange (IFX)
Interactive Financial Exchange (IFX)Interactive Financial Exchange (IFX)
Interactive Financial Exchange (IFX)
 
Terraform
TerraformTerraform
Terraform
 
Rapid Infrastructure Provisioning
Rapid Infrastructure ProvisioningRapid Infrastructure Provisioning
Rapid Infrastructure Provisioning
 
Terraform and cloud.ca
Terraform and cloud.caTerraform and cloud.ca
Terraform and cloud.ca
 
Infrastructure as Code: Introduction to Terraform
Infrastructure as Code: Introduction to TerraformInfrastructure as Code: Introduction to Terraform
Infrastructure as Code: Introduction to Terraform
 
Terraform
TerraformTerraform
Terraform
 
Terraform
TerraformTerraform
Terraform
 
Terraform at Scale
Terraform at ScaleTerraform at Scale
Terraform at Scale
 
DevOps - Infrastructure as Code by Andre Marcelo-Tanner
DevOps - Infrastructure as Code by Andre Marcelo-TannerDevOps - Infrastructure as Code by Andre Marcelo-Tanner
DevOps - Infrastructure as Code by Andre Marcelo-Tanner
 
Terraform: Cloud Configuration Management (WTC/IPC'16)
Terraform: Cloud Configuration Management (WTC/IPC'16)Terraform: Cloud Configuration Management (WTC/IPC'16)
Terraform: Cloud Configuration Management (WTC/IPC'16)
 
Terraform Introduction
Terraform IntroductionTerraform Introduction
Terraform Introduction
 
2016 - IGNITE - Terraform to go from Zero to Prod in less than 1 month and TH...
2016 - IGNITE - Terraform to go from Zero to Prod in less than 1 month and TH...2016 - IGNITE - Terraform to go from Zero to Prod in less than 1 month and TH...
2016 - IGNITE - Terraform to go from Zero to Prod in less than 1 month and TH...
 
Refactoring terraform
Refactoring terraformRefactoring terraform
Refactoring terraform
 
Rediscovering Developer Opportunities in the Philippines by Fred Tshidimba
Rediscovering Developer Opportunities in the Philippines by Fred TshidimbaRediscovering Developer Opportunities in the Philippines by Fred Tshidimba
Rediscovering Developer Opportunities in the Philippines by Fred Tshidimba
 
TerraformでECS+ECRする話
TerraformでECS+ECRする話TerraformでECS+ECRする話
TerraformでECS+ECRする話
 
Jsonnet, terraform & packer
Jsonnet, terraform & packerJsonnet, terraform & packer
Jsonnet, terraform & packer
 
Infrastructure as code with Terraform
Infrastructure as code with TerraformInfrastructure as code with Terraform
Infrastructure as code with Terraform
 
London Hug 19/5 - Terraform in Production
London Hug 19/5 - Terraform in ProductionLondon Hug 19/5 - Terraform in Production
London Hug 19/5 - Terraform in Production
 
Automation with Packer and TerraForm
Automation with Packer and TerraFormAutomation with Packer and TerraForm
Automation with Packer and TerraForm
 

Similar to Etcd terraform by Alex Somesan

CoreOS intro
CoreOS introCoreOS intro
CoreOS intro
Timo Derstappen
 
Develop and deploy using Hybrid Cloud Strategies confoo2012
Develop and deploy using Hybrid Cloud Strategies confoo2012Develop and deploy using Hybrid Cloud Strategies confoo2012
Develop and deploy using Hybrid Cloud Strategies confoo2012
Combell NV
 
Hybrid Cloud PHPUK2012
Hybrid Cloud PHPUK2012Hybrid Cloud PHPUK2012
Hybrid Cloud PHPUK2012
Combell NV
 
Multi-Region Cassandra Clusters
Multi-Region Cassandra ClustersMulti-Region Cassandra Clusters
Multi-Region Cassandra Clusters
Instaclustr
 
Aprovisionamiento multi-proveedor con Terraform - Plain Concepts DevOps day
Aprovisionamiento multi-proveedor con Terraform  - Plain Concepts DevOps dayAprovisionamiento multi-proveedor con Terraform  - Plain Concepts DevOps day
Aprovisionamiento multi-proveedor con Terraform - Plain Concepts DevOps day
Plain Concepts
 
CoreOS @Codetalks Hamburg
CoreOS @Codetalks HamburgCoreOS @Codetalks Hamburg
CoreOS @Codetalks Hamburg
Timo Derstappen
 
You know, for search. Querying 24 Billion Documents in 900ms
You know, for search. Querying 24 Billion Documents in 900msYou know, for search. Querying 24 Billion Documents in 900ms
You know, for search. Querying 24 Billion Documents in 900ms
Jodok Batlogg
 
DevOps Enabling Your Team
DevOps Enabling Your TeamDevOps Enabling Your Team
DevOps Enabling Your Team
GR8Conf
 
Cloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the CloudCloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the Cloud
petriojala123
 
OpenStack LA meetup Feb 18, 2015
OpenStack LA meetup Feb 18, 2015OpenStack LA meetup Feb 18, 2015
OpenStack LA meetup Feb 18, 2015
Tesora
 
Reusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modules
Yevgeniy Brikman
 
CoreOS @ summer meetup in Utrecht
CoreOS @ summer meetup in UtrechtCoreOS @ summer meetup in Utrecht
CoreOS @ summer meetup in Utrecht
Timo Derstappen
 
Learning Puppet basic thing
Learning Puppet basic thing Learning Puppet basic thing
Learning Puppet basic thing
DaeHyung Lee
 
Terraform for azure: the good, the bad and the ugly -
Terraform for azure: the good, the bad and the ugly -Terraform for azure: the good, the bad and the ugly -
Terraform for azure: the good, the bad and the ugly -
Giulio Vian
 
Fullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-endFullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-end
Ezequiel Maraschio
 
Introduction to Docker & CoreOS - Symfony User Group Cologne
Introduction to Docker & CoreOS - Symfony User Group CologneIntroduction to Docker & CoreOS - Symfony User Group Cologne
Introduction to Docker & CoreOS - Symfony User Group Cologne
D
 
CoreOS, or How I Learned to Stop Worrying and Love Systemd
CoreOS, or How I Learned to Stop Worrying and Love SystemdCoreOS, or How I Learned to Stop Worrying and Love Systemd
CoreOS, or How I Learned to Stop Worrying and Love Systemd
Richard Lister
 
Declarative & workflow based infrastructure with Terraform
Declarative & workflow based infrastructure with TerraformDeclarative & workflow based infrastructure with Terraform
Declarative & workflow based infrastructure with Terraform
Radek Simko
 
Infrastructure as Code with Terraform
Infrastructure as Code with TerraformInfrastructure as Code with Terraform
Infrastructure as Code with Terraform
Pedro J. Molina
 
Cassandra Summit 2014: Down with Tweaking! Removing Tunable Complexity for Ca...
Cassandra Summit 2014: Down with Tweaking! Removing Tunable Complexity for Ca...Cassandra Summit 2014: Down with Tweaking! Removing Tunable Complexity for Ca...
Cassandra Summit 2014: Down with Tweaking! Removing Tunable Complexity for Ca...
DataStax Academy
 

Similar to Etcd terraform by Alex Somesan (20)

CoreOS intro
CoreOS introCoreOS intro
CoreOS intro
 
Develop and deploy using Hybrid Cloud Strategies confoo2012
Develop and deploy using Hybrid Cloud Strategies confoo2012Develop and deploy using Hybrid Cloud Strategies confoo2012
Develop and deploy using Hybrid Cloud Strategies confoo2012
 
Hybrid Cloud PHPUK2012
Hybrid Cloud PHPUK2012Hybrid Cloud PHPUK2012
Hybrid Cloud PHPUK2012
 
Multi-Region Cassandra Clusters
Multi-Region Cassandra ClustersMulti-Region Cassandra Clusters
Multi-Region Cassandra Clusters
 
Aprovisionamiento multi-proveedor con Terraform - Plain Concepts DevOps day
Aprovisionamiento multi-proveedor con Terraform  - Plain Concepts DevOps dayAprovisionamiento multi-proveedor con Terraform  - Plain Concepts DevOps day
Aprovisionamiento multi-proveedor con Terraform - Plain Concepts DevOps day
 
CoreOS @Codetalks Hamburg
CoreOS @Codetalks HamburgCoreOS @Codetalks Hamburg
CoreOS @Codetalks Hamburg
 
You know, for search. Querying 24 Billion Documents in 900ms
You know, for search. Querying 24 Billion Documents in 900msYou know, for search. Querying 24 Billion Documents in 900ms
You know, for search. Querying 24 Billion Documents in 900ms
 
DevOps Enabling Your Team
DevOps Enabling Your TeamDevOps Enabling Your Team
DevOps Enabling Your Team
 
Cloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the CloudCloud Meetup - Automation in the Cloud
Cloud Meetup - Automation in the Cloud
 
OpenStack LA meetup Feb 18, 2015
OpenStack LA meetup Feb 18, 2015OpenStack LA meetup Feb 18, 2015
OpenStack LA meetup Feb 18, 2015
 
Reusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modules
 
CoreOS @ summer meetup in Utrecht
CoreOS @ summer meetup in UtrechtCoreOS @ summer meetup in Utrecht
CoreOS @ summer meetup in Utrecht
 
Learning Puppet basic thing
Learning Puppet basic thing Learning Puppet basic thing
Learning Puppet basic thing
 
Terraform for azure: the good, the bad and the ugly -
Terraform for azure: the good, the bad and the ugly -Terraform for azure: the good, the bad and the ugly -
Terraform for azure: the good, the bad and the ugly -
 
Fullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-endFullstack conf 2017 - Basic dev pipeline end-to-end
Fullstack conf 2017 - Basic dev pipeline end-to-end
 
Introduction to Docker & CoreOS - Symfony User Group Cologne
Introduction to Docker & CoreOS - Symfony User Group CologneIntroduction to Docker & CoreOS - Symfony User Group Cologne
Introduction to Docker & CoreOS - Symfony User Group Cologne
 
CoreOS, or How I Learned to Stop Worrying and Love Systemd
CoreOS, or How I Learned to Stop Worrying and Love SystemdCoreOS, or How I Learned to Stop Worrying and Love Systemd
CoreOS, or How I Learned to Stop Worrying and Love Systemd
 
Declarative & workflow based infrastructure with Terraform
Declarative & workflow based infrastructure with TerraformDeclarative & workflow based infrastructure with Terraform
Declarative & workflow based infrastructure with Terraform
 
Infrastructure as Code with Terraform
Infrastructure as Code with TerraformInfrastructure as Code with Terraform
Infrastructure as Code with Terraform
 
Cassandra Summit 2014: Down with Tweaking! Removing Tunable Complexity for Ca...
Cassandra Summit 2014: Down with Tweaking! Removing Tunable Complexity for Ca...Cassandra Summit 2014: Down with Tweaking! Removing Tunable Complexity for Ca...
Cassandra Summit 2014: Down with Tweaking! Removing Tunable Complexity for Ca...
 

Recently uploaded

OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 

Recently uploaded (20)

OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 

Etcd terraform by Alex Somesan

  • 1. ETCD CLUSTERS ON AWS Terraform + CoreOS for 1
  • 2. TEXT ABOUT ME ▸ Software engineer, Dev-Ops by chance ▸ Currently at reBuy.de, helping with migration to AWS ▸ Previously - 4 years at Amazon (AWS) 2
  • 3. THE PREMISE WHAT IS ETCD? ▸ Distributed key-value store ▸ Based on Raft consensus algorithm ▸ Similar to Consul and ZooKeeper ▸ Used for storing state of distributed applications (Kubernetes, Fleet, CoreUpdate) ▸ Should be treated like a database ▸ Comes bundled with CoreOS 3
  • 4. THE ENVIRONMENT TYPICAL ETCD DEPLOYMENT ▸ Odd number of instances ▸ Evenly distributed across AZs ▸ Low-latency connectivity between nodes ▸ Persistent storage (EBS) ▸ A way to determine the list of nodes 4
  • 5. THE PROCESS BOOTSTRAPPING ETCD ▸ Nodes need prior knowledge about all other nodes ▸ The bootstrap phase is a one-off scenario ▸ Has support for discovering nodes (DNS SRV records) ▸ Can use discovery for clients 5
  • 6. THE PROCESS …ON AWS ▸ Prepare CoreOS configuration (cloud-config) ▸ Launch node instances ▸ Create discovery DNS records ▸ Profit! 6
  • 7. TERRAFORM + CoreOS FINDING THE NODES ▸ Through DNS SRV records ▸ Route53 private DNS inside VPC ▸ Nodes get a stable hostname
 (not ip-172-31-2-219.eu-west-1.compute.internal) 7
  • 8. TERRAFORM + CoreOS resource "aws_route53_record" "etcd_srv_discover" {
 
 name = "_etcd-server._tcp"
 type = "SRV"
 records = ["${formatlist("0 0 2380 %s", aws_route53_record.etc_a_nodes.*.fqdn)}"]
 ttl = “300" zone_id = "${aws_route53_zone.etcd_zone.id}"
 } 
 resource "aws_route53_record" "etc_a_nodes" {
 
 count = "${var.node_count}"
 type = "A" name = "node-${count.index}"
 records = ["${aws_instance.etcd_node.*.private_ip[count.index]}"]
 ttl = 300
 zone_id = "${aws_route53_zone.etcd_zone.id}"
 } STABLE HOST NAMES 8 $ dig _etcd-server._tcp.cluster.etcd SRV _etcd-server._tcp.cluster.etcd. 183 IN SRV 0 0 2380 node-0.cluster.etcd. _etcd-server._tcp.cluster.etcd. 183 IN SRV 0 0 2380 node-1.cluster.etcd. _etcd-server._tcp.cluster.etcd. 183 IN SRV 0 0 2380 node-2.cluster.etcd.
  • 9. TERRAFORM + COREOS CONFIGURING CoreOS ▸ Uses own version of cloud-init (subset of cloud-config) ▸ Config as EC2 user-data ▸ Template data-source for user-data ▸ Has to include hostname and DNS domain for discovery 9
  • 10. TERRAFORM + CoreOS CoreOS CONFIG AS USERDATA #cloud-config
 hostname: ${node_name}
 coreos:
 update:
 reboot-strategy: "etcd-lock"
 etcd2:
 name: ${node_name}
 discovery-srv: ${cluster_domain}
 listen-peer-urls: "http://$private_ipv4:2380"
 listen-client-urls: "http://0.0.0.0:2379"
 initial-advertise-peer-urls: "http://${node_name}:2380"
 advertise-client-urls: "http://${node_name}:2379"
 units:
 - name: "etcd2.service"
 enable: false
 command: start 10 data "template_file" "userdata" {
 
 count = "${var.node_count}"
 template = "${file("${path.root}/../resources/userdata.yaml")}"
 vars {
 node_name = "node-${count.index}.${var.cluster_domain}"
 cluster_domain = "${var.cluster_domain}"
 }
 }
  • 11. TERRAFORM + CoreOS LAUNCH NODES 11 resource "aws_instance" "etcd_node" {
 
 count = "${var.node_count}"
 ami = "${data.aws_ami.coreos_ami.id}"
 instance_type = "t2.medium"
 subnet_id = "${aws_subnet.az_subnet.*.id[count.index]}"
 key_name = "${aws_key_pair.ssh-key.id}"
 user_data = "${data.template_file.userdata.*.rendered[count.index]}"
 } $ terraform apply core@node-1 ~ $ etcdctl cluster-health member 5bea3befcd2b527d is healthy: got healthy result from http://node-2.cluster.etcd:2379 member bfc4d7d3459cc4cb is healthy: got healthy result from http://node-1.cluster.etcd:2379 member d1b3f464b49063ac is healthy: got healthy result from http://node-0.cluster.etcd:2379 cluster is healthy
  • 13. TERRAFORM + CoreOS THAT'S IT! Take-aways: ▸ etcd operations are deliberately “manual” ▸ etcd requires a source-of-truth for member list (Terraform) ▸ auto-scaling possible, but discouraged ▸ Route53 useful for service discovery 13
  • 14. TERRAFORM + CoreOS QUESTIONS? Terraform module at: https://github.com/alexsomesan/tf-simple-etcd Get in touch! alex.somesan@gmail.com @ASomesan 14