Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to service discovery and self-organizing cluster orchestration. HashiCorp Serf and Consul

1,398 views

Published on

Slides by Andriy Kryachkov & Roman V. Babenko during 10th #pivorak.

Published in: Software
  • Login to see the comments

Introduction to service discovery and self-organizing cluster orchestration. HashiCorp Serf and Consul

  1. 1. Introduction to service discovery and self-organizing cluster orchestration #pivorak 2016 Andrey Kryachkov & Roman V. Babenko romanvbabenko@linkedin romanvbabenko@github romanvbabenko@twitter andreykryachkov@linkedin kryachkov@github kryachkov@twitter
  2. 2. Old School web app infrastructure Nodes Hardcoded addresses Rebuild nodes on env change No need for service discovery
  3. 3. Modern web app infrastructure Roles, not nodes No hardcoded addresses Easy failure recovery Seamless scaling up and down
  4. 4. HashiCorp Serf and Consul
  5. 5. Serf Serf is a decentralized solution for cluster membership, failure detection, and orchestration. Lightweight and highly available. https://www.serfdom.io/
  6. 6. Serf MEMBERSHIP FAILURE DETECTION EVENTS AND QUERIES
  7. 7. Serf MEMBERSHIP FAILURE DETECTION EVENTS AND QUERIES
  8. 8. Serf MEMBERSHIP FAILURE DETECTION EVENTS AND QUERIES
  9. 9. Serf MEMBERSHIP FAILURE DETECTION EVENTS AND QUERIES
  10. 10. Serf :: USE CASES Web Servers and Load Balancers Clustering Memcached or Redis Triggering Deploys Updating DNS Records Simple Observability A Building Block for Service Discovery
  11. 11. Serf :: USE CASES Web Servers and Load Balancers Clustering Memcached or Redis Triggering Deploys Updating DNS Records Simple Observability A Building Block for Service Discovery
  12. 12. Serf :: USE CASES Web Servers and Load Balancers Clustering Memcached or Redis Triggering Deploys Updating DNS Records Simple Observability A Building Block for Service Discovery
  13. 13. Serf :: USE CASES Web Servers and Load Balancers Clustering Memcached or Redis Triggering Deploys Updating DNS Records Simple Observability A Building Block for Service Discovery
  14. 14. Serf :: USE CASES Web Servers and Load Balancers Clustering Memcached or Redis Triggering Deploys Updating DNS Records Simple Observability A Building Block for Service Discovery
  15. 15. Serf :: USE CASES Web Servers and Load Balancers Clustering Memcached or Redis Triggering Deploys Updating DNS Records Simple Observability A Building Block for Service Discovery
  16. 16. Serf :: USE CASES Web Servers and Load Balancers Clustering Memcached or Redis Triggering Deploys Updating DNS Records Simple Observability A Building Block for Service Discovery
  17. 17. demo : production app serf db serf server { # ... location / { if (-f $document_root/maintenance.txt) { return 503; } } error_page 503 @maintenance; location @maintenance { rewrite ^(.*)$ /maintenance.html break; } }
  18. 18. demo : maintenance on app serf db serf # serf/member-failed.sh #!/bin/bash while read line; do ROLE=`echo $line | awk '{print $3 }'` if [ "${ROLE}" == "db" ]; then touch /usr/share/nginx/html/maintenance.txt fi done
  19. 19. demo : maintenance off app serf db serf # serf/member-join.sh #!/bin/bash while read line; do ROLE=`echo $line | awk '{print $3 }'` if [ "${ROLE}" == "db" ]; then rm /usr/share/nginx/html/maintenance.txt fi done
  20. 20. Serf in action video
  21. 21. Consul Service discovery and configuration made easy. Distributed, highly available, and datacenter-aware https://www.consul.io
  22. 22. Consul Service Discovery Health Checking Multi Datacenter Data Storage
  23. 23. Consul Service Discovery Health Checking Multi Datacenter Data Storage
  24. 24. Consul Service Discovery Health Checking Multi Datacenter Data Storage
  25. 25. Consul Service Discovery Health Checking Multi Datacenter Data Storage
  26. 26. Consul Service Discovery Health Checking Multi Datacenter Data Storage
  27. 27. Consul architecture
  28. 28. Serf vs Consul Serf Consul ● Decentralized ● Simple health checks ● Node-level membership ● Degraded performance over WAN ● More general purpose-tool ● Will operate until the last node is alive ● Requires Consul servers (1/3/5) ● Rich health checks ● Service-level membership ● Key/value storage ● Built-in multi-DC support ● Can’t operate w/o central servers quorum
  29. 29. Conclusions TREAT CONFIGURATION AS A CODE SERVICE DISCOVERY EASES SCALING AND RECOVERY SOME NICE TOOLS FOR SERVICE DISCOVERY EXIST
  30. 30. BTW: More from HashiCorp Vagrant Packer Terraform Vault Nomad Otto
  31. 31. Questions?

×