Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Service discovery in a microservice architecture using consul

21,330 views

Published on

Presentation I gave at Nextbuild 2016. Gives an overview of how Consul can be used in microservice architecture. Accompanying examples and demo can be found here: https://github.com/josdirksen/next-build-consul

Published in: Technology
  • Be the first to comment

Service discovery in a microservice architecture using consul

  1. 1. Service Discovery in a  Architecture using  Microservice Consul
  2. 2. Who Am I Currently doing Devops, Scala stu At Equeris, lean startup within Equens Docker, Consul, Scala, Cloud and other buzzwords. Email me at: I write at: Twitter: jos.dirksen@gmail.com http://www.smartjava.org @josdirksen
  3. 3. Follow along Sources, presentation etc: Demo heavy session https://github.com/josdirksen/next-build-consul
  4. 4. Microservices?
  5. 5. What are microservices? Small, ne-grained easy to replace components. Organized around capabilities. Di erent languages and backends (whatever ts best). Fault tolerant, resiliant, automated deployements. "Small Autonomous services that work together", Sam Newman
  6. 6. From three tier to Microservices
  7. 7. Running Microservices is hard Where is my other service or database? Am I healthy, is the other one healthy? Where do I store con guration? How do I handle redundancy and failover? ... , says everyone"Distributed systems are hard"
  8. 8. Service Discovery
  9. 9. Basic approach Hardcoded IP Address or LookupDNS Lookup is nice! Requires managing names (con g les), DNS Server How to handle failover? DNS
  10. 10. Now with failover Point to a loadbalancerDNS Works nicely with ! How to check health and register services? Programmatic access to LB? DNS
  11. 11. What would be nice Does : Lightweight ( , REST) support failover Has exible and manages Lookups DNS Health checking Con guration
  12. 12. Consul "Consul [..] provides an opinionated framework for service discovery and eliminates the guess-work and development e ort. Clients simply register services and then perform discovery using a DNS or HTTP interface. Other systems require a home- rolled solution." - consul.io
  13. 13. Main Features Service discovery through REST and DNS Simple registration using REST API Distributed KV store for con guration Provides extensive health checking All in one package
  14. 14. Good to know Multi DC-ready API for distributed locks Easy HA Setup Event system Consul = Zookeeper + Nagios + DNSMasq + Scriptings + ...
  15. 15. Consul Architecture
  16. 16. Service Registration Flow 1. Servicecalls Consul Agentwith registration message: . 2. Agentcommunicates registration with Consul Server 3. Agentchecks health of Service. 4. If check succeeds mark as Healthy, if not mark as Unhealthy, communicate results with Server 5. When a lookup for Serviceoccurs, only return Healthyservices http://agent_host/v1/agent/service/register
  17. 17. Sample: registration message { "Name": "service1", "address": "10.0.0.12", "port": 8080, "Check": { "http": "http://10.0.0.12:8080/health", "interval": "5s" } } Send when a new service starts up Check types: script, http, tcp, TTL, Docker
  18. 18. Sample: DNS Lookup $dig @nb-consul.local backend-service.service.consul ; <<>> DiG 9.8.3-P1 <<>> @nb-consul.local backend-service.service.consul ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27716 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;backend-service.service.consul. IN A ;; ANSWER SECTION: backend-service.service.consul. 0 IN A 10.0.9.3 backend-service.service.consul. 0 IN A 10.0.9.2 backend-service.service.consul. 0 IN A 10.0.9.4 Consul provides a DNS Server Works great with Docker ( : will show in Demo)teaser
  19. 19. Sample: REST Lookup $ curl -s http://192.168.99.106:8500/v1/catalog/service/backend-service [{ "Node": "cf2f293e423c", "Address": "192.168.99.111", "ServiceID": "backend-service", "ServiceName": "backend-service", "ServiceAddress": "10.0.9.2", "ServicePort": 8080 },{ "Node": "072b4ea1abc1", "Address": "192.168.99.112", "ServiceID": "backend-service", "ServiceName": "backend-service", "ServiceAddress": "10.0.9.3", "ServicePort": 8080 }]
  20. 20. DEMO
  21. 21. Closer look: DNS Lookup In code: resp, err := http.Get("http://backend-service:8081/") if err != nil { fmt.Println(err) } else { defer resp.Body.Close() body, _ := ioutil.ReadAll(resp.Body) w.Header().Set("Content-Type",resp.Header.Get("Content-Type")) w.Write(body) } For docker: dns: 192.168.99.106 dns_search: service.consul
  22. 22. Consul Ecosystem
  23. 23. Consul template Render template based on Consul state Setup reverse proxy: Nginx, Apache, haproxy global daemon maxconn {{key "service/haproxy/maxconn"}} defaults mode {{key "service/haproxy/mode"}}{{range ls "service/haproxy/timeouts"}} timeout {{.Key}} {{.Value}}{{end}} listen http-in bind *:8000{{range service "release.web"}} server {{.Node}} {{.Address}}:{{.Port}}{{end}}
  24. 24. Demo
  25. 25. Envconsul "III. Con g Store con g in the environment", http://12factor.net/ More settings >More complexity $ envconsul -consul demo.consul.io -prefix redis/config redis-server [opts...] Vault: help in managing secrets
  26. 26. DEMO ENVCONSUL
  27. 27. Prometheus & Grafana
  28. 28. More information links: This presentation, sources and docker stu : https://www.consul.io/ https://github.com/hashicorp/consul-template https://github.com/hashicorp/envconsul https://github.com/josdirksen/next-build-consul
  29. 29. Thank You!

×