Service discovery in Docker environments

7,493 views
7,037 views

Published on

Short presentation that focuses on a practical implementation for doing service discovery in Docker environments, using Consul.

Published in: Technology
2 Comments
20 Likes
Statistics
Notes
  • @undefined I just added the files that I used for this presentation to github. Check out https://github.com/nustiueudinastea/consul-testing.

    There is a small explanation how to run Consul. Also, check http://www.consul.io/docs/agent/basics.html and more specifically, the 'Client Addr' section; its what you are looking for. If you still cant get it to work, just email me using the address in the presentation.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I am having one hell of a time trying to find any information that details how to accomplish what this slide outlines. Is there a blog, tutorial or github project anywhere with an example?

    I would really love to know how the consul agent binds to the docker0 interface?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
7,493
On SlideShare
0
From Embeds
0
Number of Embeds
35
Actions
Shares
0
Downloads
131
Comments
2
Likes
20
Embeds 0
No embeds

No notes for slide

Service discovery in Docker environments

  1. 1. Service discovery in Docker environments Alex Giurgiu (alex@giurgiu.io)
  2. 2. Why? • Average application size has increased tremendously; from single machine to an army of machines • Big trend of splitting monolithic architectures in SOA’s • The need to see the infrastructure as a homogeneous platform rather than a mix of specialised machines.
  3. 3. Why? • Docker based architectures encourage compartmentalisation => a multitude of specialised containers that need to find each other in an automated way • Rapidly changing environments demand an effective way of managing state change => Puppet driven DNS changes don't cut it anymore
  4. 4. How?
  5. 5. How? • Plenty of tools, of various sizes and ages, among which: directory services(DNS,LDAP), etcd, skydock, smartstack, pacemaker, zookeeper, doozerd, Consul, etc. • Many ways to do it; no one-size-fits-all solution; it all depends your infrastructure • This presentation focuses on just one of the ways; a way which applies well to Docker based environments.
  6. 6. Consul
  7. 7. Consul ! • Similar to etcd(from CoreOS) • Made by Hashicorp(Vagrant, Packer and Serf) • “Built on top of a foundation of rigorous academic research” - read SWIM and RAFT papers • Consistency through a quorum of nodes(RAFT) • Excellent scalability because of its gossip based communication protocol(SWIM)
  8. 8. Features • DNS recursor • Service health checks • Support for multiple datacenters • Key/Value store • Query using DNS or HTTP 8
  9. 9. Concepts • Consul agent can act as a client or as a server • The client doest not store any data, forwards requests to server • Quorum forms between servers • Each DC acts as a separate cluster, with its own data
  10. 10. Example implementation • Simple test environment for puppet code • 4 types of containers: puppetmaster, git, jenkins and puppetvalidation • Each container registers with CONSUL when it starts • Git notifies puppetmaster of any new commits using CONSUL
  11. 11. Example implementation • each node(physical or VM) runs its own Consul agent(can be client or server) • the Consul agent binds to the docker0 interface; all containers started on the node can query it using their default gateway • when a container starts it makes a call to the HTTP api to register itself with consul using a JSON formatted string echo "Registering PM service with consul curl -X PUT -d {"ID": "puppetmaster1", ”Name”:”puppetmaster", "Port":8600} http://$GWIP:8500/v1/agent/service/register
  12. 12. • once its registered the information will be gossiped to all other Consul agents; • all containers from all nodes should now be able to access the service using DNS at <name>.server.consul or using the HTTP API. • any DNS request for names outside the .consul domain will be forwarded to a configurable external DNS server. • git notification done using the Consul key/value HTTP API
  13. 13. curl -X PUT -d 'true' http://$GWIP:8500/v1/kv/update/puppetmaster sleep 3 curl http://jenkins.service.consul:8080/job/puppet-run/build while [ true ]; do echo "Waiting for git update(index higher than $MI)" MI=`curl -s "http://$GWIP:8500/v1/kv/update/puppetmaster?index= $MI" | awk -F "," '{ print $2 }' | cut -d ":" -f 2` echo "New index is $MI" echo "Modules updated. Pulling repo." cd /etc/puppet/modules && git pull && cd - done On puppetmaster: On git:
  14. 14. Demo
  15. 15. Questions

×