Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Service discovery in
Docker environments
Alex Giurgiu
(alex@giurgiu.io)
Why?
• Average application size has increased
tremendously; from single machine to an army of
machines
• Big trend of spli...
Why?
• Docker based architectures encourage
compartmentalisation => a multitude of specialised
containers that need to find...
How?
How?
• Plenty of tools, of various sizes and ages, among
which: directory services(DNS,LDAP), etcd,
skydock, smartstack, p...
Consul
Consul
!
• Similar to etcd(from CoreOS)
• Made by Hashicorp(Vagrant, Packer and Serf)
• “Built on top of a foundation of r...
Features
• DNS recursor
• Service health checks
• Support for multiple datacenters
• Key/Value store
• Query using DNS or ...
Concepts
• Consul agent can act
as a client or as a
server
• The client doest not
store any data,
forwards requests to
ser...
Example implementation
• Simple test environment
for puppet code
• 4 types of containers:
puppetmaster, git,
jenkins and
p...
Example implementation
• each node(physical or VM) runs its own Consul agent(can be
client or server)
• the Consul agent b...
• once its registered the information will be gossiped
to all other Consul agents;
• all containers from all nodes should ...
curl -X PUT -d 'true' http://$GWIP:8500/v1/kv/update/puppetmaster
sleep 3
curl http://jenkins.service.consul:8080/job/pupp...
Demo
Questions
Service discovery in Docker environments
Upcoming SlideShare
Loading in …5
×

Service discovery in Docker environments

8,025 views

Published on

Short presentation that focuses on a practical implementation for doing service discovery in Docker environments, using Consul.

Published in: Technology
  • @undefined I just added the files that I used for this presentation to github. Check out https://github.com/nustiueudinastea/consul-testing.

    There is a small explanation how to run Consul. Also, check http://www.consul.io/docs/agent/basics.html and more specifically, the 'Client Addr' section; its what you are looking for. If you still cant get it to work, just email me using the address in the presentation.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I am having one hell of a time trying to find any information that details how to accomplish what this slide outlines. Is there a blog, tutorial or github project anywhere with an example?

    I would really love to know how the consul agent binds to the docker0 interface?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Service discovery in Docker environments

  1. 1. Service discovery in Docker environments Alex Giurgiu (alex@giurgiu.io)
  2. 2. Why? • Average application size has increased tremendously; from single machine to an army of machines • Big trend of splitting monolithic architectures in SOA’s • The need to see the infrastructure as a homogeneous platform rather than a mix of specialised machines.
  3. 3. Why? • Docker based architectures encourage compartmentalisation => a multitude of specialised containers that need to find each other in an automated way • Rapidly changing environments demand an effective way of managing state change => Puppet driven DNS changes don't cut it anymore
  4. 4. How?
  5. 5. How? • Plenty of tools, of various sizes and ages, among which: directory services(DNS,LDAP), etcd, skydock, smartstack, pacemaker, zookeeper, doozerd, Consul, etc. • Many ways to do it; no one-size-fits-all solution; it all depends your infrastructure • This presentation focuses on just one of the ways; a way which applies well to Docker based environments.
  6. 6. Consul
  7. 7. Consul ! • Similar to etcd(from CoreOS) • Made by Hashicorp(Vagrant, Packer and Serf) • “Built on top of a foundation of rigorous academic research” - read SWIM and RAFT papers • Consistency through a quorum of nodes(RAFT) • Excellent scalability because of its gossip based communication protocol(SWIM)
  8. 8. Features • DNS recursor • Service health checks • Support for multiple datacenters • Key/Value store • Query using DNS or HTTP 8
  9. 9. Concepts • Consul agent can act as a client or as a server • The client doest not store any data, forwards requests to server • Quorum forms between servers • Each DC acts as a separate cluster, with its own data
  10. 10. Example implementation • Simple test environment for puppet code • 4 types of containers: puppetmaster, git, jenkins and puppetvalidation • Each container registers with CONSUL when it starts • Git notifies puppetmaster of any new commits using CONSUL
  11. 11. Example implementation • each node(physical or VM) runs its own Consul agent(can be client or server) • the Consul agent binds to the docker0 interface; all containers started on the node can query it using their default gateway • when a container starts it makes a call to the HTTP api to register itself with consul using a JSON formatted string echo "Registering PM service with consul curl -X PUT -d {"ID": "puppetmaster1", ”Name”:”puppetmaster", "Port":8600} http://$GWIP:8500/v1/agent/service/register
  12. 12. • once its registered the information will be gossiped to all other Consul agents; • all containers from all nodes should now be able to access the service using DNS at <name>.server.consul or using the HTTP API. • any DNS request for names outside the .consul domain will be forwarded to a configurable external DNS server. • git notification done using the Consul key/value HTTP API
  13. 13. curl -X PUT -d 'true' http://$GWIP:8500/v1/kv/update/puppetmaster sleep 3 curl http://jenkins.service.consul:8080/job/puppet-run/build while [ true ]; do echo "Waiting for git update(index higher than $MI)" MI=`curl -s "http://$GWIP:8500/v1/kv/update/puppetmaster?index= $MI" | awk -F "," '{ print $2 }' | cut -d ":" -f 2` echo "New index is $MI" echo "Modules updated. Pulling repo." cd /etc/puppet/modules && git pull && cd - done On puppetmaster: On git:
  14. 14. Demo
  15. 15. Questions

×