Building Service Discovery with ZooKeeper or Consul? Here's a small comparison between two popular tools.

1. ZooKeeper
vs
Consul
Ivan Glushkov
ivan.glushkov@gmail.com
@gliush
v1.2, Nov 2014

2. ZooKeeper: Overview
ZooKeeper is a high-performance coordination service
❖ Written in Java
for distributed applications
❖ Strongly consistent (CP)
❖ Zab protocol (Paxos-like)
❖ Ensemble of servers
❖ Quorum needed (majority)
❖ Dataset must fit in memory

3. ZooKeeper: Data
❖ In-memory data
❖ Shared hierarchal namespace
(FS-like)
❖ Ephemeral nodes exists while
session is alive
❖ Access Control List (ACL) to
each node
❖ Watches on nodes.

4. ZooKeeper: API
❖ create
❖ delete
❖ exists
❖ get data
❖ set data
❖ get children
❖ sync

5. ZooKeeper: Problems
❖ Latency-dependent
❖ No multi datacenters support (very slow writes)
❖ Client number limit (out of sockets)
❖ Designed to store data on the order of kB in size
❖ Possibly lost changes while reenabling watch
❖ Need to read after every watch callback (to sync)
❖ TCP connection != service healthy (proxy)
❖ Change zk server is a problem

6. Consul
Consul is a tool for discovering and configuring services in your infrastructure
❖ Written in Go
❖ Service discovery
❖ Health checking
❖ Key/Value store
❖ Multi Datacenter support
❖ Agent/Server concept
❖ Gossip protocol for all the nodes (discovery)
❖ Consensus protocol (Raft-based) for servers
❖ 3 consistency modes
❖ Access Control List (ACL)

7. Consul: HTTP API
❖ kv - Key/Value store
❖ agent - Agent control
❖ catalog - Manages nodes and services
❖ health - Manages health checks
❖ session - Session manipulation
❖ acl - ACL creations and management
❖ event - User Events
❖ status - Consul system status

8. ZooKeeper vs Consul
With strong consistency Key/Value 3 consistency modes
HTTP API Service registering Config file or HTTP API
TCP Connection Health Check
Connection to node +
Any script on node +
HTTP API
HTTP API Service Discovery
HTTP API + DNS +
Configuration templates

9. Key/Value
❖ ZooKeeper: REST API. Needs quorum, has only strong
consistency
❖ Consul: REST API. 3 modes:
• ‘consistency’
• ‘default’ (potentially stale values = default for zk)
• ‘stale’ (read from any server, often stale values, fastest
mode)

10. Service registering
❖ ZooKeeper: K/V via HTTP request to servers
(ephemeral nodes)
❖ Consul: local agent registering
• through HTTP request
• config file

11. Health Check
❖ ZooKeeper: HTTP Connection. Might not be relevant:
• proxy holds connect
• node is alive, while the app hangs
❖ Consul: gossip for node aliveness, preconfigured health
checks (any script), HTTP API

12. Service Discovery
❖ ZooKeeper: HTTP request to the K/V
❖ Consul: DNS or HTTP API

13. More Consul pros
❖ Consul is built on top of Serf - node discovery and
orchestration tool (gossip model). It’s used for group
membership, failure detection and event broadcast:
• LAN-pool for membership and failure detection
(distributed checks, good for large clusters)
• WAN-pool for inter-datacenter consul servers
communication
❖ Consul templates to update files and possibly restart
applications (haproxy with backend servers)

14. Links
❖ Zk overview: http://zookeeper.apache.org/doc/trunk/zookeeperOver.html
❖ Zk experience: http://whilefalse.blogspot.ru/2012/12/building-global-highly-available.
html
❖ Test zk: http://aphyr.com/posts/291-call-me-maybe-zookeeper
❖ Restart zk server problem: http://zookeeper-user.578899.n2.nabble.com/How-to-replace-
a-zookeeper-server-td7472559.html
❖ Sample consul use for HAproxy configuration: https://hashicorp.com/blog/
haproxy-with-consul.html
❖ Consul templates feature: https://github.com/hashicorp/consul-template/releases
❖ Stale reads from ZooKeeper: http://stackoverflow.com/questions/5420087/
apache-zookeeper-how-do-writes-work