Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Comparing ZooKeeper and Consul


Published on

Building Service Discovery with ZooKeeper or Consul? Here's a small comparison between two popular tools.

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website!
    Are you sure you want to  Yes  No
    Your message goes here
  • Apache ZooKeeper Tutorial (Videos and Books) #ZooKeeper $7.95
    Are you sure you want to  Yes  No
    Your message goes here

Comparing ZooKeeper and Consul

  1. 1. ZooKeeper vs Consul Ivan Glushkov @gliush v1.2, Nov 2014
  2. 2. ZooKeeper: Overview ZooKeeper is a high-performance coordination service ❖ Written in Java for distributed applications ❖ Strongly consistent (CP) ❖ Zab protocol (Paxos-like) ❖ Ensemble of servers ❖ Quorum needed (majority) ❖ Dataset must fit in memory
  3. 3. ZooKeeper: Data ❖ In-memory data ❖ Shared hierarchal namespace (FS-like) ❖ Ephemeral nodes exists while session is alive ❖ Access Control List (ACL) to each node ❖ Watches on nodes.
  4. 4. ZooKeeper: API ❖ create ❖ delete ❖ exists ❖ get data ❖ set data ❖ get children ❖ sync
  5. 5. ZooKeeper: Problems ❖ Latency-dependent ❖ No multi datacenters support (very slow writes) ❖ Client number limit (out of sockets) ❖ Designed to store data on the order of kB in size ❖ Possibly lost changes while reenabling watch ❖ Need to read after every watch callback (to sync) ❖ TCP connection != service healthy (proxy) ❖ Change zk server is a problem
  6. 6. Consul Consul is a tool for discovering and configuring services in your infrastructure ❖ Written in Go ❖ Service discovery ❖ Health checking ❖ Key/Value store ❖ Multi Datacenter support ❖ Agent/Server concept ❖ Gossip protocol for all the nodes (discovery) ❖ Consensus protocol (Raft-based) for servers ❖ 3 consistency modes ❖ Access Control List (ACL)
  7. 7. Consul: HTTP API ❖ kv - Key/Value store ❖ agent - Agent control ❖ catalog - Manages nodes and services ❖ health - Manages health checks ❖ session - Session manipulation ❖ acl - ACL creations and management ❖ event - User Events ❖ status - Consul system status
  8. 8. ZooKeeper vs Consul With strong consistency Key/Value 3 consistency modes HTTP API Service registering Config file or HTTP API TCP Connection Health Check Connection to node + Any script on node + HTTP API HTTP API Service Discovery HTTP API + DNS + Configuration templates
  9. 9. Key/Value ❖ ZooKeeper: REST API. Needs quorum, has only strong consistency ❖ Consul: REST API. 3 modes: • ‘consistency’ • ‘default’ (potentially stale values = default for zk) • ‘stale’ (read from any server, often stale values, fastest mode)
  10. 10. Service registering ❖ ZooKeeper: K/V via HTTP request to servers (ephemeral nodes) ❖ Consul: local agent registering • through HTTP request • config file
  11. 11. Health Check ❖ ZooKeeper: HTTP Connection. Might not be relevant: • proxy holds connect • node is alive, while the app hangs ❖ Consul: gossip for node aliveness, preconfigured health checks (any script), HTTP API
  12. 12. Service Discovery ❖ ZooKeeper: HTTP request to the K/V ❖ Consul: DNS or HTTP API
  13. 13. More Consul pros ❖ Consul is built on top of Serf - node discovery and orchestration tool (gossip model). It’s used for group membership, failure detection and event broadcast: • LAN-pool for membership and failure detection (distributed checks, good for large clusters) • WAN-pool for inter-datacenter consul servers communication ❖ Consul templates to update files and possibly restart applications (haproxy with backend servers)
  14. 14. Links ❖ Zk overview: ❖ Zk experience: html ❖ Test zk: ❖ Restart zk server problem: a-zookeeper-server-td7472559.html ❖ Sample consul use for HAproxy configuration: haproxy-with-consul.html ❖ Consul templates feature: ❖ Stale reads from ZooKeeper: apache-zookeeper-how-do-writes-work