Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Distributed Applications with Apache Zookeeper

4,184 views

Published on

An overview of Apache Zookeeper and it's practical application for coordination of distributed systems.

Published in: Software
  • Dating direct: ❶❶❶ http://bit.ly/39mQKz3 ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❤❤❤ http://bit.ly/39mQKz3 ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • 7 Signs Your Car Battery Is About To Die And Needs To Be Replaced (or reconditioned) ◆◆◆ http://ishbv.com/ezbattery/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Distributed Applications with Apache Zookeeper

  1. 1. Building Distributed Applications with Apache Zookeeper Alex Ehrnschwender | Game Server Engineer at DeNA
  2. 2. What is Zookeeper? “ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.” Zookeeper Wiki
  3. 3. ZooKeeper: A Coordination Service for Distributed Applications Coordination & synchronization for distributed processes Logical namespacing implemented by a hierarchy (tree) of znodes Replicated in-memory over multiple hosts for reliability, availability, and performance Simple API of CRUD & basic tree operations for client integration
  4. 4. Zookeeper: Reliability & Consistency Distributed ensemble with automatic leader election through quorum Replicated in-memory on every instance with snapshot writes to disk Client TCP connection maintained to any node with failover support Guaranteed atomicity & sequential consistency
  5. 5. Zookeeper: Watches & Ephemeral nodes Underlying znodes have a data structure consisting of version numbers (cversion, aversion) & timestamps Watches ● Client-initiated subscriptions to znodes ● Changes to a watched znode trigger notification to subscribed clients Ephemeral Nodes ● Backed by a client session and deleted when client session ends ● Cannot have children
  6. 6. Zookeeper: But… why? “Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them, which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.” Zookeeper Wiki
  7. 7. Zookeeper: Advantages for Backing a Server Cluster Server workers can become cluster-aware So much out-of-the-box that would be duplicated with a custom solution Extremely fast reads (10:1 performance against writes) Small footprint - An ensemble of only 5-7 zk instances can serve the coordination needs of several large production applications Centralized event broadcasting & failure detection (heartbeat)
  8. 8. Zookeeper: Common Use Cases ● Configuration Management ● Service Discovery ● Distributed Cloud-Based File Systems ● Internal DNS Management ● Master (Leader) Election and Voting ● Messaging Queue ● Event Broadcasting & Notification
  9. 9. Use Case Example #1 - Managing Redis Shards
  10. 10. ZK Use Case Example #1 - Pinterest Pinterest stores their entire follower model inside sharded Redis instances ( ~9000 Redis shards, multiple instances per core) Shard configuration is stored and managed by Zookeeper Client lookups and watches for shard location & subsequent data retrieval Master-slave failover triggers updates to znode representation (slave address replaces master) Vertical splitting of data broadcasted to watching clients
  11. 11. Use Case Example #2 - HBase Cluster Configuration
  12. 12. Code Examples public void join(String groupName, String memberName) throws KeeperException, InterruptedException { String path = "/" + groupName + "/" + memberName; String createdPath = zk.create(path, null /* data */, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL); System.out.println("Created " + createdPath); } public void create(String groupName) throws KeeperException, InterruptedException { String path = "/" + groupName; String createdPath = zk.create(path, null /* data */, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); System.out.println("Created " + createdPath); }
  13. 13. Code Examples (cont.) public void delete(String groupName) throws KeeperException, InterruptedException { String path = "/" + groupName; try { List<String> children = zk.getChildren(path, false); for(String child : children) { zk.delete(path + "/" + child, -1); /* child */ } zk.delete(path, -1); /* parent */ } catch (KeeperException.NoNodeException e) { System.out.printf("Group %s does not existn", groupName); } } public void list(String groupName) throws KeeperException, InterruptedException { String path = "/" + groupName; try { List<String> children = zk.getChildren(path, false); for(String child : children) { System.out.println(child); } } catch (KeeperException.NoNodeException e) { System.out.printf("Group %s does not existn", groupName); } }
  14. 14. Performance Standalone ops/sec 3-Node Ensemble (ops/sec) Reference: https://wiki.apache.org/hadoop/ZooKeeper/ServiceLatencyOverview
  15. 15. Sample Configuration (zoo.cfg) tickTime=2000 dataDir=/var/lib/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
  16. 16. Exhibitor: A ZK Monitoring & Administration Tool from Netflix Centralization & externalization of zk ensemble configuration* (S3/remote FS) Web UI & REST API for ease of management Instance monitoring with automatic configuration updates Rolling ensemble changes while maintaining quorum Miscellaneous administration tasks (backup/restore, log & snapshot cleanup) * Configuration management for a configuration manager.... so meta!
  17. 17. Questions?
  18. 18. Appendix
  19. 19. Zookeeper Atomic Broadcast (ZAB) Algorithm ● Protocol for managing atomic updates to replicas ● Responsible for: o Agreeing on an ensemble leader o Synchronizing replicas o Managing transactions and broadcasts o Recovery of state ● ZXIDs & transactional ordering ● Guarantees: o Local & global primary order o Primary integrity
  20. 20. Performance
  21. 21. Performance Standalone ops/sec 3-Node Ensemble (ops/sec) Reference: https://wiki.apache.org/hadoop/ZooKeeper/ServiceLatencyOverview
  22. 22. Sample Configuration (zoo.cfg) tickTime=2000 dataDir=/var/lib/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
  23. 23. References ● http://engineering.pinterest.com/post/55272557617/building-a-follower-model-from-scratch ● http://zookeeper.apache.org/doc/trunk/zookeeperOver.html ● http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html ● https://github.com/Netflix/exhibitor/wiki ● http://www.tcs.hut.fi/Studies/T-79.5001/reports/2012-deSouzaMedeiros.pdf ● http://web.stanford.edu/class/cs347/reading/zab.pdf ● http://highscalability.com/blog/2008/7/15/zookeeper-a-reliable-scalable-distributed-coordination- syste.html ● https://wiki.apache.org/solr/SolrCloud ● http://www.slideshare.net/scottleber/apache-zookeeper

×