ZooKeeper - wait free protocol for coordinating processes

4,186 views

Published on

ZooKeeper is a service for coordinating processes within distributed systems. Stress test of the tool was applied. Reliable Multicast and Dynamic LogBack system Configuration management were implemented with ZooKeeper.
More details: http://proskurnia.in.ua/wiki/zookeeper_research

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,186
On SlideShare
0
From Embeds
0
Number of Embeds
1,751
Actions
Shares
0
Downloads
58
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

ZooKeeper - wait free protocol for coordinating processes

  1. 1. ZooKeeperIuliia Proskurnia, Zygimantas Bruzgys
  2. 2. Outline• ZooKeeper? • Experiments • ZooKeeper Service • Configuration • ZooKeeper Usage • Reliable Multicast • Key Points • Stress Test• Installation • Conclusions 2
  3. 3. Key featuresZooKeeper is: Features are:•Service for coordinating •High Throughputprocesses in distributedapplications. •Low Latency•Key/value table. •Highly Available•No locks service. •Strictly ordered access to the nodes 3
  4. 4. Architecture 4
  5. 5. ZooKeeper Service• Znodes • Guarantees: • Regular & Ephemeral • Linearizable writes• Watches • FIFO client order • Handling shared configuration• Synchronous and updates Asynchronous methods • Liveness and durability 5
  6. 6. ZooKeeper Usage • Simple Locks• Configuration Management • Simple Locks without Herd Effect• Group membership • Read/Write Locks• Leader Election • Double Barrier 6
  7. 7. Key Points• Wait-free protocol• No Locks • But...• Support Applications: • State Based System • Configuration Management • Last changes visible • Leader Election • Limited total size of data • Group Membership 7
  8. 8. InstallationTo install you need to: Sample Configuration File:•Install Java tickTime=2000 initLimit=10•Download and extract syncLimit=5 clientPort=2181ZooKeeper dataDir=/home/ubuntu/zoodata•Write a configuration file server.1=10.227.206.23:2888:3888•Write node ID in data server.2=10.58.190.233:2888:3888 server.3=10.234.121.38:2888:3888directory (filename myid)•Start ZooKeeper 8
  9. 9. Experiments 9
  10. 10. Configuration Management 10
  11. 11. Reliable Multicast: Main 11
  12. 12. Reliable Multicast: Leader Election 12
  13. 13. Reliable Multicast: Send Message 13
  14. 14. Reliable Multicast: Receive Message 14
  15. 15. Reliable Multicast: Interesting Cases• New client connects while a message is multicasted• Recipient fails before receiving a message• Leader fails 15
  16. 16. Stress Tests• Done with zk-latencies Python script• Run with the following configuration ./zk-latencies.py --servers "CONNECTION_STRING" --znode_count=ZNODE_COUNT --znode_size=100 --timeout=5000 --watch_multiple=2 16
  17. 17. Stress Tests 14000" Time#for#10.000#Async#Opera>ons# 12000" 10000" 8000" Read" 1 client,ms# 6000" Write" multiple 4000" Watch" servers 2000" 0" 1" 2" 3" 4" 5" Number#of#Servers#in#ZooKeeper#Cluster# 17
  18. 18. Stress Tests 7" Synchronous+Opera)ons+ 6" 5"ms/opera)on+ 4" Read" 1 client, 3" Write" multiple Watch" 2" servers 1" 0" 1" 2" 3" 4" Number+of+Servers+in+ZooKeeper+Cluster+ 18
  19. 19. Stress Tests 1,2" Writes+per+ms+ 1"ms/opera)on+ 0,8" 0,6" 5 servers, 0,4" 1 client 0,2" 0" 1" 2" 4" 6" 8" 10" 12" 14" 16" Thousands+of+Async+Opera)ons+ 19
  20. 20. Conclusions• ZooKeeper is: • WeShowed Several Examples: • Coordination Service • Configuration Management • Highly Available • Reliable Multicast • Low Latency • Wait-Free • Stress Tests Shows: • Prefer more servers for high availability and higher reads/write ratio • With more servers it is still low latency 20
  21. 21. ZooKeeperIuliia Proskurnia, Zygimantas Bruzgys

×