ZooKeeper - wait free protocol for coordinating processes

  • 2,951 views
Uploaded on

ZooKeeper is a service for coordinating processes within distributed systems. Stress test of the tool was applied. Reliable Multicast and Dynamic LogBack system Configuration management were …

ZooKeeper is a service for coordinating processes within distributed systems. Stress test of the tool was applied. Reliable Multicast and Dynamic LogBack system Configuration management were implemented with ZooKeeper.
More details: http://proskurnia.in.ua/wiki/zookeeper_research

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,951
On Slideshare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
50
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. ZooKeeperIuliia Proskurnia, Zygimantas Bruzgys
  • 2. Outline• ZooKeeper? • Experiments • ZooKeeper Service • Configuration • ZooKeeper Usage • Reliable Multicast • Key Points • Stress Test• Installation • Conclusions 2
  • 3. Key featuresZooKeeper is: Features are:•Service for coordinating •High Throughputprocesses in distributedapplications. •Low Latency•Key/value table. •Highly Available•No locks service. •Strictly ordered access to the nodes 3
  • 4. Architecture 4
  • 5. ZooKeeper Service• Znodes • Guarantees: • Regular & Ephemeral • Linearizable writes• Watches • FIFO client order • Handling shared configuration• Synchronous and updates Asynchronous methods • Liveness and durability 5
  • 6. ZooKeeper Usage • Simple Locks• Configuration Management • Simple Locks without Herd Effect• Group membership • Read/Write Locks• Leader Election • Double Barrier 6
  • 7. Key Points• Wait-free protocol• No Locks • But...• Support Applications: • State Based System • Configuration Management • Last changes visible • Leader Election • Limited total size of data • Group Membership 7
  • 8. InstallationTo install you need to: Sample Configuration File:•Install Java tickTime=2000 initLimit=10•Download and extract syncLimit=5 clientPort=2181ZooKeeper dataDir=/home/ubuntu/zoodata•Write a configuration file server.1=10.227.206.23:2888:3888•Write node ID in data server.2=10.58.190.233:2888:3888 server.3=10.234.121.38:2888:3888directory (filename myid)•Start ZooKeeper 8
  • 9. Experiments 9
  • 10. Configuration Management 10
  • 11. Reliable Multicast: Main 11
  • 12. Reliable Multicast: Leader Election 12
  • 13. Reliable Multicast: Send Message 13
  • 14. Reliable Multicast: Receive Message 14
  • 15. Reliable Multicast: Interesting Cases• New client connects while a message is multicasted• Recipient fails before receiving a message• Leader fails 15
  • 16. Stress Tests• Done with zk-latencies Python script• Run with the following configuration ./zk-latencies.py --servers "CONNECTION_STRING" --znode_count=ZNODE_COUNT --znode_size=100 --timeout=5000 --watch_multiple=2 16
  • 17. Stress Tests 14000" Time#for#10.000#Async#Opera>ons# 12000" 10000" 8000" Read" 1 client,ms# 6000" Write" multiple 4000" Watch" servers 2000" 0" 1" 2" 3" 4" 5" Number#of#Servers#in#ZooKeeper#Cluster# 17
  • 18. Stress Tests 7" Synchronous+Opera)ons+ 6" 5"ms/opera)on+ 4" Read" 1 client, 3" Write" multiple Watch" 2" servers 1" 0" 1" 2" 3" 4" Number+of+Servers+in+ZooKeeper+Cluster+ 18
  • 19. Stress Tests 1,2" Writes+per+ms+ 1"ms/opera)on+ 0,8" 0,6" 5 servers, 0,4" 1 client 0,2" 0" 1" 2" 4" 6" 8" 10" 12" 14" 16" Thousands+of+Async+Opera)ons+ 19
  • 20. Conclusions• ZooKeeper is: • WeShowed Several Examples: • Coordination Service • Configuration Management • Highly Available • Reliable Multicast • Low Latency • Wait-Free • Stress Tests Shows: • Prefer more servers for high availability and higher reads/write ratio • With more servers it is still low latency 20
  • 21. ZooKeeperIuliia Proskurnia, Zygimantas Bruzgys