Apache ZooKeeper TechTuesday
Upcoming SlideShare
Loading in...5
×
 

Apache ZooKeeper TechTuesday

on

  • 6,102 views

 

Statistics

Views

Total Views
6,102
Views on SlideShare
5,591
Embed Views
511

Actions

Likes
5
Downloads
165
Comments
0

9 Embeds 511

http://chess72.tistory.com 374
http://www.scoop.it 81
http://www.andreisavu.ro 35
http://andreisavu.ro 10
http://www.linkedin.com 3
http://dschool.co 3
http://coderwall.com 2
http://webcache.googleusercontent.com 2
https://www.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Apache ZooKeeper TechTuesday Apache ZooKeeper TechTuesday Presentation Transcript

    • Apache ZooKeeper Why use it? What to expect in the future? Andrei Savu @TechTuesday
    • Outline Why use it? Crash Course Practical Example What to expect in the future (3.4.0 release)? GSoC 2010 New Contrib Work in Progress
    • Crash Course
    • What is ZooKeeper? A highly available, scalable, distributed, configuration, consensus, group membership, leader election, naming and coordination service.
    • What is ZooKeeper? (2) replicated in memory tree data structure somehow similar to a file system no partial read / writes no renames ordered updates strong persistence guarantees conditional updates (version) watches for data changes ephemeral nodes generated file names
    • ZooKeeper Data Model hierarchical namespace each znode has data and children data is read and written in its entirety
    • Basic ZooKeeper API string create(path, data, acl, flags) delete(path, expected_version) stat set_data(path, data, expected_version) (data, stat) get_data(path, watch) stat exists(path, watch) string[] get_children(path, watch)
    • ZooKeeper Service Facts: 1) all servers store a copy of the data in memory 2) the leader is elected at startup 3) followers respond to clients 4) all updates go through the leader 5) responses are sent when a majority of servers have persisted the change
    • Practical Example
    • Distributed Queue (Python) http://www.cloudera.com/blog/2009/05/building-a- distributed-concurrent-queue-with-apache-zookeeper/ http://github.com/henryr/pyzk-recipes Retry operation on ConnectionLoss: http://github.com/andreisavu/pyzk-recipes
    • GSoC 2010 3 projects / 5 months
    • 1. Monitoring & Web-based interface Status: Committed to the trunk 1. JIRA: ZOOKEEPER-701 2. Progress Tracking Wiki 3. Monitoring for Ganglia, Nagios and Cacti 1. contrib / monitoring 2. 'mntr' 4letter word 4. Web interface available as a Hue application 1. contrib / huebrowser 2. complete install instructions 3. requirements: rest gateway, Hue 1.0
    • 2. Read-Only Mode Status: Under Review (Ready to be committed) 1. JIRA: ZOOKEEPER-704 2. Progress Tracking Wiki 3. Description: "When a ZooKeeper server loses contact with over half of the other servers in an ensemble ('loses a quorum'), it stops responding to client requests. For some applications, it would be beneficial if a server still responded to read requests when the quorum is lost, but caused an error condition when a write request was attempted."
    • 3. Failure Detector Model Status: Under Review 1. JIRA: ZOOKEEPER-702 2. Progress Tracking Wiki 3. Detectors: Phi Accrual, Chen, Bertier, Fixed Heartbeat 4. Why? Check the concluding remarks on the wiki. 5. Conclusion snippet: "in scenarios where we have a changing network behavior, such in a WAN, the adaptive methods can be a good pick"
    • Contrib
    • Large Scale Pub/Sub (hedwig) 1. JIRA: ZOOKEEPER-775 2. Uses ZooKeeper and BookKeeper 3. Committed to the trunk 4. Developed at Yahoo! Research 5. Used for PNUTS cross data center replication 6. http://vimeo.com/13282102
    • Work in Progress only some interesting JIRAs
    • #834 Children for ephemerals JIRA: ZOOKEEPER-834 Allow ephemeral nodes to have children owned by the same session. Useful when publishing status information. No need to do serialization for basic data structures (hash tables) Similar to /proc in *nix systems. Examples: /agent-01/ip, /agent-01/memory, /agent-01/load
    • #829 /zookeeper/sessions/*  JIRA: ZOOKEEPER-829 Requested by HBase developers: " we'd like the ability to forcible expire someone else's ZK session "
    • Plenty of bug fixes Join the community!
    • Resources http://hadoop.apache.org/zookeeper/ http://wiki.apache.org/hadoop/ZooKeeper/ProjectDescription http://wiki.apache.org/hadoop/ZooKeeper/Tao
    • Thanks! Questions?