Apache ZooKeeper TechTuesday
Upcoming SlideShare
Loading in...5

Apache ZooKeeper TechTuesday






Total Views
Views on SlideShare
Embed Views



9 Embeds 511

http://chess72.tistory.com 374
http://www.scoop.it 81
http://www.andreisavu.ro 35
http://andreisavu.ro 10
http://www.linkedin.com 3
http://dschool.co 3
http://coderwall.com 2
http://webcache.googleusercontent.com 2
https://www.linkedin.com 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Apache ZooKeeper TechTuesday Apache ZooKeeper TechTuesday Presentation Transcript

    • Apache ZooKeeper Why use it? What to expect in the future? Andrei Savu @TechTuesday
    • Outline Why use it? Crash Course Practical Example What to expect in the future (3.4.0 release)? GSoC 2010 New Contrib Work in Progress
    • Crash Course
    • What is ZooKeeper? A highly available, scalable, distributed, configuration, consensus, group membership, leader election, naming and coordination service.
    • What is ZooKeeper? (2) replicated in memory tree data structure somehow similar to a file system no partial read / writes no renames ordered updates strong persistence guarantees conditional updates (version) watches for data changes ephemeral nodes generated file names
    • ZooKeeper Data Model hierarchical namespace each znode has data and children data is read and written in its entirety
    • Basic ZooKeeper API string create(path, data, acl, flags) delete(path, expected_version) stat set_data(path, data, expected_version) (data, stat) get_data(path, watch) stat exists(path, watch) string[] get_children(path, watch)
    • ZooKeeper Service Facts: 1) all servers store a copy of the data in memory 2) the leader is elected at startup 3) followers respond to clients 4) all updates go through the leader 5) responses are sent when a majority of servers have persisted the change
    • Practical Example
    • Distributed Queue (Python) http://www.cloudera.com/blog/2009/05/building-a- distributed-concurrent-queue-with-apache-zookeeper/ http://github.com/henryr/pyzk-recipes Retry operation on ConnectionLoss: http://github.com/andreisavu/pyzk-recipes
    • GSoC 2010 3 projects / 5 months
    • 1. Monitoring & Web-based interface Status: Committed to the trunk 1. JIRA: ZOOKEEPER-701 2. Progress Tracking Wiki 3. Monitoring for Ganglia, Nagios and Cacti 1. contrib / monitoring 2. 'mntr' 4letter word 4. Web interface available as a Hue application 1. contrib / huebrowser 2. complete install instructions 3. requirements: rest gateway, Hue 1.0
    • 2. Read-Only Mode Status: Under Review (Ready to be committed) 1. JIRA: ZOOKEEPER-704 2. Progress Tracking Wiki 3. Description: "When a ZooKeeper server loses contact with over half of the other servers in an ensemble ('loses a quorum'), it stops responding to client requests. For some applications, it would be beneficial if a server still responded to read requests when the quorum is lost, but caused an error condition when a write request was attempted."
    • 3. Failure Detector Model Status: Under Review 1. JIRA: ZOOKEEPER-702 2. Progress Tracking Wiki 3. Detectors: Phi Accrual, Chen, Bertier, Fixed Heartbeat 4. Why? Check the concluding remarks on the wiki. 5. Conclusion snippet: "in scenarios where we have a changing network behavior, such in a WAN, the adaptive methods can be a good pick"
    • Contrib
    • Large Scale Pub/Sub (hedwig) 1. JIRA: ZOOKEEPER-775 2. Uses ZooKeeper and BookKeeper 3. Committed to the trunk 4. Developed at Yahoo! Research 5. Used for PNUTS cross data center replication 6. http://vimeo.com/13282102
    • Work in Progress only some interesting JIRAs
    • #834 Children for ephemerals JIRA: ZOOKEEPER-834 Allow ephemeral nodes to have children owned by the same session. Useful when publishing status information. No need to do serialization for basic data structures (hash tables) Similar to /proc in *nix systems. Examples: /agent-01/ip, /agent-01/memory, /agent-01/load
    • #829 /zookeeper/sessions/*  JIRA: ZOOKEEPER-829 Requested by HBase developers: " we'd like the ability to forcible expire someone else's ZK session "
    • Plenty of bug fixes Join the community!
    • Resources http://hadoop.apache.org/zookeeper/ http://wiki.apache.org/hadoop/ZooKeeper/ProjectDescription http://wiki.apache.org/hadoop/ZooKeeper/Tao
    • Thanks! Questions?