• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Zookeeper at the bigdata roundtable
 

Zookeeper at the bigdata roundtable

on

  • 1,014 views

Tobias Schlottkes presentation on Zookeeper at the first bigdata-roundtable in Hamburg

Tobias Schlottkes presentation on Zookeeper at the first bigdata-roundtable in Hamburg

Statistics

Views

Total Views
1,014
Views on SlideShare
1,014
Embed Views
0

Actions

Likes
0
Downloads
14
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Zookeeper at the bigdata roundtable Zookeeper at the bigdata roundtable Presentation Transcript

    • Zookeeper What it actually is. Tobias Schlottke November 2011Freitag, 2. Dezember 11
    • Zookeeper What it actually is. Tobias Schlottke November 2011Freitag, 2. Dezember 11
    • What it is „ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them, which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.“Freitag, 2. Dezember 11
    • WTF?Freitag, 2. Dezember 11
    • In simple words „A service which helps to coordinate distributed systems.“Freitag, 2. Dezember 11
    • Invented byFreitag, 2. Dezember 11
    • Invented byFreitag, 2. Dezember 11
    • More details please! • Naming nodes • Configuring nodes • Synchronizing nodes • Organize groups of nodes • Heartbeat • Democracy / Leader electionFreitag, 2. Dezember 11
    • But how to organize this stuff?Freitag, 2. Dezember 11
    • Filesystem Schema application1 applicationn application1/service1 application1/servicen application1/service1/node0001 application1/service1/node0002Freitag, 2. Dezember 11
    • „Filesystem“ Features • Organized in a node tree app1/service1 • „Ephemeral“ nodes • „Sequential“ nodes 0123 • Notification system • High availability • Exchange format does not matterFreitag, 2. Dezember 11
    • Other things you get for free • „Atomicity“ • „Consistency“ • „Reliability“ • „Timeliness“ • „Conditional updates“ (Versioning)Freitag, 2. Dezember 11
    • The Zookeeper Service ZK Server 1 ZK Server 2 ZK Server N Elected Master Slave Slave Application 1 Application NFreitag, 2. Dezember 11
    • The Zookeeper Service • All Servers store a copy of data in Mem • Master election at startup • All updates go to the leader • Responses are sent when a majority persitedFreitag, 2. Dezember 11
    • Snares • Data must fit in memory • Data size for each node should be <1MB • Client libraries not all that good so far • Ephemeral nodes are not allowed to have childrenFreitag, 2. Dezember 11
    • Usecase „I am an adserver. I deliver 20.000 ads a second and for this, you will most likely design me as a distributed service“Freitag, 2. Dezember 11
    • Adserver Usecase Database Thrift ZK Coordinator Delivery1 Delivery2 Delivery3 Delivery4Freitag, 2. Dezember 11
    • Thats it! Pretty simple, huh?Freitag, 2. Dezember 11