SlideShare a Scribd company logo
Binu George
www.globinch.com
www.java.globinch.com
Agenda
 Coordination headache in distributed applications
 What is zookeeper?
 Zookeeper Overview
 Zookeeper Service & Data Model
 ZAB (Zookeeper Atomic Broadcasting Protocol)
 Zookeeper Read, Write and Watch
 Zookeeper Use Cases
 Zookeeper Client API Example
Coordination headache in distributed
applications
 Problem of Detection
 Cluster Size Estimation
 Leader Election
 Group Membership
 Configuration Management
 Consensus among multiple parties is difficult in
dynamic distributed systems as members may
leave/fail/join anytime
 Consistency, Availability and Partition tolerance
What is Zookeeper?
 Apache Zookeeper is a highly consistent, scalable and
reliable cluster co-ordination service.
 Zookeeper is a CP system where Consistency and
Partition tolerance are guaranteed.
 Off-the-shelf support for implementing consensus,
group management, leader election etc.
 Zookeeper cluster is called Ensemble
 Ensemble has one leader and remaining followers
 Zookeeper is a Distributed Coordination Service for
Distributed Applications.
 Zookeeper is originally developed by Yahoo research
Team and became a top level Apache project.
Zookeeper is used by many enterprises including
Rackspace, Yahoo! and eBay.
Zookeeper Overview
 Zookeeper use Zookeeper Broadcasting Protocol (ZAB) for
ordering and integrity.
 Achieves High availability and performance via replication.
 Low latency and high throughput via in memory image of
data tree ,transaction logs and snapshots.
 Uses “quorum” model and will only be available if a
majority of servers are alive (by a strict majority of nodes).
 Zookeeper relies on a quorum for durability as well.
 Pipelined architecture of Zookeeper enables the execution
of operations from a single client in FIFO order.
 Ordered updates uses ZAB protocol (a leader-based atomic
broadcast protocol) for Ordering, Integrity , exactly once.
 Election of a new leader is followed by a synchronization
phase before any new changes are accepted.
Zookeeper Service & Data Model
 At any given time, one Zookeeper client is connected
to one Zookeeper server.
 If the node (to which client connected) fails, client
automatically connect to another server in ensemble
with transparent session transfer.
 Znode is an in-memory data node in Zookeeper Data
organized in a hierarchical namespace (data tree)
 The data model of ZooKeeper (znode) is essentially a
file system with a simplified API and associated meta
data and versions.
 Regular, Ephemeral and Sequential Znode are
supported.
 Each ZNode is using UNIX style notation. For
example, /A/B/C to denote the path to znode C, where
C has B as its parent and B has A as its parent.
 All znodes store data, and all znodes, except for
ephemeral znodes, can have children.
Zookeeper read/write/watch
 Clients make use of Zookeeper API to update/write
Zookeeper the data tree.
 All Updates are ordered and managed by leader with
unique zxid and are considered complete when a
quorum confirms the update.
 The leader executes update request and broadcasts
the change to the ZooKeeper state through Zab.
 Zab by default uses a simple majority quorums to
decide on a proposal.
 Read and Watch requests on the znode sent by a client
to a ZooKeeper server are processed locally.
 Watch –Allows client to get notification if an already
returned information gets updated.
Zookeeper read/write/watch
Zookeeper read/write/watch
 Read and Watch requests on the znode are
processed locally but, write requests are
propagated to leader server and go through a
consensus before a response is generated.
 Throughput of read request increases with
number of servers and throughput of write
requests, decreases with number of servers.
Zookeeper use cases
 ZooKeeper offers the library to create and manage
synchronization primitives.
 ZooKeeper avoids the single-point-of-failure. Typical
use cases ,
 Naming service
 Configuration management
 Synchronization
 Leader election
 Message Queue
 Notification system
Zookeeper Java Client API Examples
 A simple example.
 To check the complete example see at the below link.
 http://java.globinch.com/enterprise-
services/zookeeper/apache-zookeeper-explained-
tutorial-cases-zookeeper-java-api-examples/

More Related Content

What's hot

Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
Jonas Bonér
 
Apache zookeeper 101
Apache zookeeper 101Apache zookeeper 101
Apache zookeeper 101
Quach Tung
 
Apache Kafka Architecture & Fundamentals Explained
Apache Kafka Architecture & Fundamentals ExplainedApache Kafka Architecture & Fundamentals Explained
Apache Kafka Architecture & Fundamentals Explained
confluent
 
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
SANG WON PARK
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
Jurriaan Persyn
 
Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka ...
Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka ...Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka ...
Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka ...
Kai Wähner
 
Kafka replication apachecon_2013
Kafka replication apachecon_2013Kafka replication apachecon_2013
Kafka replication apachecon_2013
Jun Rao
 
OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)
OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)
OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)
SANG WON PARK
 
Kafka Tutorial - Introduction to Apache Kafka (Part 1)
Kafka Tutorial - Introduction to Apache Kafka (Part 1)Kafka Tutorial - Introduction to Apache Kafka (Part 1)
Kafka Tutorial - Introduction to Apache Kafka (Part 1)
Jean-Paul Azar
 
kafka
kafkakafka
Kafka Tutorial - basics of the Kafka streaming platform
Kafka Tutorial - basics of the Kafka streaming platformKafka Tutorial - basics of the Kafka streaming platform
Kafka Tutorial - basics of the Kafka streaming platform
Jean-Paul Azar
 
Apache Kafka
Apache KafkaApache Kafka
Apache Kafka
Saroj Panyasrivanit
 
Apache Kafka - Martin Podval
Apache Kafka - Martin PodvalApache Kafka - Martin Podval
Apache Kafka - Martin Podval
Martin Podval
 
LoadBalancer using KeepAlived
LoadBalancer using KeepAlivedLoadBalancer using KeepAlived
LoadBalancer using KeepAlived
KhushalChandak1
 
Solution Live-migrate vm in Openstack with Cpu Pinning configuratin
Solution Live-migrate vm in Openstack with Cpu Pinning configuratinSolution Live-migrate vm in Openstack with Cpu Pinning configuratin
Solution Live-migrate vm in Openstack with Cpu Pinning configuratin
Lam To
 
Geospatial Indexing at Scale: The 15 Million QPS Redis Architecture Powering ...
Geospatial Indexing at Scale: The 15 Million QPS Redis Architecture Powering ...Geospatial Indexing at Scale: The 15 Million QPS Redis Architecture Powering ...
Geospatial Indexing at Scale: The 15 Million QPS Redis Architecture Powering ...
Daniel Hochman
 
Managing 2000 Node Cluster with Ambari
Managing 2000 Node Cluster with AmbariManaging 2000 Node Cluster with Ambari
Managing 2000 Node Cluster with Ambari
DataWorks Summit
 
Ray Serve: A new scalable machine learning model serving library on Ray
Ray Serve: A new scalable machine learning model serving library on RayRay Serve: A new scalable machine learning model serving library on Ray
Ray Serve: A new scalable machine learning model serving library on Ray
Simon Mo
 
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
Amazon Web Services Korea
 
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
HostedbyConfluent
 

What's hot (20)

Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
Apache zookeeper 101
Apache zookeeper 101Apache zookeeper 101
Apache zookeeper 101
 
Apache Kafka Architecture & Fundamentals Explained
Apache Kafka Architecture & Fundamentals ExplainedApache Kafka Architecture & Fundamentals Explained
Apache Kafka Architecture & Fundamentals Explained
 
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
 
Introduction to memcached
Introduction to memcachedIntroduction to memcached
Introduction to memcached
 
Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka ...
Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka ...Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka ...
Event-Driven Stream Processing and Model Deployment with Apache Kafka, Kafka ...
 
Kafka replication apachecon_2013
Kafka replication apachecon_2013Kafka replication apachecon_2013
Kafka replication apachecon_2013
 
OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)
OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)
OLAP for Big Data (Druid vs Apache Kylin vs Apache Lens)
 
Kafka Tutorial - Introduction to Apache Kafka (Part 1)
Kafka Tutorial - Introduction to Apache Kafka (Part 1)Kafka Tutorial - Introduction to Apache Kafka (Part 1)
Kafka Tutorial - Introduction to Apache Kafka (Part 1)
 
kafka
kafkakafka
kafka
 
Kafka Tutorial - basics of the Kafka streaming platform
Kafka Tutorial - basics of the Kafka streaming platformKafka Tutorial - basics of the Kafka streaming platform
Kafka Tutorial - basics of the Kafka streaming platform
 
Apache Kafka
Apache KafkaApache Kafka
Apache Kafka
 
Apache Kafka - Martin Podval
Apache Kafka - Martin PodvalApache Kafka - Martin Podval
Apache Kafka - Martin Podval
 
LoadBalancer using KeepAlived
LoadBalancer using KeepAlivedLoadBalancer using KeepAlived
LoadBalancer using KeepAlived
 
Solution Live-migrate vm in Openstack with Cpu Pinning configuratin
Solution Live-migrate vm in Openstack with Cpu Pinning configuratinSolution Live-migrate vm in Openstack with Cpu Pinning configuratin
Solution Live-migrate vm in Openstack with Cpu Pinning configuratin
 
Geospatial Indexing at Scale: The 15 Million QPS Redis Architecture Powering ...
Geospatial Indexing at Scale: The 15 Million QPS Redis Architecture Powering ...Geospatial Indexing at Scale: The 15 Million QPS Redis Architecture Powering ...
Geospatial Indexing at Scale: The 15 Million QPS Redis Architecture Powering ...
 
Managing 2000 Node Cluster with Ambari
Managing 2000 Node Cluster with AmbariManaging 2000 Node Cluster with Ambari
Managing 2000 Node Cluster with Ambari
 
Ray Serve: A new scalable machine learning model serving library on Ray
Ray Serve: A new scalable machine learning model serving library on RayRay Serve: A new scalable machine learning model serving library on Ray
Ray Serve: A new scalable machine learning model serving library on Ray
 
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
 
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
Streaming Data Lakes using Kafka Connect + Apache Hudi | Vinoth Chandar, Apac...
 

Viewers also liked

Distributed Applications with Apache Zookeeper
Distributed Applications with Apache ZookeeperDistributed Applications with Apache Zookeeper
Distributed Applications with Apache Zookeeper
Alex Ehrnschwender
 
Introduction to ZooKeeper - TriHUG May 22, 2012
Introduction to ZooKeeper - TriHUG May 22, 2012Introduction to ZooKeeper - TriHUG May 22, 2012
Introduction to ZooKeeper - TriHUG May 22, 2012
mumrah
 
Zookeeper In Simple Words
Zookeeper In Simple WordsZookeeper In Simple Words
Zookeeper In Simple Words
Fuqiang Wang
 
Introduction to Kafka and Zookeeper
Introduction to Kafka and ZookeeperIntroduction to Kafka and Zookeeper
Introduction to Kafka and Zookeeper
Rahul Jain
 
So we're running Apache ZooKeeper. Now What? By Camille Fournier
So we're running Apache ZooKeeper. Now What? By Camille Fournier So we're running Apache ZooKeeper. Now What? By Camille Fournier
So we're running Apache ZooKeeper. Now What? By Camille Fournier
Hakka Labs
 
Introduction and Overview of Apache Kafka, TriHUG July 23, 2013
Introduction and Overview of Apache Kafka, TriHUG July 23, 2013Introduction and Overview of Apache Kafka, TriHUG July 23, 2013
Introduction and Overview of Apache Kafka, TriHUG July 23, 2013
mumrah
 
GAME 3400 Level Design - Experience Architecture
GAME 3400 Level Design - Experience ArchitectureGAME 3400 Level Design - Experience Architecture
GAME 3400 Level Design - Experience Architecture
Seth Sivak
 
Kaazing Gateway + Apache Active MQ + Javascript + Stomp
Kaazing Gateway + Apache Active MQ + Javascript + StompKaazing Gateway + Apache Active MQ + Javascript + Stomp
Kaazing Gateway + Apache Active MQ + Javascript + Stomp
Irontec
 
Zookeeper: Wait-free Coordination for Internet-scale Systems
Zookeeper: Wait-free Coordination for Internet-scale SystemsZookeeper: Wait-free Coordination for Internet-scale Systems
Zookeeper: Wait-free Coordination for Internet-scale Systems
Leandro Lera Romero
 
2017-01-26 Internet Arriskuak: Andramendi Ikastola, Gurasoen saioa
2017-01-26 Internet Arriskuak: Andramendi Ikastola, Gurasoen saioa2017-01-26 Internet Arriskuak: Andramendi Ikastola, Gurasoen saioa
2017-01-26 Internet Arriskuak: Andramendi Ikastola, Gurasoen saioa
Urko Zurutuza
 
Monitorización
MonitorizaciónMonitorización
Monitorización
Miguel Angel Nieto
 
Strata + Hadoop World 2012: HDFS: Now and Future
Strata + Hadoop World 2012: HDFS: Now and FutureStrata + Hadoop World 2012: HDFS: Now and Future
Strata + Hadoop World 2012: HDFS: Now and Future
Cloudera, Inc.
 
Conferencia 2: El esquema
Conferencia 2: El esquemaConferencia 2: El esquema
Conferencia 2: El esquema
Jorge Luis Betancourt Gonzalez
 
Replicación Mysql
Replicación MysqlReplicación Mysql
Replicación Mysql
Miguel Angel Nieto
 
Mysql Administracion
Mysql AdministracionMysql Administracion
Mysql Administracion
Miguel Angel Nieto
 
使用ZooKeeper打造軟體式負載平衡
使用ZooKeeper打造軟體式負載平衡使用ZooKeeper打造軟體式負載平衡
使用ZooKeeper打造軟體式負載平衡
Lawrence Huang
 
¿Cómo se despliega y autoescala Couchbase en Cloud? ¡Aprende de manera práctica!
¿Cómo se despliega y autoescala Couchbase en Cloud? ¡Aprende de manera práctica!¿Cómo se despliega y autoescala Couchbase en Cloud? ¡Aprende de manera práctica!
¿Cómo se despliega y autoescala Couchbase en Cloud? ¡Aprende de manera práctica!
Paradigma Digital
 
Hadoop: MapReduce para procesar grandes cantidades de datos
Hadoop: MapReduce para procesar grandes cantidades de datosHadoop: MapReduce para procesar grandes cantidades de datos
Hadoop: MapReduce para procesar grandes cantidades de datos
Raul Ochoa
 
Introduction to Terracotta
Introduction to TerracottaIntroduction to Terracotta
Introduction to Terracotta
Cris Holdorph
 
MySQL - High Availability - Load Balacing - Cluster
MySQL - High Availability - Load Balacing - ClusterMySQL - High Availability - Load Balacing - Cluster
MySQL - High Availability - Load Balacing - Cluster
Miguel Angel Nieto
 

Viewers also liked (20)

Distributed Applications with Apache Zookeeper
Distributed Applications with Apache ZookeeperDistributed Applications with Apache Zookeeper
Distributed Applications with Apache Zookeeper
 
Introduction to ZooKeeper - TriHUG May 22, 2012
Introduction to ZooKeeper - TriHUG May 22, 2012Introduction to ZooKeeper - TriHUG May 22, 2012
Introduction to ZooKeeper - TriHUG May 22, 2012
 
Zookeeper In Simple Words
Zookeeper In Simple WordsZookeeper In Simple Words
Zookeeper In Simple Words
 
Introduction to Kafka and Zookeeper
Introduction to Kafka and ZookeeperIntroduction to Kafka and Zookeeper
Introduction to Kafka and Zookeeper
 
So we're running Apache ZooKeeper. Now What? By Camille Fournier
So we're running Apache ZooKeeper. Now What? By Camille Fournier So we're running Apache ZooKeeper. Now What? By Camille Fournier
So we're running Apache ZooKeeper. Now What? By Camille Fournier
 
Introduction and Overview of Apache Kafka, TriHUG July 23, 2013
Introduction and Overview of Apache Kafka, TriHUG July 23, 2013Introduction and Overview of Apache Kafka, TriHUG July 23, 2013
Introduction and Overview of Apache Kafka, TriHUG July 23, 2013
 
GAME 3400 Level Design - Experience Architecture
GAME 3400 Level Design - Experience ArchitectureGAME 3400 Level Design - Experience Architecture
GAME 3400 Level Design - Experience Architecture
 
Kaazing Gateway + Apache Active MQ + Javascript + Stomp
Kaazing Gateway + Apache Active MQ + Javascript + StompKaazing Gateway + Apache Active MQ + Javascript + Stomp
Kaazing Gateway + Apache Active MQ + Javascript + Stomp
 
Zookeeper: Wait-free Coordination for Internet-scale Systems
Zookeeper: Wait-free Coordination for Internet-scale SystemsZookeeper: Wait-free Coordination for Internet-scale Systems
Zookeeper: Wait-free Coordination for Internet-scale Systems
 
2017-01-26 Internet Arriskuak: Andramendi Ikastola, Gurasoen saioa
2017-01-26 Internet Arriskuak: Andramendi Ikastola, Gurasoen saioa2017-01-26 Internet Arriskuak: Andramendi Ikastola, Gurasoen saioa
2017-01-26 Internet Arriskuak: Andramendi Ikastola, Gurasoen saioa
 
Monitorización
MonitorizaciónMonitorización
Monitorización
 
Strata + Hadoop World 2012: HDFS: Now and Future
Strata + Hadoop World 2012: HDFS: Now and FutureStrata + Hadoop World 2012: HDFS: Now and Future
Strata + Hadoop World 2012: HDFS: Now and Future
 
Conferencia 2: El esquema
Conferencia 2: El esquemaConferencia 2: El esquema
Conferencia 2: El esquema
 
Replicación Mysql
Replicación MysqlReplicación Mysql
Replicación Mysql
 
Mysql Administracion
Mysql AdministracionMysql Administracion
Mysql Administracion
 
使用ZooKeeper打造軟體式負載平衡
使用ZooKeeper打造軟體式負載平衡使用ZooKeeper打造軟體式負載平衡
使用ZooKeeper打造軟體式負載平衡
 
¿Cómo se despliega y autoescala Couchbase en Cloud? ¡Aprende de manera práctica!
¿Cómo se despliega y autoescala Couchbase en Cloud? ¡Aprende de manera práctica!¿Cómo se despliega y autoescala Couchbase en Cloud? ¡Aprende de manera práctica!
¿Cómo se despliega y autoescala Couchbase en Cloud? ¡Aprende de manera práctica!
 
Hadoop: MapReduce para procesar grandes cantidades de datos
Hadoop: MapReduce para procesar grandes cantidades de datosHadoop: MapReduce para procesar grandes cantidades de datos
Hadoop: MapReduce para procesar grandes cantidades de datos
 
Introduction to Terracotta
Introduction to TerracottaIntroduction to Terracotta
Introduction to Terracotta
 
MySQL - High Availability - Load Balacing - Cluster
MySQL - High Availability - Load Balacing - ClusterMySQL - High Availability - Load Balacing - Cluster
MySQL - High Availability - Load Balacing - Cluster
 

Similar to Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API Examples

Zookeeper big sonata
Zookeeper  big sonataZookeeper  big sonata
Zookeeper big sonata
Anh Le
 
Zookeeper Tutorial for beginners
Zookeeper Tutorial for beginnersZookeeper Tutorial for beginners
Zookeeper Tutorial for beginners
jeetendra mandal
 
Leo's Notes about Apache Kafka
Leo's Notes about Apache KafkaLeo's Notes about Apache Kafka
Leo's Notes about Apache Kafka
Léopold Gault
 
Apache ZooKeeper TechTuesday
Apache ZooKeeper TechTuesdayApache ZooKeeper TechTuesday
Apache ZooKeeper TechTuesday
Andrei Savu
 
ZooKeeper Partitioning - A project report
ZooKeeper Partitioning - A project reportZooKeeper Partitioning - A project report
ZooKeeper Partitioning - A project report
pramodbiligiri
 
Shopzilla On Concurrency
Shopzilla On ConcurrencyShopzilla On Concurrency
Shopzilla On Concurrency
Will Gage
 
Apache Geode Clubhouse - WAN-based Replication
Apache Geode Clubhouse - WAN-based ReplicationApache Geode Clubhouse - WAN-based Replication
Apache Geode Clubhouse - WAN-based Replication
PivotalOpenSourceHub
 
Базы данных. ZooKeeper
Базы данных. ZooKeeperБазы данных. ZooKeeper
Базы данных. ZooKeeper
Vadim Tsesko
 
Achieving mass scale with Quasar Fibers
Achieving mass scale with Quasar FibersAchieving mass scale with Quasar Fibers
Achieving mass scale with Quasar Fibers
Idan Sheinberg
 
ZeroMq ZooKeeper and FlatBuffers
ZeroMq ZooKeeper and FlatBuffersZeroMq ZooKeeper and FlatBuffers
ZeroMq ZooKeeper and FlatBuffers
Ravi Okade
 
Near Real time Indexing Kafka Messages to Apache Blur using Spark Streaming
Near Real time Indexing Kafka Messages to Apache Blur using Spark StreamingNear Real time Indexing Kafka Messages to Apache Blur using Spark Streaming
Near Real time Indexing Kafka Messages to Apache Blur using Spark Streaming
Dibyendu Bhattacharya
 
All Aboard the Databus
All Aboard the DatabusAll Aboard the Databus
All Aboard the Databus
Amy W. Tang
 
Streaming Processing with a Distributed Commit Log
Streaming Processing with a Distributed Commit LogStreaming Processing with a Distributed Commit Log
Streaming Processing with a Distributed Commit Log
Joe Stein
 
Handling Data in Mega Scale Systems
Handling Data in Mega Scale SystemsHandling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
Directi Group
 
Winter is coming? Not if ZooKeeper is there!
Winter is coming? Not if ZooKeeper is there!Winter is coming? Not if ZooKeeper is there!
Winter is coming? Not if ZooKeeper is there!
Joydeep Banik Roy
 
January 2011 HUG: Kafka Presentation
January 2011 HUG: Kafka PresentationJanuary 2011 HUG: Kafka Presentation
January 2011 HUG: Kafka Presentation
Yahoo Developer Network
 
Session 23 - Kafka and Zookeeper
Session 23 - Kafka and ZookeeperSession 23 - Kafka and Zookeeper
Session 23 - Kafka and Zookeeper
AnandMHadoop
 
Shopzilla On Concurrency
Shopzilla On ConcurrencyShopzilla On Concurrency
Shopzilla On Concurrency
Rodney Barlow
 
Sizing your alfresco platform
Sizing your alfresco platformSizing your alfresco platform
Sizing your alfresco platform
Luis Cabaceira
 
Cluster_Performance_Apache_Kafak_vs_RabbitMQ
Cluster_Performance_Apache_Kafak_vs_RabbitMQCluster_Performance_Apache_Kafak_vs_RabbitMQ
Cluster_Performance_Apache_Kafak_vs_RabbitMQ
Shameera Rathnayaka
 

Similar to Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API Examples (20)

Zookeeper big sonata
Zookeeper  big sonataZookeeper  big sonata
Zookeeper big sonata
 
Zookeeper Tutorial for beginners
Zookeeper Tutorial for beginnersZookeeper Tutorial for beginners
Zookeeper Tutorial for beginners
 
Leo's Notes about Apache Kafka
Leo's Notes about Apache KafkaLeo's Notes about Apache Kafka
Leo's Notes about Apache Kafka
 
Apache ZooKeeper TechTuesday
Apache ZooKeeper TechTuesdayApache ZooKeeper TechTuesday
Apache ZooKeeper TechTuesday
 
ZooKeeper Partitioning - A project report
ZooKeeper Partitioning - A project reportZooKeeper Partitioning - A project report
ZooKeeper Partitioning - A project report
 
Shopzilla On Concurrency
Shopzilla On ConcurrencyShopzilla On Concurrency
Shopzilla On Concurrency
 
Apache Geode Clubhouse - WAN-based Replication
Apache Geode Clubhouse - WAN-based ReplicationApache Geode Clubhouse - WAN-based Replication
Apache Geode Clubhouse - WAN-based Replication
 
Базы данных. ZooKeeper
Базы данных. ZooKeeperБазы данных. ZooKeeper
Базы данных. ZooKeeper
 
Achieving mass scale with Quasar Fibers
Achieving mass scale with Quasar FibersAchieving mass scale with Quasar Fibers
Achieving mass scale with Quasar Fibers
 
ZeroMq ZooKeeper and FlatBuffers
ZeroMq ZooKeeper and FlatBuffersZeroMq ZooKeeper and FlatBuffers
ZeroMq ZooKeeper and FlatBuffers
 
Near Real time Indexing Kafka Messages to Apache Blur using Spark Streaming
Near Real time Indexing Kafka Messages to Apache Blur using Spark StreamingNear Real time Indexing Kafka Messages to Apache Blur using Spark Streaming
Near Real time Indexing Kafka Messages to Apache Blur using Spark Streaming
 
All Aboard the Databus
All Aboard the DatabusAll Aboard the Databus
All Aboard the Databus
 
Streaming Processing with a Distributed Commit Log
Streaming Processing with a Distributed Commit LogStreaming Processing with a Distributed Commit Log
Streaming Processing with a Distributed Commit Log
 
Handling Data in Mega Scale Systems
Handling Data in Mega Scale SystemsHandling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
 
Winter is coming? Not if ZooKeeper is there!
Winter is coming? Not if ZooKeeper is there!Winter is coming? Not if ZooKeeper is there!
Winter is coming? Not if ZooKeeper is there!
 
January 2011 HUG: Kafka Presentation
January 2011 HUG: Kafka PresentationJanuary 2011 HUG: Kafka Presentation
January 2011 HUG: Kafka Presentation
 
Session 23 - Kafka and Zookeeper
Session 23 - Kafka and ZookeeperSession 23 - Kafka and Zookeeper
Session 23 - Kafka and Zookeeper
 
Shopzilla On Concurrency
Shopzilla On ConcurrencyShopzilla On Concurrency
Shopzilla On Concurrency
 
Sizing your alfresco platform
Sizing your alfresco platformSizing your alfresco platform
Sizing your alfresco platform
 
Cluster_Performance_Apache_Kafak_vs_RabbitMQ
Cluster_Performance_Apache_Kafak_vs_RabbitMQCluster_Performance_Apache_Kafak_vs_RabbitMQ
Cluster_Performance_Apache_Kafak_vs_RabbitMQ
 

Recently uploaded

20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website
Pixlogix Infotech
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
sonjaschweigert1
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
TIPNGVN2
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Rohit Gautam
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 

Recently uploaded (20)

20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial ApplicationsLarge Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 

Apache Zookeeper Explained: Tutorial, Use Cases and Zookeeper Java API Examples

  • 2. Agenda  Coordination headache in distributed applications  What is zookeeper?  Zookeeper Overview  Zookeeper Service & Data Model  ZAB (Zookeeper Atomic Broadcasting Protocol)  Zookeeper Read, Write and Watch  Zookeeper Use Cases  Zookeeper Client API Example
  • 3. Coordination headache in distributed applications  Problem of Detection  Cluster Size Estimation  Leader Election  Group Membership  Configuration Management  Consensus among multiple parties is difficult in dynamic distributed systems as members may leave/fail/join anytime  Consistency, Availability and Partition tolerance
  • 4. What is Zookeeper?  Apache Zookeeper is a highly consistent, scalable and reliable cluster co-ordination service.  Zookeeper is a CP system where Consistency and Partition tolerance are guaranteed.  Off-the-shelf support for implementing consensus, group management, leader election etc.  Zookeeper cluster is called Ensemble  Ensemble has one leader and remaining followers  Zookeeper is a Distributed Coordination Service for Distributed Applications.  Zookeeper is originally developed by Yahoo research Team and became a top level Apache project. Zookeeper is used by many enterprises including Rackspace, Yahoo! and eBay.
  • 5. Zookeeper Overview  Zookeeper use Zookeeper Broadcasting Protocol (ZAB) for ordering and integrity.  Achieves High availability and performance via replication.  Low latency and high throughput via in memory image of data tree ,transaction logs and snapshots.  Uses “quorum” model and will only be available if a majority of servers are alive (by a strict majority of nodes).  Zookeeper relies on a quorum for durability as well.  Pipelined architecture of Zookeeper enables the execution of operations from a single client in FIFO order.  Ordered updates uses ZAB protocol (a leader-based atomic broadcast protocol) for Ordering, Integrity , exactly once.  Election of a new leader is followed by a synchronization phase before any new changes are accepted.
  • 6. Zookeeper Service & Data Model  At any given time, one Zookeeper client is connected to one Zookeeper server.  If the node (to which client connected) fails, client automatically connect to another server in ensemble with transparent session transfer.  Znode is an in-memory data node in Zookeeper Data organized in a hierarchical namespace (data tree)  The data model of ZooKeeper (znode) is essentially a file system with a simplified API and associated meta data and versions.  Regular, Ephemeral and Sequential Znode are supported.  Each ZNode is using UNIX style notation. For example, /A/B/C to denote the path to znode C, where C has B as its parent and B has A as its parent.  All znodes store data, and all znodes, except for ephemeral znodes, can have children.
  • 7. Zookeeper read/write/watch  Clients make use of Zookeeper API to update/write Zookeeper the data tree.  All Updates are ordered and managed by leader with unique zxid and are considered complete when a quorum confirms the update.  The leader executes update request and broadcasts the change to the ZooKeeper state through Zab.  Zab by default uses a simple majority quorums to decide on a proposal.  Read and Watch requests on the znode sent by a client to a ZooKeeper server are processed locally.  Watch –Allows client to get notification if an already returned information gets updated.
  • 9. Zookeeper read/write/watch  Read and Watch requests on the znode are processed locally but, write requests are propagated to leader server and go through a consensus before a response is generated.  Throughput of read request increases with number of servers and throughput of write requests, decreases with number of servers.
  • 10. Zookeeper use cases  ZooKeeper offers the library to create and manage synchronization primitives.  ZooKeeper avoids the single-point-of-failure. Typical use cases ,  Naming service  Configuration management  Synchronization  Leader election  Message Queue  Notification system
  • 11. Zookeeper Java Client API Examples  A simple example.  To check the complete example see at the below link.  http://java.globinch.com/enterprise- services/zookeeper/apache-zookeeper-explained- tutorial-cases-zookeeper-java-api-examples/

Editor's Notes

  1. 1)Number of non-faulty nodes currently in the system (heartbeat.Distributed Hashtables (DHT)) Leader/master detection & availability Membership notifications etc. 2)