SlideShare a Scribd company logo
1 of 18
Jooho Lee, Jay
1. What is Hash?
2. Why we need another Hash
Algorithm?
3. Fundamentals of Consistent
Hashing
4. Strength of Consistent Hashing
Hash table
Data Structure
Hash function
Calculation
Duplicate
Trend
Poor Point of Original Hashing
Trend
Cloud Environment
Distributed Caching
Significant Point
Horizontal Scalability
Capability
Original Hashing Algorithm for Caching
(Key k, Value v)
Hash Function(k) mod N (machine or size N)
Original Hashing Algorithm for Caching
(Key k, Value v)
Hash Function(k) mod N (machine or size N)
Binary 01100001 01100010 01100011 01100100
Hex 6 1 6 2 6 3 6 4
Ascii a b c d
abcd hashes to 0
0x61626364 = 1633831724
16338831724 % 4 = 0
Abbc hashes to 3
0x61626263 = 1633837667
1633837667 % 4 = 3
0 1 2 3
abcd abbc
Binary 01100001 01100010 01100011 01100100
Hex 6 1 6 2 6 3 6 4
Ascii a b c d
abcd hashes to 0
0x61626364 = 1633831724
16338831724 % 5 = 4
Abbc hashes to 3
0x61626263 = 1633837667
1633837667 % 5 = 2 0 1 2 3
abcd abbc
4
Basic Concept
Example
Hashing machine number (j)
Hash Function(j)
Distribute keys to near machine
depending on the hash number
Clockwise of the key.
Reallocate rate
1/rn (j = machine number, r = replica
count, n = machine count)
(j,0), (j,1)….(j,r-1)
Machine 2
Machine 1
Machine 0
Hashing(Key)
Machine 2
Machine 1
Machine 0
Hashing(Key)
Crash
Few keys should be re-allocated
Easy Horizontal Scalability
Expand
Shrink
Better Performance
Similar keys in near machine or same
machine
 http://charsyam.wordpress.com/2012/11/26/%EC%9E%85-
%EA%B0%9C%EB%B0%9C-consistent-hashing-%EA%B3%BC-replication/
 http://charsyam.wordpress.com/2012/11/26/%EC%9E%85-
%EA%B0%9C%EB%B0%9C-consistent-hashing-%EA%B3%BC-replication/

More Related Content

What's hot

Query Compilation in Impala
Query Compilation in ImpalaQuery Compilation in Impala
Query Compilation in Impala
Cloudera, Inc.
 
Dreaming Infrastructure
Dreaming InfrastructureDreaming Infrastructure
Dreaming Infrastructure
kyhpudding
 

What's hot (20)

Cassandra
CassandraCassandra
Cassandra
 
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, Preset
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, PresetStreaming Data Analytics with ksqlDB and Superset | Robert Stolz, Preset
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, Preset
 
Top 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark ApplicationsTop 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark Applications
 
RocksDB detail
RocksDB detailRocksDB detail
RocksDB detail
 
Cassandra 101
Cassandra 101Cassandra 101
Cassandra 101
 
Apache Ambari: Past, Present, Future
Apache Ambari: Past, Present, FutureApache Ambari: Past, Present, Future
Apache Ambari: Past, Present, Future
 
HBase Accelerated: In-Memory Flush and Compaction
HBase Accelerated: In-Memory Flush and CompactionHBase Accelerated: In-Memory Flush and Compaction
HBase Accelerated: In-Memory Flush and Compaction
 
Apache HBase Performance Tuning
Apache HBase Performance TuningApache HBase Performance Tuning
Apache HBase Performance Tuning
 
Flink Forward San Francisco 2019: Moving from Lambda and Kappa Architectures ...
Flink Forward San Francisco 2019: Moving from Lambda and Kappa Architectures ...Flink Forward San Francisco 2019: Moving from Lambda and Kappa Architectures ...
Flink Forward San Francisco 2019: Moving from Lambda and Kappa Architectures ...
 
uReplicator: Uber Engineering’s Scalable, Robust Kafka Replicator
uReplicator: Uber Engineering’s Scalable,  Robust Kafka ReplicatoruReplicator: Uber Engineering’s Scalable,  Robust Kafka Replicator
uReplicator: Uber Engineering’s Scalable, Robust Kafka Replicator
 
Query Compilation in Impala
Query Compilation in ImpalaQuery Compilation in Impala
Query Compilation in Impala
 
Dreaming Infrastructure
Dreaming InfrastructureDreaming Infrastructure
Dreaming Infrastructure
 
Deep Dive on Amazon Aurora
Deep Dive on Amazon AuroraDeep Dive on Amazon Aurora
Deep Dive on Amazon Aurora
 
Cassandra Introduction & Features
Cassandra Introduction & FeaturesCassandra Introduction & Features
Cassandra Introduction & Features
 
Introduction to Apache Cassandra
Introduction to Apache CassandraIntroduction to Apache Cassandra
Introduction to Apache Cassandra
 
Binomial Heaps and Fibonacci Heaps
Binomial Heaps and Fibonacci HeapsBinomial Heaps and Fibonacci Heaps
Binomial Heaps and Fibonacci Heaps
 
Ozone: scaling HDFS to trillions of objects
Ozone: scaling HDFS to trillions of objectsOzone: scaling HDFS to trillions of objects
Ozone: scaling HDFS to trillions of objects
 
Hbase
HbaseHbase
Hbase
 
Bigtable and Dynamo
Bigtable and DynamoBigtable and Dynamo
Bigtable and Dynamo
 
Apache Flink and Apache Hudi.pdf
Apache Flink and Apache Hudi.pdfApache Flink and Apache Hudi.pdf
Apache Flink and Apache Hudi.pdf
 

Similar to Consistent hashing

Similar to Consistent hashing (20)

4.4 hashing02
4.4 hashing024.4 hashing02
4.4 hashing02
 
Smart City Lecture 5 - Introduction to Encryption
Smart City Lecture 5 - Introduction to EncryptionSmart City Lecture 5 - Introduction to Encryption
Smart City Lecture 5 - Introduction to Encryption
 
Hash functions
Hash functionsHash functions
Hash functions
 
Webinar: MongoDB 2.4 Feature Demo and Q&A on Hash-based Sharding
Webinar: MongoDB 2.4 Feature Demo and Q&A on Hash-based ShardingWebinar: MongoDB 2.4 Feature Demo and Q&A on Hash-based Sharding
Webinar: MongoDB 2.4 Feature Demo and Q&A on Hash-based Sharding
 
A New Key Stream Generator Based on 3D Henon map and 3D Cat map
A New Key Stream Generator Based on 3D Henon map and 3D Cat mapA New Key Stream Generator Based on 3D Henon map and 3D Cat map
A New Key Stream Generator Based on 3D Henon map and 3D Cat map
 
Locality Sensitive Hashing By Spark
Locality Sensitive Hashing By SparkLocality Sensitive Hashing By Spark
Locality Sensitive Hashing By Spark
 
Hash function landscape
Hash function landscapeHash function landscape
Hash function landscape
 
Clustering techniques
Clustering techniquesClustering techniques
Clustering techniques
 
[CB20] Cryfind : A Static Tool to Identify Cryptographic Algorithm in Binary ...
[CB20] Cryfind : A Static Tool to Identify Cryptographic Algorithm in Binary ...[CB20] Cryfind : A Static Tool to Identify Cryptographic Algorithm in Binary ...
[CB20] Cryfind : A Static Tool to Identify Cryptographic Algorithm in Binary ...
 
session 15 hashing.pptx
session 15   hashing.pptxsession 15   hashing.pptx
session 15 hashing.pptx
 
SHA512.pptx
SHA512.pptxSHA512.pptx
SHA512.pptx
 
Big spatial2014 mapreduceweights
Big spatial2014 mapreduceweightsBig spatial2014 mapreduceweights
Big spatial2014 mapreduceweights
 
A Novel Approach of Caching Direct Mapping using Cubic Approach
A Novel Approach of Caching Direct Mapping using Cubic ApproachA Novel Approach of Caching Direct Mapping using Cubic Approach
A Novel Approach of Caching Direct Mapping using Cubic Approach
 
Data Mining Lecture_4.pptx
Data Mining Lecture_4.pptxData Mining Lecture_4.pptx
Data Mining Lecture_4.pptx
 
Probabilistic Data Structures and Approximate Solutions Oleksandr Pryymak
Probabilistic Data Structures and Approximate Solutions Oleksandr PryymakProbabilistic Data Structures and Approximate Solutions Oleksandr Pryymak
Probabilistic Data Structures and Approximate Solutions Oleksandr Pryymak
 
Terracotta's OffHeap Explained
Terracotta's OffHeap ExplainedTerracotta's OffHeap Explained
Terracotta's OffHeap Explained
 
The Other HPC: High Productivity Computing in Polystore Environments
The Other HPC: High Productivity Computing in Polystore EnvironmentsThe Other HPC: High Productivity Computing in Polystore Environments
The Other HPC: High Productivity Computing in Polystore Environments
 
Engineering Fast Indexes for Big-Data Applications: Spark Summit East talk by...
Engineering Fast Indexes for Big-Data Applications: Spark Summit East talk by...Engineering Fast Indexes for Big-Data Applications: Spark Summit East talk by...
Engineering Fast Indexes for Big-Data Applications: Spark Summit East talk by...
 
Engineering fast indexes
Engineering fast indexesEngineering fast indexes
Engineering fast indexes
 
Sha
ShaSha
Sha
 

More from Jooho Lee

Practical byteman sample 20131128
Practical byteman sample 20131128Practical byteman sample 20131128
Practical byteman sample 20131128
Jooho Lee
 

More from Jooho Lee (8)

Quay 3.3 installation
Quay 3.3 installationQuay 3.3 installation
Quay 3.3 installation
 
OpenShift4 Installation by UPI on kvm
OpenShift4 Installation by UPI on kvmOpenShift4 Installation by UPI on kvm
OpenShift4 Installation by UPI on kvm
 
Hello istio
Hello istioHello istio
Hello istio
 
Docker, Atomic Host and Kubernetes.
Docker, Atomic Host and Kubernetes.Docker, Atomic Host and Kubernetes.
Docker, Atomic Host and Kubernetes.
 
OpenSCAP Overview(security scanning for docker image and container)
OpenSCAP Overview(security scanning for docker image and container)OpenSCAP Overview(security scanning for docker image and container)
OpenSCAP Overview(security scanning for docker image and container)
 
OpenSCAP Overview(security scanning for docker image and container)
OpenSCAP Overview(security scanning for docker image and container)OpenSCAP Overview(security scanning for docker image and container)
OpenSCAP Overview(security scanning for docker image and container)
 
Tcp summary
Tcp summaryTcp summary
Tcp summary
 
Practical byteman sample 20131128
Practical byteman sample 20131128Practical byteman sample 20131128
Practical byteman sample 20131128
 

Recently uploaded

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Recently uploaded (20)

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 

Consistent hashing