Distributed Hash Table and Consistent Hashing


Published on

An Introduction to Consistent Hashing and its uses

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Distributed Hash Table and Consistent Hashing

  1. 1. CloudFundoo 2012 Distributed Hash Tables and Consistent HashingDHT(Distributed Hash Table) is one of the fundamental algorithmsused in distributed scalable systems; it is used in web caching, P2Psystems, distributed file systems etc.First step in understanding DHT is Hash Tables. Hash tables needkey, value and a hash function, where hash function maps the key to alocation where the value is stored. Keys Hash Function Stored Values Key1 Value3 Key2 Value4 Key3 Value1 Value2 Key4 value = hashfunc(key)Python’s dictionary data type is implemented using hashing, see theexample below. 1. #!/usr/bin/python 2. 3. dict = {Name: Zara, Age: 11, Class: First}; 1 4. 5. dict[Age] = 12; 6. dict[School] = "State School"; 7.Copyright © CloudFundoo | cloudfundoo@gmail.com, http://cloudfundoo.wordpress.com/
  2. 2. CloudFundoo 2012 8. 9. print "dict[Age]: ", dict[Age]; 10. print "dict[School]: ", dict[School];If we have a perfect hash function we will get an O (1) performancei.e. constant time performance out of hash table while searching for a(key, value) pair, this is because hash function distributes the keysevenly across the table. One of the problem with hashing is it requireslot of memory (or space) to accommodate the entire table, even ifmost of the table is empty we need to allocate memory for entiretable, so there is waste of memory most of the time. This is called astime-space tradeoff, hashing gives best time for search at the expenseof memory.When we want to accommodate large number of keys (millions andmillions, say for the case of a cloud storage system), we will have todivide keys in to subsets, and map those subsets of keys to a bucket,each bucket can reside in a separate machine/node. You can assumebucket as a separate hash table.Distributed Hash Table Using buckets to distribute the (key, value) pair is called DHT.A simple scheme to implement DHT is by using modulus operationon key i.e. your hash function is key mod n, where n is the number ofbuckets you have. Key Space K1 Kn/3 K2n/3 Kn 2 Bucket 1 Bucket 2 Bucket 3Copyright © CloudFundoo | cloudfundoo@gmail.com, http://cloudfundoo.wordpress.com/
  3. 3. CloudFundoo 2012If you have 6 buckets then, key = 1 will go to bucket 1 since key % 6= 1, key=2 will go to bucket 2 since key % 6 = 2 and so on. We willneed a second hashing to find the actual (key, value) pair inside aparticular bucket.We can use two dictionaries to visualize DHT; here each row inClient/Proxy dictionary is equivalent to a bucket in DHT. Bucket 1 Client/Proxy Bucket 3 Bucket 0 3This scheme will work perfectly fine as long as we don’t change thenumber of buckets. This scheme starts to fail when we add/removeCopyright © CloudFundoo | cloudfundoo@gmail.com, http://cloudfundoo.wordpress.com/
  4. 4. CloudFundoo 2012buckets to/from the system. Lets add one more bucket to the system,the number of buckets is now equal to seven, i.e. n=7. The key = 7which was previously mapped to bucket 1 now map to bucket 0 sincekey % 7 is equal to 0. In order to make it still work we need to movethe data between buckets, which is going to be expensive in thishashing scheme.Let’s do some calculation, consider modulo hash function, h(key) = key mod nWhere n is the number of buckets, when we increase the number ofbuckets by one, the hash function becomes h(key) = key mod (n+1) Because of the addition of a new bucket, most of keys will hash to adifferent bucket, let’s calculate the ratio of keys moving to differentbucket, K–n keys will move to a different bucket if keys are in therange 0 to K, only the first n keys will remain in the same buckets. Soratio of keys moving to a different bucket is (K – n)/K = 1- n/KIf there are 10 buckets and 1000 keys, then 99% of keys will move toa different bucket when we add another bucket. If we are usingpython’s hash() or hashlib.md5 hashing functions, then the fraction ofkeys moving to another bucket is 4 n/(n +1)Copyright © CloudFundoo | cloudfundoo@gmail.com, http://cloudfundoo.wordpress.com/
  5. 5. CloudFundoo 2012So we need a scheme to reduce the number of keys moving to adifferent bucket, consistent hashing is a scheme for the same.Consistent HashingA ring is the core of consistent hashing; first we hash the bucket IDsto points on ring. B1 B4 B2 B3Then we hash the keys to ring, the resulting ring will look like below. B1 K1 K4 B3 B2 K3 K2 B3 5So if we want to find the bucket which stores the value correspondingto a key, we first need to hash the key to a point in that ring and thenCopyright © CloudFundoo | cloudfundoo@gmail.com, http://cloudfundoo.wordpress.com/
  6. 6. CloudFundoo 2012we need to search in the clockwise direction in the ring to find thefirst bucket in that ring, that bucket will be the one storing the valuecorresponding to the key. For key K1 value will be stored in bucketB2, for key K2 value will be stored in bucket B3 and so on.Hashing is working fine with this scheme, but we introduced thisscheme to handle addition/removal of buckets, let see how it handlesthis, this is explained in below picture. B1 K4 K1 B3 B2 K3 K2 B3So if we are removing bucket B3, key K2 seems to have a problem,let’s see how consistent hashing solves this problem, key K2 still hashto the same point in circle, while searching in the clockwise directionit sees no bucket called B3, so searches past B3 in clockwise directionand it will find bucket B4, where value corresponding to key K2 isstored. For other keys there is no problem, all remains same, key K4in bucket B1, key K1 in bucket B2 etc. So we need to move only thecontents of removed bucket to the clockwise adjacent bucket. 6Let’s see what will happen if we add a bucket, see a slightly modifieddiagram below.Copyright © CloudFundoo | cloudfundoo@gmail.com, http://cloudfundoo.wordpress.com/
  7. 7. CloudFundoo 2012 B1 K5 K1 K4 B2 B3 K3 K2 B3The additional key K5 is mapped to B1, so we have both keys K4 andK5 mapping to bucket B1, like bucket removal scenario where keysK2 and K3 maps to bucket B4 after removal. K5 B1 B5 K1 K4 B2 B3 K2 K3 B3 7Let’s add a new bucket B5, the new bucket B5 goes in between keysK4 and K5, key K4 which was previously mapped to bucket B1, nowCopyright © CloudFundoo | cloudfundoo@gmail.com, http://cloudfundoo.wordpress.com/
  8. 8. CloudFundoo 2012goes to bucket B5 and Key K5 still maps to bucket B1. So only thekeys which lie between B4 and B5 should be moved from B1 to B5.On an average the fraction of keys which we need to move betweenbuckets when one bucket is added to the system is given as 1/(n +1)So by introducing consistent hashing we reduced the fraction of keyswhich we need to move, from n/(n+1) to 1/(n+1), which is significant.There is lot of details to consistent hashing, which is not covered inthis. Consistent hashing has a great role in distributed systems likeDNS, P2P, distributed storage, and web caching systems etc,OpenStack Swift Storage and Memcached are open source projectswhich use this to achieve scalability and Availability. <EOF> 8Copyright © CloudFundoo | cloudfundoo@gmail.com, http://cloudfundoo.wordpress.com/