As part of BitTorrent's Tech Talks series, Arvid Norberg explains how BitTorrent Distributed Hash Tables work.

1. TECH TALKS
BitTorrent DHT | Arvid Norberg
BitTorrent, Inc / 2013 Tech Talks 01

2. INTRODUCTION
DHT = DISTRIBUTED HASH TABLE
(It’s like a hash table, where the payload is divided up across many nodes.)
BitTorrent, Inc / 2013 Tech Talks 02

3. INTRODUCTION
The DHT is primarily used to introduce peers to each other.
Who e
lse is
PEER on sw
arm X
?
172.4
119.7 .12.7
217.1 3.53.
3.98.
...
83
220 DHT
BitTorrent, Inc / 2013 Tech Talks 03

4. INTRODUCTION
protocol <what do nodes say to each other?>
topology <how are nodes organized?>
routing <how are target nodes found?>
routing table <representation of routing info>
traversal algorithm <search implementation?>
BitTorrent, Inc / 2013 Tech Talks 04

5. PROTOCOL
BitTorrent, Inc / 2013 Tech Talks 05

6. PROTOCOL
ping <are you still around?>
announce_peer <hey, i’m on this swarm!>
get_peers <who else is on this swarm?, or who can I ask?>
find_node <who is close to this ID?, or who can I ask?>
PEER
DHT
BitTorrent, Inc / 2013 Tech Talks 06

7. PROTOCOL
ping <are you still around?>
ONE-OFF MESSAGES
announce_peer <hey, i’m on this swarm!>
get_peers <who else is on this swarm?, or who can I ask?>
find_node <who is close to this ID?, or who can I ask?>
PEER
DHT
BitTorrent, Inc / 2013 Tech Talks 07

8. PROTOCOL
ping <are you still around?>
ONE-OFF MESSAGES
announce_peer <hey, i’m on this swarm!>
get_peers <who else is on this swarm?, or who can I ask?>
RECURSIVE
find_node <who is close to this ID?, or who can I ask?>
PEER
DHT
BitTorrent, Inc / 2013 Tech Talks 08

9. PROTOCOL
ping <are you still around?>
announce_peer <hey, i’m on this swarm!>
get_peers <who else is on this swarm?, or who can I ask?>
find_node <who is close to this ID?, or who can I ask?>
get_peers (x)
PEER nodes: <...>
DHT
BitTorrent, Inc / 2013 Tech Talks 09

10. PROTOCOL
ping <are you still around?>
announce_peer <hey, i’m on this swarm!>
get_peers <who else is on this swarm?, or who can I ask?>
find_node <who is close to this ID?, or who can I ask?>
get_ peers (x)
nod es: <...>
get_peers (x)
PEER nodes: <...>
get_pee
rs (x)
DHT
nodes:
REPEATS <...>
BitTorrent, Inc / 2013 Tech Talks 10

11. PROTOCOL
ping <are you still around?>
announce_peer <hey, i’m on this swarm!>
get_peers <who else is on this swarm?, or who can I ask?>
find_node <who is close to this ID?, or who can I ask?>
get_peers (x)
PEER nodes: <...>
get_pee
rs (x)
DHT
nodes:
TERMINATES WHEN WE <...>
RECEIVE VALUES (I.E. PEERS)
AND NO BETTER NODES
BitTorrent, Inc / 2013 Tech Talks 11

12. PROTOCOL
ping <are you still around?>
announce_peer <hey, i’m on this swarm!>
get_peers <who else is on this swarm?, or who can I ask?>
find_node <who are close to this ID?, or who can I ask?>
EACH MESSAGE INCLUDES ONES OWN NODE ID.
THIS ENABLES NODES TO LEARN ABOUT NEW
NODES AS PART OF THE PROTOCOL CHATTER.
BitTorrent, Inc / 2013 Tech Talks 12

13. PROTOCOL
bootstrap <find_nodes(self)>
refresh buckets <find_nodes (target-bucket)>
announce <get_peer(ih) + announce_peer(ih)>
BitTorrent, Inc / 2013 Tech Talks 13

14. PROTOCOL
Spoof protection
get_peers responds with a write-token
Write-token is typically a MAC of:
- source (IP, port)
- target info-hash
- local secret (which may expire in tens of minutes)
announce_peer requires a valid write token to insert the node in the peer list
BitTorrent, Inc / 2013 Tech Talks 14

15. TOPOLOGY
This section describes how nodes can
respond to recursive lookup queries like this,
with nodes closer and closer to the target.*
*Spoiler alert: it has to do with the topology
BitTorrent, Inc / 2013 Tech Talks 15

16. TOPOLOGY
The DHT is made up by all BitTorrent peers, across all swarms.
PEER PEER
PEER PEER PEER
PEER PEER PEER PEER
PEER PEER PEER PEER
PEER PEER PEER
PEER
BitTorrent, Inc / 2013 Tech Talks 16

17. TOPOLOGY
Each node has a self-assigned address, or node ID.
PEER PEER
71690...29736 24e52...a22a6
PEER PEER PEER
138b8...cea6f
caa39...9087a 09b9b...6216b
PEER PEER PEER PEER
f091c...7fc09
9a2ac...1993e 03afa...f0200 09b9b...6216b
PEER
6c6cf...10f46
PEER
f8466...b21cf
PEER PEER
5f08d...e2537
22942...48a07
PEER PEER PEER
255d5...b923d 0f35f...386ab 79280...f8bf3
PEER
2f1d5...04177
BitTorrent, Inc / 2013 Tech Talks 17

18. TOPOLOGY
Consider all nodes lined up in the node ID space....
0 2 160
BitTorrent, Inc / 2013 Tech Talks 18

19. TOPOLOGY
Consider all nodes lined up in the node ID space....
0 node-ID space 2 160
....all nodes appear evenly distributed in this space.
BitTorrent, Inc / 2013 Tech Talks 19

20. TOPOLOGY
IDs are 160 bits (20 bytes) long...
Keys in the hash table (info-hashes) are also 160 bits.
0 node-ID space 2 160
BitTorrent, Inc / 2013 Tech Talks 20

21. TOPOLOGY
IDs are 160 bits (20 bytes) long...
Keys in the hash table (info-hashes) are also 160 bits
info-hash
0 node-ID space 2 160
Nodes whose ID is close to an info-hash are responsible for
storing information about it.
BitTorrent, Inc / 2013 Tech Talks 21

22. TOPOLOGY
IDs are 160 bits (20 bytes) long...
Keys in the hash table (info-hashes) are also 160 bits
info-hash
0 node-ID space 2 160
Nodes whose ID is close to an info-hash are responsible for
storing information about it.
BitTorrent, Inc / 2013 Tech Talks 22

23. ROUTING
BitTorrent, Inc / 2013 Tech Talks 23

24. ROUTING
It is impractical for every node to know about every other node.
There are millions of nodes.
Nodes come and go constantly.
Every node specializes in knowing about all nodes close to itself.
The farther from itself, the more sparse its knowledge of nodes become.
BitTorrent, Inc / 2013 Tech Talks 24

25. ROUTING
The routing table orders nodes based on their distance from oneself.
distance distance
0 node-ID space self 2 160
BitTorrent, Inc / 2013 Tech Talks 25

26. ROUTING
The routing table orders nodes based on their distance from oneself.
distance distance
0 node-ID space self
2 160
self distance from self
This is simplified. The euclidian distance is not actually used.
BitTorrent, Inc / 2013 Tech Talks 26

27. ROUTING
The euclidian distance has a problem that it “folds” the space,
and nodes are no longer uniformly distributed (in the distance-space).
distance distance
0 node-ID space
2 160
self
BitTorrent, Inc / 2013 Tech Talks 27

28. ROUTING
The XOR distance metric is: d(a,b) = a ⊕ b
0 self distance from self
2 160
The XOR distance metric makes it so that the
distances are still uniformly distributed. It doesn't
“fold” the space the way euclidian distance does.
BitTorrent, Inc / 2013 Tech Talks 28

29. ROUTING
The distance space is divided up into buckets. Each bucket holds no more than 8 nodes.
0 self distance from self
2 160
BitTorrent, Inc / 2013 Tech Talks 29

30. ROUTING
The distance space is divided up into buckets. Each bucket holds no more than 8 nodes.
The space covered by a bucket is half as big as the previous one.
You know about more nodes close to you.
0 self 2 160
distance from self
... bucket 2 bucket 1 bucket 0
BitTorrent, Inc / 2013 Tech Talks 3-

31. ROUTING
For every hop in a recursive lookup, the nodes distance is cut in half.
Lookup complexity:
O(log n)
0 2 160
node ID space
target
This illustration is also simplified, the XOR Distance metric will
make you jump back and forth a bit.
Cutting your distance in half every hop still holds.
BitTorrent, Inc / 2013 Tech Talks 31

32. ROUTING TABLE
BitTorrent, Inc / 2013 Tech Talks 32

33. ROUTING TABLE
The XOR distance metric applied to the routing table just counts the length of the common bit-prefix.
OUR NODE ID: 10101110100010101001110...
OTHER NODE ID: 10101011010010110100101...
BitTorrent, Inc / 2013 Tech Talks 33

34. ROUTING TABLE
The XOR distance metric applied to the routing table just counts the length of the common bit-prefix.
OUR NODE ID: 10101110100010101001110...
OTHER NODE ID: 10101011010010110100101...
shared bit prefix: 5 bits
node belongs in bucket 5
BitTorrent, Inc / 2013 Tech Talks 34

35. ROUTING TABLE
A 160 bit space can be cut in half 160 times. There is a max of 160 buckets.
... prefix = 00 prefix = 0 no prefix
0 2 160
... bucket 2 bucket 1 bucket 0
BitTorrent, Inc / 2013 Tech Talks 35

36. ROUTING TABLE
View of the routing table in node ID space (instead of distance space).
bucket 0 3 4 bucket 2 bucket 1
0 self
2 160
bit 0
bit 1
bit 2
bit 3
BitTorrent, Inc / 2013 Tech Talks 36

37. ROUTING TABLE
A naïve routing table implementation would be an array of 160 buckets....
bucket 4
bucket 3
bucket 2
bucket 1
bucket 0
... bucket 5
BitTorrent, Inc / 2013 Tech Talks 37

38. ROUTING TABLE
A naïve routing table implementation would be an array of 160 buckets...
Not very efficient, since the majority of buckets will be empty.
bucket 4
bucket 3
bucket 2
bucket 1
bucket 0
... bucket 5
BitTorrent, Inc / 2013 Tech Talks 38

39. ROUTING TABLE
A typical routing table starts with only bucket 0.
When the 9th node is added, the bucket is split into bucket 0 and
bucket 1, with the nodes moved to their respective bucket.
Only the highest numbered bucket is ever split.
BitTorrent, Inc / 2013 Tech Talks 39

40. ROUTING TABLE
bucket 0
BitTorrent, Inc / 2013 Tech Talks 40

41. ROUTING TABLE
bucket 0
BitTorrent, Inc / 2013 Tech Talks 41

42. ROUTING TABLE
bucket 0
BitTorrent, Inc / 2013 Tech Talks 42

43. ROUTING TABLE
arrange nodes into the correct bucket
bucket 1 bucket 0
BitTorrent, Inc / 2013 Tech Talks 43

44. ROUTING TABLE
bucket 1 bucket 0
BitTorrent, Inc / 2013 Tech Talks 44

45. ROUTING TABLE
bucket 1 bucket 0
BitTorrent, Inc / 2013 Tech Talks 45

46. ROUTING TABLE
bucket 1 bucket 0
BitTorrent, Inc / 2013 Tech Talks 46

47. ROUTING TABLE
bucket 1 bucket 0
BitTorrent, Inc / 2013 Tech Talks 47

48. TRAVERSAL ALGORITHM
A deeper look at the get_peers and announce_peer query.
BitTorrent, Inc / 2013 Tech Talks 48

49. TRAVERSAL ALGORITHM
Pick known nodes out of routing table, close to the target we're looking up.
Sort by distance to target.
(IP, Node ID)
closer to target (IP, Node ID)
(IP, Node ID)
(IP, Node ID)
BitTorrent, Inc / 2013 Tech Talks 49

50. TRAVERSAL ALGORITHM
Send requests to 3 (or so) at a time.
s( ih)
pe er
g et_
(IP, Node ID) x
get_peers (ih)
closer to target (IP, Node ID) x
(IP, Node ID) x get
_pe
(IP, Node ID) ers
(ih)
BitTorrent, Inc / 2013 Tech Talks 50

51. TRAVERSAL ALGORITHM
Send requests to 3 (or so) at a time.
s( ....)
n ode
(IP, Node ID) x
nodes (....)
closer to target (IP, Node ID) x
(IP, Node ID) x
nod
es (
(IP, Node ID) ....)
BitTorrent, Inc / 2013 Tech Talks 51

52. TRAVERSAL ALGORITHM
Nodes are inserted in sorted order.
Nodes we already have are ignored. Nodes that don't respond, are marked as stale
(IP, Node ID)
closer to target (IP, Node ID)
(IP, Node ID)
(IP, Node ID)
(IP, Node ID) x
(IP, Node ID)
(IP, Node ID) x
(IP, Node ID) x
(IP, Node ID)
(IP, Node ID)
BitTorrent, Inc / 2013 Tech Talks 52

53. TRAVERSAL ALGORITHM
Keep requests 3 outstanding at all times.
s( ih)
pe er
g et_
(IP, Node ID) x
get_peers (ih)
closer to target (IP, Node ID) x
(IP, Node ID) x get
_pe
(IP, Node ID) ers
(ih)
(IP, Node ID) x
(IP, Node ID)
(IP, Node ID) x
(IP, Node ID) x
(IP, Node ID)
BitTorrent, Inc / 2013 Tech Talks 53

54. TRAVERSAL ALGORITHM
Terminating condition: the top 8 nodes have all been queried (and responded).
(IP, Node ID) x
closer to target (IP, Node ID) x
(IP, Node ID) x
(IP, Node ID) x
(IP, Node ID) x
(IP, Node ID) x
(IP, Node ID) x
(IP, Node ID) x
(IP, Node ID)
BitTorrent, Inc / 2013 Tech Talks 54

55. PROTOCOL
Send announce_peer to the top 8 nodes.
r( ih)
pee
n ce_ ih)
nn ou ee r(
(IP, Node ID) x
a
nc e_p ih)
nn ou ee r(
a e_p
closer to target (IP, Node ID) x u nc ih)
an no er (
_pe
(IP, Node ID) x ou nce r( ih)
ann e_ pee
(IP, Node ID) x ou nc r( ih)
a nn e_ pee
(IP, Node ID) x o unc ih)
a nn er (
_pe
(IP, Node ID) x ou nce r( ih)
ann e_ pee
(IP, Node ID) x no unc
an
(IP, Node ID) x
(IP, Node ID)
BitTorrent, Inc / 2013 Tech Talks 55