ROME: Optimising DHT-based
Peer-to-Peer Networks
James Salter and Nick Antonopoulos
Department of Computing, University of Surrey, UK
6th July 2005
The financial support of the UK Engineering and Physical Sciences Research Council (EPSRC) (for JS) is gratefully acknowledged
Outline
 Introduction and Background
 ROME Architecture
 Processes for Expansion
 Evaluation
 Conclusions and Future Work
What is P2P?
Client/ServerPeer-to-Peer
 No central server
 Node to node
connections
 Resilient
 Large #messages
 Clients send requests
to/via a central server
 Small #messages
 Single point of failure
File sharing, distributed computing, instant messaging
P2P Network Characteristics
 Nodes can act as clients and servers
 No centralised server/authority
 In Pure P2P Networks
 Network is highly dynamic
 Nodes join and leave regularly
 Large-scale
 Potentially millions of nodes
 Nodes are autonomous
 But co-operate to share/retrieve resources
Chord
 Structured Peer-to-Peer architecture
 Well-known in the research field
 Combines advantages of other P2P
architectures:
 An index of resources (e.g. Napster)
 But distributed over multiple nodes (e.g. Gnutella)
 Based on Distributed Hash Tables
 Simple lookup mechanism
 Given a key, it will return associated value(s)
Chord
20
0 1
8
12
15
2 3 4 5 6 7 8
Chord
log2(n) hops worst case
½log2(n) hops average
ROME Concept
 Message cost is proportional to number
of nodes in network (n)
 Reduce n, reduce message cost
 Goal: Keep the ring “just big enough”
 Must always support current workload
 Not unnecessarily large
 Workload should determine ring size,
not number of nodes in the network
 Adding additional functionality to Chord
ROME Architecture
ROME
Chord
Lower
Layers
Normal
Node Workload Monitoring
Zero LimitTarget Threshold
Overloaded
Node Workload Monitoring
 Node-level monitoring only shows the
status of a single node
 Ring-level monitoring would provide a
complete view
 Why not monitor at the ring level?
 Requires global knowledge
 Nodes need to report to a central entity
 Workload can fluctuate rapidly
 Many regular update messages
 Defeats the object of reducing message cost
Replace Operation
Overloaded
Replace Operation
Overloaded
Replace Operation
Add Operation
Overloaded
Add Operation
1615 17 18 19 20 22 23 24 25 2621 2714
ID: 27ID: 14
workload
ID: ??
Add Operation
Overloaded
Add Operation
Dynamic Ring Performance
 Reduction in hop count by controlling
network size based on theoretical work:
 Max hops per lookup = log2(n)
 Mean hops per lookup = ½ log2(n)
 If ring A < ring B, then log2(A) < log2(B)
 - in a static ring with correct routing information
 Does this hold true in more realistic
dynamic scenarios, with nodes
joining/leaving or failing?
Dynamic Ring: Mean Hop Count
0
1
2
3
4
5
6
3000 5000 7000 9000
Mean Node Lifetime (secs)
MeanHopCount
128 nodes
256 nodes
512 nodes
1024 nodes
Effect on Query Failure Rates
0%
20%
40%
60%
80%
100%
1000 3000 5000 7000 9000
Mean Node Lifetime (secs)
%CorrectLookups
1024 nodes
512 nodes
256 nodes
128 Nodes
64 Nodes
Reducing Ring Size with ROME
 Node Pool: 1 million nodes
 Node Capacity: 100 units
 Upper Threshold: 95% (95 units)
 Initial network-wide workload: 1 unit
 ROME Ring size: 1 node
 Chord Ring size: 1 million nodes
 What happens as we increase
network-wide workload?
Effect on Ring Size
0
200000
400000
600000
800000
1000000
1200000
0 30000000 60000000 90000000
Network-wide Workload
RingSize
Chord
ROME
Mean Hops per Lookup
0
2
4
6
8
10
12
0 30000000 60000000 90000000
Network-wide Workload
MeanHops/Lookup
Chord
ROME
Utilisation of Ring Capacity
0%
20%
40%
60%
80%
100%
0 30000000 60000000 90000000
Network-wide Workload
CapacityUtilisation
Chord
ROME
Conclusions
 Chord: DHT-based structured P2P
architecture with message cost
proportional to nodes in structure
 ROME: Reduce number of nodes in
structure, reduce message cost
 Add/replace processes on each node allow
Chord ring to grow as necessary
 Workload controls size, not nodes in network
 Smaller message costs than standard
Chord when control ring building
 What if workload decreases?
INC 2005 - ROME: Optimising DHT-based Peer-to-Peer Networks

INC 2005 - ROME: Optimising DHT-based Peer-to-Peer Networks

  • 1.
    ROME: Optimising DHT-based Peer-to-PeerNetworks James Salter and Nick Antonopoulos Department of Computing, University of Surrey, UK 6th July 2005 The financial support of the UK Engineering and Physical Sciences Research Council (EPSRC) (for JS) is gratefully acknowledged
  • 2.
    Outline  Introduction andBackground  ROME Architecture  Processes for Expansion  Evaluation  Conclusions and Future Work
  • 3.
    What is P2P? Client/ServerPeer-to-Peer No central server  Node to node connections  Resilient  Large #messages  Clients send requests to/via a central server  Small #messages  Single point of failure File sharing, distributed computing, instant messaging
  • 4.
    P2P Network Characteristics Nodes can act as clients and servers  No centralised server/authority  In Pure P2P Networks  Network is highly dynamic  Nodes join and leave regularly  Large-scale  Potentially millions of nodes  Nodes are autonomous  But co-operate to share/retrieve resources
  • 5.
    Chord  Structured Peer-to-Peerarchitecture  Well-known in the research field  Combines advantages of other P2P architectures:  An index of resources (e.g. Napster)  But distributed over multiple nodes (e.g. Gnutella)  Based on Distributed Hash Tables  Simple lookup mechanism  Given a key, it will return associated value(s)
  • 6.
  • 7.
    Chord log2(n) hops worstcase ½log2(n) hops average
  • 8.
    ROME Concept  Messagecost is proportional to number of nodes in network (n)  Reduce n, reduce message cost  Goal: Keep the ring “just big enough”  Must always support current workload  Not unnecessarily large  Workload should determine ring size, not number of nodes in the network  Adding additional functionality to Chord
  • 9.
  • 10.
    Normal Node Workload Monitoring ZeroLimitTarget Threshold Overloaded
  • 11.
    Node Workload Monitoring Node-level monitoring only shows the status of a single node  Ring-level monitoring would provide a complete view  Why not monitor at the ring level?  Requires global knowledge  Nodes need to report to a central entity  Workload can fluctuate rapidly  Many regular update messages  Defeats the object of reducing message cost
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
    Add Operation 1615 1718 19 20 22 23 24 25 2621 2714 ID: 27ID: 14 workload ID: ??
  • 17.
  • 18.
  • 19.
    Dynamic Ring Performance Reduction in hop count by controlling network size based on theoretical work:  Max hops per lookup = log2(n)  Mean hops per lookup = ½ log2(n)  If ring A < ring B, then log2(A) < log2(B)  - in a static ring with correct routing information  Does this hold true in more realistic dynamic scenarios, with nodes joining/leaving or failing?
  • 20.
    Dynamic Ring: MeanHop Count 0 1 2 3 4 5 6 3000 5000 7000 9000 Mean Node Lifetime (secs) MeanHopCount 128 nodes 256 nodes 512 nodes 1024 nodes
  • 21.
    Effect on QueryFailure Rates 0% 20% 40% 60% 80% 100% 1000 3000 5000 7000 9000 Mean Node Lifetime (secs) %CorrectLookups 1024 nodes 512 nodes 256 nodes 128 Nodes 64 Nodes
  • 22.
    Reducing Ring Sizewith ROME  Node Pool: 1 million nodes  Node Capacity: 100 units  Upper Threshold: 95% (95 units)  Initial network-wide workload: 1 unit  ROME Ring size: 1 node  Chord Ring size: 1 million nodes  What happens as we increase network-wide workload?
  • 23.
    Effect on RingSize 0 200000 400000 600000 800000 1000000 1200000 0 30000000 60000000 90000000 Network-wide Workload RingSize Chord ROME
  • 24.
    Mean Hops perLookup 0 2 4 6 8 10 12 0 30000000 60000000 90000000 Network-wide Workload MeanHops/Lookup Chord ROME
  • 25.
    Utilisation of RingCapacity 0% 20% 40% 60% 80% 100% 0 30000000 60000000 90000000 Network-wide Workload CapacityUtilisation Chord ROME
  • 26.
    Conclusions  Chord: DHT-basedstructured P2P architecture with message cost proportional to nodes in structure  ROME: Reduce number of nodes in structure, reduce message cost  Add/replace processes on each node allow Chord ring to grow as necessary  Workload controls size, not nodes in network  Smaller message costs than standard Chord when control ring building  What if workload decreases?

Editor's Notes

  • #4 What is Peer-to-Peer? It’s a network architecture (one definition – the basis of our work at least!) Traditional client/server architecture made up of a set of client machines all connected to the central server. Most of the resources (files, databases, processing, storage) users require are held on the server. Peer-to-Peer architectures (in the pure sense) are the opposite of this. All the machines in the network (often called nodes) have equal status – they host and consume resources. No central server or authority controlling the network. Problem: in a client/server architecture, its fairly easy to find the file/resource you want because they are all on the central server. In a p2p environment, since any machine can host resources, how do we find what we require? Discovering resources becomes a problem.
  • #7 Nodes are given a unique ID and arranged in a circle in ascending order. Each node is responsible for the key space between its predecessor and itself. E.g. node 8 responsible for keys hashing between 2 and 8. Lookups can start from anywhere and pass around the ring to reach the relevant node. E.g. start from node 15, looking for 6. However, that is not very efficient if there are lots of node in the ring…
  • #8 Instead, each node hosts a small routing table, called a finger table, pointing to selected other ring nodes. This allows lookups to quickly route through the ring. Chord guarantees that, provided all routing information is accurate, all lookups will complete in under log2(n) hops, with ½log2(n) hops on average. (n is the number of nodes in the network).
  • #10 Standard Chord ring architecture ROME runs on each machine Bootstrap server – needed for machines to find out addresses of nodes already in network New machine contacts BSS - Instead of being added to ring, message intercepted by ROME and machine added to pool of available machines
  • #11 If workload is between lower threshold and upper threshold, the node is experiencing normal workload No action needs to be taken (on the node)
  • #12 By monitoring at the node level, nodes only “report” if they are overloaded (like an error condition)
  • #13 Send message to bootstrap server stating capacity requirements – a machine with target capacity greater than current workload. Bootstrap server searches through node pool to find suitable node. Sends details of replacement node to ROME on current node.
  • #14 Current node sends its ID to ROME on the new node.
  • #15 This causes the nodes to be swapped and the old node is added to the bootstrap server’s node pool.
  • #17 Insert new node between the current node and its predecessor so it takes on some of the current node’s keyspace.
  • #18 Send a message to node informing it of its new ID
  • #19 The node is then inserted into the Chord ring and removed from the pool of available machines.
  • #21 Mean hop count for successful queries shows that in dynamic scenarios, the ½ log(n) calculation is still approximately correct. Out results are just a small fraction above the theoretical – e.g. 128 should have mean of 3½, our results indicate around 3¾.
  • #22 Reducing message cost is only one aspect: must ensure reducing the size of the Chord ring does not have adverse effect on ability to route a query in a dynamically changing network (e.g. nodes failing).
  • #23 We’ve shown that reducing the ring size does not have undesirable effects on hop count or query failures in dynamic environments. But what effects would ROME actually have on a ring? A simple theoretical example:
  • #24 Chord ring already contains all the nodes, so increasing the workload has no impact on the size of the Chord ring. However, ROME only adds nodes to the ring as they are required, so its ring size is kept small until we near the maximum workload.
  • #25 The ring size obviously impacts the mean hops per lookup – we see that ROME’s is less than Chord’s throughout, converging with Chord’s at the maximum workload.
  • #26 Side effect of using ROME – the capacity of the ring is more effectively utilised. In other words, more of the available capacity provided by nodes in the ring is actually used to service the workload in the network. Why not 100%? We set our upper threshold to be 95% - ROME will allow a node’s workload to grow to 95% before taking action