Your SlideShare is downloading. ×
An Adaptive Gossip-Based Dissemination Protocol for Multi-Source Message Streams
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

An Adaptive Gossip-Based Dissemination Protocol for Multi-Source Message Streams

1,055
views

Published on


0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,055
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. PULP An Adaptive Gossip-Based Dissemination Protocol forMulti-Source Message Streams Pascal FelberA.-M. Kermarrec, L. Leonini, E. Rivière, S. Voulgaris Pascal.Felber@unine.ch http://iiun.unine.ch/
  • 2. Introductionl  Epidemic protocols are widely used for information dissemination l  Algorithmic simplicity l  Robustness (failures of nodes and links) l  Adapted to large-scale dynamic networksl  Yet, there have a number of drawbacks l  Bandwidth overutilization (redundant messages) l  High message dissemination latencyl  Objectives of the PULP protocol l  Hybrid protocol bandwidth- & latency-efficient Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 2
  • 3. Epidemic Protocols: Push vs. Pulll  Push l  At 1st reception, every node forwards message to f other nodes, at most TTL times l  Low latency, high redundancyl  Pull l  Periodically, every node contacts another node and asks for missing messages l  High latency, low redundancyl  Both approaches rely on a sampling service to obtain random nodes Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 3
  • 4. Push-based Dissemination Coverage Complete disseminations 10000 nodes 100 100 80 80 60 60 40 40 20 20 20 20 0 15 0 15 20 15 10 20 15 10 10 5 5 10 5 5 TTL 0 0 FANOUT TTL 0 0 FANOUT 25 20 15 10 5 20 0 15 20 15 10 10 5 5 0 0 FANOUT TTL Avg. redundant pushes received, per peer Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 4
  • 5. Push vs. Pull: Latency & Redundancy Useful Useless Notified 10000 nodes 100 Push-based, FANOUT = 2 25000 80 20000 60 15000 40 10000 20 5000 0 0Notified nodes (percentage) Messages (useful/useless) 0 1 2 3 4 5 6 7 8 9 10 100 Push-based, FANOUT = 4 25000 80 20000 98.09% 60 15000 40 10000 20 5000 0 0 0 1 2 3 4 5 6 7 8 9 10 Pull-based 100 25000 80 20000 100% 60 15000 40 10000 20 5000 0 0 0 2 4 6 8 10 12 14 16 18 20 Cycles Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 5
  • 6. Context and Objectivesl  Dissemination of streams of small messages l  Sources may be any node l  Dissemination from all to alll  Variable publication frequency l  High frequency phases (e.g., react to event) l  Idle phases with no new messagel  Objectives l  Low network cost, proportional to actual activity l  Low latency l  Robustness to failures and churn Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 6
  • 7. The PULP Protocoll  Two-phase hybrid approach1.  Exponential growth phase (push) l  Inform sufficiently many nodes w/out redundancy2.  Quadratic shrinking phase (pull) l  Pull frequency driven by message activityl  Exploit sequences of messages l  Push messages carry information for pull phase l  Limits useless pullsl  Supports complete disseminations with low cost and low latency Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 7
  • 8. PULP: 1st Phasel  Objective: inform sufficiently many nodes with negligible redundancy l  4-5% of the network (based on observations) l  Size of network N estimated by sampling service TTL l  Choose TTL and f: c=N ∑ i=1 f i ≈ 4.5% l  Subsets of nodes reached by different messages are not correlated (random neighbor selection) l  Forwarded messages embed information about previously received messages (drive 2nd phase) Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 8
  • 9. PULP: 2nd Phasel  Objective: limit useless pulls l  The protocol uses information about missing messages (received during 1st phase) l  Pull frequency adapts according to: l  Missing messages l  Ratio of useful to useless pulls in last period l  Pull frequency increases when more messages are being disseminated l  When there is little activity, pull frequency depends on how useful previous pulls have been Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 9
  • 10. PULP: Data Structuresl  Every node maintains a sorted list of received messages l  Hp: Recent history (last messages) l  Tp ⊆ Hp: Trading window (available for others)l  Tp is embedded in messages sent by p to ql  If ∃ m ∈ Tp Hq then q can request m from p t Hp: recent history Tp: trading window old messages B3 A7 C3 B1 C6 B2 C5 A5 A6 C7 C8 B5 A9 C4 Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 10
  • 11. 3.4 Pulp: The Protocol random peers // Messages will be pulled at the next pulling periodWe now present a detailed description of the Pulp al- missing missing [ {m 2 TQ : m 2 HP } {msg} / PULP: Algorithmgorithm, which combines the push and pull componentsfor disseminating a sequence of messages in a collabo-rative and decentralized fashion. // Periodic pulling of missing elements thread PeriodicPull() do every pull seconds // Shu✏ing reduces the probability of receiving duplicates by pull Algorithm 1 shows the pseudo-code of the Pulp pro- shu✏e missing 7tocol. Each peer P maintains a history of the messages invoke Pull(missing, P, TP ) on a random node Qit has recently received, denoted as HP . It additionallymaintains a trading window, denoted nodePP containing Algorithm 1: Pulp algorithm on as T , // Invoked when a node Q requests a message from node P function Pull(requested, Q, TQ ) Variables 1st element in requested order 2 TP , or ? if nonethe list ofPmessages that are available to other nodes on H : History of (recently) received message IDs m invoke PullReply(m, P, TP ) on Qrequest. pull : Period of pull operations (initially 30s) missing: Set of message IDs known, but not yet received // Receive a reply to a pull request from node P When a message is Size of missing at the end of last node prevMissingSize: pushed to (or generated at) function PullReply(msg, Q, TQ )P for theadjust period P registers it in HP and, if the first time, if msg = ? _ m 2 HP then prevuseful : Number of useful pull replies during current prevuseless prevuseless + 1TTL has adjust period reached yet, forwards it to Fanout not been elserandom other period We stress that replies during current prevuseless : Number of useless pullobtaining the IP ad- peers. add msg to HP adjust missing missing [ {m 2 TQ : m 2 HP } {msg} /dress of randomly selectedare fixedis a trivial task thanks ( adjust , TTL and Fanout peers protocol parameters) prevuseful prevuseful + 1to Cyclon, as described inis pushed to node P by node Q // Invoked when a message Section 3.2. function Push(msg, hops, Q, TQ ) // Periodic adjustment of pulling period for node P // Forward further if needed thread AdaptFreq() if msg received for the first time then t do every adjust seconds add msg top: recent history H H P Tp: trading window if |missing| > prevMissingSize then if hops > 0 then adjust invoke 3 B1 C6 B hops-1, A P ) C8 B5 A9 old messages B3 A7 CPush(msg, 2 C5 A5 P,6TC7 on Fanout C4 pull |missing| prevMissingSize+prevuseful random peers else if |missing| > 0 ^ prevuseless  prevuseful then // Messages will be pulled at the next pulling period pull pull ⇥ 0.9 missing missing [ {m 2 TQ : m 2 HP } {msg} / elseFig. 3 Data structures of the Pulp algorithm. Note that mes- pull pull ⇥ 1.1sages// Periodic pulling of sources (here A, B, and C) and each come from multiple missing elements thread PeriodicPull() the order it received them (which isnode sorts them based on pull max( pull , pull min ) do every pull seconds pull min( pull , pull max )generally di↵erent for each node).the probability of receiving // Shu✏ing reduces prevuseless 0 duplicates by pull prevuseful 0 shu✏e missing prevMissingSize |missing| invoke Pull(missing, P, TP ) on a random node Q In forwarding a message to another peer Q, node P // Invoked when a node Q requests a message from node Palso function Pull(requested, Q, TQ ) Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber forwards the IDs of messages in its trading window 11
  • 12. Evaluationl  Conducted with SPLAYl  1000 nodes in a cluster l  Reproducing real churn (OverNet trace)l  300 nodes from PlanetLab l  Heavily loaded machinesl  Messages sent from random nodesl  Communication over UDP (i.e., unreliable)l  Metrics: number of receptions (coverage), latency, evolution of pull frequency Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 12
  • 13. Cluster 1000Number of receptions 800 600 •  1000 nodes •  200 messages (1 / 2s) 400 •  Bandwidth costs: ~30 UDP 200 messages / node / minute 0 0 100 200 300 400 500 100Pull replies (/peer, /sec.) Delays (seconds) Max 75th perc. 25th perc. 80 90th perc. 50th perc. 5th perc. 60 40 20 0 0 100 200 300 400 500 1.2 1 Useful Useless 0.8 0.6 0.4 0.2 0 0 100 200 300 400 500 Time (sec.) Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 13
  • 14. Coverage vs. Latency (1st Phase, TTL=2) Max 75th perc. 25th perc. 90th perc. 50th perc. 5th perc. 300(seconds) Delays 200 100 0duplicates (%) 4 3 Push 2 1 0 0. 1. (2 4. 6. 10 14 18 23 57 97 ) 11 94 ,4 .4 .8 .5 (4 (6 (8 (1 (1 (1 (1 ) ) ) 0) 2) 4) 6) Push coverage (%) (Fanout) Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 14
  • 15. Churn(900 Nodes + 100 Observers) Max. 75th perc. 25th perc. 90th perc. 50th perc. 5th perc. 60 No Churn 40 20 0 0 50 100 150 200 250 300 350 400 Update Time Distributions (seconds) 60 Churn x5 40 20 0 0 50 100 150 200 250 300 350 400 Churn x10 60 40 20 0 0 50 100 150 200 250 300 350 400 60 Churn x20 40 20 0 0 50 100 150 200 250 300 350 400 Time (seconds) Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 15
  • 16. PlanetLab 300 250Number of receptions •  300 nodes 200 •  200 messages (1 / 3.7s) 150 •  Bandwidth costs: ~20 UDP messages / node / minute 100 50 0Time of reception (seconds) 0 250 500 750 1000 1250 1500 1750 2000 100 90th perc. 75th perc. 25th perc. 50th perc. 5th perc. 80 60 40 20 0 0 100 200 300 400 500 600 700 Time of publication (seconds) Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 16
  • 17. Adaptation of Pull Frequency Messages 2 frequency 1.5 1 0.5 0 0 200 400 600 800 1000 1200 1400 90th perc. 50th perc. 5th perc. 75th perc. 25th perc. distribution (seconds) 30 Pulling period 20 10 0 0 200 400 600 800 1000 1200 1400 (/peer, /second) 1.5 Useful Useless Pull replies 1 0.5 0 0 200 400 600 800 1000 1200 1400 Time (seconds) Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 17
  • 18. Reaction to Message Burst Messages sent 1000 100 10 1 0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 90th perc. 50th perc. 5th perc. 75th perc. 25th perc.distribution (seconds) 30 Pulling period 20 10 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200(/peer,/second) 2 Pull replies 1.5 Useful Useless 1 0.5 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 Time (seconds) Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 18
  • 19. Only−push (Fanout=5, TTL=4): Hit and Duplicate ratios 100 Comparison with Pull-/Push-only 80 PULP (Fanout=3, TTL=3): Reception Delays Distribution Ratio 60 60 th th 40 Max 75th perc. 25th perc. 50 20 90th perc. 50 perc. 5 perc. Seconds Hit Ratio 40 Dup Ratio 0 30 0 100 200 300 20 400 500 600 10 Message sending rate 0 0 100 200 300 400 500 600 1msg /2s Only−pull: Reception Delays Distribution 80 70 60 Seconds 1msg 50 /20s 40 0 100 200 300 30 400 500 600 20 10 0 0 100 200 300 400 500 600 PULP (Fanout=3, TTL=3): Reception Delays Distribution Only−push (Fanout=5, TTL=4): Reception Delays Distribution 60 200 50 th Max 75th perc. th 25th perc. th 175 90 perc. 50 perc. 5 perc. 150Seconds Seconds 40 125 30 100 20 75 10 50 25 0 0 0 100 200 300 400 500 600 0 100 200 300 400 500 600 Only−pull: Reception Delays Distribution Only−push (Fanout=5, TTL=4): Hit and Duplicate ratios 80 70 100 60 80Seconds 50 Ratio 40 60 30 40 20 10 20 Hit Ratio Dup Ratio 0 0 0 100 200 300 400 500 600 0 100 200 300 400 500 600 Only−push (Fanout=5, TTL=4): Reception Delays Distribution Message sending rate 200 Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 19 175 1msg 150 /2s
  • 20. Conclusionl  PULP is a lightweight protocol that combines push and pull disseminationl  Handles streams of message from multiple sourcesl  Negligible amounts of redundant messagesl  Low dissemination latency thanks to adaptive pull frequencyl  Adapted to the conditions of real networksl  Efficient, robust, churn-tolerant Pulp: An Adaptive Gossip-Based Dissemination Protocol — P. Felber 20