CoolStreaming

738 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

CoolStreaming

  1. 1. Presented by:Michael Iline & Ariel KrinitsaAdvanced Topics in Communication 2/05/2011
  2. 2.  Introduction Motivation Related Work  Tree-based Protocols and Extensions  Gossip-based Protocols Design and Optimization of DONet  Node Join and Membership Management  Buffer Map Representation and Exchange  Scheduling Algorithm  Failure Recovery and Partnership Refinement Analysis of Overlay Radius Experimental Results
  3. 3.  Many multimedia applications, such as NetTV and news broadcast, involve live media streaming from a source to a large population of users. IP Multicast is probably the most efficient. But it remains confined due to the lack of incentives to install multicast capable routers and to carry multicast traffic.
  4. 4.  Application-level solution is called overlay nodes, and multicast is then achieved through data relaying among these nodes. Construction algorithms have tree structure for data delivering.
  5. 5.  This works well with dedicated infrastructure routers as in IP multicast, it often mismatches an application-level overlay with dynamic nodes.  As the autonomous overlay nodes can easily crash or leave.  A tree is highly vulnerable  Have high bandwidth and stringent continuity demands. Sophisticated structures like mesh and forest can partially solve the problem, but they are much more complex and often less scalable.
  6. 6.  A data-centric design of a streaming overlay:  availability of data guides the flow directions, not a specific overlay structure  it is more suitable for overlay with high dynamic nodes (semistatic structure)  All the nodes have strong buffering capabilities and can adaptively and intelligently determine the data forwarding directions The core operations in DONet are very simple:  easy to implement  efficient  robust and resilient
  7. 7.  The key design issues of DONet:  How the partnerships are formed  How the data availability information are encoded and exchanged  How the video data are supplied and retrieved among partners
  8. 8. A brief detour to explain base concepts and protocols CoolStreaming/DONet A Data-Driven Overlay Network for Efficient Live Media Streaming
  9. 9.  Numerous overlay multicast systems can be classified into two categories: proxy-assisted and peer-to-peer based.  Proxy-assisted  Servers or application-level proxies are strategically placed.  Peer-to-peer based  Self-organized overlay networks.  Based multimedia distribution service. DONet belongs to peer-to-peer based category.
  10. 10.  Constructing and maintaining an efficient distribution tree among the overlay nodes is a key issue to these systems. An internal node in a tree has a higher load and its leave or crash often causes buffer underflow in a large population of descendants.
  11. 11.  DONet has a simpler and straight data-driven design, which does not maintain more complex structure, nor relies on an advanced coding scheme.
  12. 12.  Implementing a gossiping protocol in DONet for membership management. What is Membership?  “Who knows whom” relation ▪ A knows C, C knows F ▪ But D does not know C, J does not know B C F I A D H B J G E CoolStreaming/DONet A Data-Driven Overlay Network for Efficient Live Media Streaming
  13. 13. a node sends a newly generated message to a set of randomly selectednodes; these nodes do similarly in the next round.
  14. 14.  Gossip-based dissemination protocols  Each member forwards the message to randomly chosen group members  Probabilistic guarantee [Reliability] ( guarantee message delivery with high probability)  Scalable  Resilient to node/link failures However, traditional gossip-based multicast protocols rely on non-scalable membership protocol  Each node has the complete view of the system  High overhead in storage and synchronization
  15. 15.  Aiming at the weakness of traditional full membership protocols, SCAMP proposes a scalable probabilistic membership protocol for gossip-based multicast Scalable, fully decentralized  Each node maintains partial, yet sufficient system view Self-reconfigurable  View size in each member can change when system size changes  Any isolated node can rejoin the system automatically with isolation recovery mechanism
  16. 16.  Basic membership management  Subscription  Un-subscription  Isolation Recovery ▪ Simply solved by using heart beating and re-subscribing Graph rebalancing mechanisms  Indirection  Lease Mechanism
  17. 17.  Each node k maintains two lists of group members  PartialView : a list containing its gossip targets  InView : a list of nodes which k is one of their gossip targets
  18. 18.  Subscription (new node join)  Contact: New nodes join the group by sending a subscription request to an arbitrary member. B request D N A C 19
  19. 19.  Subscription (new node join)  New subscription: When a node receives a new subscription request, it forwards the new node-id to all members of its own local view. It also creates c additional copies (to be discussed later). B request request D N A request request C 20
  20. 20.  Subscription (new node join)  Forwarded subscription: When a node receives a forwarded subscription ▪ With probability p = 1/(1+size of PartialViewk), add the subscriber into its PartialView ▪ Otherwise, forward the subscription to a randomly chosen node from k’s PartialView accept E B forward D N A F forward C forward G 21
  21. 21.  Subscription (new node join)  Keeping a subscription: When a node decides to keep the subscription, it integrates the new subscriber in its PartialView, and informs the subscriber to update its InView E B forward accept D N A F forward C forward G 22
  22. 22.
  23. 23.  Unsubscription (node leaves)  Views: ▪ Let PartialViewn = { i(1), i(2), … , i(L) } ▪ Let InViewn = { j(1), j(2), … ,j(L’) }  Informs nodes j…(1)~j(L’-c-1) to replace its id with i(1)~i(L’-c-1) (mod L), respectively  Informs nodes j(L’-c)~j(L’) to remove it from their lists. replace A C x x N x x remove B D 24
  24. 24.  If the leaving node has a in-degree d, the total number of arcs decreases by d+c+1  d-c-1 by replacing  (c+1)*2 by removing E[Mn-1]≈(c+1)(n-1)log(n-1) UnSubscriptions preserve the desired mean degree of arcs 25
  25. 25.  Basic membership management  Subscription  Un-subscription  Recovery Isolation ▪ Simply solved by using heart beating and re-subscribing Graph rebalancing mechanisms  Indirection  Lease Mechanism
  26. 26.  Indirection mechanisms  How would a newly joint node select a node to contact? Choosing at random uniformly among existing members requires global information.  Solution: the initial contact forwards the newcomer’s subscription request to a node which is chosen approximately at random among all existing nodes. ▪ Balance the lists by Indirection mechanisms ▪ The node which receives the subscription request forwards the “token” with a counter value ▪ Decrement the counter every hop forwarded ▪ The member where token with zero counter arrives acts as a contact node 27
  27. 27.  Lease mechanisms  Each subscription has a finite lifetime  Each node is responsible to resubscribe to a random chosen member from its PartialView ▪ Subscriber’s PartialView remains the same ▪ However, each node’s PartialView gradually changes (even there is no change to the system) Advantages  Helps to rebalance the size of partial views across group members  Removes invalid information caused by leaving the group without unsubscribing 28
  28. 28.  Distribution of partial view size 29
  29. 29.  Resilience to node failures (Full view VS Partial view) 30
  30. 30.  Pros  Fully decentralized protocol with O(logn) partial view size  With a very close performance to full membership protocol Cons  The reason why indirection does not improve the performance is not solved completely ? 34
  31. 31.  SCAMP  Membership management system for gossip-based multicast  Partial View (O(log n)) per member in average ▪ Scalable ▪ No global system size needed ▪ Self-reconfigurating ▪ Used with O(log n) gossip-based multicast  Achieve load balancing by using several techniques ▪ Indirection ▪ Distribution of contact work ▪ Lease mechanism ▪ Good to often change the view?
  32. 32. So, how does it work? SplitStream: High-bandwidth multicast in a cooperative environment
  33. 33.  There are three key modules:  Membership manager  Partnership manager  Scheduler The system diagram of a DONet node
  34. 34.  DONet node can be either a receiver or a supplier, or both. Nodes periodically exchange data availability information with a set of partners. An exception is the source node - origin node, which is always a supplier.
  35. 35.  Each DONet node has a unique identifier, such as IP address and a membership cache (mCache)containing a partial list of the identifiers. Newly node first contacts the origin node, which randomly selects a deputy node from its mCache and redirects the new node to the deputy. Use SCAM (Scalable Gossiping Membership Protocol) to distribute membership messages among nodes.
  36. 36.  Two events trigger updates of an mCache entry: 1.the membership message is to be forwarded to other nodes through gossiping 2. the node serves as a deputy and the entry is to be included in the partner candidate list.
  37. 37. Illustration of the partnership in DONet (origin node: A)
  38. 38.  A sliding window of 120-segment can effectively represent the buffer map of node. Using 120 bits to record a BM, with bit 1 indicating that a segment is available and 0 otherwise.
  39. 39. Problem: From which partner to fetch which data segment ? Constraints  Data availability  Playback deadline  Heterogeneous partner bandwidth This problem is a variation of the Parallel machine scheduling  NP-hard problem  The situation will become worse in a highly dynamic environment  Resort a simple heuristic of fast response time
  40. 40.
  41. 41.  For each expected Set i  Check availability at partners  If i exists Supplier  update the deadline [Prtnerj,Seti] Setk Setk+1 Setk+2 Setk+3 … Setk Setk+1 Setk+2 Setk+3 … Prtnra 11sec 13sec 17sec 14sec Prtnrb 7sec 7sec Prtnrc 19sec Prtnrd 8sec 15sec Prtnre 32sec Prtnrf 11sec Prtnrg 8sec
  42. 42.  Supplier Setk Prtnra Setk+1 Setk+2 Setk+3 … Setk Setk+1 Setk+2 Setk+3 … … Setk Setk+1 Setk+2 Setk+3 …Prtnra 11sec 13sec 17sec 14sec Prtnra 11sec 10sec 14sec 11secPrtnrb 7sec 7sec Prtnrb 7sec 7secPrtnrc 19sec Prtnrc 19secPrtnrd 8sec 15sec Prtnrd 8sec 15secPrtnre 32sec Prtnre 32secPrtnrf 11sec Prtnrf 11secPrtnrg 8sec Prtnrg 8sec
  43. 43.  Supplier Setk Prtnra Setk+1 Prtnrd Setk+2 Setk+3 … Setk Setk+1 Setk+2 Setk+3 … … Setk Setk+1 Setk+2 Setk+3 …Prtnra 11sec 10sec 14sec 11sec Prtnra 11sec 10sec 14sec 11secPrtnrb 7sec 7sec Prtnrb 7sec 7secPrtnrc 19sec Prtnrc 19secPrtnrd 8sec 15sec Prtnrd 8sec 10secPrtnre 32sec Prtnre 32secPrtnrf 11sec Prtnrf 11secPrtnrg 8sec Prtnrg 8sec
  44. 44.  Supplier Setk Prtnra Setk+1 Prtnrd Setk+2 Prtnra Setk+3 … Setk Setk+1 Setk+2 Setk+3 … … Setk Setk+1 Setk+2 Setk+3 …Prtnra 11sec 10sec 14sec 11sec Prtnra 11sec 10sec 14sec 5secPrtnrb 7sec 7sec Prtnrb 7sec 7secPrtnrc 19sec Prtnrc 19secPrtnrd 8sec 10sec Prtnrd 8sec 10secPrtnre 32sec Prtnre 32secPrtnrf 11sec Prtnrf 11secPrtnrg 8sec Prtnrg 8sec
  45. 45.  Supplier Setk Prtnra Setk+1 Prtnrd Setk+2 Prtnra Setk+3 Prtnrf … Setk Setk+1 Setk+2 Setk+3 … … Setk Setk+1 Setk+2 Setk+3 …Prtnra 11sec 10sec 14sec 5sec Prtnra 11sec 10sec 14sec 5secPrtnrb 7sec 7sec Prtnrb 7sec 7secPrtnrc 19sec Prtnrc 19secPrtnrd 8sec 10sec Prtnrd 8sec 10secPrtnre 32sec Prtnre 32secPrtnrf 11sec Prtnrf 11secPrtnrg 8sec Prtnrg 8sec
  46. 46.  The departure can be easily detected after an idle time of TFRC or BM exchange. An affected node can quickly react through re- scheduling using the BM information of the remaining partners. Operations to further enhance resilience:  Graceful departure: the departure message when departing  Node failure: the departure message on behalf the failed node.
  47. 47.  The departure message is gossiped similarly to the membership message. Each node periodically establish new partnerships with nodes randomly selected from its mCache. The new partner with the lowest score can be rejected to keep a stable number of partners. using function
  48. 48.  Coverage ratio for distance k  (# of neighbors: M, total nodes: N) M ( M 1) k 2 ( M 2) N 1 e  E.g. 95% nodes are covered in 6 hops when M=4, N=500 Average distance O(logN)
  49. 49.  DONet vs Tree-based overlay  Much lower outage probability
  50. 50.  PlanetLab  An open platform for developing, deploying, and accessing planetary-scale services Involved 200~300 nodes during experiment period (May to June, 2004) Streaming rate: 500 Kbps *http://www.planet-lab.org/
  51. 51.  Continuity index: number of segments that arrive before or on playback deadlines over the total number segments  Data continuity, 200 nodes, 500 kbps streaming
  52. 52.  A practical DONet implementation First version release: May, 2004 Support Real Video and Windows Media format Broadcast live sport programs at 450~755 Kbps Attached 30000 users
  53. 53.  Heterogeneous network environment  LAN, CABLE, DSL, …
  54. 54.  Present the design of DONet for live media streaming  Data-driven design  Scalable membership and partnership management algorithm  Heuristic scheduling algorithm The experiment results on PlantLab demonstrate DONet delivers quite good playback quality in a highly dynamic networks A practical implementation was also released for broadcasting live programs

×