presentation
Upcoming SlideShare
Loading in...5
×
 

presentation

on

  • 406 views

 

Statistics

Views

Total Views
406
Views on SlideShare
406
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • In all the prior work that we came across, they did not control one or more of the following conditions: . . . Because of this they failed to propose a platform for fair head-to-head comparison of different systems.
  • a) Numbers are all for uplink b) how much overprovisioning? C) no downlink constraint
  • Some frame-freeze events are more than 100 frames long. Some freeze events are couple of frames long . . . Hence we look at the cdf of the length of the frame-freeze events.
  • System A is tree-based and hence pushes data along established paths. Hence less number of duplicates. System B is mesh-based. A peer advertises the chunks that is has and then complies with requests to relay those chunks to other peers. In general, it is hard to co-ordinate to eliminate the duplicates. A lot more diversity among the parent peers for System B.
  • The nature of our test-bed allows us to include more video quality metrics in our study.
  • Before concluding, I would like to say that apart from the results in this paper, we also reported some results from the same test-runs in another paper at TRIDENTCOM. That paper nicely complements this paper and I urge you to take a look! Thank you!

presentation presentation Presentation Transcript

  • Video Quality Assessment and Comparative Evaluation of Peer-to-Peer Video Streaming Systems Aditya Mavlankar Sachin Agarwal Pierpaolo Baccichet Jatinder Pal Singh Bernd Girod Deutsche Telekom A.G., Stanford University Laboratories Stanford CA, USA Berlin, Germany
  • Outline  Introduction to P2P live video streaming  Prior work on system performance assessment  Test-bed setup  Performance of tested systems Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 2
  • P2P Live Video Streaming  Extension of P2P file-sharing  Low-cost and scalable delivery mechanism  Several deployed commercial implementations today  Increasing content / channels available Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 3
  • Related Work on Performance Assessment  Networking related metrics, e.g. bandwidth usage, packet loss, continuity index, etc. – CoolStreaming [Zhang et al., 2005]: PlanetLab – PPLive [Hei et al., 2006]: packet sniffing and crawling – SopCast [Sentinelli et al., 2007]: “watching”, PlanetLab – ...  No video PSNR results  No repeatable test conditions – Network conditions – Encoded video characteristics – Peer behavior  No fair head-to-head comparison of different systems Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 4
  • Test-Bed Setup 576 X 9 1024 X 5 Test center 2048 X 1 Berlin, Germany (15) Berlin, Germany [Emulated HS Broadband] Server 1, 2 PLR, delay, jitter and bandwidth measured for 576 X 16 representative real connections and emulated using 1024 X 5 2048 X 1 NISTNet Stanford, shaper traffic CA (22) Internet [Emulated HS Broadband] ISP Datacenter Erfurt, Germany 128 X 2 Berlin, Germany (8) 192 X 2 TU Munich, Germany (3) 576 X 2 [Emulated HS Broadband] [Real HS Broadband] 1024 X 2 3072 X 3 Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 5
  • Encoded Video Stream  La Dolce Vita (Fellini, 1960)  24 fps, 352x240 pixels  H.264/AVC video codec, 400 kbit/sec CBR bitstream, 42 dB PSNR  I B B P B B P B B P . . . (I frame every second)  H.264 bitstream wrapped in Microsoft ASF container, if required by tested system  Last frame error concealment Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 6
  • Peer Churn Model  30-minute simulation run  During each 6-minute time-slot – Peer on with probability 0.9 – Peer off with probability 0.1 – Peer can switch off for the rest of the run with probability 0.05  During last 5 minutes, peer off with probability 0.5 Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 7
  • Representative Results  Tested systems – System A: Tree-based, push approach – System B: Mesh-based, data-driven or pull approach  Emulation runs – Run 1: with traffic shaping (using NISTNet) – Run 2: without traffic shaping  Same realization of peer On-Off model for all runs Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 8
  • Pre-Roll Delay 45 40 Avg. PSNR across all clients [dB] 35 30 about 30 sec about 60 sec enough for enough for 25 System A System B (tree-based) (mesh-based) 20 Tree, w/ traffic shaping 15 Tree, w/o traffic shaping Mesh, w/ traffic shaping Mesh, w/o traffic shaping 10 0 10 20 30 40 50 60 Pre-roll delay [sec] Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 9
  • PSNR Drop (w/ traffic shaping) System A (tree-based) System B (mesh-based) 6 32 6 Avg. drop in PSNR [dB] Avg. drop in PSNR [dB] 5 5 4 4 3 3 2 2 1 1 0 0 0 20 40 0 20 40 Client ID Client ID Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 10
  • PSNR Drop (w/o traffic shaping) System A (tree-based) System B (mesh-based) 6 32 6 Avg. drop in PSNR [dB] Avg. drop in PSNR [dB] 5 5 4 4 3 3 2 2 1 1 0 0 0 20 40 0 20 40 Client ID Client ID Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 11
  • Statistics of Frame Freezes  Frames frozen (as percentage of total frames to be displayed) System A (tree-based) System B (mesh-based) Run 1 4.2% 2.0% Run 2 3.0% 0.2%  Average no. of distinct frame-freeze events per client in 30 min. System A (tree-based) System B (mesh-based) Run 1 64 23 Run 2 40 2 Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 12
  • Statistics of Frame Freezes (cont.) System A (tree- 1 based) employs 0.9 content-aware prioritization 0.8 Long frame freezes more likely with 0.7 System B (mesh- 0.6 based) cdf 0.5 0.4 0.3 System A, Run 1 System A, Run 2 0.2 System B, Run 1 System B, Run 2 0.1 0 20 40 60 80 100 120 Length of frame freeze [number of frames] Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 13
  • No. of Peers Failing to Decode a Frame System A (tree-based), Run 1 System B (mesh-based), Run 1 No. of peers failing to decode the frame No. of peers failing to decode the frame 40 40 30 30 20 20 10 10 0 0 0 10 20 30 0 10 20 30 Time [minutes] Time [minutes] Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 14
  • Redundancy, Server Load and Parent-Peer Analysis  Redundancy (bytes received in excess of required video stream bytes) – System A (tree-based): 6% in both runs – System B (mesh-based): 35% and 20% in Runs 1 (w/ traffic shaping) and 2 (w/o traffic shaping) respectively  For both Systems, peer receives on average less than 10% of its data directly from the server; slightly more for Run 2 of System B  System A (tree-based): Sustained downloads from lower number of parent peers Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 15
  • Summary  Proposed methodology allows measuring video PSNR, buffering time, frame-freeze statistics, peers failing to decode a frame, etc. beyond network usage, packet loss, etc.  Test conditions chosen by analyzing real-world conditions and experiments are repeatable  Tested three commercial-grade P2P video streaming systems  Room for improvement in current systems: – Long buffering time (10s of seconds) – Display freezes for more than 100 frames  Tested tree-based system outperforms mesh-based system: – Redundancy – Buffering time Mavlankar et al.: Comparative Evaluation of Peer-to-Peer Video Streaming Systems Jun. 25, 2008 16
  • Thank you! http://www.stanford.edu/~maditya/publication.html Related: [Agarwal, et al., TRIDENTCOM 2008]