Video Streaming Ali Saman Tosun Computer Science Department
Broadcast to True Media-on-Demand <ul><li>Broadcast (No-VoD) </li></ul><ul><ul><li>Traditional, no control </li></ul></ul>...
Streaming Stored Video   <ul><li>Streaming </li></ul><ul><li>media stored at source </li></ul><ul><li>transmitted to clien...
Streaming Video <ul><li>Client-side buffering, playout delay compensate for network-added delay, delay jitter </li></ul>co...
Smoothing Stored Video <ul><li>For prerecorded video streams: </li></ul><ul><li>All video frames stored in advance at serv...
Smoothing Constraints <ul><li>Given frame sizes { f i }  and buffer size  b </li></ul><ul><ul><li>Buffer underflow constra...
Proxy-based Video Distribution Server Proxy Client  Client Proxy adapts video Proxy caches video
Proxy Operations <ul><li>Drop frames </li></ul><ul><ul><li>Drop B,P frames if not enough bandwidth </li></ul></ul><ul><li>...
Online Smoothing <ul><li>Larger window  w  reduces burstiness, but… </li></ul><ul><ul><li>Larger buffer at the source/prox...
Online Smoothing Model <ul><li>Arrival of  A i   bits to proxy by time i in frames </li></ul><ul><li>Smoothing buffer of  ...
Online  Smoothing <ul><li>Must send enough to avoid underflow at client </li></ul><ul><ul><li>S i  must be at least  D i-w...
Online Smoothing Constraints Source/proxy has w frames ahead of current time t: L U time (in frames) number of bytes t t+w...
Smoothing  Star Wars GOP averages 2-second window 30-second window <ul><li>MPEG-1 Star Wars,12-frame group-of-pictures  </...
Prefix Caching to Avoid Start-Up Delay <ul><li>Avoid start-up delay for prerecorded streams </li></ul><ul><ul><li>Proxy ca...
Changes to Smoothing Model <ul><li>Separate parameter  s  for client start-up delay </li></ul><ul><li>Prefix cache stores ...
Scalable coding <ul><li>Typically used as Layered coding </li></ul><ul><li>A base layer </li></ul><ul><ul><li>Provides bas...
Temporal Scalability <ul><li>Frames can be dropped </li></ul><ul><ul><li>In a controlled manner </li></ul></ul><ul><ul><li...
Spatial Scalability <ul><ul><li>Base layer </li></ul></ul><ul><ul><ul><li>Downsample the original image  </li></ul></ul></...
SNR Scalability <ul><li>SNR – signal-to-noise ratio </li></ul><ul><li>Idea </li></ul><ul><ul><li>Base layer </li></ul></ul...
Multiple Description Coding <ul><li>Idea </li></ul><ul><ul><li>Encode data in two streams </li></ul></ul><ul><ul><li>Each ...
Delivery Systems Developments Network Saving network resources: Stream scheduling Several Programs or Timelines
Patching <ul><li>Server resource optimization is possible </li></ul>multicast Unicast patch stream Central server 1st clie...
Proxy Prefix Caching <ul><li>Split movie </li></ul><ul><ul><li>Prefix </li></ul></ul><ul><ul><li>Suffix </li></ul></ul><ul...
Interval Caching (IC) <ul><ul><li>caches data between requests </li></ul></ul><ul><ul><li>following  requests are thus ser...
Receiver-driven Layered Multicast (RLM) <ul><li>Requires </li></ul><ul><ul><li>IP multicast </li></ul></ul><ul><ul><li>lay...
Receiver-driven Layered Multicast (RLM)
Upcoming SlideShare
Loading in...5
×

Video Streaming Ali Saman Tosun

752

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
752
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Video Streaming Ali Saman Tosun

  1. 1. Video Streaming Ali Saman Tosun Computer Science Department
  2. 2. Broadcast to True Media-on-Demand <ul><li>Broadcast (No-VoD) </li></ul><ul><ul><li>Traditional, no control </li></ul></ul><ul><li>Pay-per-view (PPV) </li></ul><ul><ul><li>Paid specialized service </li></ul></ul><ul><li>Near Video On Demand (N-VoD) </li></ul><ul><ul><li>Same media distributed in regular time intervals </li></ul></ul><ul><ul><li>Simulated forward / backward </li></ul></ul><ul><li>True Video On Demand (T-VoD) </li></ul><ul><ul><li>Full control for the presentation, VCR capabilities </li></ul></ul><ul><ul><li>Bi-directional connection </li></ul></ul>
  3. 3. Streaming Stored Video <ul><li>Streaming </li></ul><ul><li>media stored at source </li></ul><ul><li>transmitted to client </li></ul><ul><li>streaming: client playout begins before all data has arrived </li></ul><ul><li>timing constraint for still-to-be transmitted data: in time for playout </li></ul>
  4. 4. Streaming Video <ul><li>Client-side buffering, playout delay compensate for network-added delay, delay jitter </li></ul>constant bit rate video transmission Cumulative data time variable network delay client video reception constant bit rate video playout at client client playout delay buffered video
  5. 5. Smoothing Stored Video <ul><li>For prerecorded video streams: </li></ul><ul><li>All video frames stored in advance at server </li></ul><ul><li>Prior knowledge of all frame sizes ( f i , i=1,2,..,n) </li></ul><ul><li>Prior knowledge of client buffer size ( b ) </li></ul><ul><li>workahead transmission into client buffer </li></ul>1 2 n b bytes Server Client
  6. 6. Smoothing Constraints <ul><li>Given frame sizes { f i } and buffer size b </li></ul><ul><ul><li>Buffer underflow constraint (L k = f 1 + f 2 + … + f k ) </li></ul></ul><ul><ul><li>Buffer overflow constraint (U k = min(L k + b, L n )) </li></ul></ul><ul><ul><li>Find a schedule S k between the constraints </li></ul></ul><ul><li>Algorithm minimizes peak and variability </li></ul>L U S time (in frames) number of bytes rate changes
  7. 7. Proxy-based Video Distribution Server Proxy Client Client Proxy adapts video Proxy caches video
  8. 8. Proxy Operations <ul><li>Drop frames </li></ul><ul><ul><li>Drop B,P frames if not enough bandwidth </li></ul></ul><ul><li>Quality Adaptation </li></ul><ul><ul><li>Transcoding </li></ul></ul><ul><ul><ul><li>Change quantization value </li></ul></ul></ul><ul><ul><ul><li>Most of current systems don’t support </li></ul></ul></ul><ul><li>Video staging, caching, patching </li></ul><ul><ul><li>Staging : store partial frames in proxy </li></ul></ul><ul><ul><li>Prefix caching : store first few minutes of movie </li></ul></ul><ul><ul><li>Patching : multiple users use same video </li></ul></ul>
  9. 9. Online Smoothing <ul><li>Larger window w reduces burstiness, but… </li></ul><ul><ul><li>Larger buffer at the source/proxy </li></ul></ul><ul><ul><li>Larger processing load to compute schedule </li></ul></ul><ul><ul><li>Larger playback delay at the client </li></ul></ul>b bytes Source/Proxy Client streaming video Source or proxy can delay the stream by w time units: stream with delay w
  10. 10. Online Smoothing Model <ul><li>Arrival of A i bits to proxy by time i in frames </li></ul><ul><li>Smoothing buffer of B bits at proxy </li></ul><ul><li>Smoothing window (playout delay) of w frames </li></ul><ul><li>Playout of D i-w bits by client by time i </li></ul><ul><li>Playout buffer of b bits at client </li></ul><ul><li>Transmission of S i bits by proxy by time i </li></ul>B b D i-w A i S i proxy client
  11. 11. Online Smoothing <ul><li>Must send enough to avoid underflow at client </li></ul><ul><ul><li>S i must be at least D i-w </li></ul></ul><ul><li>Cannot send more than the client can store </li></ul><ul><ul><li>S i must be at most D i-w + b </li></ul></ul><ul><li>Cannot send more than the data that has arrived </li></ul><ul><ul><li>S i must be at most A i </li></ul></ul><ul><li>Must send enough to avoid overflow at proxy </li></ul><ul><ul><li>S i must be at least A i - B </li></ul></ul>max{D i-w , A i - B}  S i <= min{D i-w + b, A i }
  12. 12. Online Smoothing Constraints Source/proxy has w frames ahead of current time t: L U time (in frames) number of bytes t t+w-1 don’t know the future Modified smoothing constraints as more frames arrive... ?
  13. 13. Smoothing Star Wars GOP averages 2-second window 30-second window <ul><li>MPEG-1 Star Wars,12-frame group-of-pictures </li></ul><ul><li>Max frame 23160 bytes, mean frame 1950 bytes </li></ul><ul><li>Client buffer b=512 kbytes </li></ul>
  14. 14. Prefix Caching to Avoid Start-Up Delay <ul><li>Avoid start-up delay for prerecorded streams </li></ul><ul><ul><li>Proxy caches initial part of popular video streams </li></ul></ul><ul><ul><li>Proxy starts satisfying client request more quickly </li></ul></ul><ul><ul><li>Proxy requests remainder of the stream from server </li></ul></ul><ul><li>smooth over large window without large delay </li></ul><ul><li>Use prefix caching to hide other Internet delays </li></ul><ul><ul><li>TCP connection from browser to server </li></ul></ul><ul><ul><li>TCP connection from player to server </li></ul></ul><ul><ul><li>Dejitter buffer at the client to tolerate jitter </li></ul></ul><ul><ul><li>Retransmission of lost packets </li></ul></ul><ul><li>apply to “point-and-click” Web video streams </li></ul>
  15. 15. Changes to Smoothing Model <ul><li>Separate parameter s for client start-up delay </li></ul><ul><li>Prefix cache stores the first w-s frames </li></ul><ul><li>Arrival vector A i includes cached frames </li></ul><ul><li>Prefix buffer does not empty after transmission </li></ul><ul><li>Send entire prefix before overflow of b s </li></ul><ul><li>Frame sizes may be known in advance (cached) </li></ul>b s b p b c A i D i-s S i
  16. 16. Scalable coding <ul><li>Typically used as Layered coding </li></ul><ul><li>A base layer </li></ul><ul><ul><li>Provides basic quality </li></ul></ul><ul><ul><li>Must always be transferred </li></ul></ul><ul><li>One or more enhancement layers </li></ul><ul><ul><li>Improve quality </li></ul></ul><ul><ul><li>Transferred if possible </li></ul></ul>Sending rate Quality Best possible quality at possible sending rate Base layer Enhancement layer
  17. 17. Temporal Scalability <ul><li>Frames can be dropped </li></ul><ul><ul><li>In a controlled manner </li></ul></ul><ul><ul><li>Frame dropping does not violate dependancies </li></ul></ul><ul><ul><li>Low gain example: B-frame dropping in MPEG-1 </li></ul></ul>
  18. 18. Spatial Scalability <ul><ul><li>Base layer </li></ul></ul><ul><ul><ul><li>Downsample the original image </li></ul></ul></ul><ul><ul><ul><li>Send like a lower resolution version </li></ul></ul></ul><ul><ul><li>Enhancement layer </li></ul></ul><ul><ul><ul><li>Subtract base layer pixels from all pixels </li></ul></ul></ul><ul><ul><ul><li>Send like a normal resolution version </li></ul></ul></ul><ul><ul><li>If enhancement layer arrives at client </li></ul></ul><ul><ul><ul><li>Decode both layers </li></ul></ul></ul><ul><ul><ul><li>Add layers </li></ul></ul></ul>Base layer Enhancement layer Better compression due to low values Less data to code 72 75 83 61 73 -1 -12 2 10
  19. 19. SNR Scalability <ul><li>SNR – signal-to-noise ratio </li></ul><ul><li>Idea </li></ul><ul><ul><li>Base layer </li></ul></ul><ul><ul><ul><li>Is regularly DCT encoded </li></ul></ul></ul><ul><ul><ul><li>A lot of data is removed using quantization </li></ul></ul></ul><ul><ul><li>Enhancement layer is regularly DCT encoded </li></ul></ul><ul><ul><ul><li>Run Inverse DCT on quantized base layer </li></ul></ul></ul><ul><ul><ul><li>Subtract from original </li></ul></ul></ul><ul><ul><ul><li>DCT encode the result </li></ul></ul></ul><ul><ul><li>If enhancement layer arrives at client </li></ul></ul><ul><ul><ul><li>Add base and enhancement layer before running Inverse DCT </li></ul></ul></ul>
  20. 20. Multiple Description Coding <ul><li>Idea </li></ul><ul><ul><li>Encode data in two streams </li></ul></ul><ul><ul><li>Each stream has acceptable quality </li></ul></ul><ul><ul><li>Both streams combined have good quality </li></ul></ul><ul><ul><li>The redundancy between both streams is low </li></ul></ul><ul><li>Problem </li></ul><ul><ul><li>The same relevant information must exist in both streams </li></ul></ul><ul><ul><li>Old problem: started for audio coding in telephony </li></ul></ul><ul><ul><li>Currently a hot topic </li></ul></ul>
  21. 21. Delivery Systems Developments Network Saving network resources: Stream scheduling Several Programs or Timelines
  22. 22. Patching <ul><li>Server resource optimization is possible </li></ul>multicast Unicast patch stream Central server 1st client 2nd client Join ! cyclic buffer
  23. 23. Proxy Prefix Caching <ul><li>Split movie </li></ul><ul><ul><li>Prefix </li></ul></ul><ul><ul><li>Suffix </li></ul></ul><ul><li>Operation </li></ul><ul><ul><li>Store prefix in prefix cache </li></ul></ul><ul><ul><ul><li>Coordination necessary! </li></ul></ul></ul><ul><ul><li>On demand </li></ul></ul><ul><ul><ul><li>Deliver prefix immediately </li></ul></ul></ul><ul><ul><ul><li>Prefetch suffix from central server </li></ul></ul></ul><ul><li>Goal </li></ul><ul><ul><li>Reduce startup latency </li></ul></ul><ul><ul><li>Hide bandwidth limitations, delay and/or jitter in backbone </li></ul></ul><ul><ul><li>Reduce load in backbone </li></ul></ul>Client Unicast Unicast Central server Prefix cache
  24. 24. Interval Caching (IC) <ul><ul><li>caches data between requests </li></ul></ul><ul><ul><li>following requests are thus served from the cache </li></ul></ul><ul><ul><li>sort intervals on length </li></ul></ul>I 32 I 33 I 21 I 11 I 31 I 12 Video clip 1 S 11 Video clip 1 S 11 S 12 Video clip 1 S 12 S 11 S 13 Video clip 2 S 22 S 21 Video clip 3 S 33 S 31 S 32 S 34 I 11 I 12 I 21 I 31 I 32 I 33
  25. 25. Receiver-driven Layered Multicast (RLM) <ul><li>Requires </li></ul><ul><ul><li>IP multicast </li></ul></ul><ul><ul><li>layered video codec (preferably exponential thickness) </li></ul></ul><ul><li>Operation </li></ul><ul><ul><li>Each video layer is one IP multicast group </li></ul></ul><ul><ul><li>Receivers join the base layer and extension layers </li></ul></ul><ul><ul><li>If they experience loss, they drop layers (leave IP multicast groups) </li></ul></ul><ul><ul><li>To add layers, they perform &quot;join experiments“ </li></ul></ul><ul><li>Advantages </li></ul><ul><ul><li>Receiver-only decision </li></ul></ul><ul><ul><li>Congestion affects only sub-tree quality </li></ul></ul><ul><ul><li>Multicast trees are pruned, sub-trees have only necessary traffic </li></ul></ul>
  26. 26. Receiver-driven Layered Multicast (RLM)
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×