This document discusses rate adaptation techniques for video streaming. It covers sender algorithms that involve chopping video frames into packets and sending at regular intervals. It also discusses adapting the transmission rate in response to network conditions using additive increase/multiplicative decrease. Methods for estimating the available bandwidth are presented, such as using packet pairs. The document also discusses using layered video encoding to adapt the video quality by sending more or fewer layers depending on the available bandwidth.

1. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Rate Adaptations

2. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
You are Here
Network
Encoder
Sender
Middlebox
Receiver
Decoder

3. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Sender’s Algorithm
open UDP socket
foreach video frame
chop into packets
add RTP header
send to network

4. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Sender’s Algorithm
open UDP socket
foreach video frame
chop into packets
add RTP header
send to network
wait for 1/fps seconds

5. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Sender’s Algorithm
open UDP socket
foreach video frame
chop into packets
add RTP header
send to network
wait for 1/fps seconds
 Send frames at
equal time
distance.

6. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Sender’s Algorithm
open UDP socket
foreach video frame
chop into packets
foreach packet
add RTP header
send to network
wait for size/bps seconds
 Send data at
constant
bandwidth.

7. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Rules
 Transmission rate should match
encoding rate
 Transmission should not be too bursty

8. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Two Approaches
1. Just send at a fix rate
 or “I hope the network can handle it”
approach

9. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Effects on TCP: Simulation
From Sisalem, Emanuel and Schulzrinne paper on
“Direct Adjustment Algorithm.”

10. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Effects on TCP

11. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
DAA Parameters
 Adaptive RTP flows
 Additive increase/multiplicative decrease
 50 kb and factor 0.875
 RTCP: min 5 sec inter-report time
 Loss thresholds: 5% and 10%
 TCP
 Immediate loss notification
 Transmission window is halved

12. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Two Approaches
1. Just send at a fixed rate
 or “I hope the network can handle it”
approach
2. Adapt transmission/encoding rate
to network condition

13. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to Adapt?
if network condition is bad
reduce rate
else if network condition is so-so
do nothing
else if network condition is good
increase rate

14. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
How to ..
 Know “network condition is bad”?
 Increase/decrease rate?

15. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
if network condition is bad
else if network condition is so-so
do nothing
else if network condition is good
Adapting Output Rate

16. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
if network condition is bad
else if network condition is so-so
do nothing
else if network condition is good
Adapting Output Rate
Multiplicative
decrease
Additive
increase

17. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Question:
What should  and  be?

18. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Observation 1
 Should never change your rate more
than an equivalent TCP stream.

19. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Observation 2
  and  should depend on network
conditions and current rate.

20. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Goal: Fair Share of Bottleneck
 let r : current rate
b : bottleneck bandwidth
S : current share

21. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
S vs 
1
S


22. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Value of 
(Assuming one receiver)

23. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Limit of 
 M : packet size
  : round trip time
 T : period between evaluation of 

24. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Loss rate vs 

loss rate
1
1

25. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Value of 
where
is the loss rate
k is a constant
(Assuming one receiver)

26. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

27. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
What is Needed?

28. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Estimating b : Packet Pair

29. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Estimating b : Packet Pair

30. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Estimating b : Packet Pair

31. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Evaluation

32. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
More TCP-Friendly Rate
Control

33. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
TCP-Equation
Window size behavior in TCP/IP with constant loss probability
T. Ott, J. Kemperman, and M. Mathis
June 1997, HPCS 1997

34. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
TCP-Equation
Equation-Based Congestion Control for Unicast Applications
Sally Floyd, Mark Handley, Jitendra Padhye, and Joerg Widmer.
August 2000. SIGCOMM 2000

35. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Rules
 Transmission rate should match
encoding rate
 Transmission should not be too bursty

36. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Rate Control
Given a rate, how to encode the
video with the given rate?

37. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Reduce Frame Rate
 Live Video
 Stored Video

38. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Reduce Frame Resolution
 Live Video
 Stored Video

39. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Increase Quantization
 Live Video
 Stored Video

40. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Drop AC components
 Live Video
 Stored Video

41. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Trouble with Stored Video
 Reducing rate requires partial decoding
and re-encoding
 Solution: Layered Video

42. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Layered Video
or “Scalable Video”

43. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Layered Video
Layer 1 Layer 2 Layer 3

44. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Layering Scheme
 Temporal (Frame Rate) Layering

45. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Layering Scheme
 Spatial (Resolution) Layering

46. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Layering Scheme
 DCT Layering; SNR (Quality) Layering
30 8
-6 -1
2 0
0 0
1 0
0 0
0 0
0 0
30 8
-6 -1
2
1
30

47. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Layering Scheme
 Fine Granularity Scalability (FGS): e.g.,
MPEG-4
1 0 0 0 0 1 0 0
1 0 0 1 0 1 0 1
1 1 1 0 0 0 1 0
1 1 0 0 0 0 0 0

48. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Layered Video
 SVC: Scalable Video Coding
 Base-layer plus enhancement layers
 Each received layer improves the quality
 Layer n+1 depends on layer n
 MDC: Multiple Description Coding
 Layers are independent
 Each received layer improves the quality

49. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Layered Video
 BUT: Layered video codecs are less
bandwidth-efficient than single-layer
codecs.

50. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Rate Adaptation
 To increase rate, send more layers
 To decrease rate, drop some layers

51. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
MS Windows Streaming
Media

52. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Intelligent Streaming
 Multiple-Bit-Rate Encoding
 Intelligent Bandwidth Control
 bit rate selection
 thinning
 Intelligent Image Processing

53. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Intelligent Streaming
 How exactly does it work?
 Sorry, it’s proprietary

54. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Is WSM TCP friendly?
Measurements of the Congestion
Responsiveness of Windows
Streaming Media
 J. Nichols et. al., NOSSDAV 2004
All figures taken from the original
NOSSDAV presentation

55. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)

56. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
340kbps clip, 725kbps bottleneck

57. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Transmission is Bursty

58. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
340kbps clip, 725kbps bottleneck

59. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
548kbps Clip - 725kbps Bottleneck

60. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
1128kbps Clip - 725kbps Bottleneck

61. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
725kbps Bottleneck
Buffering Playout
Single Encoded Bit Rate

62. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
725kbps Bottleneck
Multiple Encoded Bit Rate
Buffering Playout

63. NUS.SOC.CS5248-2014
Roger Zimmermann (based in part on slides by Ooi Wei Tsang)
Conclusion
 Two phases: buffering + playout
 Not always TCP-friendly