In the last decades, video streaming has been developing significantly. Among current technologies, HTTP Adaptive Streaming (HAS) is considered the de-facto approach in multimedia transmission over the internet. Though the majority of HAS-based media services function well even under throughput restrictions and variations, there are still significant challenges for multimedia systems, especially the tradeoff among the increasing content complexity, various time-related requirements, and Quality of Experience (QoE). Optimizing for one aspect usually negatively impacts at least one of the other two aspects. This thesis tackles critical open research questions in the context of HAS that significantly impact the QoE at the client side. The main contributions of this thesis are four-fold: - We propose Days of Future Past Plus (DoFP+) approach that leverages HTTP/3’s features to upgrade low-quality segments while downloading others. - This thesis proposes a weighted sum model, namely WISH, to provide a high QoE of the video and allow end users to express their preferences among different parameters, including data usage, stall events, and video quality. - To improve segment qualities on high-end mobile devices, this thesis introduces an ABR scheme called WISH-SR that integrates a lightweight Convolutional Neural Network (CNN) to enhance low-resolution/low-quality videos at the client side. - To improve segment qualities on high-end mobile devices, this thesis introduces an ABR scheme called WISH-SR that integrates a lightweight Convolutional Neural Network (CNN) to enhance low-resolution/low-quality videos at the client side.

1. Policy-Driven Dynamic HTTP Adaptive Streaming
Player Environment
Minh Nguyen
Supervisors:
Univ.-Prof. DI Dr. Christian Timmerer
Assoc.-Prof. DI Dr. Mathias Lux
DISSERTATION DEFENSE
University of Klagenfurt
Austria
30/06/2023
1

2. Agenda
2
Motivation
HTTP adaptive streaming
Research questions
Contributions
Concluding remarks

3. Motivation
3

4. Motivation
4
[1] Sandvine, “Global internet phenomena report 2023”, https://www.sandvine.com/phenomena
66%
Video streaming in
overall traffic [1]
Content
(complexity)
Time
(latency)
Quality of experience
(QoE)

5. HTTP Adaptive Streaming
5

6. HTTP Adaptive Streaming (HAS)
6
Server
Video
segmentation
Video encoding
...
...
...
Version 3
Version 2
Version 1
Client
Adaptive bitrate
algorithm (ABR)
Throughput
estimation
Playout buffer
Video decoding
...
HTTP GET requests
Throughput
Time

7. Research Questions
7

8. RQ #1: How to provide a generic ABR scheme for HAS players that
reflects various customer needs?
● Different requirements from customers
● Tradeoff: content, quality, and time for each use case
Research Questions
8
RQ #2: How to improve HAS players’ performance with support of new
HTTP versions?
● Utilizing HTTP/3 features
● These features include stream multiplexing, stream priority, and stream
termination

9. RQ #3: How to leverage computational power of client devices to
improve HAS players?
● Client device’s computational power is increasing significantly
● Players can improve downloaded segments by Deep Neural Networks
Research Questions
9
RQ #4: How to utilize HAS players’ information at the server to improve
HAS performance?
● Heterogeneous devices are used with specific networks
● The server needs to provide suitable quality versions for specific devices

10. Contributions
10

11. Contribution 1
RQ #2
New HTTP versions’
supports
DoFP+: An HTTP/3-Based
Adaptive Bitrate Approach Using
Retransmission Techniques
11

12. Motivation
A quality gap is a group of segments in the
buffer that
● have the same quality and
● have lower quality than 2 adjacent ones or
● have lower quality than the previous group
if it is the last one
Quality gaps decrease the QoE
12

13. Motivation
Tackle the issue by retransmission
Replace low-quality segments by higher ones
13

14. Motivation
HTTP/3 provides notable features
14
HTTP/3
Stream
priority
Stream
termination
Stream
multiplexing

15. Retransmission order analysis
● Scenario
○ Multiple quality gaps
○ Improve 1 gap
● Estimate QoE by ITU-T P.1203 [1]
15
[1] https://github.com/itu-p1203/itu-p1203
The gaps with the lowest
quality level SHOULD be
filled first.

16. Retransmission order analysis
● Scenario
○ Multiple-segment quality gap
○ Improve some segments
● Estimate QoE by ITU-T P.1203 [1]
16
[1] https://github.com/itu-p1203/itu-p1203
When the considered quality
gap has more than 1
segment, the later-played
segment SHOULD be
upgraded first.

17. DoFP+ Approach
● Target: Find the best qualities for segments in the buffer and the upcoming
segment
● Objective function
17
Average segment qualities
Qualities of segments Video instability

18. DoFP+ Approach
● Select a set of quality of segments
18
● Conditions
Download time < available time
Estimated buffer > predefined threshold (e.g 50% of buffer size)

19. Experimental results
DoFP+ downloads a moderate amount of
data
● Less than BBA-0, BBA-0-H, SARA,
SARA-H, and DoFP by > 10%
● Higher than the others
19

20. Experimental results
DoFP+ achieves higher QoE
● For ToS1 video, DoFP+ increases
the QoE score to 3.35, which is
an improvement from 4% (DoFP)
to 29% (AGG)
20

21. Contribution 2
RQ #1
A generic ABR scheme
WISH: User-centric Bitrate
Adaptation for HTTP Adaptive
Streaming on Mobile Devices
21

22. Observation
22
Higher quality
(less quality cost)
Download
high-bitrate
segment
More transferred data
(higher data cost)
More download time
(higher buffer cost)

23. Observation
23
3
2
1
Data cost Buffer cost Quality cost
Total cost
Quality version Selection

24. WISH: User-centric Bitrate Adaptation
24
● Throughput (data) cost of a bitrate is a linearly increasing function
● Buffer cost increases when the download time increases and/or the
buffer level decreases.
Throughput Cost
Buffer Cost
Bitrate
Estimated throughput
Download time of current segment
Current buffer - low threshold

25. WISH: User-centric Bitrate Adaptation
25
● Quality cost comprises two sub-penalties
○ Distortion penalty: When a quality is lower than the highest-bitrate
quality.
○ Instability penalty: When that quality is different from the average
quality of recent segments.
Quality Cost
Distortion penalty +
Instability penalty

26. WISH: User-centric Bitrate Adaptation
26
● The total cost of each quality is a weighted sum of Throughput cost,
Buffer cost, and Quality cost
● Select the quality with the lowest total cost
Total Cost = 𝛂 × Throughput Cost + 𝛃 × Buffer Cost + 𝝲 × Quality Cost
(𝛂, 𝛃, 𝝲) = (?, ?, ?)

27. Weights Determination
27
● Consider total cost as the function of bitrate
● Weights are determined by making the maximum bitrate own the lowest
cost (i.e., the derivative of ) at particular conditions:

28. Experimental results
● Comparison with state-of-the-art approaches
○ WISH achieves the highest QoE scores for all test sequences
○ WISH’s QoE scores: from 3.5 (GamePlay) to 3.7 (ToS2)
○ Other methods: < 3.40
⇒ QoE score: +18%
28

29. Experimental results
● WISH’s performance with different settings
○ Users meet their needs of data usage by varying the safe threshold ξ
○ Higher ξ ⇒ smaller γ ⇒ less priority to high bitrates
○ Higher ξ ⇒ lower bitrate, less video instability, fewer switches and stalls
29
WISH’s performance with different ξ values

30. Contribution 3
RQ #4
Advanced analytic options
integration
CADLAD: Device-Aware
Bitrate Ladder Construction
for HAS
30

31. Motivation
● Heterogeneous devices for watching video
content [1]
● Traditional HAS servers utilize static bitrate
ladder, neglecting device types and viewer’s
networks
● Some ABR algorithms (e.g., [2]) tend to
select the highest bitrate ⇒ risk of
rebuffering & no recognizable improvement
in quality for small devices
31
[1] Bitmovin, “Video Developer Report 2023,” [Online] Available: https://go.bitmovin.com/video-developer-report-2023
[2] Huang, T., Johari, R., McKeown, N., Trunnell, M., and Watson, M. A Buffer-Based Approach to Rate Adaptation: Evidence from a Large Video Streaming
Service. In ACM SIGCOMM Computer Communication Review (2014), vol. 44, ACM, pp. 187–198.

32. Motivation
32
● Quality version 1: bitrate 1, height 1, width 1
● Quality version 2: bitrate 2, height 2, width 2
● Quality version 3: bitrate 3, height 3, width 3
● …
Manifest files such as MPD (media presentation
description) hold information of quality versions is sent
from the server to the client
Bitrate X
Server

33. Common Media Client Data (CMCD)
33
Buffer
length
Measured
throughput
Top
bitrate …
bl
mtp tb
Parameters defined in CMCD
Server Client
sw
dt
Screen
width
Device
type
Proposed
parameters
CMCD specification: https://cdn.cta.tech/cta/media/media/resources/standards/pdfs/cta-5004-final.pdf
How to use CMCD? How to determine CMCD?
Bitrate
ladder

34. CMCD Parameter Determination
34
Screen
width
720p 1080p 2160p
[1]
[1] https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12
Device
type
mobile desktop TV
Top
bitrate
Average
throughput

35. Bitrate Ladder Construction
35
1. VoD scenario
b2, w2
b1, w1
b3, w3
b4 <= tb3, w4 <= sw3
b2 <= tb1, w2 <= sw1
b1, w1
b2, w2
b1, w1
b3 <= tb2, w3 <= sw2
Server
b4, w4
b3, w3
b2, w2
b1, w1
Q
u
a
l
i
t
y
v
e
r
s
i
o
n
s
(tb3, tv, sw3)
(tb1, mobile, sw1)
(tb2, desktop, sw2)
(Top bitrate, Device type, Screen width)
MPD 3
MPD 2
MPD 1

36. Bitrate Ladder Construction
36
2. Live scenario
(1) Collection
(2) Classification
(3) K-means clustering
(4) Bitrate ladder
selection
(5) Encoding

37. Bitrate Ladder Construction
37
2. Live scenario
b2, w2
b1, w1
b3, w3
b4 <= tb3, w4 <= sw3
b2 <= tb1, w2 <= sw1
b1, w1
b2, w2
b1, w1
b3 <= tb2, w3 <= sw2
MPD 3
MPD 2
MPD 1
Server

38. Experimental
results
QoE by up to 2.7x
38
VoD streaming

39. Experimental
results
QoE by up to 2.5x
39
Live streaming

40. Experimental
results
CADLAD saves data usage
40
Live streaming
VoD streaming

41. Concluding Remarks
41

42. Answering 4 research questions
42
1. How to provide a generic ABR scheme for HAS players that reflects
various customer needs?
● A Weighted Sum Model, namely WISH
● Tradeoff throughput cost, buffer cost, and quality cost
● Boost the QoE by up to 17%
● Decrease the data usage by 36%

43. 2. How to improve HAS players’ performance with support of new HTTP
versions?
● Days of Future Past Plus (DoFP+)
● Leverage HTTP/3 features (stream priority, stream multiplexing, and stream
termination)
● Improve the QoE by 33%
● Decrease # of stalls by 86%, stall duration by 92%
● Recommendation: retransmitting segments sequentially is better
Answering 4 research questions
43

44. Answering 4 research questions
3. How to leverage the computational power of client devices to improve
HAS players?
● Introduce SR-ABR Net - a lightweight DNN-based super-resolution network
● Propose WISH-SR ABR algorithm
● SR-ABR Net processes segments in real-time with 24 fps
● WISH-SR saves 43% of data usage and improves VMAF by 7%
44

45. Answering 4 research questions
4. How to utilize HAS players’ information at the server to improve HAS
performance?
● CADLAD leverages CMCD information
● CADLAD improves the QoE by 2.7x and reduces the data usage by 71%
45

46. Reviewed publications
46
H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive Video
Streaming
ACM Workshop on Packet Video, 2020 Rank B
Scalable High Efficiency Video Coding Based HTTP Adaptive Streaming over QUIC Workshop on the Evolution, Performance,
and Interoperability of QUIC, 2020
Rank B
WISH: User-Centric Bitrate Adaptation for HTTP Adaptive Streaming
on Mobile Devices
Workshop on Multimedia Signal
Processing (MMSP), 2021
Rank B
Days of Future Past: An Optimization-based Adaptive Bitrate Algorithm over HTTP/3 Workshop on Evolution, Performance and
Interoperability of QUIC, 2021
Rank B
Take the Red Pill for H3 and See How Deep the Rabbit Hole Goes Mile-High Video Conference, 2022 Rank B
Super-Resolution Based Bitrate Adaptation for HTTP Adaptive Streaming for Mobile Devices Mile-High Video Conference, 2022 Rank B
CADLAD: Device-Aware Bitrate Ladder Construction for HTTP Adaptive Streaming International Conference on Network and
Service Management (CNSM), 2022
Rank B
DoFP+: An HTTP/3-Based Adaptive Bitrate Approach Using Retransmission Techniques IEEE Access, 2022 Rank A
Performance Analysis of H2BR: HTTP/2-based Segment Upgrading to Improve the QoE in HAS Multimedia Tools and Applications, 2023 Rank A

47. THANK YOU
Q&A