QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming

All rights reserved. ©2020
QoE- and Energy-aware Content Consumption for
HTTP Adaptive Streaming
ACM MMSys ’23 - Doctoral Symposium
June 8th
, 2023 - Vancouver, Canada
Daniele Lorenzi
1

● Introduction
● Research Questions
● Existing Results
● Publications and Ongoing Work
● Schedule & Work Plan
● Q&A
Agenda
2

Introduction
3

HTTP Adaptive Streaming
4
Video
segmentation
Video encoding
...
...
...
Server Client - HAS player
Adaptive bitrate
algorithm
Throughput
estimation
Playout buffer
Video decoding
...
HTTP GET requests
Version 3
Version 2
Version 1
Internet
Throughput
Time
Timeline

Quality of Experience (QoE)
5
Average quality Start-up delay Stalls & stall duration
Downward
quality switches Video instability
...
...
... ...
...
... ...
...
+ + +
+ =
MOS (1-5)

Research Questions
6

RQ - 1 ) How to exploit features of emerging/future networking paradigms for improving
end-users’ QoE in HAS?
RQ - 2 ) How to efﬁciently deliver video representations encoded with multiple codecs for
increasing the QoE of the end user in HAS?
RQ - 3 ) How to devise energy-aware ABR techniques to jointly increase the QoE of the end user
and reduce the energy consumption in HAS systems?
RQ - 4 ) How to exploit machine learning techniques at the client side to enhance video content
and improve the QoE of the end-user in HAS?
Research Questions
7

RQ - 1 ) How to exploit features of emerging/future networking paradigms for improving end-users’
QoE in HAS?
Related Work in Literature
1
M. Nguyen, et. al. (2020) “H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive Video Streaming”. In Proceedings of the 25th Packet Video Workshop.
1–7.
2
Perna, Gianluca et. al. (2022). “A ﬁrst look at HTTP/3 adoption and performance”. Computer Communications 187, 115–124.
8
State-of-the-art:
● Exploitation of HTTP/2 features for retransmission techniques 1
● Analysis of HTTP/3 performance in default mode 2
Research gaps:
● Analyse HTTP/3 performance in “features mode”
● Exploit HTTP/3 features for jointly request next segment and buffered segments

RQ - 2 ) How to efﬁciently deliver video representations encoded with multiple codecs for increasing
the QoE of the end user in HAS?
3
Zabrovskiy, Anatoliy, et. al. (2018) “A Practical Evaluation of Video Codecs for Large-Scale HTTP Adaptive Streaming Services.” In 2018 25th IEEE International Conference on Image
Processing (ICIP). 998–1002.
4
Reznik, Yuriy A., et. al. (2019) “Optimal Multi-Codec Adaptive Bitrate Streaming”. In 2019 IEEE International Conference on Multimedia Expo Workshops (ICMEW). 348–353.
9
State-of-the-art:
● Comparison of video codecs’ performances on different contents 3
● Creation of multi-codec bitrate ladders based on network conditions and content complexity 4
Research gaps:
● Deliver video segments with different codecs over time based on segments metadata
● Analyse relationship between video quality and segment size for multiple codecs and
sequences

RQ - 3 ) How to devise energy-aware ABR techniques to jointly increase the QoE of the end user and
reduce the energy consumption in HAS systems?
5
Varghese, Benoy, et. al. (2017) “e-DASH: Modelling an energy-aware DASH player”. In IEEE 18th International Symposium on A World of Wireless, Mobile and Multimedia Networks
(WoWMoM). 1–9.
6
M. Uitto, et. al. (2018) “Towards Energy-Efﬁcient Adaptive Mpeg-Dash Streaming Using Hevc”. In IEEE International Conference on Multimedia Expo Workshops (ICMEW), pp. 1–6.
10
State-of-the-art:
● Energy-aware ABR decision based on bitrate and video brightness 5
● Encode and deliver HEVC segments 6
Research gaps:
● Consider additional video metadata (content complexity, video quality, resolution, codec)
● Analyse effectiveness of video relighting techniques to reduce display brightness impact on
energy consumption

RQ - 4 ) How to exploit machine learning techniques at the client side to enhance video content and
improve the QoE of the end-user in HAS?
7
Robin Rombach et al. “High-Resolution Image Synthesis with Latent Diffusion Models”. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(CVPR). 2022.
8
Minh Nguyen, and Ekrem Çetinkaya, et. al. (2022) “Super-Resolution Based Bitrate Adaptation for HTTP Adaptive Streaming for Mobile Devices”. In Proceedings of the 1st
Mile-High Video Conference (Denver, Colorado) (MHV ’22). Association for Computing Machinery, New York, NY, USA, 70–76.
11
State-of-the-art:
● Latent diffusion models for T2I synthesis, in-paint modiﬁcation, and super-resolution 7
● Fast super-resolution in HAS for mobile devices 8
Research gaps:
● ML-based video frame interpolation techniques for HAS
● Reduce execution time of latent diffusion models for super-resolution on the client

Existing Results
12

● State-of-the-art:
○ Quality selection for the next segment 1
○ Use of remaining throughput to retransmit
the buffered segment 2
● Proposed Method 3
:
○ Exploit HTTP/3 and QUIC’s features
○ Check for gaps in the buffer
○ Select the quality for the next segment
○ Concurrently retransmit the buffered
segments in different streams
○ Assign different priorities to the streams
1
A. Bentaleb, et. al. (2018) “A survey on bitrate adaptation schemes for streaming media over HTTP”. IEEE Communications Surveys & Tutorials 21, 1, 562–585.
2
M. Nguyen, et. al. (2020) “H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive Video Streaming”. In Proceedings of the 25th
Packet Video Workshop. 1–7.
3
M. Nguyen, and D. Lorenzi, et. al. (2022). DoFP+: An HTTP/3-Based Adaptive Bitrate Approach Using Retransmission Techniques. IEEE Access 10,
109565–109579.
13
DoFP+ (IEEE Access)

14
DoFP+ (IEEE Access): Results

● State-of-the-art:
○ Video codecs comparison 1
○ Bitrate ladder optimization 2
● Proposed Method 3
:
○ Store the multi-codec segments at the origin
○ Optimization model to select a subset of
segments to fetch at the edge
○ Edge-assisted ABR technique to maximize
delivered quality based on list of codecs
supported by clients
1
Zabrovskiy, Anatoliy, et. al. (2018) “A Practical Evaluation of Video Codecs for Large-Scale HTTP Adaptive Streaming Services.” In 2018 25th IEEE
International Conference on Image Processing (ICIP). 998–1002.
2
Reznik, Yuriy A., et. al. (2019) “Optimal Multi-Codec Adaptive Bitrate Streaming”. In 2019 IEEE International Conference on Multimedia Expo Workshops
(ICMEW). 348–353.
3
D. Lorenzi, et. al. "MCOM-Live: A Multi-Codec Optimization Model at the Edge for Live Streaming", In 29th International Conference on Multimedia
Modeling (MMM), Bergen, Norway, 2023.
15
MCOM-Live (MMM’23)

16
MCOM-Live (MMM’23): Results
The lower the better The lower the better The higher the better

Publications & Ongoing Work
17

● Published:
● D. Lorenzi, M. Nguyen, F. Tashtarian, S. Milani, H. Hellwagner, and C. Timmerer. 2021. Days of Future Past: An
Optimization-Based Adaptive Bitrate Algorithm over HTTP/3. In Proceedings of the 2021 Workshop on
Evolution, Performance and Interoperability of QUIC (Virtual Event, Germany) (EPIQ ’21).
● M. Nguyen, D. Lorenzi, F. Tashtarian, H. Hellwagner, and C. Timmerer, “DoFP+: An HTTP/3-Based Adaptive
Bitrate Approach Using Retransmission Techniques”, IEEE Access 10 (2022), 109565–109579.
● D. Lorenzi, F. Tashtarian, H. Amirpour, C. Timmerer, and H. Hellwagner, "MCOM-Live: A Multi-Codec
Optimization Model at the Edge for Live Streaming", The 29th International Conference on Multimedia
Modeling (MMM), Bergen, Norway, 2023.
● Ongoing:
● A Multi-Codec Optimization Model at the Client.
○ Target journal: IEEE Transactions on Multimedia
Bibliography
18

19
1
http://www.conferenceranks.com/
2
http://www.guide2research.com/
3
https://www.scimagojr.com/journalrank.php?type=j
Target Venues
Target Name Type Rank
ACM Multimedia Conference (MM) Conference A1 1
International Conference on Quality of Multimedia Experiences (QoMEX) Conference Top 2
Symposium on Networked Systems, Design and Implementation (NSDI) Conference A1 1
ACM Multimedia Systems (MMSys) Conference B 1
ACM Transactions on Multimedia Computing, Communications, and
Applications (ACM TOMM)
Journal Q1 3
IEEE Transactions on Multimedia Journal Q1 3

Schedule & Work Plan
20

Schedule & Work Plan
2022 2023 2024
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2
Literature
Review
IEEE Access paper
RQ - 1
RQ - 2
RQ - 3
RQ - 4
MCOM at the client
Literature
Review
ML Frame Interpolation
Technique
2025
Thesis
Energy-aware ABR
Algorithm
Literature
Review
21
Q3
MMM’23
paper
ML Super-Resolution
Technique
Energy-aware Multi-Codec
Delivery

Q&A
22

QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming

More Related Content

Similar to QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming

Recently uploaded

QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming