SlideShare a Scribd company logo
Video Coding Enhancements for HTTP Adaptive Streaming
Vignesh V Menon
Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität, Klagenfurt, Austria
10-14 Oct 2022
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 1
Outline
1 Introduction
2 Research Questions
3 Target of this study
4 Outcomes
5 Summary
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 2
Introduction
Introduction
HTTP Adaptive Streaming (HAS)1
Why Adaptive Streaming?
Adapt for a wide range of devices.
Adapt for a broad set of Internet speeds.
What HAS does?
Each source video is split into segments.
Encoded at multiple bitrates, resolutions, and codecs.
Delivered to the client based on the device capability, network speed etc.
1
Abdelhak Bentaleb et al. “A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP”. In: IEEE Communications Surveys Tutorials 21.1
(2019), pp. 562–585. doi: 10.1109/COMST.2018.2862938.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 3
Introduction
Introduction
Video Coding in HTTP Adaptive Streaming (HAS)
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 4
Research Questions
Research Questions
RQ-1 Introduction
How to efficiently provide multi-bitrate, multi-resolution representations for HAS?
The demand for high resolution and high bitrate content is rising.
More devices are introduced to cater to the demand for high-quality video content at various
resolutions and bitrates.
Goal: Reduce the increasing computational complexity (i.e., runtime) while maintaining
the compression efficiency of standalone encodings.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 5
Research Questions
Research Questions
RQ-1 Introduction
Relative encoding time of the representations normalized to encoding time of the 2160p
25Mbps representation.
500 1000 1500 2000 3000 4500 5800 7000 11600 16800 20000 25000
Bitrate (in kbps)
0
20
40
60
80
100
Relative Time Complexity (in percentage)
540p
1080p
2160p
As resolution doubles, encoding time complexity doubles!
Many encoder analysis decisions are redundant across the representations.
Multi-rate: Exploit this redundancy across representations of a resolution.
Multi-resolution: Exploit this redundancy across resolutions.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 6
Research Questions
Research Questions
RQ-1 Literature Review
How to efficiently provide multi-bitrate, multi-resolution representations for HAS?
Limit block partitioning search levels.2,3
Reuse additional available information to help with other encoding decisions.4
Multi-encoding methods combine fast multi-rate methods and multi-resolution encoding
optimizations.5
2
Damien Schroeder, Adithyan Ilangovan, and Eckehard Steinbach. “Multi-rate encoding for HEVC-based adaptive HTTP streaming with multiple resolutions”.
In: 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP). 2015, pp. 1–6. doi: 10.1109/MMSP.2015.7340822.
3
Hadi Amirpour et al. “Fast Multi-rate Encoding for Adaptive HTTP Streaming”. In: 2020 Data Compression Conference (DCC). 2020, pp. 358–358.
4
Damien Schroeder et al. “Efficient Multi-Rate Video Encoding for HEVC-Based Adaptive HTTP Streaming”. In: IEEE Transactions on Circuits and Systems
for Video Technology 28.1 (2018), pp. 143–157. doi: 10.1109/TCSVT.2016.2599028.
5
Kalyan Goswami et al. “Adaptive Multi-Resolution Encoding for ABR Streaming”. In: 2018 25th IEEE International Conference on Image Processing (ICIP).
2018, pp. 1008–1012. doi: 10.1109/ICIP.2018.8451485.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 7
Research Questions
Research Questions
RQ-2 Introduction
How to efficiently provide multi-codec representations for HAS?
HAS serves multiple types of devices:
A subset of devices can only decode the previous generation codec (e.g., AVC6
).
A subset of devices can only decode the current generation codec (e.g., HEVC7
).
A subset of devices can decode both codecs and can also seamlessly switch between them.
Representations of both codecs should be stored to address multiple clients.8
Goal: Reduce the increased computational complexity of encoding representations of mul-
tiple codecs without compromising the overall quality of such a system.
6
Thomas Wiegand et al. “Overview of the H.264/AVC video coding standard”. In: IEEE Transactions on Circuits and Systems for Video Technology 13.7
(2003), pp. 560–576. doi: 10.1109/TCSVT.2003.815165.
7
Gary J. Sullivan et al. “Overview of the high efficiency video coding (HEVC) standard”. In: IEEE Transactions on circuits and systems for video technology
22.12 (2012), pp. 1649–1668.
8
Yury Reznik et al. “Optimal Multi-Codec Adaptive Bitrate Streaming”. In: 2019 IEEE International Conference on Multimedia Expo Workshops (ICMEW).
2019, pp. 348–353. doi: 10.1109/ICMEW.2019.00066.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 8
Research Questions
Research Questions
RQ-3 Introduction
How to improve the encoder performance using content-adaptive algorithms?
Quest to achieve a perfect trade-off between perceptual quality and compression efficiency.
Allocating only the required bits for a given video based on its complexity, Content Adaptive
Encoding (CAE) can drive significant bitrate savings.9
Goal: Derive content-adaptive spatial and temporal features for the video, which we can
later use to influence encoder decisions like slice-type, quantization parameter, block par-
titioning, framerate, and much more.
9
J. Shingala and P. Dixit. “Content Adaptive Encoding: Key Decisions for an Effective Solution”. In: “online“. Feb. 2018.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 9
Research Questions
Research Questions
RQ-3 Literature Review
How to improve the encoder performance using content-adaptive algorithms?
Shot detection, content-adaptive block partitioning prediction,10 adaptive quantization pa-
rameter prediction, and much more.11
Content-adaptive controls for video encoding.12,13
10
Y. Zhang et al. “Statistical Early Termination and Early Skip Models for Fast Mode Decision in HEVC INTRA Coding”. In: ACM Trans. Multimedia Comput.
Commun. Appl. 15.3 (July 2019). issn: 1551-6857. doi: 10.1145/3321510.
11
Shingala and Dixit, “Content Adaptive Encoding: Key Decisions for an Effective Solution”.
12
Pradeep Ramachandran et al. “Content adaptive live encoding with open source codecs”. In: May 2020, pp. 345–348. doi: 10.1145/3339825.3393580.
13
Tamar Shoham et al. “Content-adaptive frame level rate control for video encoding using a perceptual video quality measure”. In: Applications of Digital
Image Processing XLII. vol. 11137. SPIE, 2019, pp. 164 –177.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 10
Research Questions
Research Questions
RQ-4 Introduction
How to improve the efficiency of per-title encoding for live-streaming applications?
For live streaming applications, the same video content is encoded (also called ”title”) at
multiple bitrates, resolutions, codecs, etc., though we are constrained in terms of encoding
time.
Encoders have presets with conservative settings biased towards the compression effi-
ciency.14
Goal:
1 Decrease the time taken to compute the convex hull for each video by analyzing it using certain
features (e.g., SI, TI information15
).
2 Study dynamically configuring the encoder parameters on the fly to sustain a target encoding
speed according to the content for efficient live streaming.
14
Ramachandran et al., “Content adaptive live encoding with open source codecs”.
15
https://github.com/Telecommunication-Telemedia-Assessment/SITI.git
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 11
Target of this study
Target of this study
Figure: The ideal video compression system for HAS envisioned in this doctoral study.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 12
Outcomes
Outcomes
Video Complexity Feature Extraction
Video Complexity Analyzer (VCA) open-source project16,17 is released, which is a practical
implementation of the video complexity feature extraction proposed in this doctoral research.
Figure: VCA in content-adaptive encoding systems.
16
https://github.com/cd-athena/VCA, last access: Aug 20, 2022
17
Vignesh V Menon et al. “VCA: Video Complexity Analyzer”. In: 13th ACM Multimedia Systems Conference (MMSys ’22). MMSys ’22. New York, NY, USA:
Association for Computing Machinery, 2022. isbn: 9781450392839. doi: 10.1145/3524273.3532896. url: https://doi.org/10.1145/3524273.3532896.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 13
Outcomes
Outcomes
Perceptually-aware Bitrate Ladder Prediction18
Figure: Perceptually-aware per-title encoding scheme.
b0 b1 b2 b3 b4 b5 b6
Bitrate
v0
v1=v0 + vJ(v0)
v2=v1 + vJ(v1)
v3=v2 + vJ(v2)
v4=v3 + vJ(v3)
v5=v4 + vJ(v4)
v6=v5 + vJ(v5)
VMAF
vmax
r1
r0
r2
r3
r4
r5
r6
18
Vignesh V Menon et al. “Perceptually-aware Per-title Encoding for Adaptive Video Streaming”. In: 2022 IEEE International Conference on Multimedia and
Expo (ICME). ICME ’22. Taiwan, 2022.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 14
Outcomes
Outcomes
Dynamic Framerate Prediction19
Figure: Block diagram of the complete variable framerate (VFR) coding scheme in the context of
per-title encoding. Here, f and ˆ
f denotes the original framerate of the title and the framerate at which
the title is encoded. d represents the frame dropping factor.
19
Vignesh V Menon et al. “CODA: Content-aware Frame Dropping Algorithm for High Frame-rate Video Streaming”. In: 2022 Data Compression Conference
(DCC). 2022, pp. 475–475. doi: 10.1109/DCC52660.2022.00086.
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 15
Summary
Summary
Table: List of Related Publications
Row Title Venue
1 Efficient Multi-Encoding Algorithms for HTTP Adaptive
Bitrate Streaming
Picture Coding Symposium (PCS) 2021
2 Efficient Content-Adaptive Feature-based Shot Detection
for HTTP Adaptive Streaming
International Conference on Image Processing (ICIP) 2021
3 INCEPT: Intra CU depth Prediction for HEVC International Workshop on Multimedia Signal Processing
(MMSP) 2021
4 OPTE: Online Per-title Encoding International Conference on Acoustics, Speech, and Signal
Processing (ICASSP) 2022
5 CODA: Content-Adaptive Frame dropping Algorithm Data Compression Conference (DCC) 2022
6 Live-PSTR: Live Per-title Encoding for Ultra HD Adaptive
Streaming
NAB Broadcast Engineering and Information Technology
(BEIT) Conference 2022
7 Perceptually-aware Per-title Encoding for Adaptive Video
Streaming
International Conference on Multimedia & Expo (ICME) 2022
8 OPSE: Online Per-Scene Encoding for Adaptive HTTP Live
Streaming
International Conference on Multimedia & Expo Workshop
(ICMEW) 2022
9 VCA: Video Complexity Analyzer Multimedia Systems Conference (MMSys) 2022
10 ETPS: Efficient Two-Pass encoding Scheme for Live Adap-
tive Streaming
ICIP 2022
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 16
Q & A
Q & A
Thank you for your attention!
Vignesh V Menon (vignesh.menon@aau.at)
Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 17

More Related Content

Similar to Doctoral Symposium presentation.pdf

Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Alpen-Adria-Universität
 
HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)
Alpen-Adria-Universität
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Alpen-Adria-Universität
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Ekrem Çetinkaya
 
OPTE: Online Per-title Encoding for Live Video Streaming
OPTE: Online Per-title Encoding for Live Video StreamingOPTE: Online Per-title Encoding for Live Video Streaming
OPTE: Online Per-title Encoding for Live Video Streaming
Alpen-Adria-Universität
 
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfOPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
Vignesh V Menon
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?
Alpen-Adria-Universität
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Alpen-Adria-Universität
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
Minh Nguyen
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Vignesh V Menon
 
HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?
Alpen-Adria-Universität
 
Analyzing Video Streaming Quality by Using Various Error Correction Methods o...
Analyzing Video Streaming Quality by Using Various Error Correction Methods o...Analyzing Video Streaming Quality by Using Various Error Correction Methods o...
Analyzing Video Streaming Quality by Using Various Error Correction Methods o...
IJERA Editor
 
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Ijripublishers Ijri
 
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Ijripublishers Ijri
 
CAdViSE or how to find the Sweet Spots of ABR Systems
CAdViSE or how to find the Sweet Spots of ABR SystemsCAdViSE or how to find the Sweet Spots of ABR Systems
CAdViSE or how to find the Sweet Spots of ABR Systems
Alpen-Adria-Universität
 
12 11 aug17 29may 7301 8997-1-ed edit satria
12 11 aug17 29may 7301 8997-1-ed edit satria12 11 aug17 29may 7301 8997-1-ed edit satria
12 11 aug17 29may 7301 8997-1-ed edit satria
IAESIJEECS
 
A Study on FFmpeg Multimedia Framework
A Study on FFmpeg Multimedia FrameworkA Study on FFmpeg Multimedia Framework
A Study on FFmpeg Multimedia Framework
ijtsrd
 
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming - Poster
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming - PosterQoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming - Poster
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming - Poster
DanieleLorenzi6
 
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
Minh Nguyen
 

Similar to Doctoral Symposium presentation.pdf (20)

Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
 
HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 
OPTE: Online Per-title Encoding for Live Video Streaming
OPTE: Online Per-title Encoding for Live Video StreamingOPTE: Online Per-title Encoding for Live Video Streaming
OPTE: Online Per-title Encoding for Live Video Streaming
 
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfOPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
 
HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?
 
Analyzing Video Streaming Quality by Using Various Error Correction Methods o...
Analyzing Video Streaming Quality by Using Various Error Correction Methods o...Analyzing Video Streaming Quality by Using Various Error Correction Methods o...
Analyzing Video Streaming Quality by Using Various Error Correction Methods o...
 
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
 
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
 
CAdViSE or how to find the Sweet Spots of ABR Systems
CAdViSE or how to find the Sweet Spots of ABR SystemsCAdViSE or how to find the Sweet Spots of ABR Systems
CAdViSE or how to find the Sweet Spots of ABR Systems
 
12 11 aug17 29may 7301 8997-1-ed edit satria
12 11 aug17 29may 7301 8997-1-ed edit satria12 11 aug17 29may 7301 8997-1-ed edit satria
12 11 aug17 29may 7301 8997-1-ed edit satria
 
A Study on FFmpeg Multimedia Framework
A Study on FFmpeg Multimedia FrameworkA Study on FFmpeg Multimedia Framework
A Study on FFmpeg Multimedia Framework
 
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming - Poster
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming - PosterQoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming - Poster
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming - Poster
 
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-driven Dynamic HTTP Adaptive Streaming Player Environment
 

More from Vignesh V Menon

Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Vignesh V Menon
 
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Vignesh V Menon
 
Green Variable framerate encoding for Adaptive Live Streaming
Green Variable framerate encoding  for Adaptive Live StreamingGreen Variable framerate encoding  for Adaptive Live Streaming
Green Variable framerate encoding for Adaptive Live Streaming
Vignesh V Menon
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
Vignesh V Menon
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
Vignesh V Menon
 
TQPM.pdf
TQPM.pdfTQPM.pdf
TQPM.pdf
Vignesh V Menon
 
CAPS_Presentation.pdf
CAPS_Presentation.pdfCAPS_Presentation.pdf
CAPS_Presentation.pdf
Vignesh V Menon
 
LiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdfLiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdf
Vignesh V Menon
 
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdfETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
Vignesh V Menon
 
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdfOPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
Vignesh V Menon
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfPerceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Vignesh V Menon
 
Video Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVideo Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdf
Vignesh V Menon
 
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingLive-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Vignesh V Menon
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
Vignesh V Menon
 
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
Vignesh V Menon
 
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
Vignesh V Menon
 

More from Vignesh V Menon (16)

Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
 
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
 
Green Variable framerate encoding for Adaptive Live Streaming
Green Variable framerate encoding  for Adaptive Live StreamingGreen Variable framerate encoding  for Adaptive Live Streaming
Green Variable framerate encoding for Adaptive Live Streaming
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
 
TQPM.pdf
TQPM.pdfTQPM.pdf
TQPM.pdf
 
CAPS_Presentation.pdf
CAPS_Presentation.pdfCAPS_Presentation.pdf
CAPS_Presentation.pdf
 
LiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdfLiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdf
 
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdfETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
 
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdfOPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfPerceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
 
Video Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVideo Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdf
 
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingLive-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
 
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
 
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
 

Recently uploaded

বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdfবাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
eBook.com.bd (প্রয়োজনীয় বাংলা বই)
 
Chapter 4 - Islamic Financial Institutions in Malaysia.pptx
Chapter 4 - Islamic Financial Institutions in Malaysia.pptxChapter 4 - Islamic Financial Institutions in Malaysia.pptx
Chapter 4 - Islamic Financial Institutions in Malaysia.pptx
Mohd Adib Abd Muin, Senior Lecturer at Universiti Utara Malaysia
 
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama UniversityNatural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
Akanksha trivedi rama nursing college kanpur.
 
MARY JANE WILSON, A “BOA MÃE” .
MARY JANE WILSON, A “BOA MÃE”           .MARY JANE WILSON, A “BOA MÃE”           .
MARY JANE WILSON, A “BOA MÃE” .
Colégio Santa Teresinha
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
Jean Carlos Nunes Paixão
 
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
mulvey2
 
The History of Stoke Newington Street Names
The History of Stoke Newington Street NamesThe History of Stoke Newington Street Names
The History of Stoke Newington Street Names
History of Stoke Newington
 
S1-Introduction-Biopesticides in ICM.pptx
S1-Introduction-Biopesticides in ICM.pptxS1-Introduction-Biopesticides in ICM.pptx
S1-Introduction-Biopesticides in ICM.pptx
tarandeep35
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
chanes7
 
Hindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdfHindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdf
Dr. Mulla Adam Ali
 
Community pharmacy- Social and preventive pharmacy UNIT 5
Community pharmacy- Social and preventive pharmacy UNIT 5Community pharmacy- Social and preventive pharmacy UNIT 5
Community pharmacy- Social and preventive pharmacy UNIT 5
sayalidalavi006
 
How to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP ModuleHow to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP Module
Celine George
 
Smart-Money for SMC traders good time and ICT
Smart-Money for SMC traders good time and ICTSmart-Money for SMC traders good time and ICT
Smart-Money for SMC traders good time and ICT
simonomuemu
 
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPLAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
RAHUL
 
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdfANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
Priyankaranawat4
 
PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.
Dr. Shivangi Singh Parihar
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
Katrina Pritchard
 
The basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptxThe basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptx
heathfieldcps1
 
Liberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdfLiberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdf
WaniBasim
 
South African Journal of Science: Writing with integrity workshop (2024)
South African Journal of Science: Writing with integrity workshop (2024)South African Journal of Science: Writing with integrity workshop (2024)
South African Journal of Science: Writing with integrity workshop (2024)
Academy of Science of South Africa
 

Recently uploaded (20)

বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdfবাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
 
Chapter 4 - Islamic Financial Institutions in Malaysia.pptx
Chapter 4 - Islamic Financial Institutions in Malaysia.pptxChapter 4 - Islamic Financial Institutions in Malaysia.pptx
Chapter 4 - Islamic Financial Institutions in Malaysia.pptx
 
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama UniversityNatural birth techniques - Mrs.Akanksha Trivedi Rama University
Natural birth techniques - Mrs.Akanksha Trivedi Rama University
 
MARY JANE WILSON, A “BOA MÃE” .
MARY JANE WILSON, A “BOA MÃE”           .MARY JANE WILSON, A “BOA MÃE”           .
MARY JANE WILSON, A “BOA MÃE” .
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
 
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
 
The History of Stoke Newington Street Names
The History of Stoke Newington Street NamesThe History of Stoke Newington Street Names
The History of Stoke Newington Street Names
 
S1-Introduction-Biopesticides in ICM.pptx
S1-Introduction-Biopesticides in ICM.pptxS1-Introduction-Biopesticides in ICM.pptx
S1-Introduction-Biopesticides in ICM.pptx
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
 
Hindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdfHindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdf
 
Community pharmacy- Social and preventive pharmacy UNIT 5
Community pharmacy- Social and preventive pharmacy UNIT 5Community pharmacy- Social and preventive pharmacy UNIT 5
Community pharmacy- Social and preventive pharmacy UNIT 5
 
How to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP ModuleHow to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP Module
 
Smart-Money for SMC traders good time and ICT
Smart-Money for SMC traders good time and ICTSmart-Money for SMC traders good time and ICT
Smart-Money for SMC traders good time and ICT
 
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPLAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
 
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdfANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
 
PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
 
The basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptxThe basics of sentences session 6pptx.pptx
The basics of sentences session 6pptx.pptx
 
Liberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdfLiberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdf
 
South African Journal of Science: Writing with integrity workshop (2024)
South African Journal of Science: Writing with integrity workshop (2024)South African Journal of Science: Writing with integrity workshop (2024)
South African Journal of Science: Writing with integrity workshop (2024)
 

Doctoral Symposium presentation.pdf

  • 1. Video Coding Enhancements for HTTP Adaptive Streaming Vignesh V Menon Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität, Klagenfurt, Austria 10-14 Oct 2022 Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 1
  • 2. Outline 1 Introduction 2 Research Questions 3 Target of this study 4 Outcomes 5 Summary Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 2
  • 3. Introduction Introduction HTTP Adaptive Streaming (HAS)1 Why Adaptive Streaming? Adapt for a wide range of devices. Adapt for a broad set of Internet speeds. What HAS does? Each source video is split into segments. Encoded at multiple bitrates, resolutions, and codecs. Delivered to the client based on the device capability, network speed etc. 1 Abdelhak Bentaleb et al. “A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP”. In: IEEE Communications Surveys Tutorials 21.1 (2019), pp. 562–585. doi: 10.1109/COMST.2018.2862938. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 3
  • 4. Introduction Introduction Video Coding in HTTP Adaptive Streaming (HAS) Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 4
  • 5. Research Questions Research Questions RQ-1 Introduction How to efficiently provide multi-bitrate, multi-resolution representations for HAS? The demand for high resolution and high bitrate content is rising. More devices are introduced to cater to the demand for high-quality video content at various resolutions and bitrates. Goal: Reduce the increasing computational complexity (i.e., runtime) while maintaining the compression efficiency of standalone encodings. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 5
  • 6. Research Questions Research Questions RQ-1 Introduction Relative encoding time of the representations normalized to encoding time of the 2160p 25Mbps representation. 500 1000 1500 2000 3000 4500 5800 7000 11600 16800 20000 25000 Bitrate (in kbps) 0 20 40 60 80 100 Relative Time Complexity (in percentage) 540p 1080p 2160p As resolution doubles, encoding time complexity doubles! Many encoder analysis decisions are redundant across the representations. Multi-rate: Exploit this redundancy across representations of a resolution. Multi-resolution: Exploit this redundancy across resolutions. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 6
  • 7. Research Questions Research Questions RQ-1 Literature Review How to efficiently provide multi-bitrate, multi-resolution representations for HAS? Limit block partitioning search levels.2,3 Reuse additional available information to help with other encoding decisions.4 Multi-encoding methods combine fast multi-rate methods and multi-resolution encoding optimizations.5 2 Damien Schroeder, Adithyan Ilangovan, and Eckehard Steinbach. “Multi-rate encoding for HEVC-based adaptive HTTP streaming with multiple resolutions”. In: 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP). 2015, pp. 1–6. doi: 10.1109/MMSP.2015.7340822. 3 Hadi Amirpour et al. “Fast Multi-rate Encoding for Adaptive HTTP Streaming”. In: 2020 Data Compression Conference (DCC). 2020, pp. 358–358. 4 Damien Schroeder et al. “Efficient Multi-Rate Video Encoding for HEVC-Based Adaptive HTTP Streaming”. In: IEEE Transactions on Circuits and Systems for Video Technology 28.1 (2018), pp. 143–157. doi: 10.1109/TCSVT.2016.2599028. 5 Kalyan Goswami et al. “Adaptive Multi-Resolution Encoding for ABR Streaming”. In: 2018 25th IEEE International Conference on Image Processing (ICIP). 2018, pp. 1008–1012. doi: 10.1109/ICIP.2018.8451485. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 7
  • 8. Research Questions Research Questions RQ-2 Introduction How to efficiently provide multi-codec representations for HAS? HAS serves multiple types of devices: A subset of devices can only decode the previous generation codec (e.g., AVC6 ). A subset of devices can only decode the current generation codec (e.g., HEVC7 ). A subset of devices can decode both codecs and can also seamlessly switch between them. Representations of both codecs should be stored to address multiple clients.8 Goal: Reduce the increased computational complexity of encoding representations of mul- tiple codecs without compromising the overall quality of such a system. 6 Thomas Wiegand et al. “Overview of the H.264/AVC video coding standard”. In: IEEE Transactions on Circuits and Systems for Video Technology 13.7 (2003), pp. 560–576. doi: 10.1109/TCSVT.2003.815165. 7 Gary J. Sullivan et al. “Overview of the high efficiency video coding (HEVC) standard”. In: IEEE Transactions on circuits and systems for video technology 22.12 (2012), pp. 1649–1668. 8 Yury Reznik et al. “Optimal Multi-Codec Adaptive Bitrate Streaming”. In: 2019 IEEE International Conference on Multimedia Expo Workshops (ICMEW). 2019, pp. 348–353. doi: 10.1109/ICMEW.2019.00066. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 8
  • 9. Research Questions Research Questions RQ-3 Introduction How to improve the encoder performance using content-adaptive algorithms? Quest to achieve a perfect trade-off between perceptual quality and compression efficiency. Allocating only the required bits for a given video based on its complexity, Content Adaptive Encoding (CAE) can drive significant bitrate savings.9 Goal: Derive content-adaptive spatial and temporal features for the video, which we can later use to influence encoder decisions like slice-type, quantization parameter, block par- titioning, framerate, and much more. 9 J. Shingala and P. Dixit. “Content Adaptive Encoding: Key Decisions for an Effective Solution”. In: “online“. Feb. 2018. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 9
  • 10. Research Questions Research Questions RQ-3 Literature Review How to improve the encoder performance using content-adaptive algorithms? Shot detection, content-adaptive block partitioning prediction,10 adaptive quantization pa- rameter prediction, and much more.11 Content-adaptive controls for video encoding.12,13 10 Y. Zhang et al. “Statistical Early Termination and Early Skip Models for Fast Mode Decision in HEVC INTRA Coding”. In: ACM Trans. Multimedia Comput. Commun. Appl. 15.3 (July 2019). issn: 1551-6857. doi: 10.1145/3321510. 11 Shingala and Dixit, “Content Adaptive Encoding: Key Decisions for an Effective Solution”. 12 Pradeep Ramachandran et al. “Content adaptive live encoding with open source codecs”. In: May 2020, pp. 345–348. doi: 10.1145/3339825.3393580. 13 Tamar Shoham et al. “Content-adaptive frame level rate control for video encoding using a perceptual video quality measure”. In: Applications of Digital Image Processing XLII. vol. 11137. SPIE, 2019, pp. 164 –177. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 10
  • 11. Research Questions Research Questions RQ-4 Introduction How to improve the efficiency of per-title encoding for live-streaming applications? For live streaming applications, the same video content is encoded (also called ”title”) at multiple bitrates, resolutions, codecs, etc., though we are constrained in terms of encoding time. Encoders have presets with conservative settings biased towards the compression effi- ciency.14 Goal: 1 Decrease the time taken to compute the convex hull for each video by analyzing it using certain features (e.g., SI, TI information15 ). 2 Study dynamically configuring the encoder parameters on the fly to sustain a target encoding speed according to the content for efficient live streaming. 14 Ramachandran et al., “Content adaptive live encoding with open source codecs”. 15 https://github.com/Telecommunication-Telemedia-Assessment/SITI.git Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 11
  • 12. Target of this study Target of this study Figure: The ideal video compression system for HAS envisioned in this doctoral study. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 12
  • 13. Outcomes Outcomes Video Complexity Feature Extraction Video Complexity Analyzer (VCA) open-source project16,17 is released, which is a practical implementation of the video complexity feature extraction proposed in this doctoral research. Figure: VCA in content-adaptive encoding systems. 16 https://github.com/cd-athena/VCA, last access: Aug 20, 2022 17 Vignesh V Menon et al. “VCA: Video Complexity Analyzer”. In: 13th ACM Multimedia Systems Conference (MMSys ’22). MMSys ’22. New York, NY, USA: Association for Computing Machinery, 2022. isbn: 9781450392839. doi: 10.1145/3524273.3532896. url: https://doi.org/10.1145/3524273.3532896. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 13
  • 14. Outcomes Outcomes Perceptually-aware Bitrate Ladder Prediction18 Figure: Perceptually-aware per-title encoding scheme. b0 b1 b2 b3 b4 b5 b6 Bitrate v0 v1=v0 + vJ(v0) v2=v1 + vJ(v1) v3=v2 + vJ(v2) v4=v3 + vJ(v3) v5=v4 + vJ(v4) v6=v5 + vJ(v5) VMAF vmax r1 r0 r2 r3 r4 r5 r6 18 Vignesh V Menon et al. “Perceptually-aware Per-title Encoding for Adaptive Video Streaming”. In: 2022 IEEE International Conference on Multimedia and Expo (ICME). ICME ’22. Taiwan, 2022. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 14
  • 15. Outcomes Outcomes Dynamic Framerate Prediction19 Figure: Block diagram of the complete variable framerate (VFR) coding scheme in the context of per-title encoding. Here, f and ˆ f denotes the original framerate of the title and the framerate at which the title is encoded. d represents the frame dropping factor. 19 Vignesh V Menon et al. “CODA: Content-aware Frame Dropping Algorithm for High Frame-rate Video Streaming”. In: 2022 Data Compression Conference (DCC). 2022, pp. 475–475. doi: 10.1109/DCC52660.2022.00086. Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 15
  • 16. Summary Summary Table: List of Related Publications Row Title Venue 1 Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming Picture Coding Symposium (PCS) 2021 2 Efficient Content-Adaptive Feature-based Shot Detection for HTTP Adaptive Streaming International Conference on Image Processing (ICIP) 2021 3 INCEPT: Intra CU depth Prediction for HEVC International Workshop on Multimedia Signal Processing (MMSP) 2021 4 OPTE: Online Per-title Encoding International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2022 5 CODA: Content-Adaptive Frame dropping Algorithm Data Compression Conference (DCC) 2022 6 Live-PSTR: Live Per-title Encoding for Ultra HD Adaptive Streaming NAB Broadcast Engineering and Information Technology (BEIT) Conference 2022 7 Perceptually-aware Per-title Encoding for Adaptive Video Streaming International Conference on Multimedia & Expo (ICME) 2022 8 OPSE: Online Per-Scene Encoding for Adaptive HTTP Live Streaming International Conference on Multimedia & Expo Workshop (ICMEW) 2022 9 VCA: Video Complexity Analyzer Multimedia Systems Conference (MMSys) 2022 10 ETPS: Efficient Two-Pass encoding Scheme for Live Adap- tive Streaming ICIP 2022 Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 16
  • 17. Q & A Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@aau.at) Vignesh V Menon Video Coding Enhancements for HTTP Adaptive Streaming 17