SlideShare a Scribd company logo
augraphics/pcs-logo
Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate
Streaming
Vignesh V Menon, Hadi Amirpour, Christian Timmerer, Mohammad Ghanbari
Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), University of Klagenfurt, Austria
Date: 02 July 2021
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 1
Outline
1 Introduction
2 State-of-the-art: Multi-Rate Encoding
3 State-of-the-art: Multi-Encoding
4 Efficient Multi-Rate Encoding
5 Efficient Multi-Encoding
6 Experimental Evaluation
7 Conclusions and Future Directions
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 2
Introduction
Introduction
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 3
Introduction
Introduction
Background of HTTP Adaptive Streaming (HAS)1
Source: https://bitmovin.com/adaptive-streaming/
Why Adaptive Streaming?
Adapt for a wide range of devices
Adapt for a broad set of Internet speeds
What HAS does?
Each source video is split into segments
Encoded at multiple bitrates, resolutions,
and codecs
Delivered to the client based on the device
capability, network speed etc.
1
A. Bentaleb et al. “A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP”. In: IEEE Communications Surveys Tutorials 21.1 (2019),
pp. 562–585. doi: 10.1109/COMST.2018.2862938.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 4
Introduction
Introduction
Encoding time of HAS representations
Relative encoding time of the representations normalized to encoding time of the 2160p
25Mbps representation.
500 1000 1500 2000 3000 4500 5800 7000 11600 16800 20000 25000
Bitrate (in kbps)
0
20
40
60
80
100
Relative Time Complexity (in percentage)
540p
1080p
2160p
As resolution doubles, encoding time complexity doubles!
Many encoder analysis decisions are redundant across the representations.
Multi-rate: Exploit this redundancy across representations of a resolution.
Multi-resolution: Exploit this redundancy across resolutions.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 5
State-of-the-art: Multi-Rate Encoding
State-of-the-art: Multi-Rate Encoding
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 6
State-of-the-art: Multi-Rate Encoding
Related Works: Multi-Rate Encoding
Single-bound2
and Double-bound3
for CU depth estimation
3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps
3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps
(a) Single-bound for CU depth estimation
3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps
3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps
(b) Double-bound for CU depth estimation
Motivation: CUs generally tend to have higher depth values (smaller CU sizes) in higher bitrate
representations.
CU depth information from the highest bitrate representation is used as the upper bound
of the CU depth estimation of dependent representations.
In double-bound, additionally, CU depth information from the lowest bitrate representation
is used as the lower bound of the CU depth estimation of dependent representations.
2
D. Schroeder, P. Rehm, and E. Steinbach. “Block structure reuse for multi-rate high efficiency video coding”. In: 2015 IEEE International Conference on
Image Processing (ICIP). Sept. 2015, pp. 3972–3976.
3
H. Amirpour et al. “Fast Multi-rate Encoding for Adaptive HTTP Streaming”. In: 2020 Data Compression Conference (DCC). 2020, pp. 358–358.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 7
State-of-the-art: Multi-Rate Encoding
Related Works: Multi-Rate Encoding
Integrated x265 mode-1 and x265 mode-24
Integrated x265 mode-1
Force the optimal decisions made in the
highest bitrate representation
the slice-type
scene-cut decisions
the quadtree structure
prediction modes
reference indices
Integrated x265 mode-2
Reuse all CUs and PUs from the highest
bitrate representation without additional
RDO. They also employ static refinement
techniques defined in x265.
If the reference depth is d, it is re-
evaluated by using the RDO against the
cost of splitting the CU (i.e., depth d +
1).
The motion vectors from the highest bi-
trate representation are used as Motion
vector predictors (MVPs).
4
K. Goswami et al. “Adaptive Multi-Resolution Encoding for ABR Streaming”. In: 2018 25th IEEE International Conference on Image Processing (ICIP). 2018,
pp. 1008–1012.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 8
State-of-the-art: Multi-Encoding
State-of-the-art: Multi-Encoding
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 9
State-of-the-art: Multi-Encoding
Related Work: Multi-Encoding
1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps
1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps
3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps
3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps
Scaled CU,PU reuse + refinement
Figure: Encoder analysis sharing method in x265 Algorithm.5
5
Aruna Mathesawaran et al. “Open source framework for reduced-complexity multi-rate HEVC encoding”. In: Applications of Digital Image Processing XLIII.
ed. by Andrew G. Tescher and Touradj Ebrahimi. Vol. 11510. International Society for Optics and Photonics. SPIE, 2020, pp. 461 –471.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 10
Efficient Multi-Rate Encoding
Efficient Multi-Rate Encoding
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 11
Efficient Multi-Rate Encoding
Efficient Multi-Rate Encoding
Prediction Mode Heuristics
Prediction Mode Heuristics
If the CU depth chosen for the dependent representations is the same as the reference represen-
tation, the following heuristics are proposed.
If Skip mode was chosen in the highest bitrate representation, RDO is evaluated for only
Merge/Skip modes.
If 2Nx2N mode was chosen in the highest bitrate representation, skip RDO for AMP modes.
If inter-prediction mode was chosen in the highest bitrate representation, skip RDO for
intra-prediction modes.
In double-bound for CU estimation, if the intra-prediction mode was chosen for the highest
and lowest bitrate representations, RDO is evaluated for only intra-prediction modes in the
intermediate representations.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 12
Efficient Multi-Rate Encoding
Efficient Multi-Rate Encoding
Motion Estimation Heuristics
Motion Estimation Heuristics
If the CU depth and PU chosen for the dependent representations is the same as the reference
representation, the following heuristics are proposed.
Force the same reference frame as that of the highest bitrate representation.
The Motion Vector Predictor (MVP) is set to be the Motion Vector (MV) of the highest
bitrate representation.
In double-bound for CU estimation, the motion search range is decreased to a smaller
window if the MV of the highest bitrate representation and the MV of the lowest bitrate
representation are close to each other. The search range is determined to be the maximum
difference between the x coordinates and y coordinates of the MVs.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 13
Efficient Multi-Rate Encoding
Efficient Multi-Rate Encoding
Proposed Multi-rate Algorithm-1 and Algorithm-2
3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps
3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps
(a) Proposed Multi-rate Algorithm-1
3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps
3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps
(b) Proposed Multi-rate Algorithm-2
− > denotes analysis sharing using single-bound for CU depth estimation and proposed
heuristics.
− > denotes analysis sharing using double-bound for CU depth estimation and proposed
heuristics.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 14
Efficient Multi-Encoding
Efficient Multi-Encoding
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 15
Efficient Multi-Encoding
Efficient Multi-Encoding
Proposed Multi-Encoding Algorithm-1
1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps
1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps
3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps
3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps
Figure: Encoder analysis sharing method in Proposed Multi-Encoding Algorithm-1.
Multi-rate Approach: Proposed Multi-rate Algorithm-2
Multi-resolution Approach: scaled CU information from highest bitrate representation of
the previous resolution is used as lower bound of CU depth estimation process.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 16
Efficient Multi-Encoding
Efficient Multi-Encoding
Proposed Multi-Encoding Algorithm-2
1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps
1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps
3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps
3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps
Scaling + Single-bound
Figure: Encoder analysis sharing method in Proposed Multi-Encoding Algorithm-2.
Multi-rate Approach: Proposed Multi-rate Algorithm-2
Multi-resolution Approach: scaled CU information from lowest bitrate representation of
the previous resolution is used as lower bound of CU depth estimation process. More rigid
bound compared to Multi-encoding Algorithm-1.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 17
Experimental Evaluation
Experimental Evaluation
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 18
Experimental Evaluation
Experimental Evaluation
Test Setup
Algorithms presented here are tested using x265 v3.46 with the veryslow preset (highest
quality preset practically usable of x265).
Test PC: Dual-processor server with Intel Xeon Gold 5218R (80 cores, frequency at 2.10
GHz).
Test Sequences: (i) 4 JVET test sequences7 and (ii) 3 professionally produced UltraHD
HDR cinematic content.8
The lowest resolution is 960 × 540, intermediate resolution is 1920 × 1080, and the highest
resolution is 3840 × 2160.
The lower resolution sources were generated from the original video source by applying
bi-cubic scaling using FFmpeg.9
6
MulticoreWare Inc. x265 HEVC Encoder/H.265 Video Codec. url: http://x265.org/.
7
Jill Boyce et al. JVET-J1010: JVET common test conditions and software reference configurations. July 2018.
8
M. H. Pinson. “The Consumer Digital Video Library [Best of the Web]”. In: IEEE Signal Processing Magazine 30.4 (2013), pp. 172–174. doi:
10.1109/MSP.2013.2258265.
9
FFmpeg. FFmpeg Documentation. url: https://ffmpeg.org/ffmpeg.html.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 19
Experimental Evaluation
Experimental Evaluation
Test Setup
Table: Bitrate representations Bi,j (in kbps) based on HLS authoring specification10
Layer (i) Resolution j = 1 j = 2 j = 3 j = 4
1 960x540 2000 1500 1000 500
2 1920x1080 7000 5800 4500 3000
3 3840x2160 25000 20000 16800 11600
Performance evaluation Metrics:
∆T : overall encoding time saving
BDRP : Average difference in bitrate with respect to reference encode to maintain the
same PSNR
BDRV : Average difference in bitrate with respect to reference encode to maintain the
same VMAF
BDRP
∆T : Trade-off between BDR and ∆T.
10
https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 20
Experimental Evaluation
Experimental Evaluation
Multi-rate Encoding
Table: Results for the multi-rate algorithms
Method ∆T BDRP
BDRP
∆T BDRV
BDRV
∆T
Single-bound for CU estimation 14.50% -0.59% -4.39% 0.10% 1.08%
Double-bound for CU estimation 26.37% 0.73% 3.44% 1.25% 5.54%
x265 Algorithm-1 18.61% 1.67% 9.52% 1.56% 8.85%
x265 Algorithm-2 55.11% 8.42% 15.62% 8.72% 16.10%
Multi-rate Algorithm-1 (ours) 26.50% -0.28% -0.84% 0.43% 2.31%
Multi-rate Algorithm-2 (ours) 37.37% 1.06% 3.36% 1.57% 4.88%
Multi-rate Algorithm-1 improves ∆T of Single-bound for CU estimation by 12% with neg-
ligible increase in BDR.
Multi-rate Algorithm-2 improves ∆T of Double-bound for CU estimation by 11% with
negligible increase in BDR.
Better BDR
∆T metric
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 21
Experimental Evaluation
Experimental Evaluation
Multi-Encoding
Scheme ∆T BDRP BDRV
State-of-the-art scheme 80.05% 13.53% 9.59%
Multi-encoding Scheme-1 39.72% 2.32% 1.55%
Multi-encoding Scheme-2 50.90% 3.45% 2.63%
State-of-the-art Multi-encoding Algorithm-1 Multi-encoding Algorithm-2
0
2
4
6
8
10
12
14
16
BDR/ T (in percentage)
BDRP/ T
BDRV/ T
State-of-the-art algorithm yields the highest ∆T.
Multi-encoding Algorithm-1 has the lowest increase in BDR and the lowest BDR
∆T metric.
Multi-encoding Algorithm-2 improves ∆T of Multi-encoding Algorithm-1 by 11% with neg-
ligible increase in BDR.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 22
Conclusions and Future Directions
Conclusions and Future Directions
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 23
Conclusions and Future Directions
Conclusions and Future Directions
Conclusions
Presented efficient multi-encoding algorithms for HTTP Adaptive Streaming, tested with the
open-source x265 HEVC encoder.
Multi-rate Encoding:
Proposed heuristics improved the encoding speed of the single-bound and double-bound
for CU estimation by an average of 12% and 11%, respectively with a negligible increase
in BD-Rate.
BDR
∆T metric for proposed algorithms is better compared to the state-of-the-art algorithms.
Multi-encoding:
Proposed two algorithms that extended the proposed multi-rate encoding algorithms across
resolutions.
BDR
∆T metric for proposed algorithms are better compared to the state-of-the-art algorithm.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 24
Conclusions and Future Directions
Conclusions and Future Directions
Future Directions
The encoding speedup can be improved by parallelizing the execution of encoding instances.
The dependent representations can be started after a delay of one frame execution time of
the reference representation.
Further work is required to extend the algorithms to resolutions, which are non-integer
power-of-2 multiples of each other.
Further work is required to investigate more recent codecs, e.g., VVC.
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 25
Conclusions and Future Directions
Q & A
Thank you for your attention!
Vignesh V Menon (vignesh.menon@aau.at)
Hadi Amirpour (hadi.amirpourazarian@aau.at)
Christian Timmerer (Christian.Timmerer@aau.at)
Mohammad Ghanbari (ghan@essex.ac.uk)
Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 26

More Related Content

Similar to IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming deployments

Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
Vignesh V Menon
 
Probabilistic Approach to Provisioning of ITV - Amos K.
Probabilistic Approach to Provisioning of ITV - Amos K.Probabilistic Approach to Provisioning of ITV - Amos K.
Probabilistic Approach to Provisioning of ITV - Amos K.
Amos Kohn
 
Probabilistic Approach to Provisioning of ITV - By Amos_Kohn
Probabilistic Approach to Provisioning of ITV - By Amos_KohnProbabilistic Approach to Provisioning of ITV - By Amos_Kohn
Probabilistic Approach to Provisioning of ITV - By Amos_Kohn
Amos Kohn
 
HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?
Alpen-Adria-Universität
 
A Real-Time Adaptive Algorithm for Video Streaming over Multiple Wireless Acc...
A Real-Time Adaptive Algorithm for Video Streaming over Multiple Wireless Acc...A Real-Time Adaptive Algorithm for Video Streaming over Multiple Wireless Acc...
A Real-Time Adaptive Algorithm for Video Streaming over Multiple Wireless Acc...
Priti Kana
 
A real time adaptive algorithm for video streaming over multiple wireless acc...
A real time adaptive algorithm for video streaming over multiple wireless acc...A real time adaptive algorithm for video streaming over multiple wireless acc...
A real time adaptive algorithm for video streaming over multiple wireless acc...
JPINFOTECH JAYAPRAKASH
 
HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?
Alpen-Adria-Universität
 
Slides
SlidesSlides
Slides
Videoguy
 
Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Machine Learning Based Video Coding Enhancements for HTTP Adaptive StreamingMachine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Alpen-Adria-Universität
 
Npma Final
Npma FinalNpma Final
Npma Final
Vasu Thiruveedula
 
dynamic media streaming over wireless and ip networks
dynamic media streaming over wireless and ip networksdynamic media streaming over wireless and ip networks
dynamic media streaming over wireless and ip networks
Naveen Dubey
 
VCIP_MCBE_presentation.pdf
VCIP_MCBE_presentation.pdfVCIP_MCBE_presentation.pdf
VCIP_MCBE_presentation.pdf
Vignesh V Menon
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Alpen-Adria-Universität
 
2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
IEEEFINALYEARSTUDENTPROJECT
 
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
IEEEFINALSEMSTUDENTSPROJECTS
 
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
IEEEGLOBALSOFTSTUDENTPROJECTS
 
CAdViSE or how to find the Sweet Spots of ABR Systems
CAdViSE or how to find the Sweet Spots of ABR SystemsCAdViSE or how to find the Sweet Spots of ABR Systems
CAdViSE or how to find the Sweet Spots of ABR Systems
Alpen-Adria-Universität
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfPerceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Vignesh V Menon
 
Xiplink hub optimization solution
Xiplink hub optimization solutionXiplink hub optimization solution
Xiplink hub optimization solution
Joshua Cohen
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
Minh Nguyen
 

Similar to IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming deployments (20)

Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
 
Probabilistic Approach to Provisioning of ITV - Amos K.
Probabilistic Approach to Provisioning of ITV - Amos K.Probabilistic Approach to Provisioning of ITV - Amos K.
Probabilistic Approach to Provisioning of ITV - Amos K.
 
Probabilistic Approach to Provisioning of ITV - By Amos_Kohn
Probabilistic Approach to Provisioning of ITV - By Amos_KohnProbabilistic Approach to Provisioning of ITV - By Amos_Kohn
Probabilistic Approach to Provisioning of ITV - By Amos_Kohn
 
HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?
 
A Real-Time Adaptive Algorithm for Video Streaming over Multiple Wireless Acc...
A Real-Time Adaptive Algorithm for Video Streaming over Multiple Wireless Acc...A Real-Time Adaptive Algorithm for Video Streaming over Multiple Wireless Acc...
A Real-Time Adaptive Algorithm for Video Streaming over Multiple Wireless Acc...
 
A real time adaptive algorithm for video streaming over multiple wireless acc...
A real time adaptive algorithm for video streaming over multiple wireless acc...A real time adaptive algorithm for video streaming over multiple wireless acc...
A real time adaptive algorithm for video streaming over multiple wireless acc...
 
HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?
 
Slides
SlidesSlides
Slides
 
Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Machine Learning Based Video Coding Enhancements for HTTP Adaptive StreamingMachine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
Machine Learning Based Video Coding Enhancements for HTTP Adaptive Streaming
 
Npma Final
Npma FinalNpma Final
Npma Final
 
dynamic media streaming over wireless and ip networks
dynamic media streaming over wireless and ip networksdynamic media streaming over wireless and ip networks
dynamic media streaming over wireless and ip networks
 
VCIP_MCBE_presentation.pdf
VCIP_MCBE_presentation.pdfVCIP_MCBE_presentation.pdf
VCIP_MCBE_presentation.pdf
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
 
2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
2014 IEEE JAVA NETWORKING COMPUTING PROJECT Cost effective resource allocatio...
 
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
2014 IEEE JAVA NETWORKING PROJECT Cost effective resource allocation of overl...
 
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
IEEE 2014 JAVA NETWORKING PROJECTS Cost effective resource allocation of over...
 
CAdViSE or how to find the Sweet Spots of ABR Systems
CAdViSE or how to find the Sweet Spots of ABR SystemsCAdViSE or how to find the Sweet Spots of ABR Systems
CAdViSE or how to find the Sweet Spots of ABR Systems
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfPerceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
 
Xiplink hub optimization solution
Xiplink hub optimization solutionXiplink hub optimization solution
Xiplink hub optimization solution
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
 

More from Vignesh V Menon

Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Vignesh V Menon
 
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Vignesh V Menon
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Vignesh V Menon
 
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdfContent_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Vignesh V Menon
 
Green Variable framerate encoding for Adaptive Live Streaming
Green Variable framerate encoding  for Adaptive Live StreamingGreen Variable framerate encoding  for Adaptive Live Streaming
Green Variable framerate encoding for Adaptive Live Streaming
Vignesh V Menon
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
Vignesh V Menon
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
Vignesh V Menon
 
TQPM.pdf
TQPM.pdfTQPM.pdf
TQPM.pdf
Vignesh V Menon
 
CAPS_Presentation.pdf
CAPS_Presentation.pdfCAPS_Presentation.pdf
CAPS_Presentation.pdf
Vignesh V Menon
 
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdfETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
Vignesh V Menon
 
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdfOPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
Vignesh V Menon
 
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfOPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
Vignesh V Menon
 
Video Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVideo Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdf
Vignesh V Menon
 
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingLive-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Vignesh V Menon
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
Vignesh V Menon
 
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
Vignesh V Menon
 

More from Vignesh V Menon (16)

Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
 
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
 
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdfContent_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
 
Green Variable framerate encoding for Adaptive Live Streaming
Green Variable framerate encoding  for Adaptive Live StreamingGreen Variable framerate encoding  for Adaptive Live Streaming
Green Variable framerate encoding for Adaptive Live Streaming
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
 
TQPM.pdf
TQPM.pdfTQPM.pdf
TQPM.pdf
 
CAPS_Presentation.pdf
CAPS_Presentation.pdfCAPS_Presentation.pdf
CAPS_Presentation.pdf
 
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdfETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
 
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdfOPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdf
 
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfOPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
 
Video Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVideo Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdf
 
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingLive-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
 
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
 

Recently uploaded

The Diamonds of 2023-2024 in the IGRA collection
The Diamonds of 2023-2024 in the IGRA collectionThe Diamonds of 2023-2024 in the IGRA collection
The Diamonds of 2023-2024 in the IGRA collection
Israel Genealogy Research Association
 
How to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP ModuleHow to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP Module
Celine George
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
chanes7
 
Types of Herbal Cosmetics its standardization.
Types of Herbal Cosmetics its standardization.Types of Herbal Cosmetics its standardization.
Types of Herbal Cosmetics its standardization.
Ashokrao Mane college of Pharmacy Peth-Vadgaon
 
Life upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for studentLife upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for student
NgcHiNguyn25
 
Azure Interview Questions and Answers PDF By ScholarHat
Azure Interview Questions and Answers PDF By ScholarHatAzure Interview Questions and Answers PDF By ScholarHat
Azure Interview Questions and Answers PDF By ScholarHat
Scholarhat
 
PIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf IslamabadPIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf Islamabad
AyyanKhan40
 
Main Java[All of the Base Concepts}.docx
Main Java[All of the Base Concepts}.docxMain Java[All of the Base Concepts}.docx
Main Java[All of the Base Concepts}.docx
adhitya5119
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
PECB
 
Top five deadliest dog breeds in America
Top five deadliest dog breeds in AmericaTop five deadliest dog breeds in America
Top five deadliest dog breeds in America
Bisnar Chase Personal Injury Attorneys
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
Nicholas Montgomery
 
writing about opinions about Australia the movie
writing about opinions about Australia the moviewriting about opinions about Australia the movie
writing about opinions about Australia the movie
Nicholas Montgomery
 
Introduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp NetworkIntroduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp Network
TechSoup
 
Liberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdfLiberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdf
WaniBasim
 
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
National Information Standards Organization (NISO)
 
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdfবাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
eBook.com.bd (প্রয়োজনীয় বাংলা বই)
 
Digital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental DesignDigital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental Design
amberjdewit93
 
CACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdfCACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdf
camakaiclarkmusic
 
PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.
Dr. Shivangi Singh Parihar
 
Assessment and Planning in Educational technology.pptx
Assessment and Planning in Educational technology.pptxAssessment and Planning in Educational technology.pptx
Assessment and Planning in Educational technology.pptx
Kavitha Krishnan
 

Recently uploaded (20)

The Diamonds of 2023-2024 in the IGRA collection
The Diamonds of 2023-2024 in the IGRA collectionThe Diamonds of 2023-2024 in the IGRA collection
The Diamonds of 2023-2024 in the IGRA collection
 
How to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP ModuleHow to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP Module
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
 
Types of Herbal Cosmetics its standardization.
Types of Herbal Cosmetics its standardization.Types of Herbal Cosmetics its standardization.
Types of Herbal Cosmetics its standardization.
 
Life upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for studentLife upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for student
 
Azure Interview Questions and Answers PDF By ScholarHat
Azure Interview Questions and Answers PDF By ScholarHatAzure Interview Questions and Answers PDF By ScholarHat
Azure Interview Questions and Answers PDF By ScholarHat
 
PIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf IslamabadPIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf Islamabad
 
Main Java[All of the Base Concepts}.docx
Main Java[All of the Base Concepts}.docxMain Java[All of the Base Concepts}.docx
Main Java[All of the Base Concepts}.docx
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
 
Top five deadliest dog breeds in America
Top five deadliest dog breeds in AmericaTop five deadliest dog breeds in America
Top five deadliest dog breeds in America
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
 
writing about opinions about Australia the movie
writing about opinions about Australia the moviewriting about opinions about Australia the movie
writing about opinions about Australia the movie
 
Introduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp NetworkIntroduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp Network
 
Liberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdfLiberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdf
 
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
 
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdfবাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
 
Digital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental DesignDigital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental Design
 
CACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdfCACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdf
 
PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.PCOS corelations and management through Ayurveda.
PCOS corelations and management through Ayurveda.
 
Assessment and Planning in Educational technology.pptx
Assessment and Planning in Educational technology.pptxAssessment and Planning in Educational technology.pptx
Assessment and Planning in Educational technology.pptx
 

IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming deployments

  • 1. augraphics/pcs-logo Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming Vignesh V Menon, Hadi Amirpour, Christian Timmerer, Mohammad Ghanbari Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), University of Klagenfurt, Austria Date: 02 July 2021 Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 1
  • 2. Outline 1 Introduction 2 State-of-the-art: Multi-Rate Encoding 3 State-of-the-art: Multi-Encoding 4 Efficient Multi-Rate Encoding 5 Efficient Multi-Encoding 6 Experimental Evaluation 7 Conclusions and Future Directions Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 2
  • 3. Introduction Introduction Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 3
  • 4. Introduction Introduction Background of HTTP Adaptive Streaming (HAS)1 Source: https://bitmovin.com/adaptive-streaming/ Why Adaptive Streaming? Adapt for a wide range of devices Adapt for a broad set of Internet speeds What HAS does? Each source video is split into segments Encoded at multiple bitrates, resolutions, and codecs Delivered to the client based on the device capability, network speed etc. 1 A. Bentaleb et al. “A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP”. In: IEEE Communications Surveys Tutorials 21.1 (2019), pp. 562–585. doi: 10.1109/COMST.2018.2862938. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 4
  • 5. Introduction Introduction Encoding time of HAS representations Relative encoding time of the representations normalized to encoding time of the 2160p 25Mbps representation. 500 1000 1500 2000 3000 4500 5800 7000 11600 16800 20000 25000 Bitrate (in kbps) 0 20 40 60 80 100 Relative Time Complexity (in percentage) 540p 1080p 2160p As resolution doubles, encoding time complexity doubles! Many encoder analysis decisions are redundant across the representations. Multi-rate: Exploit this redundancy across representations of a resolution. Multi-resolution: Exploit this redundancy across resolutions. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 5
  • 6. State-of-the-art: Multi-Rate Encoding State-of-the-art: Multi-Rate Encoding Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 6
  • 7. State-of-the-art: Multi-Rate Encoding Related Works: Multi-Rate Encoding Single-bound2 and Double-bound3 for CU depth estimation 3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps (a) Single-bound for CU depth estimation 3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps (b) Double-bound for CU depth estimation Motivation: CUs generally tend to have higher depth values (smaller CU sizes) in higher bitrate representations. CU depth information from the highest bitrate representation is used as the upper bound of the CU depth estimation of dependent representations. In double-bound, additionally, CU depth information from the lowest bitrate representation is used as the lower bound of the CU depth estimation of dependent representations. 2 D. Schroeder, P. Rehm, and E. Steinbach. “Block structure reuse for multi-rate high efficiency video coding”. In: 2015 IEEE International Conference on Image Processing (ICIP). Sept. 2015, pp. 3972–3976. 3 H. Amirpour et al. “Fast Multi-rate Encoding for Adaptive HTTP Streaming”. In: 2020 Data Compression Conference (DCC). 2020, pp. 358–358. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 7
  • 8. State-of-the-art: Multi-Rate Encoding Related Works: Multi-Rate Encoding Integrated x265 mode-1 and x265 mode-24 Integrated x265 mode-1 Force the optimal decisions made in the highest bitrate representation the slice-type scene-cut decisions the quadtree structure prediction modes reference indices Integrated x265 mode-2 Reuse all CUs and PUs from the highest bitrate representation without additional RDO. They also employ static refinement techniques defined in x265. If the reference depth is d, it is re- evaluated by using the RDO against the cost of splitting the CU (i.e., depth d + 1). The motion vectors from the highest bi- trate representation are used as Motion vector predictors (MVPs). 4 K. Goswami et al. “Adaptive Multi-Resolution Encoding for ABR Streaming”. In: 2018 25th IEEE International Conference on Image Processing (ICIP). 2018, pp. 1008–1012. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 8
  • 9. State-of-the-art: Multi-Encoding State-of-the-art: Multi-Encoding Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 9
  • 10. State-of-the-art: Multi-Encoding Related Work: Multi-Encoding 1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps 1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps 3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps Scaled CU,PU reuse + refinement Figure: Encoder analysis sharing method in x265 Algorithm.5 5 Aruna Mathesawaran et al. “Open source framework for reduced-complexity multi-rate HEVC encoding”. In: Applications of Digital Image Processing XLIII. ed. by Andrew G. Tescher and Touradj Ebrahimi. Vol. 11510. International Society for Optics and Photonics. SPIE, 2020, pp. 461 –471. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 10
  • 11. Efficient Multi-Rate Encoding Efficient Multi-Rate Encoding Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 11
  • 12. Efficient Multi-Rate Encoding Efficient Multi-Rate Encoding Prediction Mode Heuristics Prediction Mode Heuristics If the CU depth chosen for the dependent representations is the same as the reference represen- tation, the following heuristics are proposed. If Skip mode was chosen in the highest bitrate representation, RDO is evaluated for only Merge/Skip modes. If 2Nx2N mode was chosen in the highest bitrate representation, skip RDO for AMP modes. If inter-prediction mode was chosen in the highest bitrate representation, skip RDO for intra-prediction modes. In double-bound for CU estimation, if the intra-prediction mode was chosen for the highest and lowest bitrate representations, RDO is evaluated for only intra-prediction modes in the intermediate representations. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 12
  • 13. Efficient Multi-Rate Encoding Efficient Multi-Rate Encoding Motion Estimation Heuristics Motion Estimation Heuristics If the CU depth and PU chosen for the dependent representations is the same as the reference representation, the following heuristics are proposed. Force the same reference frame as that of the highest bitrate representation. The Motion Vector Predictor (MVP) is set to be the Motion Vector (MV) of the highest bitrate representation. In double-bound for CU estimation, the motion search range is decreased to a smaller window if the MV of the highest bitrate representation and the MV of the lowest bitrate representation are close to each other. The search range is determined to be the maximum difference between the x coordinates and y coordinates of the MVs. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 13
  • 14. Efficient Multi-Rate Encoding Efficient Multi-Rate Encoding Proposed Multi-rate Algorithm-1 and Algorithm-2 3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps (a) Proposed Multi-rate Algorithm-1 3840x2160 @ 25Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20Mbps 3840x2160 @ 16.8Mbps (b) Proposed Multi-rate Algorithm-2 − > denotes analysis sharing using single-bound for CU depth estimation and proposed heuristics. − > denotes analysis sharing using double-bound for CU depth estimation and proposed heuristics. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 14
  • 15. Efficient Multi-Encoding Efficient Multi-Encoding Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 15
  • 16. Efficient Multi-Encoding Efficient Multi-Encoding Proposed Multi-Encoding Algorithm-1 1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps 1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps 3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps Figure: Encoder analysis sharing method in Proposed Multi-Encoding Algorithm-1. Multi-rate Approach: Proposed Multi-rate Algorithm-2 Multi-resolution Approach: scaled CU information from highest bitrate representation of the previous resolution is used as lower bound of CU depth estimation process. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 16
  • 17. Efficient Multi-Encoding Efficient Multi-Encoding Proposed Multi-Encoding Algorithm-2 1920x1080 @ 7.0Mbps 1920x1080 @ 3.0Mbps 1920x1080 @ 5.8Mbps 1920x1080 @ 4.5Mbps 3840x2160 @ 25.0Mbps 3840x2160 @ 11.6Mbps 3840x2160 @ 20.0Mbps 3840x2160 @ 16.8Mbps Scaling + Single-bound Figure: Encoder analysis sharing method in Proposed Multi-Encoding Algorithm-2. Multi-rate Approach: Proposed Multi-rate Algorithm-2 Multi-resolution Approach: scaled CU information from lowest bitrate representation of the previous resolution is used as lower bound of CU depth estimation process. More rigid bound compared to Multi-encoding Algorithm-1. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 17
  • 18. Experimental Evaluation Experimental Evaluation Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 18
  • 19. Experimental Evaluation Experimental Evaluation Test Setup Algorithms presented here are tested using x265 v3.46 with the veryslow preset (highest quality preset practically usable of x265). Test PC: Dual-processor server with Intel Xeon Gold 5218R (80 cores, frequency at 2.10 GHz). Test Sequences: (i) 4 JVET test sequences7 and (ii) 3 professionally produced UltraHD HDR cinematic content.8 The lowest resolution is 960 × 540, intermediate resolution is 1920 × 1080, and the highest resolution is 3840 × 2160. The lower resolution sources were generated from the original video source by applying bi-cubic scaling using FFmpeg.9 6 MulticoreWare Inc. x265 HEVC Encoder/H.265 Video Codec. url: http://x265.org/. 7 Jill Boyce et al. JVET-J1010: JVET common test conditions and software reference configurations. July 2018. 8 M. H. Pinson. “The Consumer Digital Video Library [Best of the Web]”. In: IEEE Signal Processing Magazine 30.4 (2013), pp. 172–174. doi: 10.1109/MSP.2013.2258265. 9 FFmpeg. FFmpeg Documentation. url: https://ffmpeg.org/ffmpeg.html. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 19
  • 20. Experimental Evaluation Experimental Evaluation Test Setup Table: Bitrate representations Bi,j (in kbps) based on HLS authoring specification10 Layer (i) Resolution j = 1 j = 2 j = 3 j = 4 1 960x540 2000 1500 1000 500 2 1920x1080 7000 5800 4500 3000 3 3840x2160 25000 20000 16800 11600 Performance evaluation Metrics: ∆T : overall encoding time saving BDRP : Average difference in bitrate with respect to reference encode to maintain the same PSNR BDRV : Average difference in bitrate with respect to reference encode to maintain the same VMAF BDRP ∆T : Trade-off between BDR and ∆T. 10 https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 20
  • 21. Experimental Evaluation Experimental Evaluation Multi-rate Encoding Table: Results for the multi-rate algorithms Method ∆T BDRP BDRP ∆T BDRV BDRV ∆T Single-bound for CU estimation 14.50% -0.59% -4.39% 0.10% 1.08% Double-bound for CU estimation 26.37% 0.73% 3.44% 1.25% 5.54% x265 Algorithm-1 18.61% 1.67% 9.52% 1.56% 8.85% x265 Algorithm-2 55.11% 8.42% 15.62% 8.72% 16.10% Multi-rate Algorithm-1 (ours) 26.50% -0.28% -0.84% 0.43% 2.31% Multi-rate Algorithm-2 (ours) 37.37% 1.06% 3.36% 1.57% 4.88% Multi-rate Algorithm-1 improves ∆T of Single-bound for CU estimation by 12% with neg- ligible increase in BDR. Multi-rate Algorithm-2 improves ∆T of Double-bound for CU estimation by 11% with negligible increase in BDR. Better BDR ∆T metric Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 21
  • 22. Experimental Evaluation Experimental Evaluation Multi-Encoding Scheme ∆T BDRP BDRV State-of-the-art scheme 80.05% 13.53% 9.59% Multi-encoding Scheme-1 39.72% 2.32% 1.55% Multi-encoding Scheme-2 50.90% 3.45% 2.63% State-of-the-art Multi-encoding Algorithm-1 Multi-encoding Algorithm-2 0 2 4 6 8 10 12 14 16 BDR/ T (in percentage) BDRP/ T BDRV/ T State-of-the-art algorithm yields the highest ∆T. Multi-encoding Algorithm-1 has the lowest increase in BDR and the lowest BDR ∆T metric. Multi-encoding Algorithm-2 improves ∆T of Multi-encoding Algorithm-1 by 11% with neg- ligible increase in BDR. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 22
  • 23. Conclusions and Future Directions Conclusions and Future Directions Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 23
  • 24. Conclusions and Future Directions Conclusions and Future Directions Conclusions Presented efficient multi-encoding algorithms for HTTP Adaptive Streaming, tested with the open-source x265 HEVC encoder. Multi-rate Encoding: Proposed heuristics improved the encoding speed of the single-bound and double-bound for CU estimation by an average of 12% and 11%, respectively with a negligible increase in BD-Rate. BDR ∆T metric for proposed algorithms is better compared to the state-of-the-art algorithms. Multi-encoding: Proposed two algorithms that extended the proposed multi-rate encoding algorithms across resolutions. BDR ∆T metric for proposed algorithms are better compared to the state-of-the-art algorithm. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 24
  • 25. Conclusions and Future Directions Conclusions and Future Directions Future Directions The encoding speedup can be improved by parallelizing the execution of encoding instances. The dependent representations can be started after a delay of one frame execution time of the reference representation. Further work is required to extend the algorithms to resolutions, which are non-integer power-of-2 multiples of each other. Further work is required to investigate more recent codecs, e.g., VVC. Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 25
  • 26. Conclusions and Future Directions Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@aau.at) Hadi Amirpour (hadi.amirpourazarian@aau.at) Christian Timmerer (Christian.Timmerer@aau.at) Mohammad Ghanbari (ghan@essex.ac.uk) Vignesh V Menon Efficient Multi-Encoding Algorithms for HTTP Adaptive Bitrate Streaming 26