SlideShare a Scribd company logo
Perceptually-aware Live VBR Encoding Scheme for Adaptive
Video Streaming
Vignesh V Menon1, Prajit T Rajendran2, Christian Feldmann3
1
Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität, Klagenfurt, Austria
2
CEA, List, F-91120 Palaiseau, Université Paris-Saclay, France
3
Bitmovin, Klagenfurt, Austria
10 Oct 2022
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 1
Outline
1 Introduction
2 Research Goal
3 Live-VBR scheme
4 Results
5 Conclusion
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 2
Introduction
Introduction
HTTP Adaptive Streaming (HAS)1
Why Adaptive Streaming?
Adapt for a wide range of devices.
Adapt for a broad set of Internet speeds.
What HAS does?
Each source video is split into segments.
Encoded at multiple bitrates, resolutions, and codecs.
Delivered to the client based on the device capability, network speed etc.
1
A. Bentaleb et al. “A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP”. In: IEEE Communications Surveys Tutorials 21.1 (2019),
pp. 562–585. doi: 10.1109/COMST.2018.2862938.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 3
Introduction
Introduction
Motivation for Per-title bitrate ladder in Adaptive Streaming
The video sequences are encoded at a fixed set of bitrates and resolutions (referred to as the
bitrate ladder) to adapt to the network conditions of end-users.
Figure: Rate-Distortion (RD) curves of the Constant Bitrate (CBR) encoding of RushHour s000 and
YachtRide s000 video sequences (segments) of VCD dataset2
encoded at 1080p and 2160p resolutions using
x265 HEVC encoder at ultrafast preset. Here, VMAF is used as the quality metric.
2
Hadi Amirpour et al. “VCD: Video Complexity Dataset”. In: Proceedings of the 13th ACM Multimedia Systems Conference. MMSys ’22. Athlone, Ireland:
Association for Computing Machinery, 2022, 234–239. isbn: 9781450392839. doi: 10.1145/3524273.3532892. url:
https://doi.org/10.1145/3524273.3532892.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 4
Introduction
Introduction
Per-title Encoding
Though per-title encoding schemes3,4,5 enhance the quality of video delivery, determining
the convex-hull is computationally costly, making it suitable for only VoD streaming appli-
cations.
The plethora of live streaming applications call for low latency approaches to optimize video
coding.
According to the Bitmovin Video Developer Report 20216, live (low) latency is the biggest
challenge in video technology today.
3
Jan De Cock et al. “Complexity-based consistent-quality encoding in the cloud”. In: 2016 IEEE International Conference on Image Processing (ICIP). 2016,
pp. 1484–1488. doi: 10.1109/ICIP.2016.7532605.
4
Madhukar Bhat, Jean-Marc Thiesse, and Patrick Le Callet. “A Case Study of Machine Learning Classifiers for Real-Time Adaptive Resolution Prediction in
Video Coding”. In: 2020 IEEE International Conference on Multimedia and Expo (ICME). 2020, pp. 1–6. doi: 10.1109/ICME46284.2020.9102934.
5
Daniel Silhavy et al. “Machine Learning for Per-Title Encoding”. In: SMPTE Motion Imaging Journal 131.3 (2022), pp. 42–50. doi:
10.5594/JMI.2022.3154836.
6
https://go.bitmovin.com/video-developer-report, last access: Sep 15, 2022.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 5
Introduction
Introduction
Motivation for perceptually-aware bitrate ladder
Figure: RD curve of the HLS CBR encoding of Characters s000 video sequence (segment) of VCD dataset using
x265 HEVC encoder at ultrafast preset. The points with a bitrate greater than 3.6 Mbps are in the perceptually
lossless region. Hence, there is significant storage wastage while storing these representations.
Selecting similar-quality representations for the bitrate ladder does not result in improved QoE,
but it will lead to increased storage and bandwidth costs!
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 6
Introduction
Introduction
Motivation for two-pass encoding (CBR versus VBR)
Figure: Constant Bitrate (CBR) versus Variable
Bitrate (VBR) encoding.
In live streaming, Constant Bitrate (CBR) rate-
control mode is used to encode video sequences
at a fixed bitrate ladder. The consistency of
CBR makes it more reliable for time-sensitive
data transport.
In this method, there is no concern about the
bitrate exceeding internet speeds. However, this
method may result in low compression efficiency.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 7
Introduction
Introduction
Constrained Variable Bitrate (cVBR) encoding
Figure: cVBR encoding.
A ”rate factor” first-pass to identify the optimized CRF to achieve the target bitrate.
In the second-pass, the segment is encoded with the selected optimized CRF with the
maximum bitrate and maximum buffer window constraints.
The desired target bitrate is achieved with maximum compression efficiency, and min-
imum quality fluctuation.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 8
Research Goal
Research Goal
b0 b1 b2 b3 b4 b5 b6
Bitrate
v0
v1=v0 + vJ(v0)
v2=v1 + vJ(v1)
v3=v2 + vJ(v2)
v4=v3 + vJ(v3)
v5=v4 + vJ(v4)
v6=v5 + vJ(v5)
VMAF
vmax
r1
r0
r2
r3
r4
r5
r6
Figure: The ideal perceptually-aware bi-
trate ladder envisioned in this work.
Here, vJ (v0) = vJ (v1) = vJ (vM−1) =
∆VMAF
Joint optimization:
Perceptual difference of pre-defined ∆VMAF be-
tween representations.
Minimize bitrate difference between representations.
Maximize compression efficiency of representations.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 9
Live-VBR scheme
Workflow of Live-VBR
Figure: Live-VBR system envisioned in this work.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 10
Live-VBR scheme
Live-VBR
First point of the bitrate ladder
Figure: Estimation of the first point of the bi-
trate ladder. v0 is the maximum value among
the vr,b0
values output from the predicted mod-
els trained for resolutions r0, r1, .., rM−1. The
resolution corresponding to the VMAF v0 is
chosen as r0.
b0 = bmin
Determine vr,b0 ∀r ∈ R
v0 = max(vr,b0 )
r0 = arg maxr∈R(vr,b0 )
(r0, b0) is the first point of the bitrate ladder
Note
This part of the algorithm needs VMAF predic-
tion for all considered resolutions.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 11
Live-VBR scheme
Live-VBR
Remaining points of the bitrate ladder
Figure: Estimation of the (t + 1)th
point of
the bitrate ladder. bt is the minimum value
among the br,vt
values output from the pre-
dicted models trained for resolutions r0, r1, ..,
rM−1. The resolution corresponding to the bi-
trate bt is chosen as rt.
t = 1
for t ≥ 1 do
vt = vt−1 + ∆VMAF
Determine br,vt ∀r ∈ R
bt = min(br,vt )
rt = arg minr∈R(br,vt )
if bt > bmax or vt > vmax then
End of the algorithm
else
(rt, bt) is the (t + 1)th
point of the
bitrate ladder.
t = t + 1
Note
This part of the algorithm needs bitrate predic-
tion for all considered resolutions.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 12
Live-VBR scheme
Live-VBR
cVBR encoding of the bitrate ladder
Figure: Estimation of the optimized CRF to achieve the target bitrate b using a prediction model trained
for resolution r.
Optimized CRF is determined for the selected (r, b) pairs.
cVBR encoding for the (r, b, CRF) pairs is performed.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 13
Results
Results
Prediction accuracy of the models
Table: R2
score and MAE of the prediction models for various resolutions.
R2 score MAE
r 360p 432p 540p 720p 1080p 1440p 2160p 360p 432p 540p 720p 1080p 1440p 2160p
VMAF 0.821 0.852 0.882 0.906 0.910 0.906 0.930 4.860 4.899 4.832 4.393 3.838 3.490 2.941
log(b) 0.859 0.864 0.888 0.915 0.932 0.937 0.943 0.765 0.751 0.737 0.709 0.711 0.706 0.681
CRF 0.969 0.969 0.970 0.969 0.968 0.967 0.965 1.924 1.920 1.914 1.942 1.940 1.972 1.990
Note
Just three values (E, h, L) are used as the measure of video complexity. If we increase the
information measure, e.g., block-wise features), the accuracy can be improved further.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 14
Results
Results
RD plots of Live-VBR using x265
Figure: Bunny s000(E = 22.40, h = 4.70) Figure: Characters s000(E = 45.42, h = 36.88)
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 15
Results
Results
RD plots of Live-VBR using x265
Figure: Eldorado s005(E = 100.37, h = 9.23) Figure: Wood s000 (E = 124.72, h = 47.03)
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 16
Results
Results
Summary
Table: Average results of the encoding schemes compared to the HLS CBR encoding using x265 HEVC
encoder.
Method BDRP BDRV BD-PSNR BD-VMAF ∆S ∆E
Ground truth (∆VMAF=2) -23.09% -43.23% 1.34 dB 10.61 -25.99% 89.54%
Ground truth (∆VMAF=4) -28.15% -42.75% 1.70 dB 10.08 -59.07% -0.54%
Ground truth (∆VMAF=6) -25.36% -40.73% 1.67 dB 9.19 -70.50% -31.24%
Live-VBR (∆VMAF=2) -14.25% -29.14% 1.36 dB 7.82 23.57% 90.19%
Live-VBR (∆VMAF=4) -18.41% -32.48% 1.41 dB 8.31 -56.38% 0.34%
Live-VBR (∆VMAF=6) -18.80% -32.59% 1.34 dB 8.34 -68.96% -28.25%
Relative storage difference
∆S =
P
bopt
P
bref
− 1
Relative energy utilization difference
∆E =
P
E(bopt )
P
E(bref ) − 1
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 17
Conclusion
Conclusion
Presented an application of video complexity analysis, where VMAF, target bitrate, CRF
are predicted using video complexity features.
Live-VBR shall be released as an open-source software by Christmas, 2022!
GNU GPL 3.0
VCA, target encoders as sub-repositories.
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 18
Q & A
Q & A
Thank you for your attention!
Vignesh V Menon (vignesh.menon@aau.at)
Prajit T Rajendran (prajit.thazhurazhikath@cea.fr)
Christian Feldmann (christian.feldmann@bitmovin.com)
Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 19

More Related Content

Similar to LiveVBR presentation at VQEG NORM.pdf

TQPM.pdf
TQPM.pdfTQPM.pdf
TQPM.pdf
Vignesh V Menon
 
Video streaming using light-weight transcoding and in-network intelligence
Video streaming using light-weight transcoding and in-network intelligenceVideo streaming using light-weight transcoding and in-network intelligence
Video streaming using light-weight transcoding and in-network intelligence
Minh Nguyen
 
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive StreamingMiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
Alpen-Adria-Universität
 
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Vignesh V Menon
 
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Alpen-Adria-Universität
 
VCIP_MCBE_presentation.pdf
VCIP_MCBE_presentation.pdfVCIP_MCBE_presentation.pdf
VCIP_MCBE_presentation.pdf
Vignesh V Menon
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Alpen-Adria-Universität
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
Vignesh V Menon
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
Vignesh V Menon
 
Online Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingOnline Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC Streaming
Vignesh V Menon
 
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
IJECEIAES
 
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
IJECEIAES
 
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
Alpen-Adria-Universität
 
Tutorial High Efficiency Video Coding Coding - Tools and Specification.pdf
Tutorial High Efficiency Video Coding Coding - Tools and Specification.pdfTutorial High Efficiency Video Coding Coding - Tools and Specification.pdf
Tutorial High Efficiency Video Coding Coding - Tools and Specification.pdf
ssuserc5a4dd
 
How AI research is enabling next-gen codecs
How AI research is enabling next-gen codecsHow AI research is enabling next-gen codecs
How AI research is enabling next-gen codecs
Qualcomm Research
 
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdfETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
Vignesh V Menon
 
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
Alpen-Adria-Universität
 
Introduction to Video Compression Techniques - Anurag Jain
Introduction to Video Compression Techniques - Anurag JainIntroduction to Video Compression Techniques - Anurag Jain
Introduction to Video Compression Techniques - Anurag Jain
Videoguy
 
Green Variable framerate encoding for Adaptive Live Streaming
Green Variable framerate encoding  for Adaptive Live StreamingGreen Variable framerate encoding  for Adaptive Live Streaming
Green Variable framerate encoding for Adaptive Live Streaming
Vignesh V Menon
 
Paper id 2120148
Paper id 2120148Paper id 2120148
Paper id 2120148
IJRAT
 

Similar to LiveVBR presentation at VQEG NORM.pdf (20)

TQPM.pdf
TQPM.pdfTQPM.pdf
TQPM.pdf
 
Video streaming using light-weight transcoding and in-network intelligence
Video streaming using light-weight transcoding and in-network intelligenceVideo streaming using light-weight transcoding and in-network intelligence
Video streaming using light-weight transcoding and in-network intelligence
 
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive StreamingMiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
 
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
 
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low ...
 
VCIP_MCBE_presentation.pdf
VCIP_MCBE_presentation.pdfVCIP_MCBE_presentation.pdf
VCIP_MCBE_presentation.pdf
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
 
Online Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingOnline Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC Streaming
 
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
 
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
 
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
 
Tutorial High Efficiency Video Coding Coding - Tools and Specification.pdf
Tutorial High Efficiency Video Coding Coding - Tools and Specification.pdfTutorial High Efficiency Video Coding Coding - Tools and Specification.pdf
Tutorial High Efficiency Video Coding Coding - Tools and Specification.pdf
 
How AI research is enabling next-gen codecs
How AI research is enabling next-gen codecsHow AI research is enabling next-gen codecs
How AI research is enabling next-gen codecs
 
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdfETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf
 
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
 
Introduction to Video Compression Techniques - Anurag Jain
Introduction to Video Compression Techniques - Anurag JainIntroduction to Video Compression Techniques - Anurag Jain
Introduction to Video Compression Techniques - Anurag Jain
 
Green Variable framerate encoding for Adaptive Live Streaming
Green Variable framerate encoding  for Adaptive Live StreamingGreen Variable framerate encoding  for Adaptive Live Streaming
Green Variable framerate encoding for Adaptive Live Streaming
 
Paper id 2120148
Paper id 2120148Paper id 2120148
Paper id 2120148
 

More from Vignesh V Menon

Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Vignesh V Menon
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Vignesh V Menon
 
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdfContent_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Vignesh V Menon
 
Doctoral Symposium presentation.pdf
Doctoral Symposium presentation.pdfDoctoral Symposium presentation.pdf
Doctoral Symposium presentation.pdf
Vignesh V Menon
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
Vignesh V Menon
 
Video Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVideo Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdf
Vignesh V Menon
 
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingLive-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Vignesh V Menon
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
Vignesh V Menon
 
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
Vignesh V Menon
 
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
Vignesh V Menon
 

More from Vignesh V Menon (10)

Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
 
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdfContent_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
 
Doctoral Symposium presentation.pdf
Doctoral Symposium presentation.pdfDoctoral Symposium presentation.pdf
Doctoral Symposium presentation.pdf
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
 
Video Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVideo Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdf
 
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingLive-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
 
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
 
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
 

Recently uploaded

Lapbook sobre os Regimes Totalitários.pdf
Lapbook sobre os Regimes Totalitários.pdfLapbook sobre os Regimes Totalitários.pdf
Lapbook sobre os Regimes Totalitários.pdf
Jean Carlos Nunes Paixão
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
chanes7
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
Jean Carlos Nunes Paixão
 
The simplified electron and muon model, Oscillating Spacetime: The Foundation...
The simplified electron and muon model, Oscillating Spacetime: The Foundation...The simplified electron and muon model, Oscillating Spacetime: The Foundation...
The simplified electron and muon model, Oscillating Spacetime: The Foundation...
RitikBhardwaj56
 
How to Fix the Import Error in the Odoo 17
How to Fix the Import Error in the Odoo 17How to Fix the Import Error in the Odoo 17
How to Fix the Import Error in the Odoo 17
Celine George
 
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
National Information Standards Organization (NISO)
 
Chapter 4 - Islamic Financial Institutions in Malaysia.pptx
Chapter 4 - Islamic Financial Institutions in Malaysia.pptxChapter 4 - Islamic Financial Institutions in Malaysia.pptx
Chapter 4 - Islamic Financial Institutions in Malaysia.pptx
Mohd Adib Abd Muin, Senior Lecturer at Universiti Utara Malaysia
 
How to Manage Your Lost Opportunities in Odoo 17 CRM
How to Manage Your Lost Opportunities in Odoo 17 CRMHow to Manage Your Lost Opportunities in Odoo 17 CRM
How to Manage Your Lost Opportunities in Odoo 17 CRM
Celine George
 
Liberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdfLiberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdf
WaniBasim
 
Hindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdfHindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdf
Dr. Mulla Adam Ali
 
Your Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective UpskillingYour Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective Upskilling
Excellence Foundation for South Sudan
 
Azure Interview Questions and Answers PDF By ScholarHat
Azure Interview Questions and Answers PDF By ScholarHatAzure Interview Questions and Answers PDF By ScholarHat
Azure Interview Questions and Answers PDF By ScholarHat
Scholarhat
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
Nguyen Thanh Tu Collection
 
Smart-Money for SMC traders good time and ICT
Smart-Money for SMC traders good time and ICTSmart-Money for SMC traders good time and ICT
Smart-Money for SMC traders good time and ICT
simonomuemu
 
Digital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental DesignDigital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental Design
amberjdewit93
 
PIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf IslamabadPIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf Islamabad
AyyanKhan40
 
Types of Herbal Cosmetics its standardization.
Types of Herbal Cosmetics its standardization.Types of Herbal Cosmetics its standardization.
Types of Herbal Cosmetics its standardization.
Ashokrao Mane college of Pharmacy Peth-Vadgaon
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
Nicholas Montgomery
 
Introduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp NetworkIntroduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp Network
TechSoup
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
Peter Windle
 

Recently uploaded (20)

Lapbook sobre os Regimes Totalitários.pdf
Lapbook sobre os Regimes Totalitários.pdfLapbook sobre os Regimes Totalitários.pdf
Lapbook sobre os Regimes Totalitários.pdf
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
 
The simplified electron and muon model, Oscillating Spacetime: The Foundation...
The simplified electron and muon model, Oscillating Spacetime: The Foundation...The simplified electron and muon model, Oscillating Spacetime: The Foundation...
The simplified electron and muon model, Oscillating Spacetime: The Foundation...
 
How to Fix the Import Error in the Odoo 17
How to Fix the Import Error in the Odoo 17How to Fix the Import Error in the Odoo 17
How to Fix the Import Error in the Odoo 17
 
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
Pollock and Snow "DEIA in the Scholarly Landscape, Session One: Setting Expec...
 
Chapter 4 - Islamic Financial Institutions in Malaysia.pptx
Chapter 4 - Islamic Financial Institutions in Malaysia.pptxChapter 4 - Islamic Financial Institutions in Malaysia.pptx
Chapter 4 - Islamic Financial Institutions in Malaysia.pptx
 
How to Manage Your Lost Opportunities in Odoo 17 CRM
How to Manage Your Lost Opportunities in Odoo 17 CRMHow to Manage Your Lost Opportunities in Odoo 17 CRM
How to Manage Your Lost Opportunities in Odoo 17 CRM
 
Liberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdfLiberal Approach to the Study of Indian Politics.pdf
Liberal Approach to the Study of Indian Politics.pdf
 
Hindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdfHindi varnamala | hindi alphabet PPT.pdf
Hindi varnamala | hindi alphabet PPT.pdf
 
Your Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective UpskillingYour Skill Boost Masterclass: Strategies for Effective Upskilling
Your Skill Boost Masterclass: Strategies for Effective Upskilling
 
Azure Interview Questions and Answers PDF By ScholarHat
Azure Interview Questions and Answers PDF By ScholarHatAzure Interview Questions and Answers PDF By ScholarHat
Azure Interview Questions and Answers PDF By ScholarHat
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
 
Smart-Money for SMC traders good time and ICT
Smart-Money for SMC traders good time and ICTSmart-Money for SMC traders good time and ICT
Smart-Money for SMC traders good time and ICT
 
Digital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental DesignDigital Artefact 1 - Tiny Home Environmental Design
Digital Artefact 1 - Tiny Home Environmental Design
 
PIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf IslamabadPIMS Job Advertisement 2024.pdf Islamabad
PIMS Job Advertisement 2024.pdf Islamabad
 
Types of Herbal Cosmetics its standardization.
Types of Herbal Cosmetics its standardization.Types of Herbal Cosmetics its standardization.
Types of Herbal Cosmetics its standardization.
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
 
Introduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp NetworkIntroduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp Network
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
 

LiveVBR presentation at VQEG NORM.pdf

  • 1. Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming Vignesh V Menon1, Prajit T Rajendran2, Christian Feldmann3 1 Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität, Klagenfurt, Austria 2 CEA, List, F-91120 Palaiseau, Université Paris-Saclay, France 3 Bitmovin, Klagenfurt, Austria 10 Oct 2022 Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 1
  • 2. Outline 1 Introduction 2 Research Goal 3 Live-VBR scheme 4 Results 5 Conclusion Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 2
  • 3. Introduction Introduction HTTP Adaptive Streaming (HAS)1 Why Adaptive Streaming? Adapt for a wide range of devices. Adapt for a broad set of Internet speeds. What HAS does? Each source video is split into segments. Encoded at multiple bitrates, resolutions, and codecs. Delivered to the client based on the device capability, network speed etc. 1 A. Bentaleb et al. “A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP”. In: IEEE Communications Surveys Tutorials 21.1 (2019), pp. 562–585. doi: 10.1109/COMST.2018.2862938. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 3
  • 4. Introduction Introduction Motivation for Per-title bitrate ladder in Adaptive Streaming The video sequences are encoded at a fixed set of bitrates and resolutions (referred to as the bitrate ladder) to adapt to the network conditions of end-users. Figure: Rate-Distortion (RD) curves of the Constant Bitrate (CBR) encoding of RushHour s000 and YachtRide s000 video sequences (segments) of VCD dataset2 encoded at 1080p and 2160p resolutions using x265 HEVC encoder at ultrafast preset. Here, VMAF is used as the quality metric. 2 Hadi Amirpour et al. “VCD: Video Complexity Dataset”. In: Proceedings of the 13th ACM Multimedia Systems Conference. MMSys ’22. Athlone, Ireland: Association for Computing Machinery, 2022, 234–239. isbn: 9781450392839. doi: 10.1145/3524273.3532892. url: https://doi.org/10.1145/3524273.3532892. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 4
  • 5. Introduction Introduction Per-title Encoding Though per-title encoding schemes3,4,5 enhance the quality of video delivery, determining the convex-hull is computationally costly, making it suitable for only VoD streaming appli- cations. The plethora of live streaming applications call for low latency approaches to optimize video coding. According to the Bitmovin Video Developer Report 20216, live (low) latency is the biggest challenge in video technology today. 3 Jan De Cock et al. “Complexity-based consistent-quality encoding in the cloud”. In: 2016 IEEE International Conference on Image Processing (ICIP). 2016, pp. 1484–1488. doi: 10.1109/ICIP.2016.7532605. 4 Madhukar Bhat, Jean-Marc Thiesse, and Patrick Le Callet. “A Case Study of Machine Learning Classifiers for Real-Time Adaptive Resolution Prediction in Video Coding”. In: 2020 IEEE International Conference on Multimedia and Expo (ICME). 2020, pp. 1–6. doi: 10.1109/ICME46284.2020.9102934. 5 Daniel Silhavy et al. “Machine Learning for Per-Title Encoding”. In: SMPTE Motion Imaging Journal 131.3 (2022), pp. 42–50. doi: 10.5594/JMI.2022.3154836. 6 https://go.bitmovin.com/video-developer-report, last access: Sep 15, 2022. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 5
  • 6. Introduction Introduction Motivation for perceptually-aware bitrate ladder Figure: RD curve of the HLS CBR encoding of Characters s000 video sequence (segment) of VCD dataset using x265 HEVC encoder at ultrafast preset. The points with a bitrate greater than 3.6 Mbps are in the perceptually lossless region. Hence, there is significant storage wastage while storing these representations. Selecting similar-quality representations for the bitrate ladder does not result in improved QoE, but it will lead to increased storage and bandwidth costs! Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 6
  • 7. Introduction Introduction Motivation for two-pass encoding (CBR versus VBR) Figure: Constant Bitrate (CBR) versus Variable Bitrate (VBR) encoding. In live streaming, Constant Bitrate (CBR) rate- control mode is used to encode video sequences at a fixed bitrate ladder. The consistency of CBR makes it more reliable for time-sensitive data transport. In this method, there is no concern about the bitrate exceeding internet speeds. However, this method may result in low compression efficiency. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 7
  • 8. Introduction Introduction Constrained Variable Bitrate (cVBR) encoding Figure: cVBR encoding. A ”rate factor” first-pass to identify the optimized CRF to achieve the target bitrate. In the second-pass, the segment is encoded with the selected optimized CRF with the maximum bitrate and maximum buffer window constraints. The desired target bitrate is achieved with maximum compression efficiency, and min- imum quality fluctuation. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 8
  • 9. Research Goal Research Goal b0 b1 b2 b3 b4 b5 b6 Bitrate v0 v1=v0 + vJ(v0) v2=v1 + vJ(v1) v3=v2 + vJ(v2) v4=v3 + vJ(v3) v5=v4 + vJ(v4) v6=v5 + vJ(v5) VMAF vmax r1 r0 r2 r3 r4 r5 r6 Figure: The ideal perceptually-aware bi- trate ladder envisioned in this work. Here, vJ (v0) = vJ (v1) = vJ (vM−1) = ∆VMAF Joint optimization: Perceptual difference of pre-defined ∆VMAF be- tween representations. Minimize bitrate difference between representations. Maximize compression efficiency of representations. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 9
  • 10. Live-VBR scheme Workflow of Live-VBR Figure: Live-VBR system envisioned in this work. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 10
  • 11. Live-VBR scheme Live-VBR First point of the bitrate ladder Figure: Estimation of the first point of the bi- trate ladder. v0 is the maximum value among the vr,b0 values output from the predicted mod- els trained for resolutions r0, r1, .., rM−1. The resolution corresponding to the VMAF v0 is chosen as r0. b0 = bmin Determine vr,b0 ∀r ∈ R v0 = max(vr,b0 ) r0 = arg maxr∈R(vr,b0 ) (r0, b0) is the first point of the bitrate ladder Note This part of the algorithm needs VMAF predic- tion for all considered resolutions. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 11
  • 12. Live-VBR scheme Live-VBR Remaining points of the bitrate ladder Figure: Estimation of the (t + 1)th point of the bitrate ladder. bt is the minimum value among the br,vt values output from the pre- dicted models trained for resolutions r0, r1, .., rM−1. The resolution corresponding to the bi- trate bt is chosen as rt. t = 1 for t ≥ 1 do vt = vt−1 + ∆VMAF Determine br,vt ∀r ∈ R bt = min(br,vt ) rt = arg minr∈R(br,vt ) if bt > bmax or vt > vmax then End of the algorithm else (rt, bt) is the (t + 1)th point of the bitrate ladder. t = t + 1 Note This part of the algorithm needs bitrate predic- tion for all considered resolutions. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 12
  • 13. Live-VBR scheme Live-VBR cVBR encoding of the bitrate ladder Figure: Estimation of the optimized CRF to achieve the target bitrate b using a prediction model trained for resolution r. Optimized CRF is determined for the selected (r, b) pairs. cVBR encoding for the (r, b, CRF) pairs is performed. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 13
  • 14. Results Results Prediction accuracy of the models Table: R2 score and MAE of the prediction models for various resolutions. R2 score MAE r 360p 432p 540p 720p 1080p 1440p 2160p 360p 432p 540p 720p 1080p 1440p 2160p VMAF 0.821 0.852 0.882 0.906 0.910 0.906 0.930 4.860 4.899 4.832 4.393 3.838 3.490 2.941 log(b) 0.859 0.864 0.888 0.915 0.932 0.937 0.943 0.765 0.751 0.737 0.709 0.711 0.706 0.681 CRF 0.969 0.969 0.970 0.969 0.968 0.967 0.965 1.924 1.920 1.914 1.942 1.940 1.972 1.990 Note Just three values (E, h, L) are used as the measure of video complexity. If we increase the information measure, e.g., block-wise features), the accuracy can be improved further. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 14
  • 15. Results Results RD plots of Live-VBR using x265 Figure: Bunny s000(E = 22.40, h = 4.70) Figure: Characters s000(E = 45.42, h = 36.88) Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 15
  • 16. Results Results RD plots of Live-VBR using x265 Figure: Eldorado s005(E = 100.37, h = 9.23) Figure: Wood s000 (E = 124.72, h = 47.03) Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 16
  • 17. Results Results Summary Table: Average results of the encoding schemes compared to the HLS CBR encoding using x265 HEVC encoder. Method BDRP BDRV BD-PSNR BD-VMAF ∆S ∆E Ground truth (∆VMAF=2) -23.09% -43.23% 1.34 dB 10.61 -25.99% 89.54% Ground truth (∆VMAF=4) -28.15% -42.75% 1.70 dB 10.08 -59.07% -0.54% Ground truth (∆VMAF=6) -25.36% -40.73% 1.67 dB 9.19 -70.50% -31.24% Live-VBR (∆VMAF=2) -14.25% -29.14% 1.36 dB 7.82 23.57% 90.19% Live-VBR (∆VMAF=4) -18.41% -32.48% 1.41 dB 8.31 -56.38% 0.34% Live-VBR (∆VMAF=6) -18.80% -32.59% 1.34 dB 8.34 -68.96% -28.25% Relative storage difference ∆S = P bopt P bref − 1 Relative energy utilization difference ∆E = P E(bopt ) P E(bref ) − 1 Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 17
  • 18. Conclusion Conclusion Presented an application of video complexity analysis, where VMAF, target bitrate, CRF are predicted using video complexity features. Live-VBR shall be released as an open-source software by Christmas, 2022! GNU GPL 3.0 VCA, target encoders as sub-repositories. Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 18
  • 19. Q & A Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@aau.at) Prajit T Rajendran (prajit.thazhurazhikath@cea.fr) Christian Feldmann (christian.feldmann@bitmovin.com) Vignesh V Menon Perceptually-aware Live VBR Encoding Scheme for Adaptive Video Streaming 19