SlideShare a Scribd company logo
Optimal Quality and Efficiency in Adaptive Live Streaming with
JND-Aware Low latency Encoding
Vignesh V Menon1
, Jingwen Zhu2
, Prajit T Rajendran3
, Samira Afzal4
,
Klaus Schoeffmann4
, Patrick Le Callet2
, Christian Timmerer1
1
Video Communication and Applications Dept., Fraunhofer HHI, Berlin, Germany
2
Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, Nantes Universite, Nantes, France
3
CEA, List, F-91120 Palaiseau, Université Paris-Saclay, France
4
Alpen-Adria-Universität, Klagenfurt, Austria
13 Feb 2024
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 1
Outline
1 Introduction
2 JND-Aware Low latency Encoding (JALE)
3 Experimental validation
4 Conclusions
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 2
Introduction
Introduction
The cloud server can significantly streamline the encoding process by dynamically adjusting CPU
thread counts based on resolution and bitrate, accommodating diverse video qualities within an
adaptive streaming environment.
01 02 03 04 05 06 07 08 09 10 11 12
Representation ID
15
30
60
120
240
480
Encoding
speed
(in
fps)
4 threads
8 threads
16 threads
Figure: The encoding speed of each representation in HLS bitrate ladder1
for the Wood s000 sequence2
using ultrafast preset of x2653
using 4, 8, and 16 CPU threads for each representation.
1
Apple Inc. “HLS Authoring Specification for Apple Devices”. In: url:
https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices.
2
Hadi Amirpour et al. “VCD: Video Complexity Dataset”. In: Proceedings of the 13th ACM Multimedia Systems Conference. New York, NY, USA: Association
for Computing Machinery, 2022, 234–239. isbn: 9781450392839. doi: 10.1145/3524273.3532892. url: https://doi.org/10.1145/3524273.3532892.
3
VideoLAN. “x265”. In: url: https://www.videolan.org/developers/x265.html.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 3
Introduction
Optimized encoding preset
Traditional open-source encoders like x264,4 x265, and VVenC5 have pre-defined sets of
encoding parameters (termed as presets), which present a trade-off between the encoding
time and compression efficiency.6
If the encoder preset is configured such that this higher encoding speed can be reduced while
still being compatible with the expected live encoding speed, the quality of the encoded
content achieved by the encoder can be improved.
Subsequently, when the content becomes complex again, the encoder preset needs to be
reconfigured to move back to the faster configuration that achieves live encoding speed.7
By employing efficient storage techniques and removing unnecessary representations, the energy
consumption associated with storing and transmitting redundant data can be minimized.8
4
VideoLAN. “x264”. In: url: https://www.videolan.org/developers/x264.html.
5
Adam Wieckowski et al. “VVenC: An Open And Optimized VVC Encoder Implementation”. In: Proc. IEEE International Conference on Multimedia Expo
Workshops (ICMEW). 2021, pp. 1–2. doi: 10.1109/ICMEW53276.2021.9455944.
6
Dieison Silveira, Marcelo Porto, and Sergio Bampi. “Performance and energy consumption analysis of the X265 video encoder”. In: 2017 25th European
Signal Processing Conference (EUSIPCO). 2017, pp. 1519–1523. doi: 10.23919/EUSIPCO.2017.8081463.
7
Sergey Zvezdakov, Denis Kondranin, and Dmitriy Vatolin. “Machine-Learning-Based Method for Content-Adaptive Video Encoding”. In: 2021 Picture Coding
Symposium (PCS). 2021, pp. 1–5. doi: 10.1109/PCS50896.2021.9477507.
8
Quortex. Mission: Emission. https://www.quortex.io/wp-content/uploads/2022/05/WhitePaper-Mission_-Emission.pdf. [Accessed May 2023]. 2022.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 4
JND-Aware Low latency Encoding (JALE)
JALE
Figure: Live encoding using JALE envisioned in this paper.
JALE is classified into three steps:
1 video complexity feature extraction,
2 joint thread count and preset prediction,
3 perceptually-redundant representation elimination.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 5
JND-Aware Low latency Encoding (JALE) Spatiotemporal complexity feature extraction
Spatiotemporal complexity feature extraction
Three DCT-energy-based features, the average luma texture energy EY, the average gradient of
the luma texture energy h, and the average luminescence LYare used.9,10
(a) original frame (b) EY (c) h (d) LY
Figure: Heatmap depiction of the luma texture information {EY, h, LY} extracted from the second frame
of CoverSong 1080P 0a86 video of Youtube UGC Dataset.
9
Vignesh V Menon et al. “Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming”. In: Proceedings of the First International
Workshop on Green Multimedia Systems. 2023, 16–18. isbn: 9798400701962. doi: 10.1145/3593908.3593942. url:
https://doi.org/10.1145/3593908.3593942.
10
Vignesh V Menon et al. “JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming”. In: IEEE Transactions on Circuits and Systems for
Video Technology. 2023, pp. 1–1. doi: 10.1109/TCSVT.2023.3290725. url: https://doi.org/10.1109/TCSVT.2023.3290725.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 6
JND-Aware Low latency Encoding (JALE) Joint thread count and preset prediction
Joint thread count and preset prediction
Selection of the optimized thread count-preset pair for each segment per representation based
on the video content complexity is decomposed into two parts:
1 train models to predict the encoding speed for each thread count-preset pair,
2 develop a function to obtain the optimized thread count- preset pair for each representation.
The encoding speed of the tth representation of the input video segment (st) is modeled as:
st = fS(EY, h, LY, rt, bt, nt, pt) (1)
We use random forest models11 to predict the encoding speed for each thread count-preset pair.
(PT × CT) models are trained, where PT and CT represent the number of encoding presets and
the number of supported thread count per instance, respectively.
11
Leo Breiman. “Random Forests”. In: Machine Learning. Vol. 45. 2001. doi: 10.1023/A:1010933404324. url:
https://doi.org/10.1023/A:1010933404324.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 7
JND-Aware Low latency Encoding (JALE) Joint thread count and preset prediction
Joint thread count and preset prediction
The optimized thread count-preset prediction function has a look-up table of (n̂t, p̂t) pairs.
(4, medium)
(8,medium)
(12,medium)
(16,medium)
(20,medium)
(24,medium)
(4,fast)
(8,fast)
(12,fast)
(16,fast)
(20,fast)
(24,fast)
(4,faster)
(8,faster)
(12,faster)
(16,faster)
(20,faster)
(24,faster)
(4,veryfast)
(8,veryfast)
(12,veryfast)
(16,veryfast)
(20,veryfast)
(24,veryfast)
(4,superfast)
(8,superfast)
(12,superfast)
(16,superfast)
(20,superfast)
(24,superfast)
(4,ultrafast)
(8,ultrafast)
(12,ultrafast)
(16,ultrafast)
(20,ultrafast)
(24,ultrafast)
Figure: (n̂, p̂) look-up table used in the experimental validation of this paper.
The priority of (n̂t, p̂t) pairs is decided based on the following constraints:
1 the achieved encoding speed ŝt of the tth representation must be greater than or equal to
the target encoding speed sT, i.e., ŝt ≥ sT.
2 total number of CPU threads used for each representation is minimized.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 8
JND-Aware Low latency Encoding (JALE) Perceptually-redundant representation elimination
Perceptually-redundant representation elimination
Input:
q : number of representations in R
R =
Sq
t=1 {(rt , bt , n̂t , p̂t )}: representations with predicted thread count and preset
v̂t ; 1 ≤ t ≤ q: predicted VMAF
vT : maximum VMAF threshold
vJ : average target JND
Output: R̂ = (r, b, n̂, p̂): set of encoding configurations
R̂ ← {(r1, b1, n̂1, p̂1)}
u ← 1
if v̂1 ≥ vT then
return R̂
t ← 2
while t ≤ q do
if v̂t − v̂u ≥ vJ then
R̂ ← R̂ ∪ {(rt , bt , n̂t , p̂t )}
u ← t
if v̂t ≥ vT then
return R̂
t ← t + 1
return R̂
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 9
Experimental validation
Experimental parameters
Table: Experimental parameters of JALE used in this paper.
Parameter Symbol Values
Set of representations
Resolution height [pixels]
R
360 432 540 540 540 720 720 1080 1080 1440 2160 2160
Bitrate [Mbps] 0.145 0.300 0.600 0.900 1.600 2.400 3.400 4.500 5.800 8.100 11.600 16.800
Set of presets [x265] P 0 (ultrafast) – 5 (medium)
Set of supported thread counts C 4 8 12 16 20 24
Total CPU threads N 96
Encoding speed threshold [fps] sT 30
Average target JND vJ 2 4 6
Maximum VMAF threshold vT 98 96 94
Benchmark schemes:
1 Default: ultrafast preset with eight threads for each encoding instance.12
2 Bruteforce: optimized thread count-preset pair with and without JND-based representation elimi-
nation when the models are fully accurate. This is accomplished by bruteforce encoding using all
thread count-preset pairs and selecting the optimized pair.13
Hence, it is suitable only for video-on-
demand applications.
3 CAPS14
determines the optimized preset for each representation for a target encoding speed of 30 fps.
We evaluate CAPS where c=4, 8, and 16, respectively.
12
Apple Inc., “HLS Authoring Specification for Apple Devices”.
13
Jan De Cock et al. “Complexity-based consistent-quality encoding in the cloud”. In: 2016 IEEE International Conference on Image Processing (ICIP). 2016,
pp. 1484–1488. doi: 10.1109/ICIP.2016.7532605.
14
Vignesh V Menon et al. “Content-adaptive Encoder Preset Prediction for Adaptive Live Streaming”. In: 2022 Picture Coding Symposium (PCS). 2022,
pp. 253–257. doi: 10.1109/PCS56426.2022.10018034.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 10
Experimental validation
Results
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
0
1
2
3
4
5
Preset
4
8
12
16
20
24
Threads
(a) average thread count-preset
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
80
160
320
Encoding
speed
(in
fps)
Default
CAPS (c=8)
CAPS (c=16)
JALE
(b) average encoding speed
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
60
80
100
Encoding
speed
(in
fps)
Default
Bruteforce
CAPS (c=8)
CAPS (c=16)
JALE
(c) average VMAF
Figure: Results for each representation in JALE. JND-based representation elimination is not considered
in these plots.
Faster encoding presets and more computational resources are needed to encode high-
bitrate representations so that the encoding speed is above the threshold.
JALE controls the encoding speed to be greater than sT but not significantly higher than
the default encoding. This ensures higher CPU utilization when the encodings are carried
out concurrently during a live feed.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 11
Experimental validation
Results
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
60
80
100
VMAF
Default
Bruteforce (vJ=6)
CAPS (c=8)
JALE (vJ=6)
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
80
160
320
Encoding
speed
(in
fps)
Default
CAPS (c=8)
JALE (vJ=6)
(a) Basketball s000
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
60
80
100
VMAF
Default
Bruteforce (vJ=6)
CAPS (c=8)
JALE (vJ=6)
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
80
160
320
Encoding
speed
(in
fps)
Default
CAPS (c=8)
JALE (vJ=6)
(b) Characters s000
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
60
80
100
VMAF
Default
Bruteforce (vJ=6)
CAPS (c=8)
JALE (vJ=6)
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
80
160
320
Encoding
speed
(in
fps)
Default
CAPS (c=8)
JALE (vJ=6)
(c) Dolls s001
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
60
80
100
VMAF
Default
Bruteforce (vJ=6)
CAPS (c=8)
JALE (vJ=6)
0.2 0.5 1.2 3.0 8.0 16.8
Bitrate (in Mbps)
20
40
80
160
320
Encoding
speed
(in
fps)
Default
CAPS (c=8)
JALE (vJ=6)
(d) Maples s000
Figure: Rate-distortion (RD) curves of representative sequences (segments) for Default encoding (blue
line), CAPS (c = 8) encoding (red line), compared to JALE (vJ =6).
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 12
Experimental validation
Results
Table: Average results of the encoding schemes compared to the default encoding.
Method vJ BDRP BDRV BD-PSNR BD-VMAF ∆S ∆N ∆E
[%] [%] [dB] [%] [%] [%]
Bruteforce
2 -23.85 -23.02 1.21 4.61 -51.32 - 1688.91
4 -24.54 -24.27 1.25 5.26 -62.13 - 1688.91
6 -26.30 -25.77 1.32 5.53 -72.03 - 1688.91
CAPS (c = 4) - -12.31 -14.98 0.82 2.91 -0.06 -50.00 8.65
CAPS (c = 8) - -20.26 -20.95 1.06 4.03 -0.09 0 39.68
CAPS (c = 16) - -28.13 -29.35 1.62 6.04 -0.35 100.00 96.84
JALE
2 -23.30 -22.80 1.19 4.47 -51.34 -43.13 -2.69
4 -24.41 -24.08 1.25 5.05 -62.95 -54.08 -22.70
6 -25.93 -25.47 1.32 5.38 -72.70 -63.83 -37.87
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 13
Conclusions
Conclusions
We proposed JALE, a JND-aware low latency encoding scheme for adaptive live streaming
applications.
JALE jointly predicts the optimized encoder preset and CPU thread count for a given
representation for each video segment based on the video content complexity features,
target encoding speed, and the total number of available CPU threads.
The JND-based representation elimination algorithm removes perceptually redundant rep-
resentations in the bitrate ladder.
JALE yields an overall average quality improvement of 0.98 dB PSNR and 4.41 VMAF
points at the same bitrate, compared to the x265 ultrafast encoding of the reference HLS
bitrate ladder using eight CPU threads for each representation.
Considering a JND of six VMAF points, storage, thread count, and encoding time reductions
of 72.70 %, 63.83 %, and 37.87 %, respectively, are observed.
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 14
Q & A
Q & A
Thank you for your attention!
Vignesh V Menon (vignesh.menon@hhi.fraunhofer.de)
Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 15

More Related Content

Similar to Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low latency Encoding

Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Alpen-Adria-Universität
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Vignesh V Menon
 
TQPM.pdf
TQPM.pdfTQPM.pdf
TQPM.pdf
Vignesh V Menon
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video StreamingPerceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
Alpen-Adria-Universität
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
Vignesh V Menon
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfPerceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Vignesh V Menon
 
Online Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingOnline Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC Streaming
Vignesh V Menon
 
LiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdfLiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdf
Vignesh V Menon
 
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Ijripublishers Ijri
 
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Ijripublishers Ijri
 
Machine Learning approaches at video compression
Machine Learning approaches at video compression Machine Learning approaches at video compression
Machine Learning approaches at video compression
Roberto Iacoviello
 
HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?
Alpen-Adria-Universität
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
Minh Nguyen
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
Vignesh V Menon
 
INCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCINCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVC
Alpen-Adria-Universität
 
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
Alpen-Adria-Universität
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
Vignesh V Menon
 
C0161018
C0161018C0161018
C0161018
IOSR Journals
 
C0161018
C0161018C0161018
C0161018
IOSR Journals
 
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
Alpen-Adria-Universität
 

Similar to Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low latency Encoding (20)

Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
 
TQPM.pdf
TQPM.pdfTQPM.pdf
TQPM.pdf
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video StreamingPerceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfPerceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
 
Online Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingOnline Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC Streaming
 
LiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdfLiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdf
 
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
 
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...
 
Machine Learning approaches at video compression
Machine Learning approaches at video compression Machine Learning approaches at video compression
Machine Learning approaches at video compression
 
HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?HTTP Adaptive Streaming – Where Is It Heading?
HTTP Adaptive Streaming – Where Is It Heading?
 
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
MHV'22 - Super-resolution Based Bitrate Adaptation for HTTP Adaptive Streamin...
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
 
INCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCINCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVC
 
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
ComplexCTTP: Complexity Class Based Transcoding Time Prediction for Video Seq...
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
 
C0161018
C0161018C0161018
C0161018
 
C0161018
C0161018C0161018
C0161018
 
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
 

More from Alpen-Adria-Universität

Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instancesVEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
Alpen-Adria-Universität
 
GREEM: An Open-Source Energy Measurement Tool for Video Processing
GREEM: An Open-Source Energy Measurement Tool for Video ProcessingGREEM: An Open-Source Energy Measurement Tool for Video Processing
GREEM: An Open-Source Energy Measurement Tool for Video Processing
Alpen-Adria-Universität
 
VEEP: Video Encoding Energy and CO₂ Emission Prediction
VEEP: Video Encoding Energy and CO₂ Emission PredictionVEEP: Video Encoding Energy and CO₂ Emission Prediction
VEEP: Video Encoding Energy and CO₂ Emission Prediction
Alpen-Adria-Universität
 
Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive Streaming
Alpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Alpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Alpen-Adria-Universität
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Alpen-Adria-Universität
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Alpen-Adria-Universität
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Alpen-Adria-Universität
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Alpen-Adria-Universität
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video Streaming
Alpen-Adria-Universität
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Alpen-Adria-Universität
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
Alpen-Adria-Universität
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Alpen-Adria-Universität
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Alpen-Adria-Universität
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Alpen-Adria-Universität
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
Alpen-Adria-Universität
 
Immersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to HolographyImmersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to Holography
Alpen-Adria-Universität
 

More from Alpen-Adria-Universität (20)

Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instancesVEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
VEED: Video Encoding Energy and CO2 Emissions Dataset for AWS EC2 instances
 
GREEM: An Open-Source Energy Measurement Tool for Video Processing
GREEM: An Open-Source Energy Measurement Tool for Video ProcessingGREEM: An Open-Source Energy Measurement Tool for Video Processing
GREEM: An Open-Source Energy Measurement Tool for Video Processing
 
VEEP: Video Encoding Energy and CO₂ Emission Prediction
VEEP: Video Encoding Energy and CO₂ Emission PredictionVEEP: Video Encoding Energy and CO₂ Emission Prediction
VEEP: Video Encoding Energy and CO₂ Emission Prediction
 
Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive Streaming
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video Streaming
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
 
Immersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to HolographyImmersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to Holography
 

Recently uploaded

Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
christinelarrosa
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
christinelarrosa
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
ScyllaDB
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
christinelarrosa
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
UiPathCommunity
 
Day 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio FundamentalsDay 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio Fundamentals
UiPathCommunity
 

Recently uploaded (20)

Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
 
Day 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio FundamentalsDay 2 - Intro to UiPath Studio Fundamentals
Day 2 - Intro to UiPath Studio Fundamentals
 

Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low latency Encoding

  • 1. Optimal Quality and Efficiency in Adaptive Live Streaming with JND-Aware Low latency Encoding Vignesh V Menon1 , Jingwen Zhu2 , Prajit T Rajendran3 , Samira Afzal4 , Klaus Schoeffmann4 , Patrick Le Callet2 , Christian Timmerer1 1 Video Communication and Applications Dept., Fraunhofer HHI, Berlin, Germany 2 Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, Nantes Universite, Nantes, France 3 CEA, List, F-91120 Palaiseau, Université Paris-Saclay, France 4 Alpen-Adria-Universität, Klagenfurt, Austria 13 Feb 2024 Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 1
  • 2. Outline 1 Introduction 2 JND-Aware Low latency Encoding (JALE) 3 Experimental validation 4 Conclusions Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 2
  • 3. Introduction Introduction The cloud server can significantly streamline the encoding process by dynamically adjusting CPU thread counts based on resolution and bitrate, accommodating diverse video qualities within an adaptive streaming environment. 01 02 03 04 05 06 07 08 09 10 11 12 Representation ID 15 30 60 120 240 480 Encoding speed (in fps) 4 threads 8 threads 16 threads Figure: The encoding speed of each representation in HLS bitrate ladder1 for the Wood s000 sequence2 using ultrafast preset of x2653 using 4, 8, and 16 CPU threads for each representation. 1 Apple Inc. “HLS Authoring Specification for Apple Devices”. In: url: https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices. 2 Hadi Amirpour et al. “VCD: Video Complexity Dataset”. In: Proceedings of the 13th ACM Multimedia Systems Conference. New York, NY, USA: Association for Computing Machinery, 2022, 234–239. isbn: 9781450392839. doi: 10.1145/3524273.3532892. url: https://doi.org/10.1145/3524273.3532892. 3 VideoLAN. “x265”. In: url: https://www.videolan.org/developers/x265.html. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 3
  • 4. Introduction Optimized encoding preset Traditional open-source encoders like x264,4 x265, and VVenC5 have pre-defined sets of encoding parameters (termed as presets), which present a trade-off between the encoding time and compression efficiency.6 If the encoder preset is configured such that this higher encoding speed can be reduced while still being compatible with the expected live encoding speed, the quality of the encoded content achieved by the encoder can be improved. Subsequently, when the content becomes complex again, the encoder preset needs to be reconfigured to move back to the faster configuration that achieves live encoding speed.7 By employing efficient storage techniques and removing unnecessary representations, the energy consumption associated with storing and transmitting redundant data can be minimized.8 4 VideoLAN. “x264”. In: url: https://www.videolan.org/developers/x264.html. 5 Adam Wieckowski et al. “VVenC: An Open And Optimized VVC Encoder Implementation”. In: Proc. IEEE International Conference on Multimedia Expo Workshops (ICMEW). 2021, pp. 1–2. doi: 10.1109/ICMEW53276.2021.9455944. 6 Dieison Silveira, Marcelo Porto, and Sergio Bampi. “Performance and energy consumption analysis of the X265 video encoder”. In: 2017 25th European Signal Processing Conference (EUSIPCO). 2017, pp. 1519–1523. doi: 10.23919/EUSIPCO.2017.8081463. 7 Sergey Zvezdakov, Denis Kondranin, and Dmitriy Vatolin. “Machine-Learning-Based Method for Content-Adaptive Video Encoding”. In: 2021 Picture Coding Symposium (PCS). 2021, pp. 1–5. doi: 10.1109/PCS50896.2021.9477507. 8 Quortex. Mission: Emission. https://www.quortex.io/wp-content/uploads/2022/05/WhitePaper-Mission_-Emission.pdf. [Accessed May 2023]. 2022. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 4
  • 5. JND-Aware Low latency Encoding (JALE) JALE Figure: Live encoding using JALE envisioned in this paper. JALE is classified into three steps: 1 video complexity feature extraction, 2 joint thread count and preset prediction, 3 perceptually-redundant representation elimination. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 5
  • 6. JND-Aware Low latency Encoding (JALE) Spatiotemporal complexity feature extraction Spatiotemporal complexity feature extraction Three DCT-energy-based features, the average luma texture energy EY, the average gradient of the luma texture energy h, and the average luminescence LYare used.9,10 (a) original frame (b) EY (c) h (d) LY Figure: Heatmap depiction of the luma texture information {EY, h, LY} extracted from the second frame of CoverSong 1080P 0a86 video of Youtube UGC Dataset. 9 Vignesh V Menon et al. “Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming”. In: Proceedings of the First International Workshop on Green Multimedia Systems. 2023, 16–18. isbn: 9798400701962. doi: 10.1145/3593908.3593942. url: https://doi.org/10.1145/3593908.3593942. 10 Vignesh V Menon et al. “JND-aware Two-pass Per-title Encoding Scheme for Adaptive Live Streaming”. In: IEEE Transactions on Circuits and Systems for Video Technology. 2023, pp. 1–1. doi: 10.1109/TCSVT.2023.3290725. url: https://doi.org/10.1109/TCSVT.2023.3290725. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 6
  • 7. JND-Aware Low latency Encoding (JALE) Joint thread count and preset prediction Joint thread count and preset prediction Selection of the optimized thread count-preset pair for each segment per representation based on the video content complexity is decomposed into two parts: 1 train models to predict the encoding speed for each thread count-preset pair, 2 develop a function to obtain the optimized thread count- preset pair for each representation. The encoding speed of the tth representation of the input video segment (st) is modeled as: st = fS(EY, h, LY, rt, bt, nt, pt) (1) We use random forest models11 to predict the encoding speed for each thread count-preset pair. (PT × CT) models are trained, where PT and CT represent the number of encoding presets and the number of supported thread count per instance, respectively. 11 Leo Breiman. “Random Forests”. In: Machine Learning. Vol. 45. 2001. doi: 10.1023/A:1010933404324. url: https://doi.org/10.1023/A:1010933404324. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 7
  • 8. JND-Aware Low latency Encoding (JALE) Joint thread count and preset prediction Joint thread count and preset prediction The optimized thread count-preset prediction function has a look-up table of (n̂t, p̂t) pairs. (4, medium) (8,medium) (12,medium) (16,medium) (20,medium) (24,medium) (4,fast) (8,fast) (12,fast) (16,fast) (20,fast) (24,fast) (4,faster) (8,faster) (12,faster) (16,faster) (20,faster) (24,faster) (4,veryfast) (8,veryfast) (12,veryfast) (16,veryfast) (20,veryfast) (24,veryfast) (4,superfast) (8,superfast) (12,superfast) (16,superfast) (20,superfast) (24,superfast) (4,ultrafast) (8,ultrafast) (12,ultrafast) (16,ultrafast) (20,ultrafast) (24,ultrafast) Figure: (n̂, p̂) look-up table used in the experimental validation of this paper. The priority of (n̂t, p̂t) pairs is decided based on the following constraints: 1 the achieved encoding speed ŝt of the tth representation must be greater than or equal to the target encoding speed sT, i.e., ŝt ≥ sT. 2 total number of CPU threads used for each representation is minimized. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 8
  • 9. JND-Aware Low latency Encoding (JALE) Perceptually-redundant representation elimination Perceptually-redundant representation elimination Input: q : number of representations in R R = Sq t=1 {(rt , bt , n̂t , p̂t )}: representations with predicted thread count and preset v̂t ; 1 ≤ t ≤ q: predicted VMAF vT : maximum VMAF threshold vJ : average target JND Output: R̂ = (r, b, n̂, p̂): set of encoding configurations R̂ ← {(r1, b1, n̂1, p̂1)} u ← 1 if v̂1 ≥ vT then return R̂ t ← 2 while t ≤ q do if v̂t − v̂u ≥ vJ then R̂ ← R̂ ∪ {(rt , bt , n̂t , p̂t )} u ← t if v̂t ≥ vT then return R̂ t ← t + 1 return R̂ Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 9
  • 10. Experimental validation Experimental parameters Table: Experimental parameters of JALE used in this paper. Parameter Symbol Values Set of representations Resolution height [pixels] R 360 432 540 540 540 720 720 1080 1080 1440 2160 2160 Bitrate [Mbps] 0.145 0.300 0.600 0.900 1.600 2.400 3.400 4.500 5.800 8.100 11.600 16.800 Set of presets [x265] P 0 (ultrafast) – 5 (medium) Set of supported thread counts C 4 8 12 16 20 24 Total CPU threads N 96 Encoding speed threshold [fps] sT 30 Average target JND vJ 2 4 6 Maximum VMAF threshold vT 98 96 94 Benchmark schemes: 1 Default: ultrafast preset with eight threads for each encoding instance.12 2 Bruteforce: optimized thread count-preset pair with and without JND-based representation elimi- nation when the models are fully accurate. This is accomplished by bruteforce encoding using all thread count-preset pairs and selecting the optimized pair.13 Hence, it is suitable only for video-on- demand applications. 3 CAPS14 determines the optimized preset for each representation for a target encoding speed of 30 fps. We evaluate CAPS where c=4, 8, and 16, respectively. 12 Apple Inc., “HLS Authoring Specification for Apple Devices”. 13 Jan De Cock et al. “Complexity-based consistent-quality encoding in the cloud”. In: 2016 IEEE International Conference on Image Processing (ICIP). 2016, pp. 1484–1488. doi: 10.1109/ICIP.2016.7532605. 14 Vignesh V Menon et al. “Content-adaptive Encoder Preset Prediction for Adaptive Live Streaming”. In: 2022 Picture Coding Symposium (PCS). 2022, pp. 253–257. doi: 10.1109/PCS56426.2022.10018034. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 10
  • 11. Experimental validation Results 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 0 1 2 3 4 5 Preset 4 8 12 16 20 24 Threads (a) average thread count-preset 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 80 160 320 Encoding speed (in fps) Default CAPS (c=8) CAPS (c=16) JALE (b) average encoding speed 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 60 80 100 Encoding speed (in fps) Default Bruteforce CAPS (c=8) CAPS (c=16) JALE (c) average VMAF Figure: Results for each representation in JALE. JND-based representation elimination is not considered in these plots. Faster encoding presets and more computational resources are needed to encode high- bitrate representations so that the encoding speed is above the threshold. JALE controls the encoding speed to be greater than sT but not significantly higher than the default encoding. This ensures higher CPU utilization when the encodings are carried out concurrently during a live feed. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 11
  • 12. Experimental validation Results 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 60 80 100 VMAF Default Bruteforce (vJ=6) CAPS (c=8) JALE (vJ=6) 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 80 160 320 Encoding speed (in fps) Default CAPS (c=8) JALE (vJ=6) (a) Basketball s000 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 60 80 100 VMAF Default Bruteforce (vJ=6) CAPS (c=8) JALE (vJ=6) 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 80 160 320 Encoding speed (in fps) Default CAPS (c=8) JALE (vJ=6) (b) Characters s000 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 60 80 100 VMAF Default Bruteforce (vJ=6) CAPS (c=8) JALE (vJ=6) 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 80 160 320 Encoding speed (in fps) Default CAPS (c=8) JALE (vJ=6) (c) Dolls s001 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 60 80 100 VMAF Default Bruteforce (vJ=6) CAPS (c=8) JALE (vJ=6) 0.2 0.5 1.2 3.0 8.0 16.8 Bitrate (in Mbps) 20 40 80 160 320 Encoding speed (in fps) Default CAPS (c=8) JALE (vJ=6) (d) Maples s000 Figure: Rate-distortion (RD) curves of representative sequences (segments) for Default encoding (blue line), CAPS (c = 8) encoding (red line), compared to JALE (vJ =6). Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 12
  • 13. Experimental validation Results Table: Average results of the encoding schemes compared to the default encoding. Method vJ BDRP BDRV BD-PSNR BD-VMAF ∆S ∆N ∆E [%] [%] [dB] [%] [%] [%] Bruteforce 2 -23.85 -23.02 1.21 4.61 -51.32 - 1688.91 4 -24.54 -24.27 1.25 5.26 -62.13 - 1688.91 6 -26.30 -25.77 1.32 5.53 -72.03 - 1688.91 CAPS (c = 4) - -12.31 -14.98 0.82 2.91 -0.06 -50.00 8.65 CAPS (c = 8) - -20.26 -20.95 1.06 4.03 -0.09 0 39.68 CAPS (c = 16) - -28.13 -29.35 1.62 6.04 -0.35 100.00 96.84 JALE 2 -23.30 -22.80 1.19 4.47 -51.34 -43.13 -2.69 4 -24.41 -24.08 1.25 5.05 -62.95 -54.08 -22.70 6 -25.93 -25.47 1.32 5.38 -72.70 -63.83 -37.87 Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 13
  • 14. Conclusions Conclusions We proposed JALE, a JND-aware low latency encoding scheme for adaptive live streaming applications. JALE jointly predicts the optimized encoder preset and CPU thread count for a given representation for each video segment based on the video content complexity features, target encoding speed, and the total number of available CPU threads. The JND-based representation elimination algorithm removes perceptually redundant rep- resentations in the bitrate ladder. JALE yields an overall average quality improvement of 0.98 dB PSNR and 4.41 VMAF points at the same bitrate, compared to the x265 ultrafast encoding of the reference HLS bitrate ladder using eight CPU threads for each representation. Considering a JND of six VMAF points, storage, thread count, and encoding time reductions of 72.70 %, 63.83 %, and 37.87 %, respectively, are observed. Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 14
  • 15. Q & A Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@hhi.fraunhofer.de) Vignesh V Menon Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resolution Encoding 15