SlideShare a Scribd company logo
1 of 16
Download to read offline
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live
Streaming
Vignesh V Menon1, Hadi Amirpour1, Mohammad Ghanbari1,2, and Christian Timmerer1
1
Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität, Klagenfurt, Austria
2
School of Computer Science and Electronic Engineering, University of Essex, UK
16-19 October 2022
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 1
Outline
1 Introduction
2 Research Problem
3 ETPS
4 Evaluation
5 Conclusions
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 2
Introduction
Introduction
CBR versus VBR
Figure: Constant Bitrate (CBR) versus Variable
Bitrate (VBR) encoding.
In live streaming, Constant Bitrate (CBR) rate-
control mode is used to encode video sequences
at a fixed bitrate ladder. The consistency of
CBR makes it more reliable for time-sensitive
data transport.
In this method, there is no concern about the
bitrate exceeding internet speeds. However, this
method may result in low compression efficiency.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 3
Introduction
Introduction
Variable Bitrate (VBR) encoding
Figure: Two-pass encoding.
VBR encoding schemes are implemented as two-pass encoding, also known as multi-pass
encoding used to retain the best compression efficiency during encoding.
In the first-pass of two-pass encoding, the input data from the video is analyzed (and stored
in a log file). The collected data from the first-pass is used to achieve the best encoding
compression efficiency in the second-pass.
Two-pass encoding is usually controlled by the average bitrate setting, bitrate range setting
(minimal and maximal allowed bitrate), or target video file size.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 4
Introduction
Introduction
Constrained Variable Bitrate (cVBR) encoding
Figure: cVBR encoding.
A ”rate factor” first-pass to identify the optimized CRF to achieve the target bitrate.
In the second-pass, the segment is encoded with the selected optimized CRF with the
maximum bitrate and maximum buffer window constraints.
The desired target bitrate is achieved with maximum compression efficiency, and min-
imum quality fluctuation.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 5
Research Problem
Research Problem
In two-pass encoding, the encoder processes all segments for the stream twice.
Due to the latency introduced by the first-pass to achieve the optimal encoding parameters
for the second-pass, two-pass encoding is not used in live streaming applications.
This paper targets a two-pass encoding scheme optimized for live streaming applications. The
target average bitrate (bT ) is aimed to be achieved uniformly throughout the streaming session
with the highest possible compression efficiency.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 6
ETPS
ETPS
Figure: ETPS architecture.
The first-pass predicts the optimized CRF to achieve the target bitrate.
In the second-pass, the segment is encoded with the selected, optimized CRF with the
maximum bitrate and maximum buffer window constraints.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 7
ETPS Phase 1: Video Complexity Feature Extraction
ETPS
Phase 1: Video Complexity Feature Extraction
Compute texture energy per block
A DCT-based energy function is used to determine the block-wise feature of each frame
defined as:
Hk =
w−1
X
i=0
w−1
X
j=0
e|( ij
wh
)2−1|
|DCT(i, j)| (1)
where wxw is the size of the block, and DCT(i, j) is the (i, j)th DCT component when
i + j > 0, and 0 otherwise.
The energy values of blocks in a frame is averaged to determine the energy per frame1,.2
E =
C−1
X
k=0
Hp,k
C · w2
(2)
1
Michael King et al. “A New Energy Function for Segmentation and Compression”. In: 2007 IEEE International Conference on Multimedia and Expo. 2007,
pp. 1647–1650. doi: 10.1109/ICME.2007.4284983.
2
Vignesh V Menon et al. “Efficient Content-Adaptive Feature-Based Shot Detection for HTTP Adaptive Streaming”. In: 2021 IEEE International Conference
on Image Processing (ICIP). 2021, pp. 2174–2178. doi: 10.1109/ICIP42928.2021.9506092.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 8
ETPS Phase 1: Video Complexity Feature Extraction
ETPS
Phase 1: Video Complexity Feature Extraction
hp: SAD of the block level energy values of frame p to that of the previous frame p − 1.
hp =
C−1
X
k=0
| Hp,k, Hp−1,k |
C · w2
(3)
where C denotes the number of blocks in frame p.
Latency
Speed of feature extraction = 370fps for UHD video with 8 CPU threads and x86 SIMD
optimization
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 9
ETPS Phase 2: CRF Prediction
ETPS
Phase 2: CRF Prediction
Hidden Layer
E R4
Hidden Layer
E R4
Input Layer
E R3
Output Layer
E R1
E
h
log(b)
ĉ
A shallow neural network is trained for each resolu-
tion r and framerate f , which determines the opti-
mized CRF (ĉ) based on E, h, and log(b) for every
video segment.
The input vector passed to the network is [E, h, log(bT )].
The target bitrate input (bT ) is passed in logarithmic scale to the network to reduce internal
covariate shift.3
Rectified Linear Unit (ReLU)4 is used as the activation function and Adam5 is used as
optimizer with a learning rate of 10−4.
3
Sergey Ioffe and Christian Szegedy. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”. In: Proceedings of the
32nd International Conference on Machine Learning. Vol. 37. Lille, France: PMLR, 2015, pp. 448–456.
4
Xavier Glorot, Antoine Bordes, and Y. Bengio. “Deep Sparse Rectifier Neural Networks”. In: vol. 15. Jan. 2010.
5
Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: 3rd International Conference on Learning Representations, ICLR
2015, San Diego, CA, USA, May 7-9, 2015. 2015.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 10
Evaluation Test Methodology
Test Methodology
Table: Encoder Settings.
Encoder x265 v3.5
Preset ultrafast
Segment length 4 seconds
Resolution Ultra HD (2160p)
Framerate 30 fps
Target bitrates 8, 12, 17, and 20 Mbps
Keras6 is used as the machine learning framework.
E and h features are extracted from the video segments using VCA7 run in eight CPU
threads
6
Nikhil Ketkar. “Introduction to Keras”. In: Oct. 2017, pp. 95–109. isbn: 978-1-4842-2765-7. doi: 10.1007/978-1-4842-2766-4_7.
7
Vignesh V Menon et al. “VCA: Video Complexity Analyzer”. In: 13th ACM Multimedia Systems Conference (MMSys ’22). 2022. isbn: 9781450392839. doi:
10.1145/3524273.3532896. url: https://doi.org/10.1145/3524273.3532896.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 11
Evaluation Test Methodology
Test Methodology
Criteria:
Root Mean Squared Error (RMSE) of cG and the predicted CRF (ĉ), i.e., || cG − ĉ ||2.
The first-pass time, i.e., latency in streaming introduced due to the time taken to extract
E and h, and the inference time of the neural network to predict optimized CRF.
Bjøntegaard delta rates8 BDRP and BDRV to measure the average increase in bitrate of
the representations compared with that of the reference encoding scheme to maintain the
same PSNR and VMAF, respectively.
∆T: the overall difference in encoding time of ETPS with the reference encoding scheme.
The results are compared with CBR encoding and two-pass average bitrate encoding schemes9.
8
G. Bjontegaard. “Calculation of average PSNR differences between RD-curves”. In: VCEG-M33 (2001).
9
https://x265.readthedocs.io/en/master/cli.html#quality-rate-control-and-rate-distortion-options, last access: Jun 30, 2022.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 12
Evaluation Experimental Results
Experimental Results
Table: Results of ETPS against Constant Bitrate (CBR) encoding and two-pass average bitrate encoding
schemes.
Constant bitrate (CBR) Two-pass average bitrate
Dataset Video E h || cG − ĉ ||2 τp (in ms) BDRP BDRV ∆T BDRP BDRV ∆T
MCML10 Basketball 15.75 9.69 0.86 8 -17.80% -14.36% 0.43% -0.48% 0.50% -45.15%
MCML Bunny 23.03 4.89 1.58 7 -9.92% -9.09% 0.78% 0.04% -1.14% -43.88%
MCML Crowd 33.11 7.03 1.58 7 -7.63% -1.47% 0.56% 0.75% 1.05% -43.34%
MCML Dolls 19.91 12.22 0.86 8 -5.73% -4.05% 1.01% 1.21% 1.97% -45.26%
MCML Flowers 12.01 10.47 0.50 7 -8.44% -7.12% -0.59% 0.38% 0.49% -43.90%
MCML Park 26.07 27.34 0.71 9 -5.16% -2.62% 1.10% 0.16% 0.87% -44.27%
SJTU11 BundNightScape 54.90 11.62 1.93 8 -14.63% -13.51% -1.62% -1.11% -1.27% -40.91%
SJTU CampfireParty 51.50 42.38 1.58 8 -4.66% -3.66% -1.72% -0.51% -0.35% -43.04%
SJTU Runners 104.30 17.44 1.32 7 -8.14% -2.44% -1.34% -0.77% -1.08% -42.69%
SJTU TallBuildings 94.35 7.70 1.65 8 -17.30% -12.54% -1.02% 0.89% 0.59% -43.37%
SJTU TrafficAndBuilding 60.54 11.55 1.87 8 -15.41% -9.91% -0.43% 0.06% -2.42% -43.96%
SJTU TrafficFlow 58.93 8.61 1.32 8 -15.66% -1.17% 0.74% 1.19% 1.43% -43.86%
SJTU TreeShade 80.19 15.83 0.50 8 -11.82% -6.85% -0.79% 0.52% 0.59% -44.75%
SJTU Wood 114.26 8.83 0.71 8 -10.14% -21.58% 0.52% 1.34% 1.88% -44.50%
Average 1.02 8 -10.89% -8.60% 0.65% 0.26% 0.38% -43.78%
10
Manri Cheon and Jong-Seok Lee. “Subjective and Objective Quality Assessment of Compressed 4K UHD Videos for Immersive Experience”. In: IEEE
Transactions on Circuits and Systems for Video Technology 28.7 (2018), pp. 1467–1480. doi: 10.1109/TCSVT.2017.2683504.
11
L. Song et al. “The SJTU 4K Video Sequence Dataset”. In: Fifth International Workshop on Quality of Multimedia Experience (QoMEX2013) (July 2013).
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 13
Evaluation Experimental Results
Experimental Results
MCML sequences are sequentially encoded using ETPS at 8, 12, 17, and 20 Mbps, and the
achieved average bitrate is recorded.
The average target bitrate is consistently achieved irrespective of the spatial and temporal
complexities of the segments.
0 4 8 12 16 20 24 28 32
Time (in seconds)
8
10
12
14
16
18
20
Average
bitrate
(in
Mbps)
8 Mbps
12 Mbps
17 Mbps
20 Mbps
Figure: The average bitrate in a streaming session where MCML sequences are encoded using ETPS and
are streamed at intervals of four seconds for target average bitrates 8, 12, 17, 20 Mbps.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 14
Conclusions
Conclusions
This paper proposed ETPS, a low-latency two-pass constrained variable bitrate (cVBR)
encoding scheme for live streaming applications.
ETPS includes a prediction algorithm for optimized CRF at a given target bitrate, resolution,
and framerate of video segments.
DCT-energy-based features are used to determine segments’ spatial and temporal complex-
ity.
Live streaming using ETPS has similar compression efficiency as that of the two-pass average
bitrate encoding scheme, but with a significant encoding time-saving of 43.78%.
Compared to CBR encoding, ETPS yields bitrate saving of 10.89% and 8.60% to maintain
the same PSNR and VMAF, respectively, with an additional encoding time of 0.65%.
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 15
Q & A
Q & A
Thank you for your attention!
Vignesh V Menon (vignesh.menon@aau.at)
Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 16

More Related Content

Similar to ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf

OPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfOPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfVignesh V Menon
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video StreamingPerceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video StreamingAlpen-Adria-Universität
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfPerceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfVignesh V Menon
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Vignesh V Menon
 
Efficient bitrate ladder construction for live video streaming
Efficient bitrate ladder construction for live video streamingEfficient bitrate ladder construction for live video streaming
Efficient bitrate ladder construction for live video streamingMinh Nguyen
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCVignesh V Menon
 
INCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCINCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCAlpen-Adria-Universität
 
Machine Learning approaches at video compression
Machine Learning approaches at video compression Machine Learning approaches at video compression
Machine Learning approaches at video compression Roberto Iacoviello
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdfVignesh V Menon
 
Performance Evaluation of Percent Root Mean Square Difference for ECG Signals...
Performance Evaluation of Percent Root Mean Square Difference for ECG Signals...Performance Evaluation of Percent Root Mean Square Difference for ECG Signals...
Performance Evaluation of Percent Root Mean Square Difference for ECG Signals...CSCJournals
 
International Journal of Engineering Research and Development
International Journal of Engineering Research and DevelopmentInternational Journal of Engineering Research and Development
International Journal of Engineering Research and DevelopmentIJERD Editor
 
Online Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingOnline Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingVignesh V Menon
 
Data detection with a progressive parallel ici canceller in mimo ofdm
Data detection with a progressive parallel ici canceller in mimo ofdmData detection with a progressive parallel ici canceller in mimo ofdm
Data detection with a progressive parallel ici canceller in mimo ofdmeSAT Publishing House
 
Rate Distortion Performance for Joint Source Channel Coding of JPEG image Ove...
Rate Distortion Performance for Joint Source Channel Coding of JPEG image Ove...Rate Distortion Performance for Joint Source Channel Coding of JPEG image Ove...
Rate Distortion Performance for Joint Source Channel Coding of JPEG image Ove...CSCJournals
 
LiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdfLiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdfVignesh V Menon
 
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...IJECEIAES
 
Optimal coding unit decision for early termination in high efficiency video c...
Optimal coding unit decision for early termination in high efficiency video c...Optimal coding unit decision for early termination in high efficiency video c...
Optimal coding unit decision for early termination in high efficiency video c...IJECEIAES
 

Similar to ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf (20)

OPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdfOPTE: Online Per-title Encoding for Live Video Streaming.pdf
OPTE: Online Per-title Encoding for Live Video Streaming.pdf
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video StreamingPerceptually-aware Per-title Encoding for Adaptive Video Streaming
Perceptually-aware Per-title Encoding for Adaptive Video Streaming
 
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdfPerceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
Perceptually-aware Per-title Encoding for Adaptive Video Streaming.pdf
 
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
Energy-efficient Adaptive Video Streaming with Latency-Aware Dynamic Resoluti...
 
CAPS_Presentation.pdf
CAPS_Presentation.pdfCAPS_Presentation.pdf
CAPS_Presentation.pdf
 
Efficient bitrate ladder construction for live video streaming
Efficient bitrate ladder construction for live video streamingEfficient bitrate ladder construction for live video streaming
Efficient bitrate ladder construction for live video streaming
 
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVCIEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
IEEE MMSP'21: INCEPT: Intra CU Depth Prediction for HEVC
 
INCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCINCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVC
 
Machine Learning approaches at video compression
Machine Learning approaches at video compression Machine Learning approaches at video compression
Machine Learning approaches at video compression
 
Green_VCA_presentation.pdf
Green_VCA_presentation.pdfGreen_VCA_presentation.pdf
Green_VCA_presentation.pdf
 
Performance Evaluation of Percent Root Mean Square Difference for ECG Signals...
Performance Evaluation of Percent Root Mean Square Difference for ECG Signals...Performance Evaluation of Percent Root Mean Square Difference for ECG Signals...
Performance Evaluation of Percent Root Mean Square Difference for ECG Signals...
 
40120140504006
4012014050400640120140504006
40120140504006
 
International Journal of Engineering Research and Development
International Journal of Engineering Research and DevelopmentInternational Journal of Engineering Research and Development
International Journal of Engineering Research and Development
 
Online Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingOnline Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC Streaming
 
Data detection with a progressive parallel ici canceller in mimo ofdm
Data detection with a progressive parallel ici canceller in mimo ofdmData detection with a progressive parallel ici canceller in mimo ofdm
Data detection with a progressive parallel ici canceller in mimo ofdm
 
Rate Distortion Performance for Joint Source Channel Coding of JPEG image Ove...
Rate Distortion Performance for Joint Source Channel Coding of JPEG image Ove...Rate Distortion Performance for Joint Source Channel Coding of JPEG image Ove...
Rate Distortion Performance for Joint Source Channel Coding of JPEG image Ove...
 
LiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdfLiveVBR presentation at VQEG NORM.pdf
LiveVBR presentation at VQEG NORM.pdf
 
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Vide...
 
H43044145
H43044145H43044145
H43044145
 
Optimal coding unit decision for early termination in high efficiency video c...
Optimal coding unit decision for early termination in high efficiency video c...Optimal coding unit decision for early termination in high efficiency video c...
Optimal coding unit decision for early termination in high efficiency video c...
 

More from Vignesh V Menon

Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Vignesh V Menon
 
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdfContent_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdfVignesh V Menon
 
Green Variable framerate encoding for Adaptive Live Streaming
Green Variable framerate encoding  for Adaptive Live StreamingGreen Variable framerate encoding  for Adaptive Live Streaming
Green Variable framerate encoding for Adaptive Live StreamingVignesh V Menon
 
Doctoral Symposium presentation.pdf
Doctoral Symposium presentation.pdfDoctoral Symposium presentation.pdf
Doctoral Symposium presentation.pdfVignesh V Menon
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfVignesh V Menon
 
Video Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVideo Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVignesh V Menon
 
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingLive-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingVignesh V Menon
 
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...Vignesh V Menon
 
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...Vignesh V Menon
 

More from Vignesh V Menon (10)

Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
Gain of Grain: A Film Grain Handling Toolchain for VVC-based Open Implementat...
 
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdfContent_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
Content_adaptive_video_coding_for_HTTP_Adaptive_Streaming.pdf
 
Green Variable framerate encoding for Adaptive Live Streaming
Green Variable framerate encoding  for Adaptive Live StreamingGreen Variable framerate encoding  for Adaptive Live Streaming
Green Variable framerate encoding for Adaptive Live Streaming
 
JASLA_presentation.pdf
JASLA_presentation.pdfJASLA_presentation.pdf
JASLA_presentation.pdf
 
Doctoral Symposium presentation.pdf
Doctoral Symposium presentation.pdfDoctoral Symposium presentation.pdf
Doctoral Symposium presentation.pdf
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
 
Video Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdfVideo Complexity Dataset (VCD).pdf
Video Complexity Dataset (VCD).pdf
 
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive StreamingLive-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
Live-PSTR: Live Per-Title Encoding for Ultra HD Adaptive Streaming
 
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
IEEE PCS'21: Efficient multi-encoding for large-scale HTTP Adaptive Streaming...
 
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
IEEE ICIP'22:Efficient Content-Adaptive Feature-based Shot Detection for HTTP...
 

Recently uploaded

Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptxAnalyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptxLimon Prince
 
When Quality Assurance Meets Innovation in Higher Education - Report launch w...
When Quality Assurance Meets Innovation in Higher Education - Report launch w...When Quality Assurance Meets Innovation in Higher Education - Report launch w...
When Quality Assurance Meets Innovation in Higher Education - Report launch w...Gary Wood
 
diagnosting testing bsc 2nd sem.pptx....
diagnosting testing bsc 2nd sem.pptx....diagnosting testing bsc 2nd sem.pptx....
diagnosting testing bsc 2nd sem.pptx....Ritu480198
 
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...Nguyen Thanh Tu Collection
 
Trauma-Informed Leadership - Five Practical Principles
Trauma-Informed Leadership - Five Practical PrinciplesTrauma-Informed Leadership - Five Practical Principles
Trauma-Informed Leadership - Five Practical PrinciplesPooky Knightsmith
 
DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUM
DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUMDEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUM
DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUMELOISARIVERA8
 
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽中 央社
 
How To Create Editable Tree View in Odoo 17
How To Create Editable Tree View in Odoo 17How To Create Editable Tree View in Odoo 17
How To Create Editable Tree View in Odoo 17Celine George
 
Book Review of Run For Your Life Powerpoint
Book Review of Run For Your Life PowerpointBook Review of Run For Your Life Powerpoint
Book Review of Run For Your Life Powerpoint23600690
 
Graduate Outcomes Presentation Slides - English (v3).pptx
Graduate Outcomes Presentation Slides - English (v3).pptxGraduate Outcomes Presentation Slides - English (v3).pptx
Graduate Outcomes Presentation Slides - English (v3).pptxneillewis46
 
SURVEY I created for uni project research
SURVEY I created for uni project researchSURVEY I created for uni project research
SURVEY I created for uni project researchCaitlinCummins3
 
How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17Celine George
 
UChicago CMSC 23320 - The Best Commit Messages of 2024
UChicago CMSC 23320 - The Best Commit Messages of 2024UChicago CMSC 23320 - The Best Commit Messages of 2024
UChicago CMSC 23320 - The Best Commit Messages of 2024Borja Sotomayor
 
Stl Algorithms in C++ jjjjjjjjjjjjjjjjjj
Stl Algorithms in C++ jjjjjjjjjjjjjjjjjjStl Algorithms in C++ jjjjjjjjjjjjjjjjjj
Stl Algorithms in C++ jjjjjjjjjjjjjjjjjjMohammed Sikander
 
8 Tips for Effective Working Capital Management
8 Tips for Effective Working Capital Management8 Tips for Effective Working Capital Management
8 Tips for Effective Working Capital ManagementMBA Assignment Experts
 
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...Nguyen Thanh Tu Collection
 
Major project report on Tata Motors and its marketing strategies
Major project report on Tata Motors and its marketing strategiesMajor project report on Tata Motors and its marketing strategies
Major project report on Tata Motors and its marketing strategiesAmanpreetKaur157993
 

Recently uploaded (20)

VAMOS CUIDAR DO NOSSO PLANETA! .
VAMOS CUIDAR DO NOSSO PLANETA!                    .VAMOS CUIDAR DO NOSSO PLANETA!                    .
VAMOS CUIDAR DO NOSSO PLANETA! .
 
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptxAnalyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
Analyzing and resolving a communication crisis in Dhaka textiles LTD.pptx
 
When Quality Assurance Meets Innovation in Higher Education - Report launch w...
When Quality Assurance Meets Innovation in Higher Education - Report launch w...When Quality Assurance Meets Innovation in Higher Education - Report launch w...
When Quality Assurance Meets Innovation in Higher Education - Report launch w...
 
diagnosting testing bsc 2nd sem.pptx....
diagnosting testing bsc 2nd sem.pptx....diagnosting testing bsc 2nd sem.pptx....
diagnosting testing bsc 2nd sem.pptx....
 
ESSENTIAL of (CS/IT/IS) class 07 (Networks)
ESSENTIAL of (CS/IT/IS) class 07 (Networks)ESSENTIAL of (CS/IT/IS) class 07 (Networks)
ESSENTIAL of (CS/IT/IS) class 07 (Networks)
 
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
 
Trauma-Informed Leadership - Five Practical Principles
Trauma-Informed Leadership - Five Practical PrinciplesTrauma-Informed Leadership - Five Practical Principles
Trauma-Informed Leadership - Five Practical Principles
 
DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUM
DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUMDEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUM
DEMONSTRATION LESSON IN ENGLISH 4 MATATAG CURRICULUM
 
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
 
How To Create Editable Tree View in Odoo 17
How To Create Editable Tree View in Odoo 17How To Create Editable Tree View in Odoo 17
How To Create Editable Tree View in Odoo 17
 
Book Review of Run For Your Life Powerpoint
Book Review of Run For Your Life PowerpointBook Review of Run For Your Life Powerpoint
Book Review of Run For Your Life Powerpoint
 
Graduate Outcomes Presentation Slides - English (v3).pptx
Graduate Outcomes Presentation Slides - English (v3).pptxGraduate Outcomes Presentation Slides - English (v3).pptx
Graduate Outcomes Presentation Slides - English (v3).pptx
 
SURVEY I created for uni project research
SURVEY I created for uni project researchSURVEY I created for uni project research
SURVEY I created for uni project research
 
How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17How to Send Pro Forma Invoice to Your Customers in Odoo 17
How to Send Pro Forma Invoice to Your Customers in Odoo 17
 
Including Mental Health Support in Project Delivery, 14 May.pdf
Including Mental Health Support in Project Delivery, 14 May.pdfIncluding Mental Health Support in Project Delivery, 14 May.pdf
Including Mental Health Support in Project Delivery, 14 May.pdf
 
UChicago CMSC 23320 - The Best Commit Messages of 2024
UChicago CMSC 23320 - The Best Commit Messages of 2024UChicago CMSC 23320 - The Best Commit Messages of 2024
UChicago CMSC 23320 - The Best Commit Messages of 2024
 
Stl Algorithms in C++ jjjjjjjjjjjjjjjjjj
Stl Algorithms in C++ jjjjjjjjjjjjjjjjjjStl Algorithms in C++ jjjjjjjjjjjjjjjjjj
Stl Algorithms in C++ jjjjjjjjjjjjjjjjjj
 
8 Tips for Effective Working Capital Management
8 Tips for Effective Working Capital Management8 Tips for Effective Working Capital Management
8 Tips for Effective Working Capital Management
 
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT TOÁN 2024 - TỪ CÁC TRƯỜNG, TRƯỜNG...
 
Major project report on Tata Motors and its marketing strategies
Major project report on Tata Motors and its marketing strategiesMajor project report on Tata Motors and its marketing strategies
Major project report on Tata Motors and its marketing strategies
 

ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdf

  • 1. ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming Vignesh V Menon1, Hadi Amirpour1, Mohammad Ghanbari1,2, and Christian Timmerer1 1 Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität, Klagenfurt, Austria 2 School of Computer Science and Electronic Engineering, University of Essex, UK 16-19 October 2022 Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 1
  • 2. Outline 1 Introduction 2 Research Problem 3 ETPS 4 Evaluation 5 Conclusions Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 2
  • 3. Introduction Introduction CBR versus VBR Figure: Constant Bitrate (CBR) versus Variable Bitrate (VBR) encoding. In live streaming, Constant Bitrate (CBR) rate- control mode is used to encode video sequences at a fixed bitrate ladder. The consistency of CBR makes it more reliable for time-sensitive data transport. In this method, there is no concern about the bitrate exceeding internet speeds. However, this method may result in low compression efficiency. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 3
  • 4. Introduction Introduction Variable Bitrate (VBR) encoding Figure: Two-pass encoding. VBR encoding schemes are implemented as two-pass encoding, also known as multi-pass encoding used to retain the best compression efficiency during encoding. In the first-pass of two-pass encoding, the input data from the video is analyzed (and stored in a log file). The collected data from the first-pass is used to achieve the best encoding compression efficiency in the second-pass. Two-pass encoding is usually controlled by the average bitrate setting, bitrate range setting (minimal and maximal allowed bitrate), or target video file size. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 4
  • 5. Introduction Introduction Constrained Variable Bitrate (cVBR) encoding Figure: cVBR encoding. A ”rate factor” first-pass to identify the optimized CRF to achieve the target bitrate. In the second-pass, the segment is encoded with the selected optimized CRF with the maximum bitrate and maximum buffer window constraints. The desired target bitrate is achieved with maximum compression efficiency, and min- imum quality fluctuation. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 5
  • 6. Research Problem Research Problem In two-pass encoding, the encoder processes all segments for the stream twice. Due to the latency introduced by the first-pass to achieve the optimal encoding parameters for the second-pass, two-pass encoding is not used in live streaming applications. This paper targets a two-pass encoding scheme optimized for live streaming applications. The target average bitrate (bT ) is aimed to be achieved uniformly throughout the streaming session with the highest possible compression efficiency. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 6
  • 7. ETPS ETPS Figure: ETPS architecture. The first-pass predicts the optimized CRF to achieve the target bitrate. In the second-pass, the segment is encoded with the selected, optimized CRF with the maximum bitrate and maximum buffer window constraints. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 7
  • 8. ETPS Phase 1: Video Complexity Feature Extraction ETPS Phase 1: Video Complexity Feature Extraction Compute texture energy per block A DCT-based energy function is used to determine the block-wise feature of each frame defined as: Hk = w−1 X i=0 w−1 X j=0 e|( ij wh )2−1| |DCT(i, j)| (1) where wxw is the size of the block, and DCT(i, j) is the (i, j)th DCT component when i + j > 0, and 0 otherwise. The energy values of blocks in a frame is averaged to determine the energy per frame1,.2 E = C−1 X k=0 Hp,k C · w2 (2) 1 Michael King et al. “A New Energy Function for Segmentation and Compression”. In: 2007 IEEE International Conference on Multimedia and Expo. 2007, pp. 1647–1650. doi: 10.1109/ICME.2007.4284983. 2 Vignesh V Menon et al. “Efficient Content-Adaptive Feature-Based Shot Detection for HTTP Adaptive Streaming”. In: 2021 IEEE International Conference on Image Processing (ICIP). 2021, pp. 2174–2178. doi: 10.1109/ICIP42928.2021.9506092. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 8
  • 9. ETPS Phase 1: Video Complexity Feature Extraction ETPS Phase 1: Video Complexity Feature Extraction hp: SAD of the block level energy values of frame p to that of the previous frame p − 1. hp = C−1 X k=0 | Hp,k, Hp−1,k | C · w2 (3) where C denotes the number of blocks in frame p. Latency Speed of feature extraction = 370fps for UHD video with 8 CPU threads and x86 SIMD optimization Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 9
  • 10. ETPS Phase 2: CRF Prediction ETPS Phase 2: CRF Prediction Hidden Layer E R4 Hidden Layer E R4 Input Layer E R3 Output Layer E R1 E h log(b) ĉ A shallow neural network is trained for each resolu- tion r and framerate f , which determines the opti- mized CRF (ĉ) based on E, h, and log(b) for every video segment. The input vector passed to the network is [E, h, log(bT )]. The target bitrate input (bT ) is passed in logarithmic scale to the network to reduce internal covariate shift.3 Rectified Linear Unit (ReLU)4 is used as the activation function and Adam5 is used as optimizer with a learning rate of 10−4. 3 Sergey Ioffe and Christian Szegedy. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”. In: Proceedings of the 32nd International Conference on Machine Learning. Vol. 37. Lille, France: PMLR, 2015, pp. 448–456. 4 Xavier Glorot, Antoine Bordes, and Y. Bengio. “Deep Sparse Rectifier Neural Networks”. In: vol. 15. Jan. 2010. 5 Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015. 2015. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 10
  • 11. Evaluation Test Methodology Test Methodology Table: Encoder Settings. Encoder x265 v3.5 Preset ultrafast Segment length 4 seconds Resolution Ultra HD (2160p) Framerate 30 fps Target bitrates 8, 12, 17, and 20 Mbps Keras6 is used as the machine learning framework. E and h features are extracted from the video segments using VCA7 run in eight CPU threads 6 Nikhil Ketkar. “Introduction to Keras”. In: Oct. 2017, pp. 95–109. isbn: 978-1-4842-2765-7. doi: 10.1007/978-1-4842-2766-4_7. 7 Vignesh V Menon et al. “VCA: Video Complexity Analyzer”. In: 13th ACM Multimedia Systems Conference (MMSys ’22). 2022. isbn: 9781450392839. doi: 10.1145/3524273.3532896. url: https://doi.org/10.1145/3524273.3532896. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 11
  • 12. Evaluation Test Methodology Test Methodology Criteria: Root Mean Squared Error (RMSE) of cG and the predicted CRF (ĉ), i.e., || cG − ĉ ||2. The first-pass time, i.e., latency in streaming introduced due to the time taken to extract E and h, and the inference time of the neural network to predict optimized CRF. Bjøntegaard delta rates8 BDRP and BDRV to measure the average increase in bitrate of the representations compared with that of the reference encoding scheme to maintain the same PSNR and VMAF, respectively. ∆T: the overall difference in encoding time of ETPS with the reference encoding scheme. The results are compared with CBR encoding and two-pass average bitrate encoding schemes9. 8 G. Bjontegaard. “Calculation of average PSNR differences between RD-curves”. In: VCEG-M33 (2001). 9 https://x265.readthedocs.io/en/master/cli.html#quality-rate-control-and-rate-distortion-options, last access: Jun 30, 2022. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 12
  • 13. Evaluation Experimental Results Experimental Results Table: Results of ETPS against Constant Bitrate (CBR) encoding and two-pass average bitrate encoding schemes. Constant bitrate (CBR) Two-pass average bitrate Dataset Video E h || cG − ĉ ||2 τp (in ms) BDRP BDRV ∆T BDRP BDRV ∆T MCML10 Basketball 15.75 9.69 0.86 8 -17.80% -14.36% 0.43% -0.48% 0.50% -45.15% MCML Bunny 23.03 4.89 1.58 7 -9.92% -9.09% 0.78% 0.04% -1.14% -43.88% MCML Crowd 33.11 7.03 1.58 7 -7.63% -1.47% 0.56% 0.75% 1.05% -43.34% MCML Dolls 19.91 12.22 0.86 8 -5.73% -4.05% 1.01% 1.21% 1.97% -45.26% MCML Flowers 12.01 10.47 0.50 7 -8.44% -7.12% -0.59% 0.38% 0.49% -43.90% MCML Park 26.07 27.34 0.71 9 -5.16% -2.62% 1.10% 0.16% 0.87% -44.27% SJTU11 BundNightScape 54.90 11.62 1.93 8 -14.63% -13.51% -1.62% -1.11% -1.27% -40.91% SJTU CampfireParty 51.50 42.38 1.58 8 -4.66% -3.66% -1.72% -0.51% -0.35% -43.04% SJTU Runners 104.30 17.44 1.32 7 -8.14% -2.44% -1.34% -0.77% -1.08% -42.69% SJTU TallBuildings 94.35 7.70 1.65 8 -17.30% -12.54% -1.02% 0.89% 0.59% -43.37% SJTU TrafficAndBuilding 60.54 11.55 1.87 8 -15.41% -9.91% -0.43% 0.06% -2.42% -43.96% SJTU TrafficFlow 58.93 8.61 1.32 8 -15.66% -1.17% 0.74% 1.19% 1.43% -43.86% SJTU TreeShade 80.19 15.83 0.50 8 -11.82% -6.85% -0.79% 0.52% 0.59% -44.75% SJTU Wood 114.26 8.83 0.71 8 -10.14% -21.58% 0.52% 1.34% 1.88% -44.50% Average 1.02 8 -10.89% -8.60% 0.65% 0.26% 0.38% -43.78% 10 Manri Cheon and Jong-Seok Lee. “Subjective and Objective Quality Assessment of Compressed 4K UHD Videos for Immersive Experience”. In: IEEE Transactions on Circuits and Systems for Video Technology 28.7 (2018), pp. 1467–1480. doi: 10.1109/TCSVT.2017.2683504. 11 L. Song et al. “The SJTU 4K Video Sequence Dataset”. In: Fifth International Workshop on Quality of Multimedia Experience (QoMEX2013) (July 2013). Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 13
  • 14. Evaluation Experimental Results Experimental Results MCML sequences are sequentially encoded using ETPS at 8, 12, 17, and 20 Mbps, and the achieved average bitrate is recorded. The average target bitrate is consistently achieved irrespective of the spatial and temporal complexities of the segments. 0 4 8 12 16 20 24 28 32 Time (in seconds) 8 10 12 14 16 18 20 Average bitrate (in Mbps) 8 Mbps 12 Mbps 17 Mbps 20 Mbps Figure: The average bitrate in a streaming session where MCML sequences are encoded using ETPS and are streamed at intervals of four seconds for target average bitrates 8, 12, 17, 20 Mbps. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 14
  • 15. Conclusions Conclusions This paper proposed ETPS, a low-latency two-pass constrained variable bitrate (cVBR) encoding scheme for live streaming applications. ETPS includes a prediction algorithm for optimized CRF at a given target bitrate, resolution, and framerate of video segments. DCT-energy-based features are used to determine segments’ spatial and temporal complex- ity. Live streaming using ETPS has similar compression efficiency as that of the two-pass average bitrate encoding scheme, but with a significant encoding time-saving of 43.78%. Compared to CBR encoding, ETPS yields bitrate saving of 10.89% and 8.60% to maintain the same PSNR and VMAF, respectively, with an additional encoding time of 0.65%. Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 15
  • 16. Q & A Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@aau.at) Vignesh V Menon ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming 16