SlideShare a Scribd company logo
1 of 66
Download to read offline
FaME-ML: Fast Multirate Encoding for HTTP
Adaptive Streaming Using Machine Learning
December 2nd
, 2020
IEEE VCIP
1
Ekrem Çetinkaya, Hadi Amirpour, Christian Timmerer, and Mohammad Ghanbari
● Introduction
● FaME-ML
● Experimental Results
● Conclusion
● Q & A
Agenda
2
Introduction
3
Video Streaming
4
Video Streaming
Share in the Internet Traffic
82%
4
Video Streaming
Share in the Internet Traffic
82%
Content Characteristics
4
Video Streaming
Share in the Internet Traffic
82%
Content Characteristics
1 Million
minutes
Video Streamed Every Second
4
Video Streaming
Share in the Internet Traffic
82%
Content Characteristics
1 Million
minutes
Video Streamed Every Second
As of 2021
* Cisco VNI Forecast Highlights (2021)
4
HTTP Adaptive Streaming (HAS)
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
Very Nice Video 3500Kbps
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
Very Nice Video 3500Kbps
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
Very Nice Video 3500Kbps
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
Very Nice Video 3500Kbps
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
Very Nice Video 3500Kbps
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
Very Nice Video 3500Kbps
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
Play
Very Nice Video 3500Kbps
Play
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
PlayPlay
Very Nice Video 1500Kbps
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
PlayPlay
Very Nice Video 1500Kbps
5
HTTP Adaptive Streaming (HAS)
Very Nice Video
PlayPlay
Very Nice Video 1500Kbps
5
Multi-rate Encoding
6
1500
kbps
2000
kbps
5000
kbps
3500
kbps
Source Video
HTTP Server
Encoding x4
Multi-rate Encoding
6
1500
kbps
2000
kbps
5000
kbps
3500
kbps
Source Video
HTTP Server
Encoding x4
Block Partitioning
7
Block Partitioning
7
Block Partitioning
7
Block Partitioning
PSNR
Bitrate
0
7
Block Partitioning
PSNR
Bitrate
0
1 1 1 1
7
Block Partitioning
PSNR
Bitrate
0
1 1 1 1
2 2 2 2
7
Block Partitioning
PSNR
Bitrate
0
1 1 1 1
7
Block Partitioning
PSNR
Bitrate
0
1 1 1 1
2 2 2 2
7
Block Partitioning
PSNR
Bitrate
0
1 1 1 1
2 2 2 2
3 3 3 3
7
CTU Search Window Bound
QP 22 QP 38QP 30
● Finding = CTUs tend to get higher depth levels as the quality goes up
Upper1 Lower2
8
1 Schroeder, Damien, et al. "Efficient multi-rate video encoding
for HEVC-based adaptive HTTP streaming." IEEE Transactions
on Circuits and systems for Video Technology 28.1 (2016): 143-
157.
2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari,
"Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020
Data Compression Conference (DCC), Snowbird, UT, USA, 2020,
pp. 358-358
CTU Search Window Bound
QP 22 QP 38QP 30
Depth = [0 1 2 3]
● Finding = CTUs tend to get higher depth levels as the quality goes up
Upper1 Lower2
8
1 Schroeder, Damien, et al. "Efficient multi-rate video encoding
for HEVC-based adaptive HTTP streaming." IEEE Transactions
on Circuits and systems for Video Technology 28.1 (2016): 143-
157.
2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari,
"Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020
Data Compression Conference (DCC), Snowbird, UT, USA, 2020,
pp. 358-358
CTU Search Window Bound
QP 22 QP 38QP 30
2
Depth = [0 1 2 3]
● Finding = CTUs tend to get higher depth levels as the quality goes up
Upper1 Lower2
8
1 Schroeder, Damien, et al. "Efficient multi-rate video encoding
for HEVC-based adaptive HTTP streaming." IEEE Transactions
on Circuits and systems for Video Technology 28.1 (2016): 143-
157.
2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari,
"Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020
Data Compression Conference (DCC), Snowbird, UT, USA, 2020,
pp. 358-358
CTU Search Window Bound
QP 22 QP 38QP 30
2
3
Depth = [0 1 2 3]
● Finding = CTUs tend to get higher depth levels as the quality goes up
Upper1 Lower2
8
1 Schroeder, Damien, et al. "Efficient multi-rate video encoding
for HEVC-based adaptive HTTP streaming." IEEE Transactions
on Circuits and systems for Video Technology 28.1 (2016): 143-
157.
2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari,
"Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020
Data Compression Conference (DCC), Snowbird, UT, USA, 2020,
pp. 358-358
CTU Search Window Bound
QP 22 QP 38QP 30
2
3
Depth = [0 1 2 3]
Depth = [0 1 2 3]
● Finding = CTUs tend to get higher depth levels as the quality goes up
Upper1 Lower2
8
1 Schroeder, Damien, et al. "Efficient multi-rate video encoding
for HEVC-based adaptive HTTP streaming." IEEE Transactions
on Circuits and systems for Video Technology 28.1 (2016): 143-
157.
2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari,
"Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020
Data Compression Conference (DCC), Snowbird, UT, USA, 2020,
pp. 358-358
CTU Search Window Bound
QP 22 QP 38
1
QP 30
2
3
Depth = [0 1 2 3]
Depth = [0 1 2 3]
● Finding = CTUs tend to get higher depth levels as the quality goes up
Upper1 Lower2
8
1 Schroeder, Damien, et al. "Efficient multi-rate video encoding
for HEVC-based adaptive HTTP streaming." IEEE Transactions
on Circuits and systems for Video Technology 28.1 (2016): 143-
157.
2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari,
"Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020
Data Compression Conference (DCC), Snowbird, UT, USA, 2020,
pp. 358-358
CTU Search Window Bound
QP 22 QP 38
1
QP 30
2
3
2
Depth = [0 1 2 3]
Depth = [0 1 2 3]
● Finding = CTUs tend to get higher depth levels as the quality goes up
Upper1 Lower2
8
1 Schroeder, Damien, et al. "Efficient multi-rate video encoding
for HEVC-based adaptive HTTP streaming." IEEE Transactions
on Circuits and systems for Video Technology 28.1 (2016): 143-
157.
2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari,
"Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020
Data Compression Conference (DCC), Snowbird, UT, USA, 2020,
pp. 358-358
Problem & Solution
● Existing methods typically use the highest
quality representation as the reference
● Cannot reduce the parallel encoding time
● The highest quality representation is the
bottleneck
● Use the lowest quality representation as the
reference
● Utilize machine learning for better
performance
● Focus on parallel encoding time
○ Reduce the encoding-time of the highest
complexity representations
○ Eliminate the encoding-time bottleneck
9
Normalized time-complexity of different quality
representations using three different encoding
methods
1 Schroeder, Damien, et al. "Efficient multi-rate video encoding
for HEVC-based adaptive HTTP streaming." IEEE Transactions
on Circuits and systems for Video Technology 28.1 (2016): 143-
157.
2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari,
"Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020
Data Compression Conference (DCC), Snowbird, UT, USA, 2020,
pp. 358-358
1 2
FaME-ML
10
Features
11
Features
● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD)
11
FRD
5
Features
● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD)
11
FRD FV
5 5
● Variance of pixels = Inside the CU (FV)
Features
● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD)
11
FRD FV FMV
5 5 1
● Variance of pixels = Inside the CU (FV)
● Motion vectors = Average magnitude of MVs inside the CU (FMV)
Features
● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD)
11
FRD FV FMV FD
5 5 1 1
● Variance of pixels = Inside the CU (FV)
● Motion vectors = Average magnitude of MVs inside the CU (FMV)
● Depth level = CU split decision for given depth level (FD)
Features
● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD)
11
FRD FV FMV FD FQP
5 5 1 1 1
● Variance of pixels = Inside the CU (FV)
● Motion vectors = Average magnitude of MVs inside the CU (FMV)
● Depth level = CU split decision for given depth level (FD)
● Frame level QP = QP value for the given frame (FQP)
Features
● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD)
11
FRD FV FMV FD FQP FPU
5 5 1 1 1 1
● Variance of pixels = Inside the CU (FV)
● Motion vectors = Average magnitude of MVs inside the CU (FMV)
● Depth level = CU split decision for given depth level (FD)
● Frame level QP = QP value for the given frame (FQP)
● PU decision = PU split decision for the given CU (FPU)
Features
● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD)
11
FRD FV FMV FD FQP FPU
5 5 1 1 1 1
F =
● Variance of pixels = Inside the CU (FV)
● Motion vectors = Average magnitude of MVs inside the CU (FMV)
● Depth level = CU split decision for given depth level (FD)
● Frame level QP = QP value for the given frame (FQP)
● PU decision = PU split decision for the given CU (FPU)
14
Training Dataset
● 12 Test sequences defined in HEVC CTC3
● YUV information are extracted for each CU
○ 64x64 size for D0 and 32x32 size for D1
● Sequences are encoded with HEVC reference software (HM 16.21)4
○ Encoding information are extracted and saved for QP38
○ 64x64 size for D0 and 32x32 size for D1
○ Features are individually min-max normalized in video level
○ Depth values are saved as targets for remaining QPs
● 90 % of frames for training set (259,200 CTUs)
● 10 % of frames for validation set (28,800 CTUs)
12
3 F. Bossen et al., “Common test conditions and software
reference configurations,” JCTVC-L1100, vol. 12, p. 7, 2013
4 https://vcgit.hhi.fraunhofer.de/jct-vc/HM
Convolutional Neural Network (CNN)
13
Y,U,V input sizes are halved and red part is dismissed in the Depth 1 classifier.
Overall Method
● Encode the lowest quality representation
with HEVC reference software
○ Save the encoding information
● Pass YUV information to texture
processing CNN and get an intermediate
decisions
● Combine the intermediate decision with
feature vector and pass through a fully
connected layer to get the final decision
● Apply CNN for bottleneck quality levels in
parallel encoding scenario
○ Depth 0 and Depth 1 for QP22
○ Depth 0 for QP26
Overall Method
● Encode the lowest quality representation
with HEVC reference software
○ Save the encoding information
● Pass YUV information to texture
processing CNN and get an intermediate
decisions
● Combine the intermediate decision with
feature vector and pass through a fully
connected layer to get the final decision
● Apply CNN for bottleneck quality levels in
parallel encoding scenario
○ Depth 0 and Depth 1 for QP22
○ Depth 0 for QP26
Overall Method
QP38
HEVC
● Encode the lowest quality representation
with HEVC reference software
○ Save the encoding information
● Pass YUV information to texture
processing CNN and get an intermediate
decisions
● Combine the intermediate decision with
feature vector and pass through a fully
connected layer to get the final decision
● Apply CNN for bottleneck quality levels in
parallel encoding scenario
○ Depth 0 and Depth 1 for QP22
○ Depth 0 for QP26
Overall Method
QP38
QP34QP30
HEVC
HEVCHEVC
● Encode the lowest quality representation
with HEVC reference software
○ Save the encoding information
● Pass YUV information to texture
processing CNN and get an intermediate
decisions
● Combine the intermediate decision with
feature vector and pass through a fully
connected layer to get the final decision
● Apply CNN for bottleneck quality levels in
parallel encoding scenario
○ Depth 0 and Depth 1 for QP22
○ Depth 0 for QP26
Overall Method
QP38
CNN
QP34QP22 QP26 QP30
HEVC
HEVCHEVC
CNN
HEVC HEVC
● Encode the lowest quality representation
with HEVC reference software
○ Save the encoding information
● Pass YUV information to texture
processing CNN and get an intermediate
decisions
● Combine the intermediate decision with
feature vector and pass through a fully
connected layer to get the final decision
● Apply CNN for bottleneck quality levels in
parallel encoding scenario
○ Depth 0 and Depth 1 for QP22
○ Depth 0 for QP26
Experimental Results
15
Experiment Settings
● 8 Test sequences from SVT 5 and JVET 6 datasets
● Five QP levels [38, 34, 30, 26, 22]
● Low-Delay P configuration
● Bjontegaard Delta 7 Rate with PSNR and VMAF 8 are calculated
● Encoding performance is compared with HEVC reference software (HM 16.21)4 and the
lower bound approach 2
○ Lower bound = Minimum CTU depth search value is limited by the lowest quality
representation
16
5 L. Haglund, “The SVT high definition multi format test
set,”SwedishTelevision Stockholm, 2006
6 K. Suehring and X. Li, “JVET common test conditions and
software reference configurations,”JVET-B1010, 2016
7 G. Bjontegaard, “Calculation of average PSNR differences
between RD-curves,”VCEG-M33, 2001.
8 Z.Li,A.Aaron,I.Katsavounidis,A.Moorthy, and
M.Manohara,“Towards practical perceptual video quality
metric,”[Online]https://netflixtechblog.com/toward-a-
practical-perceptual-video-quality-metric-653f208b9652,2016
Encoding Results
● Compared with the HM
● Calculated over five QP levels
● ΔT is the difference between the maximum time complexity of each method
● BDRP and BDRV are the Bjontegaard Delta rates with PSNR and VMAF respectively
● 41 % time saving for parallel encoding (difference between the highest time complexity
representations)
17
Encoding Time Graph
18
0.59
0.88
1.00
Conclusion
19
Conclusion
● Machine learning based approach for fast multi-rate encoding
○ Focus on the parallel encoding performance
● The lowest quality representation is used as the reference
● CNN is used for CU split decision for a given depth level
● Method is applied on the highest two complexity representations
○ Bottleneck encoding times are reduced with minimal quality increase
● 41 % time saving for parallel encoding with 0.88 % bitrate increase in average
20
Thank you
21
ekrem@itec.aau.at

More Related Content

What's hot

Press Release of 131st WG11 (MPEG) Meeting
Press Release of 131st WG11 (MPEG) MeetingPress Release of 131st WG11 (MPEG) Meeting
Press Release of 131st WG11 (MPEG) MeetingAlpen-Adria-Universität
 
A Distributed Delivery Architecture for User Generated Content Live Streaming...
A Distributed Delivery Architecture for User Generated Content Live Streaming...A Distributed Delivery Architecture for User Generated Content Live Streaming...
A Distributed Delivery Architecture for User Generated Content Live Streaming...Alpen-Adria-Universität
 
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive StreamingMiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive StreamingAlpen-Adria-Universität
 
WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile D...
WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile D...WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile D...
WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile D...Minh Nguyen
 
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...Alpen-Adria-Universität
 
INCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCINCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCAlpen-Adria-Universität
 
Towards Optimal Multirate Encoding for HTTP Adaptive Streaming
Towards Optimal Multirate Encoding for HTTP Adaptive StreamingTowards Optimal Multirate Encoding for HTTP Adaptive Streaming
Towards Optimal Multirate Encoding for HTTP Adaptive StreamingAlpen-Adria-Universität
 
Bandwidth Prediction in Low-Latency Chunked Streaming
Bandwidth Prediction in Low-Latency Chunked StreamingBandwidth Prediction in Low-Latency Chunked Streaming
Bandwidth Prediction in Low-Latency Chunked StreamingAlpen-Adria-Universität
 
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...Alpen-Adria-Universität
 
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video StreamingCSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video StreamingAlpen-Adria-Universität
 
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video StreamingES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video StreamingAlpen-Adria-Universität
 
Video complexity analyzer (VCA) for streaming applications
 Video complexity analyzer (VCA) for streaming applications Video complexity analyzer (VCA) for streaming applications
Video complexity analyzer (VCA) for streaming applicationsAlpen-Adria-Universität
 
FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machin...
FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machin...FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machin...
FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machin...Alpen-Adria-Universität
 
H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive...
H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive...H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive...
H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive...Alpen-Adria-Universität
 
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...Minh Nguyen
 
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...Alpen-Adria-Universität
 
CAdViSE: Cloud based Adaptive Video Streaming Evaluation Framework for the Au...
CAdViSE: Cloud based Adaptive Video Streaming Evaluation Framework for the Au...CAdViSE: Cloud based Adaptive Video Streaming Evaluation Framework for the Au...
CAdViSE: Cloud based Adaptive Video Streaming Evaluation Framework for the Au...Alpen-Adria-Universität
 

What's hot (20)

SLFC: Scalable Light Field Coding
SLFC: Scalable Light Field CodingSLFC: Scalable Light Field Coding
SLFC: Scalable Light Field Coding
 
Press Release of 131st WG11 (MPEG) Meeting
Press Release of 131st WG11 (MPEG) MeetingPress Release of 131st WG11 (MPEG) Meeting
Press Release of 131st WG11 (MPEG) Meeting
 
What’s new in MPEG?
What’s new in MPEG?What’s new in MPEG?
What’s new in MPEG?
 
A Distributed Delivery Architecture for User Generated Content Live Streaming...
A Distributed Delivery Architecture for User Generated Content Live Streaming...A Distributed Delivery Architecture for User Generated Content Live Streaming...
A Distributed Delivery Architecture for User Generated Content Live Streaming...
 
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive StreamingMiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
MiPSO: Multi-Period Per-Scene Optimization For HTTP Adaptive Streaming
 
WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile D...
WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile D...WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile D...
WISH: User-centric Bitrate Adaptation for HTTP Adaptive Streaming on Mobile D...
 
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of th...
 
INCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVCINCEPT: Intra CU Depth Prediction for HEVC
INCEPT: Intra CU Depth Prediction for HEVC
 
Towards Optimal Multirate Encoding for HTTP Adaptive Streaming
Towards Optimal Multirate Encoding for HTTP Adaptive StreamingTowards Optimal Multirate Encoding for HTTP Adaptive Streaming
Towards Optimal Multirate Encoding for HTTP Adaptive Streaming
 
Bandwidth Prediction in Low-Latency Chunked Streaming
Bandwidth Prediction in Low-Latency Chunked StreamingBandwidth Prediction in Low-Latency Chunked Streaming
Bandwidth Prediction in Low-Latency Chunked Streaming
 
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
Where to Encode: A Performance Analysis of Intel x86 and Arm-based Amazon EC2...
 
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video StreamingCSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
CSDN: CDN-Aware QoE Optimization in SDN-Assisted HTTP Adaptive Video Streaming
 
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video StreamingES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
ES-HAS: An Edge- and SDN-Assisted Framework for HTTP Adaptive Video Streaming
 
Video complexity analyzer (VCA) for streaming applications
 Video complexity analyzer (VCA) for streaming applications Video complexity analyzer (VCA) for streaming applications
Video complexity analyzer (VCA) for streaming applications
 
FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machin...
FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machin...FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machin...
FAUST: Fast Per-Scene Encoding Using Entropy-Based Scene Detection and Machin...
 
H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive...
H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive...H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive...
H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive...
 
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
EPIQ'21: Days of Future Past: An Optimization-based Adaptive Bitrate Algorith...
 
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
A Channel Allocation Algorithm for Cognitive Radio Users Based on Channel Sta...
 
CAdViSE: Cloud based Adaptive Video Streaming Evaluation Framework for the Au...
CAdViSE: Cloud based Adaptive Video Streaming Evaluation Framework for the Au...CAdViSE: Cloud based Adaptive Video Streaming Evaluation Framework for the Au...
CAdViSE: Cloud based Adaptive Video Streaming Evaluation Framework for the Au...
 
HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?HTTP Adaptive Streaming – Quo Vadis?
HTTP Adaptive Streaming – Quo Vadis?
 

Similar to FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning

Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentAlpen-Adria-Universität
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentMinh Nguyen
 
Encoding stored video for stremming applications ieee paper ppt
Encoding stored video for stremming applications ieee paper pptEncoding stored video for stremming applications ieee paper ppt
Encoding stored video for stremming applications ieee paper pptNavin Kumar
 
Motion Vector Recovery for Real-time H.264 Video Streams
Motion Vector Recovery for Real-time H.264 Video StreamsMotion Vector Recovery for Real-time H.264 Video Streams
Motion Vector Recovery for Real-time H.264 Video StreamsIDES Editor
 
LwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
LwTE-Live: Light-weight Transcoding at the Edge for Live StreamingLwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
LwTE-Live: Light-weight Transcoding at the Edge for Live StreamingAlpen-Adria-Universität
 
CADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive Streaming
CADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive StreamingCADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive Streaming
CADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive StreamingMinh Nguyen
 
Encoding at Scale for Live Video Streaming
Encoding at Scale for Live Video StreamingEncoding at Scale for Live Video Streaming
Encoding at Scale for Live Video StreamingRay Adensamer
 
Online Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingOnline Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingVignesh V Menon
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Alpen-Adria-Universität
 
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive StreamingQoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive StreamingDanieleLorenzi6
 
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...Alpen-Adria-Universität
 
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...FIAT/IFTA
 
Design and Implementation of HDMI Transmitter
Design and Implementation of HDMI TransmitterDesign and Implementation of HDMI Transmitter
Design and Implementation of HDMI TransmitterIJERA Editor
 
H.265ImprovedCE_over_H.264-HarmonicMay2014Final
H.265ImprovedCE_over_H.264-HarmonicMay2014FinalH.265ImprovedCE_over_H.264-HarmonicMay2014Final
H.265ImprovedCE_over_H.264-HarmonicMay2014FinalDonald Pian
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningEkrem Çetinkaya
 
Video Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingVideo Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingAlpen-Adria-Universität
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfVignesh V Menon
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningAlpen-Adria-Universität
 

Similar to FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning (20)

Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
 
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player EnvironmentPolicy-Driven Dynamic HTTP Adaptive Streaming Player Environment
Policy-Driven Dynamic HTTP Adaptive Streaming Player Environment
 
Encoding stored video for stremming applications ieee paper ppt
Encoding stored video for stremming applications ieee paper pptEncoding stored video for stremming applications ieee paper ppt
Encoding stored video for stremming applications ieee paper ppt
 
Midtem_19082004
Midtem_19082004Midtem_19082004
Midtem_19082004
 
Motion Vector Recovery for Real-time H.264 Video Streams
Motion Vector Recovery for Real-time H.264 Video StreamsMotion Vector Recovery for Real-time H.264 Video Streams
Motion Vector Recovery for Real-time H.264 Video Streams
 
LwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
LwTE-Live: Light-weight Transcoding at the Edge for Live StreamingLwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
LwTE-Live: Light-weight Transcoding at the Edge for Live Streaming
 
CADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive Streaming
CADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive StreamingCADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive Streaming
CADLAD: Device-aware Bitrate Ladder Construction for HTTP Adaptive Streaming
 
Encoding at Scale for Live Video Streaming
Encoding at Scale for Live Video StreamingEncoding at Scale for Live Video Streaming
Encoding at Scale for Live Video Streaming
 
Online Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC StreamingOnline Bitrate ladder prediction for Adaptive VVC Streaming
Online Bitrate ladder prediction for Adaptive VVC Streaming
 
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, S...
 
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive StreamingQoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
QoE- and Energy-aware Content Consumption for HTTP Adaptive Streaming
 
H263.ppt
H263.pptH263.ppt
H263.ppt
 
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...Optimizing  QoE and Latency of  Live Video Streaming Using  Edge Computing  a...
Optimizing QoE and Latency of Live Video Streaming Using Edge Computing a...
 
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
Are you Digitized Files Really OK? Levels of QC and Film Digitization (SCHALL...
 
Design and Implementation of HDMI Transmitter
Design and Implementation of HDMI TransmitterDesign and Implementation of HDMI Transmitter
Design and Implementation of HDMI Transmitter
 
H.265ImprovedCE_over_H.264-HarmonicMay2014Final
H.265ImprovedCE_over_H.264-HarmonicMay2014FinalH.265ImprovedCE_over_H.264-HarmonicMay2014Final
H.265ImprovedCE_over_H.264-HarmonicMay2014Final
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 
Video Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive StreamingVideo Coding Enhancements for HTTP Adaptive Streaming
Video Coding Enhancements for HTTP Adaptive Streaming
 
Research@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdfResearch@Lunch_Presentation.pdf
Research@Lunch_Presentation.pdf
 
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine LearningVideo Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
Video Coding Enhancements for HTTP Adaptive Streaming Using Machine Learning
 

More from Alpen-Adria-Universität

Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingAlpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Alpen-Adria-Universität
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...Alpen-Adria-Universität
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...Alpen-Adria-Universität
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Alpen-Adria-Universität
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Alpen-Adria-Universität
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamAlpen-Adria-Universität
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingAlpen-Adria-Universität
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...Alpen-Adria-Universität
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesAlpen-Adria-Universität
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Alpen-Adria-Universität
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsAlpen-Adria-Universität
 
Immersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to HolographyImmersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to HolographyAlpen-Adria-Universität
 
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...Alpen-Adria-Universität
 
HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)Alpen-Adria-Universität
 
How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and SolutionsHow to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and SolutionsAlpen-Adria-Universität
 
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumMPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumAlpen-Adria-Universität
 
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingCollaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingAlpen-Adria-Universität
 
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsOTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsAlpen-Adria-Universität
 
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingAlpen-Adria-Universität
 

More from Alpen-Adria-Universität (20)

Content-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive StreamingContent-adaptive Video Coding for HTTP Adaptive Streaming
Content-adaptive Video Coding for HTTP Adaptive Streaming
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Video...
 
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...Empowerment of Atypical Viewers  via Low-Effort Personalized Modeling  of Vid...
Empowerment of Atypical Viewers via Low-Effort Personalized Modeling of Vid...
 
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...Optimizing Video Streaming  for Sustainability and Quality: The Role of Prese...
Optimizing Video Streaming for Sustainability and Quality: The Role of Prese...
 
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
Energy-Efficient Multi-Codec Bitrate-Ladder Estimation for Adaptive Video Str...
 
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...Machine Learning Based Resource Utilization Prediction in the Computing Conti...
Machine Learning Based Resource Utilization Prediction in the Computing Conti...
 
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming StreamEvaluation of Quality of Experience of ABR Schemes in Gaming Stream
Evaluation of Quality of Experience of ABR Schemes in Gaming Stream
 
Multi-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video StreamingMulti-access Edge Computing for Adaptive Video Streaming
Multi-access Edge Computing for Adaptive Video Streaming
 
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...
 
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and StrategiesEnergy Consumption in Video Streaming: Components, Measurements, and Strategies
Energy Consumption in Video Streaming: Components, Measurements, and Strategies
 
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
Exploring the Energy Consumption of Video Streaming: Components, Challenges, ...
 
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming ApplicationsSARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
SARENA: SFC-Enabled Architecture for Adaptive Video Streaming Applications
 
Immersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to HolographyImmersive Video Delivery: From Omnidirectional Video to Holography
Immersive Video Delivery: From Omnidirectional Video to Holography
 
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
LLL-CAdViSE: Live Low-Latency Cloud-based Adaptive Video Streaming Evaluation...
 
HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)HTTP Adaptive Streaming – Quo Vadis? (2023)
HTTP Adaptive Streaming – Quo Vadis? (2023)
 
How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and SolutionsHow to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions
 
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumMPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
MPEC2: Multilayer and Pipeline Video Encoding on the Computing Continuum
 
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video StreamingCollaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
Collaborative Edge-Assisted Systems for HTTP Adaptive Video Streaming
 
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog EnvironmentsOTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
OTEC: An Optimized Transcoding Task Scheduler for Cloud and Fog Environments
 
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live StreamingETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
ETPS: Efficient Two-pass Encoding Scheme for Adaptive Live Streaming
 

Recently uploaded

UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...Karmanjay Verma
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 

Recently uploaded (20)

UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 

FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning

  • 1. FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning December 2nd , 2020 IEEE VCIP 1 Ekrem Çetinkaya, Hadi Amirpour, Christian Timmerer, and Mohammad Ghanbari
  • 2. ● Introduction ● FaME-ML ● Experimental Results ● Conclusion ● Q & A Agenda 2
  • 5. Video Streaming Share in the Internet Traffic 82% 4
  • 6. Video Streaming Share in the Internet Traffic 82% Content Characteristics 4
  • 7. Video Streaming Share in the Internet Traffic 82% Content Characteristics 1 Million minutes Video Streamed Every Second 4
  • 8. Video Streaming Share in the Internet Traffic 82% Content Characteristics 1 Million minutes Video Streamed Every Second As of 2021 * Cisco VNI Forecast Highlights (2021) 4
  • 10. HTTP Adaptive Streaming (HAS) Very Nice Video Play 5
  • 11. HTTP Adaptive Streaming (HAS) Very Nice Video Play 5
  • 12. HTTP Adaptive Streaming (HAS) Very Nice Video Play 5
  • 13. HTTP Adaptive Streaming (HAS) Very Nice Video Play 5
  • 14. HTTP Adaptive Streaming (HAS) Very Nice Video Play 5
  • 15. HTTP Adaptive Streaming (HAS) Very Nice Video Play Very Nice Video 3500Kbps 5
  • 16. HTTP Adaptive Streaming (HAS) Very Nice Video Play Very Nice Video 3500Kbps Play 5
  • 17. HTTP Adaptive Streaming (HAS) Very Nice Video Play Very Nice Video 3500Kbps Play 5
  • 18. HTTP Adaptive Streaming (HAS) Very Nice Video Play Very Nice Video 3500Kbps Play 5
  • 19. HTTP Adaptive Streaming (HAS) Very Nice Video Play Very Nice Video 3500Kbps Play 5
  • 20. HTTP Adaptive Streaming (HAS) Very Nice Video Play Very Nice Video 3500Kbps Play 5
  • 21. HTTP Adaptive Streaming (HAS) Very Nice Video Play Very Nice Video 3500Kbps Play 5
  • 22. HTTP Adaptive Streaming (HAS) Very Nice Video PlayPlay Very Nice Video 1500Kbps 5
  • 23. HTTP Adaptive Streaming (HAS) Very Nice Video PlayPlay Very Nice Video 1500Kbps 5
  • 24. HTTP Adaptive Streaming (HAS) Very Nice Video PlayPlay Very Nice Video 1500Kbps 5
  • 35. Block Partitioning PSNR Bitrate 0 1 1 1 1 2 2 2 2 3 3 3 3 7
  • 36. CTU Search Window Bound QP 22 QP 38QP 30 ● Finding = CTUs tend to get higher depth levels as the quality goes up Upper1 Lower2 8 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143- 157. 2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358
  • 37. CTU Search Window Bound QP 22 QP 38QP 30 Depth = [0 1 2 3] ● Finding = CTUs tend to get higher depth levels as the quality goes up Upper1 Lower2 8 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143- 157. 2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358
  • 38. CTU Search Window Bound QP 22 QP 38QP 30 2 Depth = [0 1 2 3] ● Finding = CTUs tend to get higher depth levels as the quality goes up Upper1 Lower2 8 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143- 157. 2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358
  • 39. CTU Search Window Bound QP 22 QP 38QP 30 2 3 Depth = [0 1 2 3] ● Finding = CTUs tend to get higher depth levels as the quality goes up Upper1 Lower2 8 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143- 157. 2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358
  • 40. CTU Search Window Bound QP 22 QP 38QP 30 2 3 Depth = [0 1 2 3] Depth = [0 1 2 3] ● Finding = CTUs tend to get higher depth levels as the quality goes up Upper1 Lower2 8 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143- 157. 2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358
  • 41. CTU Search Window Bound QP 22 QP 38 1 QP 30 2 3 Depth = [0 1 2 3] Depth = [0 1 2 3] ● Finding = CTUs tend to get higher depth levels as the quality goes up Upper1 Lower2 8 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143- 157. 2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358
  • 42. CTU Search Window Bound QP 22 QP 38 1 QP 30 2 3 2 Depth = [0 1 2 3] Depth = [0 1 2 3] ● Finding = CTUs tend to get higher depth levels as the quality goes up Upper1 Lower2 8 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143- 157. 2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358
  • 43. Problem & Solution ● Existing methods typically use the highest quality representation as the reference ● Cannot reduce the parallel encoding time ● The highest quality representation is the bottleneck ● Use the lowest quality representation as the reference ● Utilize machine learning for better performance ● Focus on parallel encoding time ○ Reduce the encoding-time of the highest complexity representations ○ Eliminate the encoding-time bottleneck 9 Normalized time-complexity of different quality representations using three different encoding methods 1 Schroeder, Damien, et al. "Efficient multi-rate video encoding for HEVC-based adaptive HTTP streaming." IEEE Transactions on Circuits and systems for Video Technology 28.1 (2016): 143- 157. 2 H. Amirpour, E. Çetinkaya, C. Timmerer and M. Ghanbari, "Fast Multi-rate Encoding for Adaptive HTTP Streaming," 2020 Data Compression Conference (DCC), Snowbird, UT, USA, 2020, pp. 358-358 1 2
  • 46. Features ● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD) 11 FRD 5
  • 47. Features ● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD) 11 FRD FV 5 5 ● Variance of pixels = Inside the CU (FV)
  • 48. Features ● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD) 11 FRD FV FMV 5 5 1 ● Variance of pixels = Inside the CU (FV) ● Motion vectors = Average magnitude of MVs inside the CU (FMV)
  • 49. Features ● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD) 11 FRD FV FMV FD 5 5 1 1 ● Variance of pixels = Inside the CU (FV) ● Motion vectors = Average magnitude of MVs inside the CU (FMV) ● Depth level = CU split decision for given depth level (FD)
  • 50. Features ● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD) 11 FRD FV FMV FD FQP 5 5 1 1 1 ● Variance of pixels = Inside the CU (FV) ● Motion vectors = Average magnitude of MVs inside the CU (FMV) ● Depth level = CU split decision for given depth level (FD) ● Frame level QP = QP value for the given frame (FQP)
  • 51. Features ● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD) 11 FRD FV FMV FD FQP FPU 5 5 1 1 1 1 ● Variance of pixels = Inside the CU (FV) ● Motion vectors = Average magnitude of MVs inside the CU (FMV) ● Depth level = CU split decision for given depth level (FD) ● Frame level QP = QP value for the given frame (FQP) ● PU decision = PU split decision for the given CU (FPU)
  • 52. Features ● RD Cost = Number of bits to encode the given CU and four sub-CUs (FRD) 11 FRD FV FMV FD FQP FPU 5 5 1 1 1 1 F = ● Variance of pixels = Inside the CU (FV) ● Motion vectors = Average magnitude of MVs inside the CU (FMV) ● Depth level = CU split decision for given depth level (FD) ● Frame level QP = QP value for the given frame (FQP) ● PU decision = PU split decision for the given CU (FPU) 14
  • 53. Training Dataset ● 12 Test sequences defined in HEVC CTC3 ● YUV information are extracted for each CU ○ 64x64 size for D0 and 32x32 size for D1 ● Sequences are encoded with HEVC reference software (HM 16.21)4 ○ Encoding information are extracted and saved for QP38 ○ 64x64 size for D0 and 32x32 size for D1 ○ Features are individually min-max normalized in video level ○ Depth values are saved as targets for remaining QPs ● 90 % of frames for training set (259,200 CTUs) ● 10 % of frames for validation set (28,800 CTUs) 12 3 F. Bossen et al., “Common test conditions and software reference configurations,” JCTVC-L1100, vol. 12, p. 7, 2013 4 https://vcgit.hhi.fraunhofer.de/jct-vc/HM
  • 54. Convolutional Neural Network (CNN) 13 Y,U,V input sizes are halved and red part is dismissed in the Depth 1 classifier.
  • 55. Overall Method ● Encode the lowest quality representation with HEVC reference software ○ Save the encoding information ● Pass YUV information to texture processing CNN and get an intermediate decisions ● Combine the intermediate decision with feature vector and pass through a fully connected layer to get the final decision ● Apply CNN for bottleneck quality levels in parallel encoding scenario ○ Depth 0 and Depth 1 for QP22 ○ Depth 0 for QP26
  • 56. Overall Method ● Encode the lowest quality representation with HEVC reference software ○ Save the encoding information ● Pass YUV information to texture processing CNN and get an intermediate decisions ● Combine the intermediate decision with feature vector and pass through a fully connected layer to get the final decision ● Apply CNN for bottleneck quality levels in parallel encoding scenario ○ Depth 0 and Depth 1 for QP22 ○ Depth 0 for QP26
  • 57. Overall Method QP38 HEVC ● Encode the lowest quality representation with HEVC reference software ○ Save the encoding information ● Pass YUV information to texture processing CNN and get an intermediate decisions ● Combine the intermediate decision with feature vector and pass through a fully connected layer to get the final decision ● Apply CNN for bottleneck quality levels in parallel encoding scenario ○ Depth 0 and Depth 1 for QP22 ○ Depth 0 for QP26
  • 58. Overall Method QP38 QP34QP30 HEVC HEVCHEVC ● Encode the lowest quality representation with HEVC reference software ○ Save the encoding information ● Pass YUV information to texture processing CNN and get an intermediate decisions ● Combine the intermediate decision with feature vector and pass through a fully connected layer to get the final decision ● Apply CNN for bottleneck quality levels in parallel encoding scenario ○ Depth 0 and Depth 1 for QP22 ○ Depth 0 for QP26
  • 59. Overall Method QP38 CNN QP34QP22 QP26 QP30 HEVC HEVCHEVC CNN HEVC HEVC ● Encode the lowest quality representation with HEVC reference software ○ Save the encoding information ● Pass YUV information to texture processing CNN and get an intermediate decisions ● Combine the intermediate decision with feature vector and pass through a fully connected layer to get the final decision ● Apply CNN for bottleneck quality levels in parallel encoding scenario ○ Depth 0 and Depth 1 for QP22 ○ Depth 0 for QP26
  • 61. Experiment Settings ● 8 Test sequences from SVT 5 and JVET 6 datasets ● Five QP levels [38, 34, 30, 26, 22] ● Low-Delay P configuration ● Bjontegaard Delta 7 Rate with PSNR and VMAF 8 are calculated ● Encoding performance is compared with HEVC reference software (HM 16.21)4 and the lower bound approach 2 ○ Lower bound = Minimum CTU depth search value is limited by the lowest quality representation 16 5 L. Haglund, “The SVT high definition multi format test set,”SwedishTelevision Stockholm, 2006 6 K. Suehring and X. Li, “JVET common test conditions and software reference configurations,”JVET-B1010, 2016 7 G. Bjontegaard, “Calculation of average PSNR differences between RD-curves,”VCEG-M33, 2001. 8 Z.Li,A.Aaron,I.Katsavounidis,A.Moorthy, and M.Manohara,“Towards practical perceptual video quality metric,”[Online]https://netflixtechblog.com/toward-a- practical-perceptual-video-quality-metric-653f208b9652,2016
  • 62. Encoding Results ● Compared with the HM ● Calculated over five QP levels ● ΔT is the difference between the maximum time complexity of each method ● BDRP and BDRV are the Bjontegaard Delta rates with PSNR and VMAF respectively ● 41 % time saving for parallel encoding (difference between the highest time complexity representations) 17
  • 65. Conclusion ● Machine learning based approach for fast multi-rate encoding ○ Focus on the parallel encoding performance ● The lowest quality representation is used as the reference ● CNN is used for CU split decision for a given depth level ● Method is applied on the highest two complexity representations ○ Bottleneck encoding times are reduced with minimal quality increase ● 41 % time saving for parallel encoding with 0.88 % bitrate increase in average 20