Advertisement

May. 5, 2023•0 likes## 0 likes

•20 views## views

Be the first to like this

Show More

Total views

0

On Slideshare

0

From embeds

0

Number of embeds

0

Download to read offline

Report

Education

For adaptive streaming applications, low-complexity and accurate video complexity features are necessary to analyze the video content in real time, which ensures fast and compression-efficient video streaming without disruptions. State-of-the-art video complexity features are Spatial Information (SI) and Temporal Information (TI) features which do not correlate well with the encoding parameters in adaptive streaming applications. To this light, Video Complexity Analyzer (VCA) was introduced, determining the features based on Discrete Cosine Transform (DCT)-energy. This paper presents optimizations on VCA for faster and energy-efficient video complexity analysis. Experimental results show that VCA v2.0, using eight CPU threads, Single Instruction Multiple Data (SIMD), and low-pass DCT optimization, determines seven complexity features of Ultra High Definition 8-bit videos with better accuracy at a speed of up to 292.68 fps and an energy consumption of 97.06% lower than the reference SITI implementation.

Vignesh V MenonFollow

Advertisement

Advertisement

Advertisement

TQPM.pdfVignesh V Menon

OPSE: Online Per-Scene Encoding for Adaptive HTTP Live StreamingAlpen-Adria-Universität

OPSE_Online Per-Scene Encoding for Adaptive HTTP Live Streaming.pdfVignesh V Menon

OPTE: Online Per-title Encoding for Live Video Streaming.pdfVignesh V Menon

Video complexity analyzer (VCA) for streaming applicationsAlpen-Adria-Universität

ETPS_Efficient_Two_pass_Encoding_Scheme_for_Adaptive_Streaming.pdfVignesh V Menon

- Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming Vignesh V Menon1, Christian Feldmann2, Klaus Schoeffmann1, Mohammad Ghanbari3, Christian Timmerer1 1 Christian Doppler Laboratory ATHENA, Alpen-Adria-Universität, Klagenfurt, Austria 2 Bitmovin, Klagenfurt, Austria 3 School of Computer Science and Electronic Engineering, University of Essex, UK 7 – 10 June 2023 Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 1
- Outline 1 Background 2 SITI 3 Video Complexity Features 4 Accuracy Analysis 5 Performance Optimizations 6 Conclusions Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 2
- Background Background Why VCA? The optimal encoding parameters depend on the video content complexity. There is a need to extract content features that can represent the video content complexity to predict the optimal encoding parameters for that video content. Encoder Analysis stats Encoded bitstream Input video Video complexity analysis Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 3
- SITI SITI1 Spatial Information (SI) indicates the maximum spatial detail in a video. SI = max{std[Sobel(F(x, y, t))]} (1) where F(x, y, t) represents each video frame. Temporal Information (TI) is the maximum temporal variation between two successive frames. D(x, y, t) = F(x, y, t)) − F(x, y, t − 1)). (2) TI = max{std[D(x, y, t)]} (3) 1 https://github.com/Telecommunication-Telemedia-Assessment/SITI, last access: Apr 30, 2023. Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 4
- Video Complexity Features Texture Energy Texture Energy Compute texture energy per block A DCT-based energy function is used to determine the block-wise feature of each frame defined as: HY ,p,k = w−1 X i=0 w−1 X j=0 e|( ij wh )2−1| |DCT(i, j)| (4) where wxw is the size of the block, and DCT(i, j) is the (i, j)th DCT component when i + j > 0, and 0 otherwise. The energy values of blocks in a frame are averaged to determine the energy per frame.2 EY = C−1 X k=0 HY ,p,k C · w2 (5) 2 Michael King, Zinovi Tauber, and Ze-Nian Li. “A New Energy Function for Segmentation and Compression”. In: 2007 IEEE International Conference on Multimedia and Expo. 2007, pp. 1647–1650. doi: 10.1109/ICME.2007.4284983. Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 5
- Video Complexity Features Texture energy gradient Texture energy gradient hp: SAD of the block level energy values of frame p to that of the previous frame p − 1. h = C−1 X k=0 | HY ,p,k, HY ,p−1,k | C · w2 (6) where C denotes the number of blocks in frame p. Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 6
- Video Complexity Features Luminescence Luminescence The luminescence of non-overlapping blocks k of each frame p is defined as: LY ,k = p DCT(0, 0) (7) where DCT(0, 0) is the DC component in the DCT calculation. The block-wise luminescence is averaged per frame denoted as LY as shown below. LY = K−1 X k=0 LY ,p,k C · w2 (8) where C denotes the number of blocks in frame p. Chroma features VCA also determines chroma texture energy EU and EV (for U and V planes), and the chromi- nance LU and LV (for U and V planes). Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 7
- Video Complexity Features Example heatmap Video Complexity Features (a) Original video frame (b) Heatmap of L (c) Heatmap of E (d) Heatmap of h Figure: Example heatmap of Luminescence (L), spatial texture (E) and temporal activity (h) features of the 2nd frame of CoverSong 1080P 0a86 video of Youtube UGC dataset extracted using VCA. Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 8
- Accuracy Analysis Accuracy Analysis Correlation of spatial complexity features with the ground truth Bitrate in All Intra configuration3 is considered as the ground truth of the spatial complexity. Figure: PCC between SI and EY , respectively, with bitrate in All Intra configuration with medium preset of x265 encoder with ultrafast preset of x265 encoder for the VCD dataset.4 3 Jill Boyce et al. JVET-J1010: JVET common test conditions and software reference configurations. July 2018. 4 Hadi Amirpour et al. “VCD: Video Complexity Dataset”. In: Proceedings of the 13th ACM Multimedia Systems Conference. MMSys ’22. Athlone, Ireland: Association for Computing Machinery, 2022, 234–239. isbn: 9781450392839. doi: 10.1145/3524273.3532892. url: https://doi.org/10.1145/3524273.3532892. Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 9
- Accuracy Analysis Accuracy Analysis Correlation of complexity features with bitrate in the Low Delay P picture (LDP) configuration Figure: PCC between SI, EY , TI and h with bitrate in the Low Delay P picture configuration with ultrafast preset of x265 encoder for the VCD dataset. EY and h strongly correlate with the encoding bitrate. Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 10
- Performance Optimizations Performance Optimizations Multi-threading optimizations Creates multiple threads within a VCA execution instance, which executes independently but concurrently, sharing process resources. Independent threads carry out DCT-energy computation per block. x86 SIMD optimizations SIMD optimization5 of DCT functions implemented as intrinsic and assembly codes for x86 architecture. 5 Praveen Kumar Tiwari et al. “Accelerating x265 with intel® advanced vector extensions 512”. In: White Paper on the Intel Developers Page (2018). Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 11
- Performance Optimizations Performance Optimizations Low-pass DCT optimization Unlike SI, the EY feature exhibits better correlation across resolutions. In VCA v2.0, the complexity features are evaluated on the video spatially downsampled by a factor of two. (a) (b) Figure: PCC between the spatial complexity features (a) SI and (b) EY across multiple resolutions for the VCD dataset. Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 12
- Performance Optimizations Performance Optimizations Processing time results Figure: Processing time of video content complexity analysis methods. m1: without any performance optimizations m2: with SIMD m3: with SIMD and low-pass DCT m4: with SIMD , low-pass DCT and multi-threading (2 threads) m5: with SIMD , low-pass DCT and multi-threading (4 threads) m6: with SIMD , low-pass DCT and multi-threading (8 threads) Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 13
- Performance Optimizations Performance Optimizations Energy consumption results Figure: Energy consumption of video content complexity analysis methods. m1: without any performance optimizations m2: with SIMD m3: with SIMD and low-pass DCT m4: with SIMD , low-pass DCT and multi-threading (2 threads) m5: with SIMD , low-pass DCT and multi-threading (4 threads) m6: with SIMD , low-pass DCT and multi-threading (8 threads) Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 14
- Conclusions Conclusions VCA is an open-source video complexity analyzer library published under the GNU GPLv3 license. Low-complexity DCT-based energy features are extracted using VCA v2.0, which encoders use to derive decisions like bitrate-ladder, frame-type, block-partitioning, and much more. Multi-threading, x86 SIMD, and low-pass DCT optimizations improve the energy efficiency of the VCA implementation. Compared to the state-of-the-art SITI implementation of video complexity analysis, VCA v2.0 yields a better estimation of video complexity, with an energy consumption reduction of 97.06%. Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 15
- Q & A Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@aau.at) VCA Vignesh V Menon Green Video Complexity Analysis for Efficient Encoding in Adaptive Video Streaming 16

Advertisement