Successfully reported this slideshow.

INCEPT: Intra CU Depth Prediction for HEVC

0

Share

1 of 22
1 of 22

INCEPT: Intra CU Depth Prediction for HEVC

0

Share

Download to read offline

High Efficiency Video Coding (HEVC) improves the encoding efficiency by utilizing sophisticated tools such as flexible Coding Tree Unit (CTU) partitioning. The Coding Unit (CU) can be split recursively into four equally sized CUs ranging from 64×64 to 8×8 pixels. At each depth level (or CU size), intra prediction via exhaustive mode search was exploited in HEVC to improve the encoding efficiency and result in a very high encoding time complexity. This paper proposes an Intra CU Depth Prediction (INCEPT) algorithm, which limits Rate-Distortion Optimization (RDO) for each CTU in HEVC by utilizing the spatial correlation with the neighboring CTUs, which is computed using a DCT energy-based feature. Thus, INCEPT reduces the number of candidate CU sizes required to be considered for each CTU in HEVC intra coding. Experimental results show that the INCEPT algorithm achieves a better trade-off between the encoding efficiency and encoding time saving (i.e., BDR/∆T) than the benchmark algorithms. While BDR/∆T is 12.35% and 9.03% for the benchmark algorithms, it is 5.49% for the proposed algorithm. As a result, INCEPT achieves a 23.34% reduction in encoding time on average while incurring only a 1.67% increase in bit rate than the original coding in the x265 HEVC open-source encoder.

High Efficiency Video Coding (HEVC) improves the encoding efficiency by utilizing sophisticated tools such as flexible Coding Tree Unit (CTU) partitioning. The Coding Unit (CU) can be split recursively into four equally sized CUs ranging from 64×64 to 8×8 pixels. At each depth level (or CU size), intra prediction via exhaustive mode search was exploited in HEVC to improve the encoding efficiency and result in a very high encoding time complexity. This paper proposes an Intra CU Depth Prediction (INCEPT) algorithm, which limits Rate-Distortion Optimization (RDO) for each CTU in HEVC by utilizing the spatial correlation with the neighboring CTUs, which is computed using a DCT energy-based feature. Thus, INCEPT reduces the number of candidate CU sizes required to be considered for each CTU in HEVC intra coding. Experimental results show that the INCEPT algorithm achieves a better trade-off between the encoding efficiency and encoding time saving (i.e., BDR/∆T) than the benchmark algorithms. While BDR/∆T is 12.35% and 9.03% for the benchmark algorithms, it is 5.49% for the proposed algorithm. As a result, INCEPT achieves a 23.34% reduction in encoding time on average while incurring only a 1.67% increase in bit rate than the original coding in the x265 HEVC open-source encoder.

More Related Content

More from Alpen-Adria-Universität

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

INCEPT: Intra CU Depth Prediction for HEVC

  1. 1. INCEPT: Intra CU Depth Prediction for HEVC Vignesh V Menon, Hadi Amirpour, Christian Timmerer and Mohammad Ghanbari Christian Doppler Laboratory ATHENA, Institute of Information Technology (ITEC), University of Klagenfurt, Austria 06 October 2021 Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 1
  2. 2. Outline 1 Introduction 2 INCEPT Algorithm 3 Evaluation 4 Conclusions and Future Directions Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 2
  3. 3. Introduction Introduction Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 3
  4. 4. Introduction Introduction Background of High Efficiency Video Coding (HEVC)3 The compression efficiency of HEVC has been improved greatly compared to AVC1 with the adoption of numerous advanced tools. HEVC has a flexible quad-tree coding block partitioning structure and uses the Coding Units (CUs) and Prediction Units (PUs) concept to manage the partition.2 Each video frame is divided into non-overlapping predefined size Coding Tree Unit (CTUs), and each CTU is then recursively divided into CUs. A CU can be further partitioned into PUs and Transform Units (TUs). The partitioning is recursive within a quad-tree hierarchy making the codec computation- ally intensive. 1 T. Wiegand et al. “Overview of the H.264/AVC video coding standard”. In: IEEE Transactions on Circuits and Systems for Video Technology 13.7 (2003), pp. 560–576. 2 Ekrem Çetinkaya et al. “CTU depth decision algorithms for HEVC: A survey”. In: Signal Processing: Image Communication 99 (2021), p. 116442. issn: 0923-5965. doi: https://doi.org/10.1016/j.image.2021.116442. url: https://www.sciencedirect.com/science/article/pii/S0923596521002113. 3 G. J. Sullivan et al. “Overview of the high efficiency video coding (HEVC) standard”. In: IEEE Transactions on circuits and systems for video technology 22.12 (2012), pp. 1649–1668. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 4
  5. 5. Introduction Introduction CU partitioning in HEVC 32x32 16x16 8x8 depth 1 depth 0 depth 2 depth 3 Figure: An example of the CU partitioning structure of a CTU and its corresponding quad-tree structure. The white nodes represent CUs that have been partitioned. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 5
  6. 6. Introduction Introduction Intra Coding of High Efficiency Video Coding (HEVC)6 Intra frames are essential to conditions such as mobile devices with limited computational power, transmission over error-prone channels, and frequent random access.4 In intra coding, PU’s size is generally equal to the corresponding CU. In addition, up to 35 prediction modes, including DC mode, Planar mode, and 33 angular modes, are provided for intra prediction in HEVC, enabling more accurate predictions. The increase in CU depth and prediction modes also causes higher coding complexity than Advanced Video Coding (AVC).5 4 Yun Zhang et al. “Statistical Early Termination and Early Skip Models for Fast Mode Decision in HEVC INTRA Coding”. In: ACM Trans. Multimedia Comput. Commun. Appl. 15.3 (July 2019). issn: 1551-6857. doi: 10.1145/3321510. url: https://doi.org/10.1145/3321510. 5 Wiegand et al., “Overview of the H.264/AVC video coding standard”. 6 Sullivan et al., “Overview of the high efficiency video coding (HEVC) standard”. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 6
  7. 7. Introduction Intra CU Depth Estimation Start CTU Depth i ∈ [dmin, dmax] No d > dmax End CTU PU mode decisions i = i + 1 for next CU No Yes Yes Figure: Quad-tree CU algorithm for partitioning of a CTU. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 7
  8. 8. INCEPT Algorithm INCEPT Algorithm Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 8
  9. 9. INCEPT Algorithm Phase 1: Feature Extraction INCEPT Algorithm Phase 1: Feature Extraction Compute texture energy per Coding Tree Unit (CTU) A DCT-based energy function is used to determine the block-wise feature of each frame defined as: EDCT = w X i=1 h X j=1 e|( ij wh )2−1| |DCT(i − 1, j − 1)| (1) where w and h are the width and height of the block, and DCT(i, j) is the (i, j)th DCT component when i + j > 2, and 0 otherwise. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 9
  10. 10. INCEPT Algorithm Phase 1: Feature Extraction INCEPT Algorithm Phase 2: CU Depth Prediction For each CTU: Inputs: HL, HA, HAL : weighted DCT energy of the neighboring CTUs mL, mA, mAL : mean of the CU depths of the neighboring CTUs vL, vA, vAL : variance of the CU depths of the neighboring CTUs Output: dmin and dmax Step 1: Compute Hmin and Hmax , mmin and mmax and vmin and vmax Hmin = min(HL, HA, HAL) Hmax = max(HL, HA, HAL) mmin = min(mL, mA, mAL) mmax = max(mL, mA, mAL) vmin = min(vL, vA, vAL) vmax = max(vL, vA, vAL) Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 10
  11. 11. INCEPT Algorithm Phase 1: Feature Extraction INCEPT Algorithm Phase 2: CU Depth Prediction Step 2: Compute HC . Step 3: Determine dmin and dmax if HC < Hmin then dmax = dmmin + vmin 2 + φe else dmax = dmmax + vmax 2 + φe if HC > Hmax then dmin = bmmax − vmin 2 + ψc else dmin = bmmin − vmax 2 + ψc Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 11
  12. 12. INCEPT Algorithm Phase 1: Feature Extraction INCEPT Algorithm Depth prediction accuracy Table: Depth prediction accuracy of INCEPT algorithm over JVET sequences and QPs. Video QP22 QP26 QP30 QP34 Average CatRobot 97.93% 95.98% 95.34% 94.76% 96.00% DaylightRoad2 98.04% 97.86% 97.02% 96.39% 97.33% FoodMarket4 99.32% 99.1% 98.79% 97.34% 98.64% ParkRunning3 96.89% 95.34% 94.71% 94.03% 95.24% Average 98.05% 97.07% 96.46% 95.63% 96.80% Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 12
  13. 13. Evaluation Evaluation Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 13
  14. 14. Evaluation Evaluation Test Methodology Test videos: (i) JVET test sequences,7 (i) MCML test sequences,8 and (ii) SJTU test sequences9 representing various types of contents. System: Dual-processor server with Intel Xeon Gold 5218R (80 cores, 2.10 GHz) The presented algorithms were implemented in x265 v3.410 and tested with the veryslow preset. Sequences were encoded with ALL intra configuration and QPs ∀{22, 26, 30, 34}. The lower resolution sources were generated from the original video source by applying bi-cubic scaling using FFmpeg.11 7 Jill Boyce et al. JVET-J1010: JVET common test conditions and software reference configurations. July 2018. 8 Manri Cheon and Jong-Seok Lee. “Subjective and Objective Quality Assessment of Compressed 4K UHD Videos for Immersive Experience”. In: IEEE Transactions on Circuits and Systems for Video Technology 28.7 (2018), pp. 1467–1480. doi: 10.1109/TCSVT.2017.2683504. 9 L. Song et al. “The SJTU 4K Video Sequence Dataset”. In: Fifth International Workshop on Quality of Multimedia Experience (QoMEX2013) (July 2013). 10 MulticoreWare Inc. x265 HEVC Encoder/H.265 Video Codec. url: http://x265.org/. 11 FFmpeg. FFmpeg Documentation. url: https://ffmpeg.org/ffmpeg.html. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 14
  15. 15. Evaluation Evaluation Test Methodology Metrics: ∆T: the cumulative time savings for all bitrate representations compared with the stand- alone encoding Bjøntegaard delta rates,12 BDRP and BDRV : average increase in bitrate of the represen- tations compared with that of the stand-alone encoding to maintain the same PSNR and VMAF. BDRP ∆T and BDRV ∆T are calculated to compare the performance of the algorithms. The lower is the value of BDR ∆T , the better is the performance of the algorithm. 12 G. Bjontegaard. “Calculation of average PSNR differences between RD-curves”. In: VCEG-M33 (2001). Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 15
  16. 16. Evaluation Evaluation Experimental Results Table: ∆T and BDR comparison between the INCEPT algorithm and the benchmark algorithms. ADTS13 SCDP14 INCEPT Video ∆T BDRP BDRV ∆T BDRP BDRV ∆T BDRP BDRV CatRobot 13.74% 2.36% 2.02% 24.97% 3.89% 3.71% 24.75% 3.08% 3.25% DaylightRoad 16.38% 1.25% 1.19% 26.30% 3.15% 2.44% 26.20% 1.72% 1.54% FoodMarket 16.15% 1.06% 1.12% 19.00% 2.56% 1.26% 20.09% 1.40% 0.72% Basketball 13.75% 1.96% 1.68% 18.69% 4.82% 3.29% 19.13% 2.16% 1.88% Bunny 15.40% 1.98% 2.03% 18.11% 3.32% 3.09% 18.69% 1.67% 1.69% Lake 13.01% 1.08% 0.97% 22.54% 2.95% 2.12% 22.89% 1.19% -2.25% BundNightScape 16.68% 1.08% 0.99% 28.18% 2.95% 4.10% 28.02% 1.43% 1.73% CampfireParty 12.27% 0.78% 1.12% 22.11% 1.88% 2.64% 23.41% 0.82% 1.36% Fountains 17.61% 0.88% 1.07% 25.12% 2.72% 2.66% 26.90% 1.53% 1.61% Average 15.00% 1.38% 1.35% 22.78% 3.14% 2.81% 23.34% 1.67% 1.28% 13 Xin Lu, Chang Yu, and Xuesong Jin. “A fast HEVC intra-coding algorithm based on texture homogeneity and spatio-temporal correlation”. In: EURASIP Journal on Advances in Signal Processing 37 (2018). doi: https://doi.org/10.1186/s13634-018-0558-4. 14 Zhang et al., “Statistical Early Termination and Early Skip Models for Fast Mode Decision in HEVC INTRA Coding”. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 16
  17. 17. Evaluation Evaluation Experimental Results SCDP ADTS INCEPT 0 2 4 6 8 10 12 14 BDRv / T 12.35% 9.03% 5.49% Figure: Comparison of BDRV ∆T with benchmark algorithms Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 17
  18. 18. Evaluation Evaluation Experimental Results QP22 QP26 QP30 QP34 0 5 10 15 20 25 30 35 T (%) 28.60% 25.54% 23.34% 21.56% Figure: Average time saving using INCEPT algorithm for various QPs Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 18
  19. 19. Conclusions and Future Directions Conclusions and Future Directions Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 19
  20. 20. Conclusions and Future Directions Conclusions We proposed fast intra CU depth prediction algorithm for HEVC encoding. We analyzed the algorithm by comparing against two benchmark algorithms after integrat- ing them into the x265 open-source HEVC encoder. Experimental results demonstrate that the proposed INCEPT algorithm decreased the over- all encoding time by 23.34% with a negligible increase in bitrate. BDR ∆T metric is better for INCEPT compared to the two benchmark algorithms. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 20
  21. 21. Conclusions and Future Directions Future Directions More encoding time can be saved by reducing the time taken for PU mode decisions for each CU15,16,17. The proposed INCEPT algorithm can be easily extended for the VVC standard18. 15 Lu, Yu, and Jin, “A fast HEVC intra-coding algorithm based on texture homogeneity and spatio-temporal correlation”. 16 Jinzheng Lu and Yixian Li. “Fast Algorithm for CU Partitioning and Mode Selection in HEVC Intra Prediction”. In: 2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI). 2019, pp. 1–5. doi: 10.1109/CISP-BMEI48845.2019.8966035. 17 Tao Zhang et al. “Fast Intra-Mode and CU Size Decision for HEVC”. In: IEEE Transactions on Circuits and Systems for Video Technology 27.8 (2017), pp. 1714–1726. doi: 10.1109/TCSVT.2016.2556518. 18 Gary Sullivan. “Versatile Video Coding (VVC) Arrives”. In: 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP). 2020, pp. 1–1. doi: 10.1109/VCIP49819.2020.9301847. Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 21
  22. 22. Conclusions and Future Directions Q & A Thank you for your attention! Vignesh V Menon (vignesh.menon@aau.at) Hadi Amirpour (hadi.amirpourazarian@aau.at) Christian Timmerer (Christian.Timmerer@aau.at) Mohammad Ghanbari (ghan@essex.ac.uk) Vignesh V Menon INCEPT: Intra CU Depth Prediction for HEVC 22

×