Comparative Evaluation of VVC, HEVC, H.264, AV1, and VP9 Standards

1. Comparative Evaluation of VVC, HEVC, H.264, AV1, and VP9
Standards
A. Selected Software Implementations
• Versatile video coding Test Model (VTM v3.2) instead of JEM and BMS
• Academia-driven HEVC Test Model (HM 16.17), industry-driven x265 (v2.8) (two
pass mode)
• Joint Model (JM 19.0) and the industry-driven x264 software (v r2935).
• AV1 (v1.0.0 ) and VP9 (v1.7.0-1652-g6b02a12)
B. Selected Test Sequences
• The experiments were conducted on interactive video sequences with different
content to address Low-Delay video applications e.g. video surveillance
Table 1: Test sequences Figure 2: Spatial and temporal information
indices of the test sequences
C. Configuration Setup
• Default main Low-Delay (P) HM-like configuration files (Academia software).
• GOP size of 1 (i.e., I-P-P-P... pattern) command line (Industry software)
Table 1: Example of AV1 Command line Low-Delay Configuration.
METHODOLGY
RESULTS AND DISCUSSION
• In this work a comparative study was carried out on interactive video sequences for
VVC, HEVC, JM, VP9, and AV1 standards with different representative software.
• The conducted experiments reproduce the fact that each newly born standard
outperforms its predecessor in terms of compression efficiency.
• The results indicate that further research should be carried out to identify the error
robustness of each of the aforementioned standard.
• Studying the effect of error on the video bitstream will help us develop new
techniques to overcome packet losses introduced in an imperfect medium such as
in VANETs.
CONCLUSION
I. Mansri, N. Doghmane, N. Kouadria
Laboratory of Automatic and Signals of Annaba (LASA), Electronics Department
Badji Mokhtar University Annaba - Sidi Amar
Annaba 23000, Algeria
Islem.mansri@univ-annaba.org
Owing to the fact that video is a bandwidth-greedy media stream, Codec such as AV1,
as well as (Versatile Video Coding) VVC have emerged. This work provides a
comparative assessment of the compression efficiency for the above-mentioned
standards according to its prominent predecessors VP9, HEVC, and H.264. Seven
encoder implementations were used for benchmarking AV1, VP9, VVC, HEVC, and
H.264. i.e. AV1, VP9, VTM, (HM and x265), as well as (JM and x264) respectively.
Results show that VVC outperforms all the representative encoders in the compression
efficiency at the expanse of computational complexity (encoding run time).
ABSTRACT
Figure 3. Rate-Distortion (R-D) curves for several software implementations.
Table 2: Summarized BD-BR saving for Low-Delay Experimental Results (1-Pass
Mode)
• VTM > AV1 > HM > x265 > VP9 > JM > x264 in compression efficiency.
Figure 4. Average bitrate saving for each quadrant of the SI/TI plan.
• Important Bitrate reduction when dealing with smooth regions (U-HD 4k and 8k)
• VVC vs HEVC (block partitioning QUADTREE PLUS BINARY TREE QTBT).
Table 3: Computational Complexity in terms of the factor of encoding run time
• It requires large computing time for some encoders to achieve the desired bitrate
reduction (caused by the introduction of new tools).
8th Algerian Thematic School on Signal Processing & its Applications
(ATSPA 2019). June 23-27, 2019, Annaba, ALGERIA
Class Sequence name Resolution
E
FourPeople 1820 x 720
60 fps
Johnny 1820 x 720
60 fps
KristenAndSara 1820 x 720
60 fps
E’
Vidyo1 1820 x 720
60 fps
Vidyo3 1820 x 720
60 fps
Vidyo4
1820 x 720
60 fps
CODEC Version Parameters
AV1 v1.0.0
--cpu-used=0 --threads=0 --profile=0 --lag-in-frames=0 --min-q=$QP --
max-q=$QP --auto-alt-ref=1 --passes=$P --kf-max-dist=9999 --kf-min-
dist=9999 --drop-frame=0 --static-thresh=0 --arnr-maxframes=7 --arnr-
strength=5 --sharpness=0 --undershoot-pct=100 --overshoot-pct=100 --
frame-parallel=0 --tile-columns=0 --end-usage=q --cq-level=$QP --
tune=psnr
• The emergence of codecs such as HEVC, VVC, and AV1 led researchers to study
their compression capabilities with reference to their predecessors i.e. H.264 as a
predecessor for HEVC and VVC as well as VP9 for AV1. However, in the literature,
numerous works were conducted to benchmark the afore-mentioned video codecs.
The conducted works were based on random access video applications i.e. live
video streaming.
• To this end, it is worthwhile to mention that this work deals with the objective
evaluation of the above mentioned standards by means of peak to signal noise ratio
(PSNR) with regard to low delay applications (Real-Time video streaming).
• It is self evident that different software implementations provide different degrees of
optimizations. However, this raises the issues regarding its error resilience
performance to network errors.
Figure 1: Simulated framework for video quality evaluation under error prone networks
PROBLEM
CODEC VVC HM x265 JM X264 VP9 AV1
VVC -26,28 % -28,28 % -54,15 % -56,67 % -49,43 % -18,39 %
HM 36,01 % -3.26 % -38.14 % -41.90 % -31.12 % 10,87 %
x265 39,96 % 3.5 % -36.97 % -41.49 % -30.23 % 10,51 %
JM 120,28 % 63,70 % 60,08 % -6,04 % 9,59 % 71,71 %
x264 133,14 % 73,97 % 72,42 % 6,54 % 13,99 % 75,64 %
VP9 98,30 % 45,95 % 44,43 % -8,35 % -11,90 % 57,50 %
AV1 22,82 % -9,01 % -8,17 % -41,26 % -42,48 % -36,27 %
VVC HM AV1P1 VP9P1 JM x264P1 x265P1
VVC ~ 1,32 ~0,64 ~75,2 ~5,9 ~6427,30 ~1424,74
HM ~0,75 ~0,48 ~56,96 ~4,46 ~4868,74 ~1079,25
AV1 ~1,54 ~2,04 ~116,47 9,12 ~9955,75 ~2206,89