• Save
Subjective quality evaluation of the upcoming HEVC video compression standard
Upcoming SlideShare
Loading in...5
×
 

Subjective quality evaluation of the upcoming HEVC video compression standard

on

  • 15,913 views

Slides of my presentation at SPIE Optics+Photonics 2012 Applications of Digital Image Processing XXXV, San Diego, August 12-16, 2012

Slides of my presentation at SPIE Optics+Photonics 2012 Applications of Digital Image Processing XXXV, San Diego, August 12-16, 2012
Paper available at: http://infoscience.epfl.ch/record/180494

Statistics

Views

Total Views
15,913
Views on SlideShare
10,233
Embed Views
5,680

Actions

Likes
11
Downloads
0
Comments
3

9 Embeds 5,680

http://www.scoop.it 5408
http://www.linkedin.com 229
http://pinterest.com 16
http://webcache.googleusercontent.com 10
http://translate.googleusercontent.com 7
https://twitter.com 5
http://sonar9.com 2
http://twitter.com 2
http://localhost 1
More...

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Challenging to encode since it has relatively highSI and TI indexesArtifacts are more visible in the upper left corner due to higher sensitivity of the human visual system in low intensity areas (Weber law)Blockiness was perceived in AVC encoded sequences while the content was smoothed out in HEVC encoded sequences, which is less annoying
  • Relatively easy to encode since it has a small TI indexBit rates as low as 5 Mbps and 2 Mbps for AVC and HEVC, respectively, are evaluated as transparent (60% bit rate reduction)
  • HEVC exhibits a significant improvement over AVC and very low bit rates can be achieved due to the absence of noise in the original contentA bit rate as low as 1.2 Mbps is perceived as transparent with HEVC while the same bit rate for AVC is evaluated as annoying
  • Results for Sintel2 are in the linear phase, where an increase in PSNR induces an increase in perceived qualityResults for Traffic are mostly in the saturation phase, where no significant gain in quality is perceived for an increase in PSNR For PeopleOnStreet, most results are on the upper part of the linear phase and a few are on the saturation phase

Subjective quality evaluation of the upcoming HEVC video compression standard Subjective quality evaluation of the upcoming HEVC video compression standard Presentation Transcript

  • Subjective quality evaluation of the upcoming HEVC video compression standardPhilippe Hanhart, Martin Rerabek, Francesca De Simone, and Touradj Ebrahimi
  • Outline• Introduction• Dataset• Test methodology• Test equipment and environment• Results• Conclusion 14/8/2012 SPIE Optics and Photonic 2012 2
  • Outline• Introduction• Dataset• Test methodology• Test equipment and environment• Results• Conclusion 14/8/2012 SPIE Optics and Photonic 2012 3
  • Next-generation video compression• Motivation – Large quantity of video material over broadcast channels, digital networks, and packaged media – Demand for increased resolution and higher quality – Evolution of video acquisition and display technologies faster than network capabilities New video coding standard with higher efficiency than H.264/AVC is needed!• Goals – Better coding efficiency at higher resolutions – Suitability to a wide variety of applications (mobile TV, home cinema, UHDTV) 14/8/2012 SPIE Optics and Photonic 2012 4
  • Joint Collaborative Team on Video Coding (JCT-VC)• Video coding experts from ITU-T Study Group 16 (VCEG) + ISO/IEC JTC 1/SC 29/WG 11 (MPEG) = the Joint Collaborative Team on Video Coding (JCT-VC)• The JCT-VC standardization effort is being referred to as High Efficiency Video Coding (HEVC) Test Model under HEVC Test Model (HM) Call for Proposals Consideration publicly available (January 2010) (July 2010) (January 2011)Each proponent had to: • Based on key elements • Integrates the latest• Develop and submit a binary of some of the best developments executable of the proposed codec proposals • Steady simplification and• Encode and decode test material efficiency improvement• Evaluate objective quality using • Starting point for the definition of the new • New version available at PNSR• Provide and algorithmic standard each JCT-VC meeting cycle description of the codec 14/8/2012 SPIE Optics and Photonic 2012 5
  • Introduction• Double the compression efficiency is expected in HEVC versus AVC• Can be reliably compared only by means of subjective tests• Compression efficiency especially for resolutions beyond HDTV• Subjective evaluations of HEVC have been so-far limited up to HDTV resolution• This paper provides results of subjective evaluation on 4K/QFHD video content – Professional high-performance 4K/QFHD LCD reference monitor 14/8/2012 SPIE Optics and Photonic 2012 6
  • Outline• Introduction• Dataset• Test methodology• Test equipment and environment• Results• Conclusion 14/8/2012 SPIE Optics and Photonic 2012 7
  • Contents• PeopleOnStreet (3840x2160@30fps) • Traffic (3840x2048@30fps)• Sintel2 (3840x1744@24fps) • Sintel39* (3840x1744@24fps) *Only used for training 14/8/2012 SPIE Optics and Photonic 2012 8
  • Spatial information and temporal information indexes 30 Sintel2 Sintel39 25 PeopleOnStreet TI 20 Traffic 15 0 5 10 15 SI• Different spatio-temporal characteristics – Sintel2 and Sintel39: large TI values – Traffic: small TI index (easier to encode) 14/8/2012 SPIE Optics and Photonic 2012 9
  • Encoding• Codecs – AVC (JM 18.3) – HEVC (HM 6.1.1)• Configuration – Random Access – GOP size: 8 pictures – Intra Period: 1s 24 pictures @24fps and 32 @30fps• Hierarchical B-pictures – QP increase of 1 between each Temporal Level – Coding Order: 0 8 4 2 1 3 6 5 7 14/8/2012 SPIE Optics and Photonic 2012 10
  • Encoding• 5 different bit rates for each content and codec – Both natural and synthetic contents• Fixed QPs (no Rate Control in HM 6) – Typical QPs range for AVC: 25 - 37 – Upper bit rate limit: 20 Mbps• Bit rates selected based on expert viewing sessions – Select lower/upper bounds for each content separately – Targeting realistic bit rates – Try to cover the full quality scale *BR expressed in Mbps 14/8/2012 SPIE Optics and Photonic 2012 11
  • Outline• Introduction• Dataset• Test methodology• Test equipment and environment• Results• Conclusion 14/8/2012 SPIE Optics and Photonic 2012 12
  • Methodology 1• Double Stimulus Impairment Scale (DSIS) Variant II 100 Imperceptible 90 Reference Test Reference Test Video Video Video Video 80 Perceptible annoying but not 70 60 annoying Slightly 50 40 2s 5s 2s 5s 2s 5s 2s 5s 6s Annoying (TOT= 34 s) 30 20 “Rate the level of annoyance of the visual defects that you see annoying Very 10 in stimulus B, knowing that A is the reference video.” 0 14/8/2012 SPIE Optics and Photonic 2012 13
  • Sessions• ITU BT-500 – One test session should not last more than 30 minutes – Alternate as many different contents as possible• Short sequences => evaluation task requires a lot of attention – Test sessions no longer than 15 minutes each – Each session is followed by a resting phase• Details – Never the same content in consecutive presentations – Randomization to avoid possible effect of content presentation order – Dummy sequences at beginning of 1st session to stabilize observers’ rating – Reference versus reference stimulus to check subject’s reliability 14/8/2012 SPIE Optics and Photonic 2012 14
  • Timing• One DSIS Variant II presentation – Theoretically: 34 s – Loading time for each video file in Media Player Classic: 4 s • Concatenate ‘A’ + ref seq + ‘B’ + test seq in a single avi file – Total time: 46 s• Sequences – 30 test sequences (2 codecs × 3 contents × 5 bit rates) – Split in 2 test sessions• Sessions – Training session: 10 min – 1st test session: 2 dummies + 1 ref vs ref + 15 stimuli = 18 x 46 s = 14 min – 2nd test session: 15 stimuli = 15 x 46 s = 11.5 min 14/8/2012 SPIE Optics and Photonic 2012 15
  • Test• Test planning – 2 days, 3 time slots per day – 6 subjects per slot, split in 2 groups of 3 subjects each• Subjects – 36 naive people – 30% of the observers were female – Age ranged from 20 to 61 years old• Screening – Correct visual acuity: Snellen charts – Correct color vision: Ishiara charts• Training – Oral instructions to explain the task and the meaning of each label reported on the scale – Viewing session to allow the viewer familiarizing with the assessment procedure – Training samples have quality levels representative of the labels reported on the rating scales 14/8/2012 SPIE Optics and Photonic 2012 16
  • Outline• Introduction• Dataset• Test methodology• Test equipment and environment• Results• Conclusion 14/8/2012 SPIE Optics and Photonic 2012 17
  • Laboratory for subjective video quality assessment• Server with SSD to read and play in real time 3840x2160@30fps YUV 4:2:0 raw video (i.e., 373.25 MB/s!)• 56-inch professional high-performance 4K/QFHD LCD reference monitor Sony Trimaster SRM-L560• ITU BT.500-11 compliant test environment – 3 subjects in front of the screen – Viewing distance ≈ 3.5 x screen height 14/8/2012 SPIE Optics and Photonic 2012 18
  • Outline• Introduction• Dataset• Test methodology• Test equipment and environment• Results• Conclusion 14/8/2012 SPIE Optics and Photonic 2012 19
  • PeopleOnStreet PeopleOnStreet 100 90 80 70 60MOS 50 40 30 20 10 AVC HEVC 0 4 6 8 10 12 14 16 18 20 22 bitrate [Mbps] • At similar bit rates, HEVC outperforms AVC in 4 / 5 cases • Bit rate reduction – BD-PSNR: 27.5% – BD-MOS: 50.8% 14/8/2012 SPIE Optics and Photonic 2012 20
  • Traffic Traffic 100 90 80 70 60MOS 50 40 30 20 10 AVC HEVC 0 2 4 6 8 10 12 14 16 bitrate [Mbps] • At similar bit rates, HEVC outperforms AVC in 1 / 4 cases • Bit rate reduction – BD-PSNR: 37.7% – BD-MOS: 74.0% 14/8/2012 SPIE Optics and Photonic 2012 21
  • Sintel2 Sintel2 100 90 80 70 60MOS 50 40 30 20 10 AVC HEVC 0 0.5 1 1.5 2 2.5 3 3.5 bitrate [Mbps] • At similar bit rates, HEVC outperforms AVC in 4 / 4 cases • Bit rate reduction – BD-PSNR: 68.0% – BD-MOS: 74.7% 14/8/2012 SPIE Optics and Photonic 2012 22
  • Subjective versus objective results PSNR highly correlated with perceived quality, as long as saturation limits are considered14/8/2012 SPIE Optics and Photonic 2012 23
  • Outline• Introduction• Dataset• Test methodology• Test equipment and environment• Results• Conclusion 14/8/2012 SPIE Optics and Photonic 2012 24
  • Conclusion• First comparison of HEVC vs AVC on resolution beyond HDTV• In general, HEVC significantly outperform AVC for similar bit rates• Substantial improvement in compression performance – Natural contents: 51 to 74% bit rate reduction – Synthetic content: 75% bit rate reduction• Objective gains based on BD-PSNR are pessimistic compared to actual gains measured with subjective rating• The upcoming HEVC video compression standard seems to be one of the key elements towards a wide deployment of UHDTV 14/8/2012 SPIE Optics and Photonic 2012 25
  • Thank you for your attention!14/8/2012 SPIE Optics and Photonic 2012 26