Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Current developments in Video Quality :
From the emerging HEVC standard to temporal video
quality assessment and classific...
Dr. Harilaos Koumaras
• Research Interests
▫ Video Quality, QoE, Prediction and Assessment
Methods, Video Coding Standards...
Outline
• Advances on Video Encoding
▫ HEVC (H.265)
▫ Performance Expectations
▫ Main Characteristics
▫ Performance Analys...
Outline
• Advances on Video Encoding
▫ HEVC (H.265)
▫ Performance Expectations
▫ Main Characteristics
▫ Performance Analys...
Video Coding - Timeline
5
Dr. Harilaos Koumaras
Codec Evolution (milestones)
• Transition from MPEG2 to H.264/AVC
– ~2:1 improvement in performance
– Significantly more c...
Codec Comparison
http://www.compression.ru/video/codec_comparison/index_en.html
7
Dr. Harilaos Koumaras
Outline
• Advances on Video Encoding
▫ HEVC (H.265)
▫ Performance Expectations
▫ Main Characteristics
▫ Performance Analys...
HEVC – The project objectives
• Quality
▫ To increase the quality of the coded signal
• Compression
▫ To enhance the compr...
Outline
• Advances on Video Encoding
▫ HEVC (H.265)
▫ Performance Expectations
▫ Main Characteristics
▫ Performance Analys...
Reference Machine
• The Reference Machine used to test new algorithms
against is the test model developed by members of
JC...
HEVC – New Key Features
• Flexible block structure to support arbitrary min & max
unit sizes
▫ Coding Unit (CU)
▫ Predicti...
Reference Configurations
• 6 reference
configurations are
specified
▫ (a) High Efficiency
(HE) and Low
Complexity (LC)
set...
HEVC – Unit Definition (CU)
• Coding Unit (CU)
• CU is the basic processing block
▫ Used for quad-tree based segmentation ...
CU example #2
• LCU=64
• Max Hier. Depth=3
• Split Flag 0/1
15
Dr. Harilaos Koumaras
Benefits of CU structure
• Supports large CU size
▫ Virtually no limit to maximum
size
▫ Maximum of 128x128 used in
CfP su...
HEVC – Unit Definition (PU)
• Prediction Unit (PU)
• PU is the basic unit for prediction
▫ Largest allowed PU size is equa...
HEVC – Unit Definition (TU)
• Transform Unit
• TU is the basic unit for transform and quantization
▫ May exceed size of PU...
Relationship of CU, PU and TU
CU
PU
TU
2Nx2N 2NxN Nx2N NxN 2NxnU 2NxnD nLx2N nRx2N
Symmetric type Asymmetric type
TU size ...
Intra Prediction
• The current intra prediction in HM unified two
directional intra prediction methods
▫ Arbitrary Directi...
Unified Intra Prediction
• In current HM, unified intra prediction
provides up to 34 directional prediction
modes for diff...
Intra Prediction Example
• When the intra prediction angle is
positive, (blue lines), only the
samples from the main array...
Inter Prediction
• Instead of adaptive
interpolation filter, HM
adopts the fixed 12-tap
DCT-based interpolation
filter to ...
Outline
• Advances on Video Encoding
▫ HEVC (H.265)
▫ Performance Expectations
▫ Main Characteristics
▫ Performance Analys...
Performance Results
• Four Test Signals
• The video clips were encoded from their original
uncompressed YUV format to
• IS...
Experimental Results
26
Dr. Harilaos Koumaras
Performance Outcomes
• Based on experimental data :
• HEVC can retain the same video quality level as
AVC
• HEVC achieves ...
Outline
• Advances on Video Encoding
▫ HEVC (H.265)
▫ Performance Expectations
▫ Main Characteristics
▫ Performance Analys...
Objective Assessment (SSIM)
• Maintain the same quality level
29
Dr. Harilaos Koumaras
.. HEVC @ Low Bit Rates ??
• AVC‟s decoded video for
QP equal to 51, proves
not viewable, while
HEVC achieves
acceptable v...
HEVC – The project milestones
• Quality
▫ Maintains the same quality as AVC/H.264
▫ Can achieve acceptable quality at very...
Outline
• Advances on Video Encoding
▫ HEVC (H.265)
▫ Performance Expectations
▫ Main Characteristics
▫ Performance Analys...
Video Quality Degradation
• Lossy compression methods introduce
distortions whose visibility depends highly on
the content...
Video Quality Evaluation
Subjective Methods
• Subjective experiments
▫ An audience is questioned for the perceived
quality...
Video Quality Evaluation
Subjective Methods
• The subjective test methods
▫ Proposed by ITU and VQEG
▫ ITU-R Rec. T.500-11...
Video Quality Evaluation
Subjective Methods
• Double Stimulus Impairment Scale (DSIS)
▫ Scene pairs (reference scene is al...
Video Quality Evaluation
Subjective Methods
• Single Stimulus Continuous Quality Evaluation
(SSCQE)
▫ The viewers watch a ...
Video Quality Evaluation
Objective Methods
Subjective Methods
Time consuming, Expensive, Require Sophisticated Equipment
O...
Video Quality Evaluation
Objective Methods
Objective Methods
“Psychophysical approach",
where metric design is based on
mo...
Objective Methods
- Advantages
• Faster
• Cheaper
• Economically Affordable
• No audience is required
• No statistical ana...
Objective Methods - Cons
• Full Reference Methods
 Initial undistorted clips are not always available.
 Synchronization ...
Video Quality Evaluation
Practical Limitations
The 3G/4G vision is the provision of audiovisual
content at various quality...
Need for predicting Video Quality
There is need for developing methods for predicting
quickly and easily the video quality...
Assuming that available network resources can be
efficiently confronted by traffic control
techniques
Video Quality is mos...
Our contribution…
Evaluation of VQ today:
- Subjective procedures
(time-consuming, expensive)
- Objective procedures
(many...
Our contribution (2)
• Harilaos Koumaras, Fidel Liberal, Lingfen Sun, “Quality of experience issues in multimedia provisio...
… more prediction methods??
• YES!
• Our current works is focused on video quality
fingerprint
• We try to specify each vi...
Video Fingerprint
• We were looking for a method of describing the
temporal and spatial video quality
characteristics of e...
Video Fingerprint
• Video Fingerprint
combines:
▫ Temporal Video
quality aspects of the
test signal (TSSIM,
MovIe)
▫ Spati...
Proposed Method
Initial Encoding
at a predefined
Resolution and
Bit Rate
Generation
of
Fingerprint
Mapping of each
fingerp...
Fingerprint Characteristics
Dr. Harilaos Koumaras
51
Spatial Spatial Spatial Spatial
Spatial Spatial Spatial Spatial
Tempo...
Fingerprint Pros and Cons
Pros Cons
• Unique Characterization of
each test signal
• Both temporal and spatial
description
...
Next Steps…
• Extensive testing of Video Quality Fingerprint
• Extension of the fingerprint to uncompressed
signal
• If su...
References
• [1] JCTVC-A119, Video coding technology proposal by Tandberg, Nokia, and Ericsson
• [2] JCTVC-A124, Video cod...
Upcoming SlideShare
Loading in …5
×

Current developments in video quality: From the emerging HEVC standard to temporal video quality assessment and classification

2,530 views

Published on

This is a presentation given at my invited talk at FTW, Vienna, Austria in 2012.

Published in: Technology

Current developments in video quality: From the emerging HEVC standard to temporal video quality assessment and classification

  1. 1. Current developments in Video Quality : From the emerging HEVC standard to temporal video quality assessment and classification Dr. Harilaos G. Koumaras Friday 22 June 2012 FTW, Vienna, Austria 1 Dr. Harilaos Koumaras
  2. 2. Dr. Harilaos Koumaras • Research Interests ▫ Video Quality, QoE, Prediction and Assessment Methods, Video Coding Standards, IMS • EC Research Activities ▫ Coordination and Participation in FP7 ▫ ICT and SEC ▫ Independent Expert Objective 1.5 • National Research Activities ▫ Evaluator of the Entrepreneurship Contest “Η Ελλάδα Καινοτομεί” 2 Dr. Harilaos Koumaras
  3. 3. Outline • Advances on Video Encoding ▫ HEVC (H.265) ▫ Performance Expectations ▫ Main Characteristics ▫ Performance Analysis ▫ Perceived Performance Analysis • Advances on Video Quality Prediction ▫ Current Situation ▫ Enhanced content classification (uncompressed) ▫ Enhanced prediction (uncompressed->compressed) 3 Dr. Harilaos Koumaras
  4. 4. Outline • Advances on Video Encoding ▫ HEVC (H.265) ▫ Performance Expectations ▫ Main Characteristics ▫ Performance Analysis ▫ Perceived Performance Analysis • Advances on Video Quality Prediction ▫ Current Situation ▫ Enhanced content classification (uncompressed) ▫ Enhanced prediction (uncompressed->compressed) 4 Dr. Harilaos Koumaras
  5. 5. Video Coding - Timeline 5 Dr. Harilaos Koumaras
  6. 6. Codec Evolution (milestones) • Transition from MPEG2 to H.264/AVC – ~2:1 improvement in performance – Significantly more complex, but neutralized by Moore’s law • Next generation H.265/HEVC in development – 30-50% improvement in performance – Significantly more complex yet • Performance strongly dependent on encoder 6 Dr. Harilaos Koumaras
  7. 7. Codec Comparison http://www.compression.ru/video/codec_comparison/index_en.html 7 Dr. Harilaos Koumaras
  8. 8. Outline • Advances on Video Encoding ▫ HEVC (H.265) ▫ Performance Expectations ▫ Main Characteristics ▫ Performance Analysis ▫ Perceived Performance Analysis • Advances on Video Quality Prediction ▫ Current Situation ▫ Enhanced content classification (uncompressed) ▫ Enhanced prediction (uncompressed->compressed) 8 Dr. Harilaos Koumaras
  9. 9. HEVC – The project objectives • Quality ▫ To increase the quality of the coded signal • Compression ▫ To enhance the compression ratio ▫ A good compression rate is important to be able to use the video at limitations such as Internet connection bandwidth, radio bandwidth or limited space of DVD and Blue-Ray discs. • Computational Complexity ▫ Keep it as low as possible ▫ Support real-time applications, which rely on fast decoding process. • Improvements in one of these three factors often comes at the cost of the others. 9 Dr. Harilaos Koumaras
  10. 10. Outline • Advances on Video Encoding ▫ HEVC (H.265) ▫ Performance Expectations ▫ Main Characteristics ▫ Performance Analysis ▫ Perceived Performance Analysis • Advances on Video Quality Prediction ▫ Current Situation ▫ Enhanced content classification (uncompressed) ▫ Enhanced prediction (uncompressed->compressed) 10 Dr. Harilaos Koumaras
  11. 11. Reference Machine • The Reference Machine used to test new algorithms against is the test model developed by members of JCT-VC. • The test model is written in C++ and have the currently best algorithms chosen by JCT-VC meetings implemented. • At first (up to version 0.9) the test model was called TMuC (Test Model under Consideration). • When JCT-VC officially adopted the test model the name changed to HM (HEVC test Model). • Currently HM has reached version 7 (!) 11 Dr. Harilaos Koumaras
  12. 12. HEVC – New Key Features • Flexible block structure to support arbitrary min & max unit sizes ▫ Coding Unit (CU) ▫ Prediction Unit (PU) ▫ Transform Unit (TU) • Consistent syntax representation, independent of size • Asymmetric motion partitions • Greater than ¼ pixel motion accuracy with new interpolation filter • Large integer transforms up to 64x64 • New rotational transform • New motion vector prediction method • New in-loop filtering methods • New intra-coding prediction methods • New entropy coding with explicit scan order signaling 12 Dr. Harilaos Koumaras
  13. 13. Reference Configurations • 6 reference configurations are specified ▫ (a) High Efficiency (HE) and Low Complexity (LC) settings; ▫ (b) Intra Only, Random Access, and Low Delay settings. • Six test conditions (or configurations) are formed by picking up one from the first group and one form the second group. 13 Dr. Harilaos Koumaras
  14. 14. HEVC – Unit Definition (CU) • Coding Unit (CU) • CU is the basic processing block ▫ Used for quad-tree based segmentation of regions ▫ Plays a similar role to macroblock ▫ Can take various sizes  Always power of 2 size  Always square shape • Range of allowed sizes specified in Sequence Parameter Set ▫ Largest CU (LCU) ▫ Maximum hierarchical depth ▫ Easily adapted for various applications • Recursive structure with split flag ▫ Single 2Nx2N or four NxN LCU size = 128 (N=64), maximum hierarchical depth = 5 14 Dr. Harilaos Koumaras
  15. 15. CU example #2 • LCU=64 • Max Hier. Depth=3 • Split Flag 0/1 15 Dr. Harilaos Koumaras
  16. 16. Benefits of CU structure • Supports large CU size ▫ Virtually no limit to maximum size ▫ Maximum of 128x128 used in CfP submissions • Flexible structure ▫ Can be optimized for content, device or application • Size-independent syntax ▫ Each CU has an identical syntax regardless of its size ▫ Reduces complexity of parsing 64 64 32 32 16 16 8 8 Resolution: 1920x1080 LCU size : 64 Maximum depth =2 LCU size : 64 Maximum depth = 4 Resolution: 1920x1080 Resolution: 352x288 LCU size : 16 Maximum depth =2 32 32 16 16 8 8 4 4 16 Dr. Harilaos Koumaras
  17. 17. HEVC – Unit Definition (PU) • Prediction Unit (PU) • PU is the basic unit for prediction ▫ Largest allowed PU size is equal to the CU size ▫ Other allowed PU sizes depend on prediction type  Includes asymmetric splitting options for inter-prediction 2Nx2N NxN 2Nx2N Nx2N 2NxnU2NxN NxN 2NxnD nLx2N nRx2N Intra Inter 2Nx2N Skip • Example of 128x128 CU – Skip: PU = 128x128 – Intra: PU = 128x128 or 64x64 – Inter: PU = 128x128, 128x64, 64x128, 64x64, 128x32, 128x96, 32x128 or 96x128 17 Dr. Harilaos Koumaras
  18. 18. HEVC – Unit Definition (TU) • Transform Unit • TU is the basic unit for transform and quantization ▫ May exceed size of PU, but not CU • Only two TU options are allowed, signalled by transform unit size flag ▫ Transform unit size flag = 0  2Nx2N - same as CU ▫ Transform unit size flag = 1  square units of smaller size  NxN when PU splitting is symmetric  N/2xN/2 when PU splitting is asymmetric 2N 2N 0 1 2 3 N N 2N 2N N/2 N/2 transform unit size flag = 0 transform unit size flag = 0transform unit size flag = 1 transform unit size flag = 1 (a) 2Nx2N, 2NxN, Nx2N, NxN case (b) 2NxnU, 2NxnD, nLx2N, nRxN case 18 Dr. Harilaos Koumaras
  19. 19. Relationship of CU, PU and TU CU PU TU 2Nx2N 2NxN Nx2N NxN 2NxnU 2NxnD nLx2N nRx2N Symmetric type Asymmetric type TU size flag = 0 TU size flag = 1 TU TU size flag = 0 TU size flag = 1 19 Dr. Harilaos Koumaras
  20. 20. Intra Prediction • The current intra prediction in HM unified two directional intra prediction methods ▫ Arbitrary Direction Intra (ADI) introduced in JCTVC-A124 ▫ Angular Intra Prediction introduced in JCTVC- A119 • With simplification for parallel processing possibility, leading to a simplified unified intra prediction (JCTVC-B100, JCTVC-C042). 20 Dr. Harilaos Koumaras
  21. 21. Unified Intra Prediction • In current HM, unified intra prediction provides up to 34 directional prediction modes for different PUs. • With the PU size of 4×4, 8×8, 16×16, 32×32, 64×64, there are 17, 34, 34, 34, and 5 prediction modes available respectively. • The prediction directions in the unified intra prediction have the angles of +/- [0, 2, 5, 9, 13, 17, 21, 26, 32]/32. • The angle is given by displacement of the bottom row of the PU and the reference row above the PU in case of vertical prediction, or displacement of the rightmost column of the PU and the reference column left from the PU in case of horizontal prediction. • Figure 1 shows an example of prediction directions for 32×32 block size. PU size Number of prediction modes 4×4 17 8×8 34 16×16 34 32×32 34 64×64 5 21 Dr. Harilaos Koumaras
  22. 22. Intra Prediction Example • When the intra prediction angle is positive, (blue lines), only the samples from the main array are used for prediction. • When the intra prediction angle is negative, (red lines), a per-sample test should be performed to determine whether samples from the main or the side array should be used for prediction 22 Dr. Harilaos Koumaras
  23. 23. Inter Prediction • Instead of adaptive interpolation filter, HM adopts the fixed 12-tap DCT-based interpolation filter to provide fractional pel accuracy interpolation by replacing the combination of Wiener and bilinear filters with a set of interpolation filters at the desired fractional accuracy. • More specifically, only one filtering procedure is needed to provide the interpolation pixel to any pixel accuracy, instead of a combination of 6-tap and bilinear filtering procedures in H.264/AVC. 23 Dr. Harilaos Koumaras
  24. 24. Outline • Advances on Video Encoding ▫ HEVC (H.265) ▫ Performance Expectations ▫ Main Characteristics ▫ Performance Analysis ▫ Perceived Performance Analysis • Advances on Video Quality Prediction ▫ Current Situation ▫ Enhanced content classification (uncompressed) ▫ Enhanced prediction (uncompressed->compressed) 24 Dr. Harilaos Koumaras
  25. 25. Performance Results • Four Test Signals • The video clips were encoded from their original uncompressed YUV format to • ISO AVC Main Profile (MP) • Random Access Profile (RAP) • Random Access Low Complexity Profile (RALCP) • Low Delay Profile (LDP) • Low Delay P Profile (LDPP) • For HEVC the Test Model (HM) Reference Software v5.1 Frames Resolution Apocalypto Trailer 990 352x288 Batman Trailer 913 352x288 Bubbles 501 416x240 Horses 300 416x240 25 Dr. Harilaos Koumaras
  26. 26. Experimental Results 26 Dr. Harilaos Koumaras
  27. 27. Performance Outcomes • Based on experimental data : • HEVC can retain the same video quality level as AVC • HEVC achieves 32% to 62% improvement in the compression and coding efficiency • Complexity doubles 27 Dr. Harilaos Koumaras
  28. 28. Outline • Advances on Video Encoding ▫ HEVC (H.265) ▫ Performance Expectations ▫ Main Characteristics ▫ Performance Analysis ▫ Perceived Performance Analysis • Advances on Video Quality Prediction ▫ Current Situation ▫ Enhanced content classification (uncompressed) ▫ Enhanced prediction (uncompressed->compressed) 28 Dr. Harilaos Koumaras
  29. 29. Objective Assessment (SSIM) • Maintain the same quality level 29 Dr. Harilaos Koumaras
  30. 30. .. HEVC @ Low Bit Rates ?? • AVC‟s decoded video for QP equal to 51, proves not viewable, while HEVC achieves acceptable video quality and smooth playback. (b) (c) Decoded frame of Race Horses signal for HEVC RAP and AVC MP (QP=51) (a) Initial Video, (b) HEVC RAP, (c) AVC MP 30 Dr. Harilaos Koumaras
  31. 31. HEVC – The project milestones • Quality ▫ Maintains the same quality as AVC/H.264 ▫ Can achieve acceptable quality at very low bit rates • Compression ▫ Doubles the compression efficiency in comparison to AVC/H.264 ▫ Can achieve very high compression at acceptable quality • Computational Complexity ▫ Depending on the profile used, it may be double in comparison to AVC/H.264. 31 Dr. Harilaos Koumaras
  32. 32. Outline • Advances on Video Encoding ▫ HEVC (H.265) ▫ Performance Expectations ▫ Main Characteristics ▫ Performance Analysis ▫ Perceived Performance Analysis • Advances on Video Quality Prediction ▫ Current Situation ▫ Enhanced content classification (uncompressed) ▫ Enhanced prediction (uncompressed->compressed) 32 Dr. Harilaos Koumaras
  33. 33. Video Quality Degradation • Lossy compression methods introduce distortions whose visibility depends highly on the content. • These artifacts result in perceived quality degradation. • The parameters with strong influence on the video quality are the encoding-related ▫ (the bit rate, the frame rate and the resolution) • Thus, the issue of the user satisfaction in correlation with the encoding parameters has been raised. 33 Dr. Harilaos Koumaras
  34. 34. Video Quality Evaluation Subjective Methods • Subjective experiments ▫ An audience is questioned for the perceived quality evaluation of a encoded signal ▫ Subjective experiments to date are the only widely recognized method of determining the actual perceived quality ▫ They are complex and time-consuming, both in their preparation and execution. 34 Dr. Harilaos Koumaras
  35. 35. Video Quality Evaluation Subjective Methods • The subjective test methods ▫ Proposed by ITU and VQEG ▫ ITU-R Rec. T.500-11 (2002) and ITU-T Rec. P.910 (1999) ▫ involve an audience of people, who watch a video sequence and score its quality as perceived by them, under specific and controlled watching conditions. ▫ Afterwards, the statistical analysis of the collected data is used for the evaluation of the perceived quality.  The Mean Opinion Score (MOS) is regarded as the most reliable method of quality measurement and has been applied on the most known subjective techniques 35 Dr. Harilaos Koumaras
  36. 36. Video Quality Evaluation Subjective Methods • Double Stimulus Impairment Scale (DSIS) ▫ Scene pairs (reference scene is always first) ▫ Overall impression scale of impairment • Single Stimulus (SS) Methods ▫ Multiple separate scenes are shown ▫ Three different scoring methods are used:  Adjectival: the aforementioned 5-grade impairment scale, however half-grades may be allowed.  Numerical: an 11-grade numerical scale, useful if a reference is not available.  Non-categorical: a continuous scale with no numbers or a large range, e.g. 0-100 36 Dr. Harilaos Koumaras
  37. 37. Video Quality Evaluation Subjective Methods • Single Stimulus Continuous Quality Evaluation (SSCQE) ▫ The viewers watch a program of typically 20–30 minutes without the original reference to be shown ▫ The subjects/viewers using a slider continuously rate the instantaneously PQoS on scale from „bad‟ to „excellent‟ (0 to 100). • Double Stimulus Continuous Quality Scale (DSCQS) ▫ Scene pairs (reference scene is always first) ▫ The subjects/viewers using a slider continuously rate the instantaneously PQoS on scale from „bad‟ to „excellent‟ (0 to 100) 37 Dr. Harilaos Koumaras
  38. 38. Video Quality Evaluation Objective Methods Subjective Methods Time consuming, Expensive, Require Sophisticated Equipment Objective Methods Exploiting Mathematical Models for Emulating the Results of Subjective Procedures 38 Dr. Harilaos Koumaras
  39. 39. Video Quality Evaluation Objective Methods Objective Methods “Psychophysical approach", where metric design is based on models of the human visual system. (e.g. VQ metric by Stefan Winkler, VDP by Daly, VDM by Lubin…) “Engineering approach", where metrics make assumptions about the artifacts that are introduced by the compression process or transmission link. (e.g. SSIM metric Z. Wang, DVQ metric by Watson…) • Full-reference metrics (frame-by-frame comparison between a reference video and the video under test) • No-reference metrics (no need of reference information) • Reduced-reference metrics (extract a number of features from the reference and degraded video (e.g. amount of motion, spatial detail) 39 Dr. Harilaos Koumaras
  40. 40. Objective Methods - Advantages • Faster • Cheaper • Economically Affordable • No audience is required • No statistical analysis is needed • … 40 Dr. Harilaos Koumaras
  41. 41. Objective Methods - Cons • Full Reference Methods  Initial undistorted clips are not always available.  Synchronization predicaments between the undistorted and the distorted signal (which may have experienced frame loss) • Reduced Reference  Very few implementations  Similar problems to Full Reference • No Reference  Usually Codec Specific 41 Dr. Harilaos Koumaras
  42. 42. Video Quality Evaluation Practical Limitations The 3G/4G vision is the provision of audiovisual content at various quality and price levels (P. Seeling, 2004) All the aforementioned subjective/objective post- encoding methods require repeating tests in order to determine the encoding parameters that satisfy a specific level of user satisfaction. 42 Dr. Harilaos Koumaras
  43. 43. Need for predicting Video Quality There is need for developing methods for predicting quickly and easily the video quality level. These methods will enable the determination of the specific encoding parameters that will satisfy a certain quality level. 43 Dr. Harilaos Koumaras
  44. 44. Assuming that available network resources can be efficiently confronted by traffic control techniques Video Quality is mostly depended on i) Encoding parameters ii) Video content 44 Dr. Harilaos Koumaras
  45. 45. Our contribution… Evaluation of VQ today: - Subjective procedures (time-consuming, expensive) - Objective procedures (many repeated tests required) We have proposed: - Objective pre-encoding evaluation method - For ISO MPEG-4/AVC Clips - Based on a single metric and a single test encoding process Post-encoding evaluation 45 Dr. Harilaos Koumaras
  46. 46. Our contribution (2) • Harilaos Koumaras, Fidel Liberal, Lingfen Sun, “Quality of experience issues in multimedia provision” (Editorial), Int. Journal of Telecommunications Systems, Springer, pp.1-3, DOI: 10.1007/s11235-010-9349-4, March 2012 • Julien Arnaud, Daniel Négru, Mamadou Sidibé, Julien Pauty, Harilaos Koumaras, “Adaptive IPTV services based on a novel IP Multimedia Subsystem”, Multimedia Tools & Applications, Springer, Vol. 55(2),333-352, DOI: 10.1007/s11042-010-0576-1, November 2011 • H. Koumaras, C-H Lin, C-K Shieh, A. Kourtis , “A Framework for End-to-End Video Quality Prediction of MPEG Video”, Journal of Visual Communication and Image Representation, Elsevier, July 2009, • M. Sidibe, H. Koumaras, I. Kofler, A. Mehaoua, A. Kourtis, C. Timmerer, “A Novel Cross Layer Monitoring Architecture for Media Services Adaption Based on Network QoS to Perceived QoS Mapping”, International Journal of Signal, Image and Video Processing Special Issue on “Multimedia Semantics, Adaptation & Personalization”, Vol.2, No.4, pp307-320, DOI 10.1007/s11760-008-0083-2, ISSN-1863-1703, December 2008 • Harilaos Koumaras, Michail-Alexandros Kourtis, Drakoulis Martakos and Christian, “Impact of H.264 Advanced Video Coding Inter-Frame Block Sizes on Video Quality” The International Conference on Computer Vision Theory and Applications VISAPP 2012, Rome, Italy, February 24-26. • H. Koumaras, Julien Arnaud, Daniel Negru, A. Kourtis, ” An Experimental Approach of Video Quality Level Dependence on Video Content Dynamics” , MobiMedia 2009, 3rd European Symposium on Mobile Media Delivery (EUMOB), London, U.K., 7-9 September 2009 • M. Sibide, H. Koumaras, G. Xilouris, “A Perceived Quality-aware Cross Layer Monitoring Framework for Real- Time Media Content Adaptation”, The IEEE Global Information Infrastructure Symposium (IEEE GIIS 2009) June 23- 26, 2009, Hammamet, Tunisia • H. Koumaras, A. Kourtis, “Video Quality Prediction Based on the Spatial and Temporal Classification of the Uncompressed Content” , The 18th Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Comm. (PIMRC), Athens, Greece, 3-7 Sep. 2007 • H. Koumaras, A. Kourtis, C-H Lin, C-K Shieh, “A Theoretical Framework for End-to-End Video Quality Prediction of MPEG-based Sequences” BEST PAPER AWARD • The Third Inter. Conf. on Networking and Services – ICNS07, Athens, Greece, June 19-25 2007 • H. Koumaras, T. Pliakas, A. Kourtis, “A Novel Method for Pre-Encoding Video Quality Prediction”, IST Mobile Summit 2007, Budapest, Hungary, 1-5 July 2007 • G. Gardikis, H.Koumaras, G.Xilouris, E.Pallis, A.Kourtis , “Real-time, Dynamic Resource Allocation in DVB- S.2/RCS Networks” , Eighth International Symposium on Interworking, Santiago, Chile, January 15 – 19, 2007 46 Dr. Harilaos Koumaras
  47. 47. … more prediction methods?? • YES! • Our current works is focused on video quality fingerprint • We try to specify each video clip/test signal with a unique fingerprint, which describes its unique characteristics. 47 Dr. Harilaos Koumaras
  48. 48. Video Fingerprint • We were looking for a method of describing the temporal and spatial video quality characteristics of each video signal • We have proposed a 2-D metric (called video fingerprint), which provides a perceived and unique classification of the video content characteristics • The form of each fingerprint is unique and representative of each video signal. Dr. Harilaos Koumaras 48
  49. 49. Video Fingerprint • Video Fingerprint combines: ▫ Temporal Video quality aspects of the test signal (TSSIM, MovIe) ▫ Spatial Video Quality aspects of the test signals (SSIM, VQM) Dr. Harilaos Koumaras 49 Temporal Video Quality Metric SpatialVideoQualityMetric Low Temporal High Spatial HighTemporal High Spatial Low Temporal Low Spatial High Temporal Low Spatial
  50. 50. Proposed Method Initial Encoding at a predefined Resolution and Bit Rate Generation of Fingerprint Mapping of each fingerprint according to reference Video Quality vs. Bit Rate curves Dr. Harilaos Koumaras 50 i) Low Bit Rate ii) Low Resolution iii) Fast process
  51. 51. Fingerprint Characteristics Dr. Harilaos Koumaras 51 Spatial Spatial Spatial Spatial Spatial Spatial Spatial Spatial Temporal Temporal Temporal Temporal Temporal Temporal Temporal Temporal Homogeneous Low Spatial Low Temporal Homogeneous Low Spatial High Temporal Homogeneous High Spatial High Temporal Homogeneous High Spatial Low Temporal Heterogeneous Mixed Spatial Mixed Temporal Heterogeneous Low Spatial Mixed Temporal Heterogeneous High Spatial Mixed Temporal Heterogeneous Mixed Spatial High Temporal
  52. 52. Fingerprint Pros and Cons Pros Cons • Unique Characterization of each test signal • Both temporal and spatial description • Homogeneous and Heterogeneous classification of the videos • One test encoding is necessary • Full reference comparison with reference signal • Difficult decision on highly heterogeneous videos 52 Dr. Harilaos Koumaras
  53. 53. Next Steps… • Extensive testing of Video Quality Fingerprint • Extension of the fingerprint to uncompressed signal • If succeeded, we will provide a method for video quality prediction directly from the uncompressed Dr. Harilaos Koumaras 53
  54. 54. References • [1] JCTVC-A119, Video coding technology proposal by Tandberg, Nokia, and Ericsson • [2] JCTVC-A124, Video coding technology proposal by Samsung (and BBC) • [3] JCTVC-B093, Simplified angular intra prediction • [4] JCTVC-B100, Unification of the Directional Intra Prediction Methods in TMuC • [5] JCTVC-B118, Angular intra prediction and ADI simplification • [6] JCTVC-C042, TE5: Results for Simplification of Unified Intra Prediction • [7] JCTVC-C207, Encoder improvement of unified intra prediction • [8] H. Koumaras, A. Kourtis, D. Martakos, J. Lauterjung, “Quantified PQoS Assessment Based on Fast Estimation of the Spatial and Temporal Activity Level”, Multimedia Tools and Applications, Springer Editions, Published online (IF 0.416) • [9] H. Koumaras, A. Kourtis, D. Martakos, “Evaluation of Video Quality Based on Objectively Estimated Metric”, Journal of Communications and Networking, KICS, (IF 0.479) • Vol. 7(3), Sep 2005, Technically cosponsored by IEEE ComSoc • [10] Harilaos Koumaras, Michail-Alexandros Kourtis, Spyros Mantzouratos, Drakoulis Martakos,, “Quantitative Performance Evaluation Of the Emerging HEVC/H.265 Video Codec”, QoEMCS 2012 workshop, Euro ITV 2012, Berlin, Germany, 04 – 06 July 2012. (Accepted) • [11] Harilaos Koumaras, Michail-Alexandros Kourtis, Drakoulis Martakos, “Benchmarking the Encoding Efficiency of H.265/HEVC and H.264/AVC”, Future Network & Mobile Summit 2012 4 – 6 July 2012, Berlin, Germany, July 4-6. ( 54 Dr. Harilaos Koumaras

×