• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
PERCEVAL - Perceptual and Cognitive Quality Evaluation Techniques for Audiovisual Systems, Ulrich Reiter, NTNU
 

PERCEVAL - Perceptual and Cognitive Quality Evaluation Techniques for Audiovisual Systems, Ulrich Reiter, NTNU

on

  • 501 views

VERDIKT conference 2013

VERDIKT conference 2013

Statistics

Views

Total Views
501
Views on SlideShare
501
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    PERCEVAL - Perceptual and Cognitive Quality Evaluation Techniques for Audiovisual Systems, Ulrich Reiter, NTNU PERCEVAL - Perceptual and Cognitive Quality Evaluation Techniques for Audiovisual Systems, Ulrich Reiter, NTNU Presentation Transcript

    • 1 PERCEVAL - Perceptual and Cognitive Quality Evaluation Techniques for Audiovisual Systems Ulrich Reiter ulrich.reiter@ntnu.no
    • 2 Outline • • • • • • • PERCEVAL overview Motivation (long duration) Method description (long duration) Experiment (long duration) Results (long duration) Conclusions Food for thought Ulrich Reiter – PERCEVAL
    • 3 Outline • • • • • • • PERCEVAL overview Motivation (long duration) Method description (long duration) Experiment (long duration) Results (long duration) Conclusions Food for thought Ulrich Reiter – PERCEVAL
    • 4 Multimedia systems rely on compression and encoding to speed up transmission Compression reduces the file size by removing redundant or unessential information As a result, media quality goes down and the audio and video streams can become misaligned Ulrich Reiter – PERCEVAL
    • 5 Perceptual consequences of distortion: - - Face identification Scene/object recognition Speech comprehension Audiovisual speech integration Temporal order of events Audiovisual temporal integration? Long duration quality perception? Ulrich Reiter – PERCEVAL
    • 6 PERCEVAL project Ulrich Reiter – PERCEVAL
    • 7 Outline • • • • • • • PERCEVAL overview Motivation (long duration) Method description (long duration) Experiment (long duration) Results (long duration) Conclusions Food for thought Ulrich Reiter – PERCEVAL
    • 8 Questions to be answered Do quality expectations decrease over time and with increased involvement in the content? How fast can users notice quality changes and at which quality level does this happen? Is the quality level at which the change is noticed similar to the desired quality level? Ulrich Reiter – PERCEVAL
    • 9 Quality assessment methods for long duration AV content Ulrich Reiter – PERCEVAL
    • 10 SSCQE (Single Stimulus Continuous Quality Evaluation) Ulrich Reiter – PERCEVAL
    • 11 Outline • • • • • • • PERCEVAL overview Motivation (long duration) Method description (long duration) Experiment (long duration) Results (long duration) Conclusions Food for thought Ulrich Reiter – PERCEVAL
    • 12 Method description Quality adjustment Ulrich Reiter – PERCEVAL
    • 13 Outline • • • • • • • PERCEVAL overview Motivation (long duration) Method description (long duration) Experiment (long duration) Results (long duration) Conclusions Food for thought Ulrich Reiter – PERCEVAL
    • 14 Experimental conditions Ulrich Reiter – PERCEVAL
    • 15 Stimulus material 30min 55sec long HD resolution (1080p) H.264/AVC compression – Quantization parameter 11 versions, JND Conversion to 16 bit YUV 4:2:0 Original audio track Ulrich Reiter – PERCEVAL
    • 16 Experimental procedure Participants 22 subjects (8 female, 14 male) in the age of 24-61 years Test procedure and duration Quality degradation initiated every 3 mins One JND drop every 10 sec Ulrich Reiter – PERCEVAL
    • 17 Outline • • • • • • • PERCEVAL overview Motivation (long duration) Method description (long duration) Experiment (long duration) Results (long duration) Conclusions Food for thought Ulrich Reiter – PERCEVAL
    • 18 Questions to be answered Do the quality expectations decrease over time and with increased involvement in the content? How fast can participants notice quality changes and at which quality level does this happen? Is the quality level at which the change is noticed similar to the quality level set by the participant? Ulrich Reiter – PERCEVAL
    • 19 Data preparation Three subsets of data (20x10 each) created: 1) average quality level of the last minute of each 3min time slot 2) response time to the automatic quality degradation right after the start of each 3 min time slot 3) quality level at the time when a user reacted to quality change Ulrich Reiter – PERCEVAL
    • 20 Results for subset data 1) Main effects plot for quality levels averaged over last minute of each 3 min time section. Ulrich Reiter – PERCEVAL
    • 21 Results for subset data 2) Main effects plot for reaction time to quality changes. Ulrich Reiter – PERCEVAL
    • 22 Results for subset data 3) Main effects plot for quality levels at time when a user reacted to quality change Ulrich Reiter – PERCEVAL
    • 23 Results cont. QLRT: Quality Level at Reaction Time AQL: Absolute Quality Level Comparison of sensitivity to quality changes under different conditions. Ulrich Reiter – PERCEVAL
    • 24 Outline • • • • • • • PERCEVAL overview Motivation (long duration) Method description (long duration) Experiment (long duration) Results (long duration) Conclusions Food for thought Ulrich Reiter – PERCEVAL
    • 25 Conclusions Time dimension does not influence participants’ expectations with respect to the perceived quality. The reaction time to quality changes is relatively constant across all time intervals. Participants are less sensitive to quality changes when the process is controlled externally than when they are in charge of the quality adjustment. Ulrich Reiter – PERCEVAL
    • 26 Outline • • • • • • • PERCEVAL overview Motivation (long duration) Method description (long duration) Experiment (long duration) Results (long duration) Conclusions Food for thought Ulrich Reiter – PERCEVAL
    • 27 Food for thought • Content dependency – Ideal bandwidth trade-off for audio vs. video quality depends on content type • Impact of emotional state, fatigue – On perceived quality, on quality requirements / expectations – Use of EEG and body sensors • Data analysis of non-traditional data – E.g. quantitative and qualitative, combined – Multivariate statistical methods Ulrich Reiter – PERCEVAL
    • 28 Thanks for your attention! ulrich.reiter@ntnu.no Ulrich Reiter – PERCEVAL
    • 29 References [1] http://perceval.no [2] ITU-R Recommendation BT.500-7: Methodology for the Subjective Assessment of the Quality of Television Pictures. International Telecommunication Union, Geneva (1996) [3] N. Staelens, S. Moens, W. Van den Broeck, I. Marien, B. Vermuelen, P. Lambert, R. Van de Walle and P. Demeester, “Assessing Quality of Experience of IPTV and Video on Demand Services in Real-Life Environments”, IEEE Transactions on Broadcasting, Vol.56, No.4, Dec. 2010. [4] Borowiak, A., Reiter, U., Svensson, U.P.: Quality Evaluation of Long Duration Audiovisual Content. In: The 9th Annual IEEE Consumer Communications and Networking Conference – Special Session on Quality of Experience (QoE) for Multimedia Communications, pp. 353--357, Las Vegas (2012) [5] Yang, X., Tan, Y., Ling, N.: Rate control for H.264 with two-step quantization parameter determination but single-pass encoding. In: EURASIP Journal on Applied Signal Processing, pp. 1--13, (2006) [6] Wang, H., Qian, X., Liu, G.: Inter Mode Decision Based on Just Noticeable Difference Profile. In: Proceedings of 2010 IEEE 17th International Conference on Image Processing, Hong Kong (2010) Thanks to Adam Borowiak and Ragnhild Eg for providing figures and slides! Ulrich Reiter – PERCEVAL