Successfully reported this slideshow.
Your SlideShare is downloading. ×

Pqs recommendation-sensory-experience

Ad

RECOMMENDATIONS FOR THE
SUBJECTIVE EVALUATION OF
SENSORY EXPERIENCE
Benjamin Rainer, Christian Timmerer,
and Markus Waltl

Ad

TABLE OF CONTENTS
 Introduction
 Motivation
 Test Setup
 Single Stimulus Studies
 SSCQE + ACR
 Double Stimulus Studi...

Ad

INTRODUCTION
• Multimedia enriched by sensory effects like wind,
vibration and light
 For quality degradations by lowerin...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Upcoming SlideShare
Goal setting
Goal setting
Loading in …3
×

Check these out next

1 of 17 Ad
1 of 17 Ad
Advertisement

More Related Content

Advertisement

Pqs recommendation-sensory-experience

  1. 1. RECOMMENDATIONS FOR THE SUBJECTIVE EVALUATION OF SENSORY EXPERIENCE Benjamin Rainer, Christian Timmerer, and Markus Waltl
  2. 2. TABLE OF CONTENTS  Introduction  Motivation  Test Setup  Single Stimulus Studies  SSCQE + ACR  Double Stimulus Studies  DCR  DSCQS + single stimulus
  3. 3. INTRODUCTION • Multimedia enriched by sensory effects like wind, vibration and light  For quality degradations by lowering the bitrate of the actual video sequence  For genres like action, documentary, commercial, and sports  Intensity of emotions are influenced • Passive emotions are decreased • Active emotions are increased
  4. 4. MOTIVATION  How has the test setup and design to be, for subjectively assessing the QoE of sensory experience?  Which stimulus methods are useful for subjectively assessing sensory experience?  Single stimulus?  Double stimulus?  Does mixing up single stimulus and double stimulus end up in a bias when assessing sensory experience?
  5. 5. TEST CONDITIONS, TEST SETUP, AND TEST DESGIN  In-Lab assessments  Setup according to ITU-R BT.500-11/13  Ergonomic placement of keyboard, mouse and seat  Test setup
  6. 6. TEST CONDITIONS, TEST SETUP, AND TEST DESGIN  Textual introduction  Each participant is presented the same introduction  Allows for training phases  Includes demographic questions aswell as the possiblity for the participants to give feedback  Supports different test methods
  7. 7. SINGLE STIMULUS EVALUATION - SSCQE + ACR  Single Stimulus Continuous Quality Evaluation with categorical rating scale  Combined with an Absolute Category Rating (ACR)
  8. 8. SINGLE STIMULUS EVALUATION - SSCQE + ACR  24 Participants  11 female  13 male  Assess whether sensory effects unmask quality degradations  Quality in terms of bitrate  Dedicated voting device  Stressed the participants for the SSCQE part  No intiuitive handling of the voting device  More intuitive voting devices should be used • Indirect devices e.g., EEG  Assessing Sensory Experience with SSCQE is not recommended
  9. 9. SINGLE STIMULUS EVALUATION - SSCQE + ACR Earth Sequence Babylon A.D. Sequence
  10. 10. DOUBLE STIMULUS EVALUATIONS - DCR  Degradation Category Rating  Double stimulus with a categorical scale • veryannoying (1), annoying (2), imperceptible (3), littleenhancement (4) andbigenhancement (5)  Assess the enhancement of sensory effects  No randomization of reference and test condition  Reference can be easily identified (no effects)  25 Participants  12 female  13 male
  11. 11. DOUBLE STIMULUS EVALUATIONS - DCR
  12. 12. DOUBLE STIMULUS EVALUATIONS - DSCQS  Double Stimulus Continuous Quality Scale  Modified by combining it with a single stimulus • After each stimulus presentation emotion rating • Double stimulus for the enhancement of QoE  The rating scale for emotions and the enhancement in QoE was on an interval of [0,100]
  13. 13. DOUBLE STIMULUS EVALUATIONS - DSCQS  In total 68 participants  36 female  32 male  Conducted at three sites  AAU (Austria)  UOW (Australia)  RMIT (Australia)  For identyfing a bias of the combined test method studies from UOW and RMIT were used
  14. 14. DOUBLE STIMULUS EVALUATIONS - DSCQS  No significant difference between the means of RMIT and UoW
  15. 15. CONCLUSIONS  Single stimulus method with categorical rating scales are suitable for assessing the sensory experience  SSCQE may stress the participants, the voting device plays an important role  Even with an intuitive voting device particpants may be stressed due to effects and the task of the continuous voting  Indirect voting devices such as EEG should be used
  16. 16. CONCLUSIONS  The DSCQS was modified to fit the needs of an subjective quality evaluation in the domain of sensory experience  The DCR method was used as defined, only the rating scale had to be changed slightly (from impairment to enhancement)  When combining stimulus methods caution is advised that no bias is introduced
  17. 17. THANK YOU! QA

×