Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
RECOMMENDATIONS FOR THE
SUBJECTIVE EVALUATION OF
SENSORY EXPERIENCE
Benjamin Rainer, Christian Timmerer,
and Markus Waltl
TABLE OF CONTENTS
 Introduction
 Motivation
 Test Setup
 Single Stimulus Studies
 SSCQE + ACR
 Double Stimulus Studi...
INTRODUCTION
• Multimedia enriched by sensory effects like wind,
vibration and light
 For quality degradations by lowerin...
MOTIVATION
 How has the test setup and design to be, for subjectively
assessing the QoE of sensory experience?
 Which st...
TEST CONDITIONS, TEST SETUP, AND
TEST DESGIN
 In-Lab assessments
 Setup according to ITU-R BT.500-11/13
 Ergonomic plac...
TEST CONDITIONS, TEST SETUP, AND
TEST DESGIN
 Textual introduction
 Each participant is presented the same introduction
...
SINGLE STIMULUS EVALUATION -
SSCQE + ACR
 Single Stimulus Continuous Quality Evaluation
with categorical rating scale
 C...
SINGLE STIMULUS EVALUATION -
SSCQE + ACR
 24 Participants
 11 female
 13 male
 Assess whether sensory effects unmask q...
SINGLE STIMULUS EVALUATION -
SSCQE + ACR
Earth Sequence Babylon A.D. Sequence
DOUBLE STIMULUS EVALUATIONS -
DCR
 Degradation Category Rating
 Double stimulus with a categorical scale
• veryannoying ...
DOUBLE STIMULUS EVALUATIONS -
DCR
DOUBLE STIMULUS EVALUATIONS -
DSCQS
 Double Stimulus Continuous Quality Scale
 Modified by combining it with a single st...
DOUBLE STIMULUS EVALUATIONS -
DSCQS
 In total 68 participants
 36 female
 32 male
 Conducted at three sites
 AAU (Aus...
DOUBLE STIMULUS EVALUATIONS -
DSCQS
 No significant difference between the means of RMIT and UoW
CONCLUSIONS
 Single stimulus method with categorical rating
scales are suitable for assessing the sensory
experience
 SS...
CONCLUSIONS
 The DSCQS was modified to fit the needs of an
subjective quality evaluation in the domain of
sensory experie...
THANK YOU!
QA
Upcoming SlideShare
Loading in …5
×

Pqs recommendation-sensory-experience

518 views

Published on

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Pqs recommendation-sensory-experience

  1. 1. RECOMMENDATIONS FOR THE SUBJECTIVE EVALUATION OF SENSORY EXPERIENCE Benjamin Rainer, Christian Timmerer, and Markus Waltl
  2. 2. TABLE OF CONTENTS  Introduction  Motivation  Test Setup  Single Stimulus Studies  SSCQE + ACR  Double Stimulus Studies  DCR  DSCQS + single stimulus
  3. 3. INTRODUCTION • Multimedia enriched by sensory effects like wind, vibration and light  For quality degradations by lowering the bitrate of the actual video sequence  For genres like action, documentary, commercial, and sports  Intensity of emotions are influenced • Passive emotions are decreased • Active emotions are increased
  4. 4. MOTIVATION  How has the test setup and design to be, for subjectively assessing the QoE of sensory experience?  Which stimulus methods are useful for subjectively assessing sensory experience?  Single stimulus?  Double stimulus?  Does mixing up single stimulus and double stimulus end up in a bias when assessing sensory experience?
  5. 5. TEST CONDITIONS, TEST SETUP, AND TEST DESGIN  In-Lab assessments  Setup according to ITU-R BT.500-11/13  Ergonomic placement of keyboard, mouse and seat  Test setup
  6. 6. TEST CONDITIONS, TEST SETUP, AND TEST DESGIN  Textual introduction  Each participant is presented the same introduction  Allows for training phases  Includes demographic questions aswell as the possiblity for the participants to give feedback  Supports different test methods
  7. 7. SINGLE STIMULUS EVALUATION - SSCQE + ACR  Single Stimulus Continuous Quality Evaluation with categorical rating scale  Combined with an Absolute Category Rating (ACR)
  8. 8. SINGLE STIMULUS EVALUATION - SSCQE + ACR  24 Participants  11 female  13 male  Assess whether sensory effects unmask quality degradations  Quality in terms of bitrate  Dedicated voting device  Stressed the participants for the SSCQE part  No intiuitive handling of the voting device  More intuitive voting devices should be used • Indirect devices e.g., EEG  Assessing Sensory Experience with SSCQE is not recommended
  9. 9. SINGLE STIMULUS EVALUATION - SSCQE + ACR Earth Sequence Babylon A.D. Sequence
  10. 10. DOUBLE STIMULUS EVALUATIONS - DCR  Degradation Category Rating  Double stimulus with a categorical scale • veryannoying (1), annoying (2), imperceptible (3), littleenhancement (4) andbigenhancement (5)  Assess the enhancement of sensory effects  No randomization of reference and test condition  Reference can be easily identified (no effects)  25 Participants  12 female  13 male
  11. 11. DOUBLE STIMULUS EVALUATIONS - DCR
  12. 12. DOUBLE STIMULUS EVALUATIONS - DSCQS  Double Stimulus Continuous Quality Scale  Modified by combining it with a single stimulus • After each stimulus presentation emotion rating • Double stimulus for the enhancement of QoE  The rating scale for emotions and the enhancement in QoE was on an interval of [0,100]
  13. 13. DOUBLE STIMULUS EVALUATIONS - DSCQS  In total 68 participants  36 female  32 male  Conducted at three sites  AAU (Austria)  UOW (Australia)  RMIT (Australia)  For identyfing a bias of the combined test method studies from UOW and RMIT were used
  14. 14. DOUBLE STIMULUS EVALUATIONS - DSCQS  No significant difference between the means of RMIT and UoW
  15. 15. CONCLUSIONS  Single stimulus method with categorical rating scales are suitable for assessing the sensory experience  SSCQE may stress the participants, the voting device plays an important role  Even with an intuitive voting device particpants may be stressed due to effects and the task of the continuous voting  Indirect voting devices such as EEG should be used
  16. 16. CONCLUSIONS  The DSCQS was modified to fit the needs of an subjective quality evaluation in the domain of sensory experience  The DCR method was used as defined, only the rating scale had to be changed slightly (from impairment to enhancement)  When combining stimulus methods caution is advised that no bias is introduced
  17. 17. THANK YOU! QA

×