Quality of Experience The Good, The Bad, and The Ugly!


Published on

Slides presented at NTNU Audio Visual Forum International Seminar, May 4th, 2009, Trondheim, Norway

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Quality of Experience The Good, The Bad, and The Ugly!

  1. 1. Quality of Experience The Good, The Bad, and The Ugly! Touradj Ebrahimi Adjunct Professor at NTNU/Q2S Professor at EPFL
  2. 2. A few initial words <ul><li>This talk is inspired from another talk I gave with the same approach (The Good, The Bad, and The Ugly!) on the future of multimedia retrieval research at CBMI, June 2008, in London, UK </li></ul><ul><li>This talk also revisits some issues addressed at a previous talk I gave at Yokosuka Research Park (YRP), in Japan, on February 2001, where I suggested the use of Quality of Experience (QoE) as opposed to Quality of Service in communication </li></ul>
  3. 3. Pointers to these talks <ul><li>CBMI 2008 talk </li></ul><ul><ul><li>Title: Future of Multimedia retrieval Research - The Good, The Bad, and The Ugly! </li></ul></ul><ul><ul><li>Slides of presentation: http://tinyurl.com/d2sxcw </li></ul></ul><ul><li>YRP 2001 workshop talk </li></ul><ul><ul><li>Title: Quality of Experience – A new look into quality and its impact in future personal communication </li></ul></ul><ul><ul><li>Slides of presentation: http://tinyurl.com/d5ebdf </li></ul></ul><ul><ul><li>Video of presentation: http://www.youtube.com/mmspgroup </li></ul></ul>
  4. 4. Communication – a definition <ul><li>Communication is about sharing experience (real or imaginary) with others </li></ul><ul><li>It all started with story telling and wall drawing around the fire in the caves of early men </li></ul><ul><li>Modern communication systems are mostly evolved versions of the good old story telling and wall drawing, which hopefully offer increasingly richer experience </li></ul><ul><li>The degree of richness of the experience is measured by Quality of Experience (QoE) </li></ul>
  5. 5. Evolutions versus Revolutions in communication <ul><li>Evolution: A given modality of communication improves itself in terms of Quality of Experience : </li></ul><ul><ul><li>B&W TV </li></ul></ul><ul><ul><li>Color TV </li></ul></ul><ul><ul><li>Stereo and CD quality audio TV </li></ul></ul><ul><ul><li>HDTV </li></ul></ul><ul><li>Revolution: New modalities of communication and expression are introduced bringing new dimensions in Quality of Experience </li></ul><ul><ul><li>Photography </li></ul></ul><ul><ul><li>Cinema </li></ul></ul><ul><ul><li>Internet </li></ul></ul>
  6. 6. Some of the major evolutions and revolutions in communication <ul><li>Story telling and cave drawing </li></ul><ul><li>Books and written press </li></ul><ul><li>Photography </li></ul><ul><li>Telegraph </li></ul><ul><li>Telephone </li></ul><ul><li>Radio and music recording </li></ul><ul><li>Cinema </li></ul><ul><li>Television and video recording </li></ul><ul><li>Internet (including VOIP, IPTV, etc.) </li></ul><ul><li>Mobile communication </li></ul><ul><li>Social networking (Web 2.0) </li></ul><ul><li>(What is next?) </li></ul>
  7. 7. Quality of Experience <ul><li>A terminology and concept used in many fields: </li></ul><ul><ul><li>Religion: S. Crites (1971) </li></ul></ul><ul><ul><li>Marketing: SW Brown, TA Swartz (1989) </li></ul></ul><ul><ul><li>Philosophy: G. Harman (1990) </li></ul></ul><ul><ul><li>Psychology: U. Schiefele (1996) </li></ul></ul><ul><ul><li>Design: L Alben (1996) </li></ul></ul><ul><ul><li>Internet: A. Van Moorsel (2001) </li></ul></ul><ul><ul><li>… and many many other… </li></ul></ul>
  8. 8. The Good
  9. 9. The Good <ul><li>Digital world has (re-)discovered the notion of quality </li></ul><ul><ul><li>Lower quality content is less and less tolerated by end-users </li></ul></ul><ul><ul><li>Digital technology can now rival and even surpass the old analog systems performance while remaining cost effective </li></ul></ul><ul><li>Increasing interest in QoE </li></ul><ul><ul><li>Extending from device- and system-centric quality optimization to end-to-end and especially user-centric optimization </li></ul></ul>
  10. 10. The Good <ul><li>Challenges in QoE currently under investigation: </li></ul><ul><ul><li>Reliable quality assessment methods and metrics for speech </li></ul></ul><ul><ul><li>Reliable quality assessment methods and metrics for pictures </li></ul></ul><ul><ul><li>Reliable quality assessment methods and metrics for audio </li></ul></ul><ul><ul><li>Reliable quality assessment methods and metrics for video </li></ul></ul><ul><ul><li>Reliable quality metrics with full, no, and reduced reference </li></ul></ul><ul><ul><li>Device and content independent quality metrics </li></ul></ul><ul><li>Increased interest in workshops and conferences around the notion of quality assessment and metrics </li></ul><ul><ul><li>QoMEX: International Workshop on Quality of Multimedia Experience (http://www.qomex.org) </li></ul></ul><ul><ul><li>VPQM: International Workshop on Video Processing and Quality Metrics for Consumer Electronics (http://www.vpqm.org) </li></ul></ul><ul><ul><li>IMQA: International Workshop on Image Media Quality and its Applications ( http://www.mi.tj.chiba-u.jp/imqa2008/) </li></ul></ul>
  11. 11. The Good <ul><li>Standardization efforts in quality assessment and metrics </li></ul><ul><ul><li>Video Quality Experts Group (VQEG) </li></ul></ul><ul><ul><li>ITU-T SG 12 (Performance, QoS and QoE) </li></ul></ul><ul><ul><li>ITU-R WP6C (Prog. production and quality assessment) </li></ul></ul><ul><ul><li>JPEG (Advanced Image Coding - AIC) </li></ul></ul><ul><ul><li>MPEG (High performance Video Coding – HVC) </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><li>Increased interest from industry </li></ul>
  12. 12. The Bad
  13. 13. The Bad <ul><li>Challenges in QoE not sufficiently addressed </li></ul><ul><ul><li>Content-dependent quality assessment methods and metrics </li></ul></ul><ul><ul><li>Context-dependent quality assessment methods and metrics </li></ul></ul><ul><ul><li>Quality assessment methods and metrics beyond AV (haptics, …) </li></ul></ul><ul><ul><li>Multi-modal quality assessment methods and metrics (AV, … ) </li></ul></ul><ul><ul><li>3D quality assessment methods and metrics (3D sound, 3D video, …) </li></ul></ul><ul><ul><li>HDR content quality assessment methods and metrics </li></ul></ul><ul><ul><li>Interactivity quality metrics (closely related to usability) </li></ul></ul><ul><ul><li>Naturalness and Realism quality metrics </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><li>Lack of quality certification mechanisms </li></ul><ul><ul><li>Similar to ISO 9000 series </li></ul></ul><ul><li>Lack of better coordination between experts and disciplines relevant to QoE </li></ul>
  14. 14. The Ugly
  15. 15. The Ugly <ul><li>Lack of a widely accepted definition of Quality of Experience </li></ul><ul><ul><li>Some definitions are even contradictory! </li></ul></ul><ul><li>Increasing usage of ‘QoE’ as a buzz word similar to the usage of ‘ Multimedia’ </li></ul><ul><li>Lack of research funding and coordination especially from the European Commission </li></ul><ul><ul><li>Several past attempts to get funding have faced failure </li></ul></ul><ul><ul><li>Lack of vision, understanding, clarity? </li></ul></ul><ul><ul><li>Costly strategic error for countries and programs not paying sufficient attention to this important topic? </li></ul></ul><ul><li>Current efforts on QoE are still too fragmented </li></ul>
  16. 16. Thank you for your attention Questions?