T.Gy. Intrernetes médiakommunikáció. 2009.05. 13.


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

T.Gy. Intrernetes médiakommunikáció. 2009.05. 13.

  1. 1. Internetes médiakommunikáció Az Internetes Médiakommunikáció minőségének megítélése, mérése, a rendszer tervezési alapjai Takács György 12. előadás 2009. 05. 13. T.Gy. Intrernetes médiakommunikáció. 2009.05. 13.
  2. 2. What is QoE? <ul><li>The overall acceptability of an application or service, as perceived subjectively by the end user. </li></ul><ul><li>NOTES </li></ul><ul><li>1 Quality of Experience includes the complete end-to-end system effects (client, terminal, network, services infrastructure, etc). </li></ul><ul><li>2 Overall acceptability may be influenced by user expectations and context. ~From ITU-T Recommendation P.10/G.100~ </li></ul>
  3. 5. Factors affecting QoE of IPTV services
  4. 7. Why QoE evaluation? <ul><li>Recent survey indicates that…. </li></ul><ul><li>84% of IPTV providers think video quality monitoring is critical or very important </li></ul><ul><li>77% say video quality is a main reason for customer churn </li></ul><ul><li>78% say video quality problems are a main reason for high volumes of customer service calls </li></ul><ul><li>~From http://telephonyonline.com/iptv/news/iptv-video-quality-0219/~ </li></ul><ul><li>There surely is a market need for QoE evaluation methods/tools! </li></ul>
  5. 8. Quality monitoring as well as quality design <ul><li>Conventional telephone services over PSTN are relatively stable after a call is connected. </li></ul><ul><li>In PSTN services, service providers established telephone systems </li></ul><ul><li>with static quality design, then, monitor/manage the “availability” and “functionality” </li></ul><ul><li>IP networks (core/access/home-NWs) and IP-based terminals are time-variant systems. </li></ul><ul><li>In IPTV services, media quality varies depending on coding performance at H/E, traffic conditions in networks, and terminal performance </li></ul>
  6. 9. What to measure? <ul><li>Pre-transmission domain </li></ul><ul><li>Encode quality </li></ul><ul><li>System failure (blackout, frame freezing etc.) </li></ul><ul><li>Network domain </li></ul><ul><li>• Packet loss </li></ul><ul><li>• Delay </li></ul><ul><li>• Delay jitter </li></ul><ul><li>Post-transmission domain </li></ul><ul><li>Error recovery performance such as FEC (Forward Error Correction) </li></ul><ul><li>Decode quality (including PLC: Packet Loss Concealment) </li></ul><ul><li>D/A conversion performance </li></ul><ul><li>Color management performance </li></ul><ul><li>Display performance </li></ul>
  7. 10. When to measure? <ul><li>QoE measurement is necessary in the following phases: </li></ul><ul><li>Before the service is developed: </li></ul><ul><li>Individual quality factors (coding distortion etc.) </li></ul><ul><li>Quality planning </li></ul><ul><li>After the service is developed, but before the service is delivered: </li></ul><ul><li>Off-service quality check </li></ul><ul><li>After the service is delivered: </li></ul><ul><li>In-service quality monitoring at user end </li></ul><ul><li>In-service network mid-point monitoring </li></ul><ul><li>On-site quality check (on claim) </li></ul>
  8. 11. Where to measure?
  9. 12. How to measure? <ul><li>Subjective quality assessment is time consuming and expensive. </li></ul><ul><li>It cannot be applied to on-site and/or real-time measurement, which is essential in quality monitoring. </li></ul><ul><li>Objective quality assessment, which estimates QoE by subjective testing, is the solution! </li></ul>
  10. 13. Overview of objective quality assessment models <ul><li>From the viewpoint of input information into a model, objective quality assessment models can be categorized as follows: </li></ul><ul><li>Media-layer models </li></ul><ul><li>Parametric packet-layer models </li></ul><ul><li>Bitstream-layer models </li></ul><ul><li>Parametric planning models </li></ul><ul><li>Hybrid models </li></ul>
  11. 14. Media-layer models
  12. 15. Current standards of media-layer models (video) <ul><li>ITU-T standardized Rec. J.144, which recommends four fullreference models for Standard Definition TV, in 2004. </li></ul><ul><ul><li>Coding distortion only (no transmission errors such as packet loss) </li></ul></ul><ul><li>VQEG finalized its evaluation on full-reference and reducedreference models applicable to QCIF ~ VGA. The draft Recommendation has been consented at ITU-T SG9 as Rec. J.247, which includes four models, and Rec. J.246, respectively in May 2008. </li></ul><ul><ul><li>Packet-loss degradation as well as coding distortion is in the scope of Recommendation. </li></ul></ul><ul><li>ITU-T SG9 considered that current no-reference technologies seemed to be premature in terms of quality-estimation accuracy. </li></ul>
  13. 16. Parametric packet-layer models <ul><li>Only uses packet-header information. </li></ul><ul><li>Light-weight implementation. </li></ul><ul><li>Can be applied even if payload is encrypted. </li></ul>
  14. 17. Bitstream-layer models <ul><li>Use up to coded bitstream (payload), but not decoder output. </li></ul><ul><li>Can take into account the content dependence of the impact of packet loss. </li></ul>
  15. 18. Standardization activities for packet-layer and bitstream-layer models <ul><li>Parametric packet-layer models </li></ul><ul><li>ITU-T SG12 Q.14 & VQEG </li></ul><ul><li>• P.NAMS (non-intrusive parametric model for the assessment of performance of multimedia streaming) </li></ul><ul><li>Bitstream-layer models </li></ul><ul><li>ITU-T SG12 Q.14 & VQEG </li></ul><ul><li>• P.NBAMS (non-intrusive parametric bitstream model for the assessment of performance of multimedia streaming) </li></ul>
  16. 19. Parametric planning models <ul><li>Use quality planning parameters as input. </li></ul><ul><li>ITU-T Recommendation G.1070 for videophone applications. </li></ul><ul><li>ITU-T SG12 Q.13 is working on G.OMVAS (opinion model for video and audio streaming applications). </li></ul>
  17. 20. Present state of multimedia quality assessment methods Future trends of quality assessment research <ul><li>Multimodality </li></ul><ul><li>Multiparty </li></ul><ul><li>Wideband </li></ul><ul><li>Examples of NTT’s studies on multimedia quality assessment for audiovisual communication services </li></ul>
  18. 21. Relevant ITU-T/R Recommendations on Multimedia Quality Assessment <ul><li>Multimedia quality assessment is at a reasonably advanced stage. </li></ul>
  19. 22. Existing Subjective Multimedia Quality Assessment Methods (P.911) <ul><li>Multimedia quality is assessed in a similar way to individual audio/video qualities. </li></ul><ul><li>Absolute Category Rating (ACR) </li></ul><ul><li>Degradation Category Rating (DCR) </li></ul><ul><li>Pair Comparison Method (PC) </li></ul><ul><li>Single Stimulus Continuous Quality </li></ul><ul><li>Evaluation (SSCQE) </li></ul><ul><li>Assessment paying attention to the cross-modal influences is important. </li></ul><ul><li>Interactions between differing quality levels in different modalities </li></ul>
  20. 25. Existing Subjective Multimedia Quality Assessment Methods (P.920) <ul><li>Several category judgment scales are used to evaluate multimedia quality. </li></ul><ul><li>Overall audiovisual quality </li></ul><ul><li>Individual audio/video qualities </li></ul><ul><li>Effort needed to interrupt </li></ul><ul><li>Communication difficulty </li></ul><ul><li>Acceptability of communication </li></ul><ul><li>Communication quality depends on tasks used in conversational test. </li></ul><ul><li>Assessment considering interactivity and usability is also important. </li></ul>
  21. 26. Existing Concept Model of Objective Multimedia Quality Evaluation (J.148) <ul><li>Basic components of an objective multimedia quality model are defined. </li></ul>
  22. 27. Examples of Multimedia Quality Evaluation Model
  23. 28. Framework for Conducting Quality Assessment Research
  24. 29. Key Words for Future Trends of Multimedia Quality Assessment (1/3) <ul><li>Multimodality </li></ul><ul><li>Combination of multiple media such as audio, video, text, graphics, fax, and telephony in the communication of information </li></ul><ul><li>Key points of quality assessment </li></ul><ul><li>√ Intermedia synchronization </li></ul><ul><li>√ Intermedia quality balance </li></ul>
  25. 30. Key Words for Future Trends of Multimedia Quality Assessment (2/3) <ul><li>Multiparty </li></ul><ul><li>Communication style extending from 1:1 to N:N (e.g., instant messaging, teleconferencing, and distributed collaboration services) </li></ul><ul><li>Key points of quality assessment </li></ul><ul><li>√ Interdestination synchronization </li></ul><ul><li>√ Interdestination quality balance </li></ul>
  26. 31. Key Words for Future Trends of Multimedia Quality Assessment (3/3) <ul><li>Wideband </li></ul><ul><li>Telecommunications applications having more bandwidth available for higher-quality multimodal services </li></ul><ul><li>Key points of quality assessment </li></ul><ul><li>√ Psychological factors: We need to assess the richness of high-quality services not only on a one-dimensional scale, like MOS, but also on a multi-dimensional scale. </li></ul>
  27. 32. Examples of NTT’s Studies on Quality Assessment for “Multimodality” (1/3)
  28. 33. Examples of NTT’s Studies on Quality Assessment for “Multimodality” (2/3)
  29. 34. Examples of NTT’s Studies on Quality Assessment for “Multimodality” (3/3)
  30. 35. Examples of NTT’s Studies on Quality Assessment for “Multiparty” (1/2) Quality imbalance is one of the multiparty quality degradation factors.
  31. 36. Examples of NTT’s Studies on Quality Assessment for “Multiparty” (2/2) <ul><li>Overall quality is strongly affected by the inferior quality at another point. </li></ul><ul><li>Other-point audiovisual quality depends on conversation task or roles. </li></ul>
  32. 37. Examples of NTT’s Studies on Quality Assessment for “Wideband” (1/3) <ul><li>Interactive multimodal quality can be evaluated using a multi-dimensional scale of psychological factors. </li></ul>
  33. 38. Examples of NTT’s Studies on Quality Assessment for “Wideband” (2/3) <ul><li>Psychological factors were extracted by using the semantic differential (SD) technique and factor analysis. </li></ul><ul><li>Subject’s impression of an audiovisual communication service was evaluated on the basis of 25 pairs of bipolar adjectives on a seven-grade comparison scale. </li></ul>
  34. 39. Examples of NTT’s Studies on Quality Assessment for “Wideband” (3/3) <ul><li>Multimedia quality was formulated as a function of two psychological factors expressing an aesthetic feeling and a feeling of activity . </li></ul>