IPTV QoE Monitoring
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

IPTV QoE Monitoring

  • 11,066 views
Uploaded on

IPTV QoE and QoS Monitoring, the standards, Monitoring goals, hierarchies and domains.

IPTV QoE and QoS Monitoring, the standards, Monitoring goals, hierarchies and domains.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
11,066
On Slideshare
11,010
From Embeds
56
Number of Embeds
9

Actions

Shares
Downloads
419
Comments
3
Likes
13

Embeds 56

http://www.slideshare.net 33
http://honey-knowledge.blogspot.com 14
http://static.slidesharecdn.com 2
http://www.lmodules.com 2
http://www.dsp-ip.com 1
http://honey-knowledge.blogspot.in 1
http://honeypresentations.blogspot.ca 1
http://honey-knowledge.blogspot.co.uk 1
https://www.linkedin.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • When the destination receives a picture, speech, video, video+ voice the other side expects to see a clear, good picture and as close as to the real one. The destination wants to recognize the person from his/her voice and recognize the person and watch his movement as natural as possible.

Transcript

  • 1. IPTV QoS Monitoring Fast Forward Your Development www.dsp-ip.com
  • 2. Overview • QoS and QoE • Measurement and Monitoring Goals ▫ Lab, pre-deployment, Monitoring, failure analysis • Measurement Methods ▫ Objective, Subjective, payload/codec based • Monitoring Levels ▫ Transport, Transaction, content • Monitoring Domains ▫ Head-end, network, Home Fast Forward Your Development www.dsp-ip.com
  • 3. Challenges in providing QoE Name Challenge Answer network Overcome Packet loss, jitter, Improved protocols network congestion QoS mapping schemes Bandwidth No bandwidth in the network and at increased capacity inside the the home home and the network level Monitoring Measure hundreds of video streams Video over IP monitoring, in Gb Eth & IP environment new protocols & devices CRM costs Lower MTBF, call center calls and Automatic maintenance, “truck rolls” Automatic firmware update Home network monitoring Fast Forward Your Development www.dsp-ip.com
  • 4. User quality measurements • What does subscribers want? ▫ Content  Content quality  Content pricing Technical  Content availability Factors ▫ Viewing experience  Video Quality  Channel Zapping delay ▫ Integration of services  Usability  One bill  New services ▫ Customer services Fast Forward Your Development www.dsp-ip.com
  • 5. Voice/Video Quality? We can look at quality in different ways: 1. Perceptual subjective Quality (MOS) 2. Numerical Quality parameters (PSNR) 3. How to connect (relation matrix between 1 and 2) Fast Forward Your Development www.dsp-ip.com
  • 6. MEASUREMENT & MONITORING In the Lab & In The Fields For Pre-Deployment/monitoring/Failure Analysis Fast Forward Your Development www.dsp-ip.com
  • 7. Measurement & Monitoring phases Design Lab Testing Deployment Phase Pre Analysis Deployment Problem Solving Testing Tuning Pre- 24/7 Deployment Monitoring Phase Fast Forward Your Development www.dsp-ip.com
  • 8. Measurement & Monitoring phases • Design & Lab testing ▫ Simulation and Emulation of the network ▫ Lab and testing tools • Pre Deployment Stage ▫ Work on actual network ▫ Load testing ▫ Lab, testing, diagnosing and monitoring tools • Deployment (production) Phase ▫ Mostly monitoring (probes) equipment, management systems, data filtering and diagnostics equipment Fast Forward Your Development www.dsp-ip.com
  • 9. MEASUREMENT TAXONOMY Subjective Objective Payload based, codec aware, codec anaware Fast Forward Your Development www.dsp-ip.com
  • 10. Measurement methods review • Subjective ▫ Accurate ▫ Expensive, not for monitoring • Objective ▫ Repeatable ▫ For both testing and monitoring Fast Forward Your Development www.dsp-ip.com
  • 11. Multimedia monitoring methods Broadcast HSI and World Data World Subjective Objective Network MOS BT500 Codec aware Monitoring (Voice) (Video) Packet based Delay, Jitter Payload Packet loss VQS Full Telchemy Codec independent Reference Reduced Packet based V-Factor Reference J.144 PSNR No Reference VQI MDI Testing Monitoring Fast Forward Your Development www.dsp-ip.com
  • 12. Objective methods Objective Payload Codec aware Codec independent Network Packet based Packet based Monitoring Fast Forward Your Development www.dsp-ip.com
  • 13. Payload Based Methods Payload Full Reference Reduced Reference J.144 PSNR No Reference Fast Forward Your Development www.dsp-ip.com
  • 14. Full Reference: Video Quality Assessment ITU-T J.144 and ITU-R BT.1683 • Full-reference perceptual models • Digital TV • Rec. 601 image resolution (PAL/NTSC) • Bit rates: 768 kbps ~ 5 Mbps • Compression errors Fast Forward Your Development www.dsp-ip.com
  • 15. Voice Quality Assessment – with/out reference • ITU-T P.862 (Feb 2001) - Full Reference ▫ Full-reference perceptual model (PESQ) ▫ Signal-based measurement ▫ Narrow-band telephony and speech codecs ▫ P.862.1 provides output mapping for prediction on MOS scale • ITU-T P.563 (May 2004) ▫ No-reference perceptual model ▫ Signal-based measurement ▫ Narrow-band telephony applications Fast Forward Your Development www.dsp-ip.com
  • 16. Voice Quality Assessment • ITU-T P.862.2 (Nov 2005): ▫ Extension of ITU-T P.862 ▫ Wide-band telephony and speech codecs (5 ~7Khz) • ITU-T P.VTQ (on-going): ▫ Targeted at VoIP applications ▫ Minimum performance framework for no-reference packet-based measurement ▫ Models analyze packet statistics; speech payload is assumed ▫ Uses P.862 as a measurement reference Fast Forward Your Development www.dsp-ip.com
  • 17. Codec Aware Methods Codec aware Packet based VQS Telchemy V-Factor VQI Fast Forward Your Development www.dsp-ip.com
  • 18. Packet – Codec Aware • Monitoring technique • Codec dependent • Incorporates network parameters data with codec behavior data • Scales- could monitor thousands of channels • Examples: The need a codec aware metrics 35 ▫ VQS (Telchemy) 30 ▫ VQI(Brix) 25 Robust PSNR (dB) ▫ V-Factor (QoSMetrics) 20 15 codec 10 Problem area 5 0 0 “Raw” 5 10 15 20 codec Packet Loss (%) Fast Forward Your Development www.dsp-ip.com
  • 19. Packet – Codec aware 100 Packet Loss/Discard Rate 80 Packet loss/discard typically occurs in high density periods 60 40 20 0 0 10 20 30 40 50 Base quality level 5 Time depends on frame rate, Mean Opinion Score codec type, bit rate 4 3 Average can be Impact of Burst of misleading 2 Packet Loss 1 Subjective 0 5 10 15 20 compensation for Poor quality 5-8 Time variance between during burst of seconds 15-30 human and testing loss/discards seconds equipment view of loss Fast Forward Your Development www.dsp-ip.com
  • 20. Example V-Factor • Based on MPQM (Moving Picture Quality Metrics) – high quality video measurement standard • V = f(QER, PLR, R) ▫ QER – relative video codec quality ▫ PLR – Packet loss ratio (based on actual packet loss, jitter data and jitter buffer model) ▫ R – Image complexity factor (2-3) • Adopted by Spirnet Fast Forward Your Development www.dsp-ip.com
  • 21. Packet – Codec Independent • Monitoring only • Codec independent • Based on network parameters data only • Scales - could monitor thousands of channels • Examples: ▫ MDI  IneoQuest  standardized by IETF Fast Forward Your Development www.dsp-ip.com
  • 22. MONITORING LEVELS Fast Forward Your Development www.dsp-ip.com
  • 23. Measurement Levels • Transport Level • Service (transaction) Level • Media Quality Level Add Your Text Content Quality Add Your Text Video Stream Quality Add Your Text Transport Quality Add Your Text Transaction Quality Fast Forward Your Development www.dsp-ip.com
  • 24. Transport Level • Example: ▫ Packet Loss ▫ Loss Patterns ▫ Jitter ▫ Delay • Well understood • Defined by ITU and IETF Fast Forward Your Development www.dsp-ip.com
  • 25. Transaction Level • Examples ▫ Post-dial delay in PSTN/mobile networks ▫ Video start time • Requires understanding in both network monitoring and signaling (IGMP, SIP) and in media coding (analysis of the media to discover dial tone or I frame) Fast Forward Your Development www.dsp-ip.com
  • 26. Channel zapping delay • Multicast saves bandwidth but creates signaling delays: ▫ Multicast Leave + ▫ Multicast Join + ▫ First I Frame + ▫ Up to 2 seconds buffering time First frame viewed Total Channel zapping Latency Leave latency First I Buffering Frame latency Join latency Signaling Latency Media Latency Fast Forward Your Development www.dsp-ip.com
  • 27. Microsoft improved channel zapping • Unicast server is used to serve the stream till the second multicast stream has an “I Frame” ▫ Immediate I Frame ▫ Shorter buffering time I Frame unicast First frame viewed I Frame Multicast Total Channel zapping Latency Leave latency unicast stream Join latency Multicast stream Shorter buffering time Fast Forward Your Development www.dsp-ip.com
  • 28. VoD Signaling • Unicast server is used to serve the stream till the second multicast stream has an “I Frame” ▫ Immediate I Frame ▫ Shorter buffering time First frame viewed Total VoD Signaling Latency Play to server Buffering latency Fast Forward Your Development www.dsp-ip.com
  • 29. Content Level • Content quality is a payload based measurement. ▫ Requires decoding of the video stream ▫ Understanding of the buffering and error concealment algorithms of the decoder • CPU intensive – Does not scale • Accurate • Used mostly is Lab equipment and diagnostic equipment • Examples: ▫ PSNR ▫ ITU-T J.144 • Usually requires the reference (original) stream • Tests: ▫ Source artifacts ▫ Source quality Fast Forward Your Development www.dsp-ip.com
  • 30. Standardization landscape ATIS ITU ITU DSL IIF – IPTV Study Group 12 VQEG – Video DSL Forum Interoperability Quality Expert Forum Algorithms for Group TR-64, TR-69 end-to-end LAN and WAN Video performance QoS Metrics transmission measurement based monitoring Standardization performance on Subjective tests standards Database Used for Used in Monitoring Diagnostics / Lab Fast Forward Your Development www.dsp-ip.com
  • 31. Example: ATIS IIF Quality Metrics VSTQ - Video Service Transmission Quality ▫ Transmission Quality - codec/ content independent ▫ Based on the rate and distribution of effective packet loss and discard VSPQ - Video Service Picture Quality ▫ Estimated viewing quality ▫ Considers the impact of VSTQ, video codec type and rate, resolution VSAQ - Video Service Audio Quality ▫ audio listening quality ▫ Considers the impact of VSTQ, audio codec type, sample rate, ….. VSMQ - Video Service Multimedia Quality ▫ overall user experience ▫ Combined effect of VSPQ, VSAQ, audio-video synchronization.. VSCQ - Video Service Control (Plane) Quality ▫ Considers responsiveness and reliability of control plane (trick play) Fast Forward Your Development www.dsp-ip.com
  • 32. Monitoring levels • J.144 and PSNR examines the video content only (payload measurements) • TR101290 examines only transport stream data and coherence without examining the video content • V-Factor and VQS looks at packet loss, jitter and loss patterns data and incorporate it with codec information and video header information • MDI – Examines only packet loss and packet loss patterns without considering the codec or video information TR101290 MPEG2TS MDI V-Factor, VQS Headers J.144, PSNR Video payload Fast Forward Your Development www.dsp-ip.com
  • 33. MEASUREMENT DOMAINS Headend Network VoD Home Fast Forward Your Development www.dsp-ip.com
  • 34. DVB TR 101290 • DVB Transport stream monitoring standard • Alerts and KPI: ▫ TS sync loss ▫ CRC error ▫ Sync byte error ▫ PCR error ▫ PAT error ▫ PCR ▫ Continuity counter errors discontinuity ▫ PMT error ▫ PCR accuracy ▫ PID error error ▫ Transport error ▫ PTS error Fast Forward Your Development www.dsp-ip.com
  • 35. Network measurement points • Head-end • Network Probe interface ▫ VoD Servers  GBE, OC3/12, STM1/4 ▫ Encoders • Raw Video input ▫ TS muxers  MPEG2 TS - DVB ASI, DVBoIP • Network core  Digital and analog video ▫ Routers • Network edge ▫ ATM switches ▫ GBE Switches • Access ▫ DSLAM / BRAS • CPE ▫ STBs ▫ Home network Source: Brix-Video Quality Measurement Algorithms WP Fast Forward Your Development www.dsp-ip.com
  • 36. Headend • Video source quality • Video encoding quality • MPEG2 Transport stream parameters ▫ TR100290  1st-3rd level priority alerts • Inputs ▫ Raw analog video in a variety of forms (SD/HD) ▫ DVB-ASI ▫ DVBoIP ▫ DVBoATM (AAL1, AAL5) Fast Forward Your Development www.dsp-ip.com
  • 37. VoD • Pre-deployment ▫ RTSP signaling latency (delay) ▫ Performance under stress (user load) ▫ Trick Mode (FF, REW)  Signaling delay  Video quality  Buffer state (overflow and underflow) • Monitoring ▫ Loss of data (under stress) • Interfaces: ▫ GBE ▫ 10GBE Fast Forward Your Development www.dsp-ip.com
  • 38. Home network monitoring challenges • Scalability – Millions of STBs ▫ Cost – Monitoring hardware and/or software costs must not add to STB costs (multiply by number of STBs) ▫ Data – Large amounts of data which needs RT processing, filtering, logging and analysis • Unclear network structure • Non “standard” equipment • Installation, support and debugging problems Fast Forward Your Development www.dsp-ip.com
  • 39. Home network monitoring roots • Home network monitoring is currently a minor feature in IPTV “Support Automation” • Support Automation handles: ▫ Installation ▫ Preventive maintenance ▫ STB and CPE remote management (TR69 and more) ▫ Service verification for field technician ▫ CRM support ▫ Interaction logging ▫ Home Network monitoring and repair Fast Forward Your Development www.dsp-ip.com
  • 40. CPE network management • 2Wire solution -Three data views  OSS/BSS  Customer care  Data Analysis • Other companies in this field ▫ SupportSoft ▫ Enure ▫ Motive Fast Forward Your Development www.dsp-ip.com
  • 41. Low level network monitoring Fast Forward Your Development www.dsp-ip.com
  • 42. Vendors specialization Lab Testing Monitoring Incumbent Telecom Cable Telecom Cable Agilent Rhode & Agilent Thomson Ixia Schwartz JDSU/ Snell&Wilcox Spirnet Acterna Spirnet New Telecom Telecom entrants Shenick Telchemy Bridge Technologies iBrix QoSMetrix Fast Forward Your Development www.dsp-ip.com
  • 43. DSP-IP Contact information For Course materials & lecture request contact: Adi Yakov Training and Courses Manager adi@dsp-ip.com +972-9-8651933 For projects development services contact: Alona Ashkenazi Technology Management Services : alona@dsp-ip.com +972-9-8850956 www.dsp-ip.com Mail : info@dsp-ip.com Phone: +972-9-8850956, Fax : +972-50- 8962910 Fast Forward Your Development www.dsp-ip.com