Your SlideShare is downloading. ×
IPTV QoE Monitoring
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

IPTV QoE Monitoring


Published on

IPTV QoE and QoS Monitoring, the standards, Monitoring goals, hierarchies and domains.

IPTV QoE and QoS Monitoring, the standards, Monitoring goals, hierarchies and domains.

Published in: Technology

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • When the destination receives a picture, speech, video, video+ voice the other side expects to see a clear, good picture and as close as to the real one. The destination wants to recognize the person from his/her voice and recognize the person and watch his movement as natural as possible.
  • Transcript

    • 1. IPTV QoS Monitoring Fast Forward Your Development
    • 2. Overview • QoS and QoE • Measurement and Monitoring Goals ▫ Lab, pre-deployment, Monitoring, failure analysis • Measurement Methods ▫ Objective, Subjective, payload/codec based • Monitoring Levels ▫ Transport, Transaction, content • Monitoring Domains ▫ Head-end, network, Home Fast Forward Your Development
    • 3. Challenges in providing QoE Name Challenge Answer network Overcome Packet loss, jitter, Improved protocols network congestion QoS mapping schemes Bandwidth No bandwidth in the network and at increased capacity inside the the home home and the network level Monitoring Measure hundreds of video streams Video over IP monitoring, in Gb Eth & IP environment new protocols & devices CRM costs Lower MTBF, call center calls and Automatic maintenance, “truck rolls” Automatic firmware update Home network monitoring Fast Forward Your Development
    • 4. User quality measurements • What does subscribers want? ▫ Content  Content quality  Content pricing Technical  Content availability Factors ▫ Viewing experience  Video Quality  Channel Zapping delay ▫ Integration of services  Usability  One bill  New services ▫ Customer services Fast Forward Your Development
    • 5. Voice/Video Quality? We can look at quality in different ways: 1. Perceptual subjective Quality (MOS) 2. Numerical Quality parameters (PSNR) 3. How to connect (relation matrix between 1 and 2) Fast Forward Your Development
    • 6. MEASUREMENT & MONITORING In the Lab & In The Fields For Pre-Deployment/monitoring/Failure Analysis Fast Forward Your Development
    • 7. Measurement & Monitoring phases Design Lab Testing Deployment Phase Pre Analysis Deployment Problem Solving Testing Tuning Pre- 24/7 Deployment Monitoring Phase Fast Forward Your Development
    • 8. Measurement & Monitoring phases • Design & Lab testing ▫ Simulation and Emulation of the network ▫ Lab and testing tools • Pre Deployment Stage ▫ Work on actual network ▫ Load testing ▫ Lab, testing, diagnosing and monitoring tools • Deployment (production) Phase ▫ Mostly monitoring (probes) equipment, management systems, data filtering and diagnostics equipment Fast Forward Your Development
    • 9. MEASUREMENT TAXONOMY Subjective Objective Payload based, codec aware, codec anaware Fast Forward Your Development
    • 10. Measurement methods review • Subjective ▫ Accurate ▫ Expensive, not for monitoring • Objective ▫ Repeatable ▫ For both testing and monitoring Fast Forward Your Development
    • 11. Multimedia monitoring methods Broadcast HSI and World Data World Subjective Objective Network MOS BT500 Codec aware Monitoring (Voice) (Video) Packet based Delay, Jitter Payload Packet loss VQS Full Telchemy Codec independent Reference Reduced Packet based V-Factor Reference J.144 PSNR No Reference VQI MDI Testing Monitoring Fast Forward Your Development
    • 12. Objective methods Objective Payload Codec aware Codec independent Network Packet based Packet based Monitoring Fast Forward Your Development
    • 13. Payload Based Methods Payload Full Reference Reduced Reference J.144 PSNR No Reference Fast Forward Your Development
    • 14. Full Reference: Video Quality Assessment ITU-T J.144 and ITU-R BT.1683 • Full-reference perceptual models • Digital TV • Rec. 601 image resolution (PAL/NTSC) • Bit rates: 768 kbps ~ 5 Mbps • Compression errors Fast Forward Your Development
    • 15. Voice Quality Assessment – with/out reference • ITU-T P.862 (Feb 2001) - Full Reference ▫ Full-reference perceptual model (PESQ) ▫ Signal-based measurement ▫ Narrow-band telephony and speech codecs ▫ P.862.1 provides output mapping for prediction on MOS scale • ITU-T P.563 (May 2004) ▫ No-reference perceptual model ▫ Signal-based measurement ▫ Narrow-band telephony applications Fast Forward Your Development
    • 16. Voice Quality Assessment • ITU-T P.862.2 (Nov 2005): ▫ Extension of ITU-T P.862 ▫ Wide-band telephony and speech codecs (5 ~7Khz) • ITU-T P.VTQ (on-going): ▫ Targeted at VoIP applications ▫ Minimum performance framework for no-reference packet-based measurement ▫ Models analyze packet statistics; speech payload is assumed ▫ Uses P.862 as a measurement reference Fast Forward Your Development
    • 17. Codec Aware Methods Codec aware Packet based VQS Telchemy V-Factor VQI Fast Forward Your Development
    • 18. Packet – Codec Aware • Monitoring technique • Codec dependent • Incorporates network parameters data with codec behavior data • Scales- could monitor thousands of channels • Examples: The need a codec aware metrics 35 ▫ VQS (Telchemy) 30 ▫ VQI(Brix) 25 Robust PSNR (dB) ▫ V-Factor (QoSMetrics) 20 15 codec 10 Problem area 5 0 0 “Raw” 5 10 15 20 codec Packet Loss (%) Fast Forward Your Development
    • 19. Packet – Codec aware 100 Packet Loss/Discard Rate 80 Packet loss/discard typically occurs in high density periods 60 40 20 0 0 10 20 30 40 50 Base quality level 5 Time depends on frame rate, Mean Opinion Score codec type, bit rate 4 3 Average can be Impact of Burst of misleading 2 Packet Loss 1 Subjective 0 5 10 15 20 compensation for Poor quality 5-8 Time variance between during burst of seconds 15-30 human and testing loss/discards seconds equipment view of loss Fast Forward Your Development
    • 20. Example V-Factor • Based on MPQM (Moving Picture Quality Metrics) – high quality video measurement standard • V = f(QER, PLR, R) ▫ QER – relative video codec quality ▫ PLR – Packet loss ratio (based on actual packet loss, jitter data and jitter buffer model) ▫ R – Image complexity factor (2-3) • Adopted by Spirnet Fast Forward Your Development
    • 21. Packet – Codec Independent • Monitoring only • Codec independent • Based on network parameters data only • Scales - could monitor thousands of channels • Examples: ▫ MDI  IneoQuest  standardized by IETF Fast Forward Your Development
    • 22. MONITORING LEVELS Fast Forward Your Development
    • 23. Measurement Levels • Transport Level • Service (transaction) Level • Media Quality Level Add Your Text Content Quality Add Your Text Video Stream Quality Add Your Text Transport Quality Add Your Text Transaction Quality Fast Forward Your Development
    • 24. Transport Level • Example: ▫ Packet Loss ▫ Loss Patterns ▫ Jitter ▫ Delay • Well understood • Defined by ITU and IETF Fast Forward Your Development
    • 25. Transaction Level • Examples ▫ Post-dial delay in PSTN/mobile networks ▫ Video start time • Requires understanding in both network monitoring and signaling (IGMP, SIP) and in media coding (analysis of the media to discover dial tone or I frame) Fast Forward Your Development
    • 26. Channel zapping delay • Multicast saves bandwidth but creates signaling delays: ▫ Multicast Leave + ▫ Multicast Join + ▫ First I Frame + ▫ Up to 2 seconds buffering time First frame viewed Total Channel zapping Latency Leave latency First I Buffering Frame latency Join latency Signaling Latency Media Latency Fast Forward Your Development
    • 27. Microsoft improved channel zapping • Unicast server is used to serve the stream till the second multicast stream has an “I Frame” ▫ Immediate I Frame ▫ Shorter buffering time I Frame unicast First frame viewed I Frame Multicast Total Channel zapping Latency Leave latency unicast stream Join latency Multicast stream Shorter buffering time Fast Forward Your Development
    • 28. VoD Signaling • Unicast server is used to serve the stream till the second multicast stream has an “I Frame” ▫ Immediate I Frame ▫ Shorter buffering time First frame viewed Total VoD Signaling Latency Play to server Buffering latency Fast Forward Your Development
    • 29. Content Level • Content quality is a payload based measurement. ▫ Requires decoding of the video stream ▫ Understanding of the buffering and error concealment algorithms of the decoder • CPU intensive – Does not scale • Accurate • Used mostly is Lab equipment and diagnostic equipment • Examples: ▫ PSNR ▫ ITU-T J.144 • Usually requires the reference (original) stream • Tests: ▫ Source artifacts ▫ Source quality Fast Forward Your Development
    • 30. Standardization landscape ATIS ITU ITU DSL IIF – IPTV Study Group 12 VQEG – Video DSL Forum Interoperability Quality Expert Forum Algorithms for Group TR-64, TR-69 end-to-end LAN and WAN Video performance QoS Metrics transmission measurement based monitoring Standardization performance on Subjective tests standards Database Used for Used in Monitoring Diagnostics / Lab Fast Forward Your Development
    • 31. Example: ATIS IIF Quality Metrics VSTQ - Video Service Transmission Quality ▫ Transmission Quality - codec/ content independent ▫ Based on the rate and distribution of effective packet loss and discard VSPQ - Video Service Picture Quality ▫ Estimated viewing quality ▫ Considers the impact of VSTQ, video codec type and rate, resolution VSAQ - Video Service Audio Quality ▫ audio listening quality ▫ Considers the impact of VSTQ, audio codec type, sample rate, ….. VSMQ - Video Service Multimedia Quality ▫ overall user experience ▫ Combined effect of VSPQ, VSAQ, audio-video synchronization.. VSCQ - Video Service Control (Plane) Quality ▫ Considers responsiveness and reliability of control plane (trick play) Fast Forward Your Development
    • 32. Monitoring levels • J.144 and PSNR examines the video content only (payload measurements) • TR101290 examines only transport stream data and coherence without examining the video content • V-Factor and VQS looks at packet loss, jitter and loss patterns data and incorporate it with codec information and video header information • MDI – Examines only packet loss and packet loss patterns without considering the codec or video information TR101290 MPEG2TS MDI V-Factor, VQS Headers J.144, PSNR Video payload Fast Forward Your Development
    • 33. MEASUREMENT DOMAINS Headend Network VoD Home Fast Forward Your Development
    • 34. DVB TR 101290 • DVB Transport stream monitoring standard • Alerts and KPI: ▫ TS sync loss ▫ CRC error ▫ Sync byte error ▫ PCR error ▫ PAT error ▫ PCR ▫ Continuity counter errors discontinuity ▫ PMT error ▫ PCR accuracy ▫ PID error error ▫ Transport error ▫ PTS error Fast Forward Your Development
    • 35. Network measurement points • Head-end • Network Probe interface ▫ VoD Servers  GBE, OC3/12, STM1/4 ▫ Encoders • Raw Video input ▫ TS muxers  MPEG2 TS - DVB ASI, DVBoIP • Network core  Digital and analog video ▫ Routers • Network edge ▫ ATM switches ▫ GBE Switches • Access ▫ DSLAM / BRAS • CPE ▫ STBs ▫ Home network Source: Brix-Video Quality Measurement Algorithms WP Fast Forward Your Development
    • 36. Headend • Video source quality • Video encoding quality • MPEG2 Transport stream parameters ▫ TR100290  1st-3rd level priority alerts • Inputs ▫ Raw analog video in a variety of forms (SD/HD) ▫ DVB-ASI ▫ DVBoIP ▫ DVBoATM (AAL1, AAL5) Fast Forward Your Development
    • 37. VoD • Pre-deployment ▫ RTSP signaling latency (delay) ▫ Performance under stress (user load) ▫ Trick Mode (FF, REW)  Signaling delay  Video quality  Buffer state (overflow and underflow) • Monitoring ▫ Loss of data (under stress) • Interfaces: ▫ GBE ▫ 10GBE Fast Forward Your Development
    • 38. Home network monitoring challenges • Scalability – Millions of STBs ▫ Cost – Monitoring hardware and/or software costs must not add to STB costs (multiply by number of STBs) ▫ Data – Large amounts of data which needs RT processing, filtering, logging and analysis • Unclear network structure • Non “standard” equipment • Installation, support and debugging problems Fast Forward Your Development
    • 39. Home network monitoring roots • Home network monitoring is currently a minor feature in IPTV “Support Automation” • Support Automation handles: ▫ Installation ▫ Preventive maintenance ▫ STB and CPE remote management (TR69 and more) ▫ Service verification for field technician ▫ CRM support ▫ Interaction logging ▫ Home Network monitoring and repair Fast Forward Your Development
    • 40. CPE network management • 2Wire solution -Three data views  OSS/BSS  Customer care  Data Analysis • Other companies in this field ▫ SupportSoft ▫ Enure ▫ Motive Fast Forward Your Development
    • 41. Low level network monitoring Fast Forward Your Development
    • 42. Vendors specialization Lab Testing Monitoring Incumbent Telecom Cable Telecom Cable Agilent Rhode & Agilent Thomson Ixia Schwartz JDSU/ Snell&Wilcox Spirnet Acterna Spirnet New Telecom Telecom entrants Shenick Telchemy Bridge Technologies iBrix QoSMetrix Fast Forward Your Development
    • 43. DSP-IP Contact information For Course materials & lecture request contact: Adi Yakov Training and Courses Manager +972-9-8651933 For projects development services contact: Alona Ashkenazi Technology Management Services : +972-9-8850956 Mail : Phone: +972-9-8850956, Fax : +972-50- 8962910 Fast Forward Your Development