Microsoft PowerPoint - IIT - VoIP Performance Management

Uploaded on


  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Managing the Performance of Enterprise and Residential VoIP Services Alan Clark Telchemy
  • 2. Outline • Problems affecting Residential VoIP and Teleworkers • VoIP Performance and Quality Challenges • VoIP Performance Measurement and Problem Diagnosis • Passive Monitoring using Embedded Agents • Active Testing using Downloadable Agents • Looking forward to IPTV
  • 3. Problems affecting Residential VoIP and Teleworkers • Net Neutrality issues – Deliberate downgrading of priority – Artificial congestion • Residential gateways – Performance – Difficult to control bandwidth allocation – “Bugs” affecting SIP/RTP connectivity • Home networks – Home environment – Unexpected sources of congestion
  • 4. VoIP Performance and Quality Challenges • Lost packets and jitter due to network congestion – Typically time varying/ transient • Packet loss due to layer 1-2 problems – Duplex mismatch, bit errors….. • Delay – High delays lead to conversational problems – Even small delays make echo more obvious • Echo – Many phone services suffer from echo – Not obvious if delay is very short • Signal levels – High – clipping, Low – noisy speech, gaps • Noise levels – Obvious but – should this be a factor in QoE metrics?
  • 5. Jitter measurements can be misleading!!! 150 Average jitter level (PPDV) = 4.5mS Peak jitter level = 60mS 125 Delay (mS) 100 75 50 0 0.5 1 1.5 2 Time (Seconds)
  • 6. Packet Loss is time varying 50 500mS Avge Packet Loss Rate 40 Average packet loss rate = 2.1% Peak packet loss = 30% 30 20 10 0 30 35 40 45 50 55 60 65 70 Time (seconds)
  • 7. Passive Monitoring using Embedded Agents • Basic premise – Measure performance close to the user • Integrate a performance monitoring agent into the IP endpoint (IP phone) – Send fewer but more meaningful reports • Capture information and report info on transient problems – Incorporate signal, noise, echo information but avoid the need to decode the signal • Leverage data already available from the endpoint’s codec/ echo canceller – Conversational quality calculation requires some information from the remote endpoint • Need combined information exchange/ reporting protocol
  • 8. VoIP Performance Reporting Architecture End of call QoS report QoS Embedded RTCP XR Embedded Agent Agent RTCP XR RTCP XR exchanged every 5-15 seconds during call
  • 9. How to measure performance? • Subjective quality – “MOS” – Listening quality – Conversational quality • Objective measurement
  • 10. What is MOS? • Extract from an ITU subjective test 50 40 • Mean Opinion Score (MOS) was 30 2.4 Votes 20 10 • 1=Unacceptable • 2=Poor 0 1 2 3 4 5 • 3=Fair Opinion Score • 4=Good • 5=Excellent
  • 11. Objectively estimating MOS • VQmon – Passive measurement algorithm, widely used in VoIP test tools and IP phones/ gateways (tested vs P.562) • ITU G.107 – Transmission planning model – Used for monitoring but does not comply with P.562 • ITU P.562 – Standard for testing IP based passive measurement algorithms • ITU P.563 – Voice sample based passive measurement algorithm, very compute intensive – Only accurate when averaged over many calls • ITU P.862 – PESQ – Full reference
  • 12. E model Packet loss rate Ie mapping Calculate R-CQ MOS-CQ Signal level Calculate Noise level Ro, Is Remote Echo Return Loss Calculate Id Symmetric One Way Delay
  • 13. Problems with the E Model • Additivity problem – Impairments aren’t additive • Inaccurate for time varying loss • Inaccurate for consecutive loss • Conversational quality only
  • 14. VQmon computational model Perceptual model ETSI TS 101 329-5 Burst loss rate Ie mapping Calculate R-LQ Gap loss rate MOS-LQ Consecutive Non-linear Loss Model Impairment Combination Signal level model Calculate Noise level Ro, Is Remote Echo Calculate Return Loss Calculate R-CQ Id MOS-CQ Symmetric One Way Delay
  • 15. Impact of TS 101 329-5 model 4 al Estimated Subjective Test Score VQmon - models impact I de 3.5 of transient IP problems 3 2.5 2 “E Model” - based on average packet loss rate 1.5 1.5 2 2.5 3 3.5 4 Actual Subjective Test Score Independent test by France Telecom and University of Bochum
  • 16. VQmon Packet Markov Model Stream Gather detailed packet loss info in real time Loss/ Discard events Discarded Jitter buffer MOS-LQ MOS-CQ R-LQ CODEC CODEC, Metrics R-CQ Signal/ calculation Burst statistics Noise/ Echo level Diagnostic data
  • 17. VoIP Performance Reporting Architecture End of call SIP QoS report SIP QoS VQmon RTCP XR VQmon Agent Agent RTCP XR RTCP XR exchanged every 5-15 seconds during call
  • 18. RTCP XR Reports Loss Rate Discard Rate Burst Density Gap Density Burst Duration (mS) Gap Duration (mS) Round Trip Delay (mS) End System Delay (mS) Signal level RERL Noise Level Gmin R Factor Ext R MOS-LQ MOS-CQ Rx Config - Jitter Buffer Nominal Jitter Buffer Max Jitter Buffer Abs Max
  • 19. Information exchange using RTCP XR Remote Echo Return Loss Remote End System Delay Remote Signal Level VQmon RTCP XR VQmon Agent Agent RTCP XR
  • 20. RTCP XR as a reporting protocol Probe or Analyzer -can incorporate signal, noise, echo, delay from XR reports - Can report for both midpoints and endpoints RTCP XR VQmon VQmon Agent Agent RTCP XR
  • 21. SIP as a reporting protocol “Collector” (e.g. SQmediator) SIP RTCP Summary Reports SIP QoS RTCP XR VQmon/EP VQmon/EP RTCP XR
  • 22. SIP RTCP Summary PUBLISH SIP/2.0 ……… Content-Type: application/rtcpxr VQSessionReport LocalMetrics: TimeStamps=START:10012004.18.23.43 STOP:10012004.18.26.02 SessionDesc=PT:0 PD:G.711 SR:8000 FD:20 FPP:2 PLC:3 SSUP:on ……… Signal=SL:2 NL:10 RERL:14 QualityEst=RLQ:90 RCQ:85 EXTR:90 MOSLQ:3.4 MOSCQ:3.3 QoEEstAlg:VQMonv2.1
  • 23. Passive Monitoring – Pro’s and Con’s • Pro’s – Understand problems affecting actual calls – Capture information that can be post-analyzed to address customer problem reports – Generates virtually no overhead traffic • Con’s – Can’t detect a problem until it impacts a live call – Limited ability to run additional diagnostic tests to isolate and diagnose problem – Can’t deal with failed connections
  • 24. Downloadable Active Test Agents • Test agents (VoIP, IPTV…) downloadable from web server • Agents run tests against a network based server • Agents initiate tests – minimizes firewall problems • Results retained by server • Agents can be static or transient
  • 25. Downloadable Agent - Architecture Agent Agent Delivery Builder Server Agent Request/ Agent Agent Download Agent Config Agent Config Config Config User/Test ID Server Agent Active Test(s) Results Database
  • 26. Example VoIP test process • Call connect – Try SIP on port 5060 – Try SIP on alternate port – Try encrypted SIP on alternate port – Agent reports what it had to do in order to connect • Media path quality – Establish bidirectional RTP stream using voice payloads, measure quality – Establish bidirectional “non-RTP” stream, measure quality
  • 27. What can active tests discover? • Call connect tests – Service provider may be blocking SIP – Residential gateways can (mistakenly) block SIP – Problem accessing server – …….. • Media path quality – Network congestion – Layer 1-2 problems, duplex mismatch, bit error problems…… – Net (non) neutrality
  • 28. Use Active or Passive tests? • Need both • Active tests – Predeployment testing – SLA monitoring – Troubleshooting • Passive monitoring – Service assurance – SLA monitoring – Customer service
  • 29. Looking forward to IPTV • IP Video is much more sensitive to transmission impairments than VoIP • Much more complex video structure makes quality estimation more difficult • IPTV is usually encrypted, which makes quality estimation even more difficult • Making IPTV work over existing DSL infrastructure • Service providers need to prove that IP based services work
  • 30. Summary • Problems affecting Residential VoIP and Teleworkers • VoIP Performance and Quality Challenges • VoIP Performance Measurement and Problem Diagnosis • Passive Monitoring using Embedded Agents • Active Testing using Downloadable Agents • Looking forward to IPTV