Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

On Losses, Pauses and Jumps and the Wideband E-Model

383 views

Published on

A Presentation about my work in France Telecom R&D.

Published in: Engineering, Technology, Business
  • Be the first to comment

  • Be the first to like this

On Losses, Pauses and Jumps and the Wideband E-Model

  1. 1. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim ON LOSSES, PAUSES AND JUMPS AND THE WIDEBAND E-MODEL Adil Raja Anna Jagodzinska Vincent Barriac France Telecom R&D, TECH/OPERA/MOV
  2. 2. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim OUTLINE 1 BACKGROUND AND MOTIVATION 2 VOIP SIMULATION 3 METHODOLOGY 4 PREPARATION OF THE TEST MATERIAL 5 INTRODUCTION TO GP 6 EXPERIMENTAL SETUP 7 RESULTS AND ANALYSIS Comparison With Multiple Linear Regression Comparison With E-Model Performance Evaluation Against Data From Auditory Tests 8 CONCLUSIONS
  3. 3. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim OUTLINE 1 BACKGROUND AND MOTIVATION 2 VOIP SIMULATION 3 METHODOLOGY 4 PREPARATION OF THE TEST MATERIAL 5 INTRODUCTION TO GP 6 EXPERIMENTAL SETUP 7 RESULTS AND ANALYSIS Comparison With Multiple Linear Regression Comparison With E-Model Performance Evaluation Against Data From Auditory Tests 8 CONCLUSIONS
  4. 4. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim OUTLINE 1 BACKGROUND AND MOTIVATION 2 VOIP SIMULATION 3 METHODOLOGY 4 PREPARATION OF THE TEST MATERIAL 5 INTRODUCTION TO GP 6 EXPERIMENTAL SETUP 7 RESULTS AND ANALYSIS Comparison With Multiple Linear Regression Comparison With E-Model Performance Evaluation Against Data From Auditory Tests 8 CONCLUSIONS
  5. 5. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim OUTLINE 1 BACKGROUND AND MOTIVATION 2 VOIP SIMULATION 3 METHODOLOGY 4 PREPARATION OF THE TEST MATERIAL 5 INTRODUCTION TO GP 6 EXPERIMENTAL SETUP 7 RESULTS AND ANALYSIS Comparison With Multiple Linear Regression Comparison With E-Model Performance Evaluation Against Data From Auditory Tests 8 CONCLUSIONS
  6. 6. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim OUTLINE 1 BACKGROUND AND MOTIVATION 2 VOIP SIMULATION 3 METHODOLOGY 4 PREPARATION OF THE TEST MATERIAL 5 INTRODUCTION TO GP 6 EXPERIMENTAL SETUP 7 RESULTS AND ANALYSIS Comparison With Multiple Linear Regression Comparison With E-Model Performance Evaluation Against Data From Auditory Tests 8 CONCLUSIONS
  7. 7. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim OUTLINE 1 BACKGROUND AND MOTIVATION 2 VOIP SIMULATION 3 METHODOLOGY 4 PREPARATION OF THE TEST MATERIAL 5 INTRODUCTION TO GP 6 EXPERIMENTAL SETUP 7 RESULTS AND ANALYSIS Comparison With Multiple Linear Regression Comparison With E-Model Performance Evaluation Against Data From Auditory Tests 8 CONCLUSIONS
  8. 8. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim OUTLINE 1 BACKGROUND AND MOTIVATION 2 VOIP SIMULATION 3 METHODOLOGY 4 PREPARATION OF THE TEST MATERIAL 5 INTRODUCTION TO GP 6 EXPERIMENTAL SETUP 7 RESULTS AND ANALYSIS Comparison With Multiple Linear Regression Comparison With E-Model Performance Evaluation Against Data From Auditory Tests 8 CONCLUSIONS
  9. 9. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim OUTLINE 1 BACKGROUND AND MOTIVATION 2 VOIP SIMULATION 3 METHODOLOGY 4 PREPARATION OF THE TEST MATERIAL 5 INTRODUCTION TO GP 6 EXPERIMENTAL SETUP 7 RESULTS AND ANALYSIS Comparison With Multiple Linear Regression Comparison With E-Model Performance Evaluation Against Data From Auditory Tests 8 CONCLUSIONS
  10. 10. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim BACKGROUND VoIP listening quality is not only distorted by packet loss and codec related impairments only. Temporal discontinuities such as pauses and jumps (packet discards) also play a role. (S. Voran, 03) Packet loss happens due to network congestion. Jumps and Pauses happen due to the jitter/jitter buffer interaction.
  11. 11. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim BACKGROUND LOSS, PAUSE AND JUMPS 0 100 200 300 400 500 −6 −4 −2 0 2 4 6 8 10 x 10 −3 1 32 FIGURE: A sequence of 3 frames
  12. 12. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim BACKGROUND LOSS, PAUSE AND JUMPS 0 100 200 300 400 500 −6 −4 −2 0 2 4 6 8 10 x 10 −3 1 E 3 FIGURE: Loss
  13. 13. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim BACKGROUND LOSS, PAUSE AND JUMPS 0 100 200 300 400 500 −6 −4 −2 0 2 4 6 8 10 x 10 −3 1 E 2 FIGURE: Pause
  14. 14. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim BACKGROUND LOSS, PAUSE AND JUMPS 0 100 200 300 400 500 −6 −4 −2 0 2 4 6 8 10 x 10 −3 1 3 4 FIGURE: Jump
  15. 15. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim VOIP SIMULATION Sender ReceiverJitter Loss FIGURE: VoIP Simulation
  16. 16. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim WEIBULL DISTRIBUTION VoIP jitter is a self-similar phenomenon that can be modeled by a heavy tailed distribution. Notable distributions are: Weibull (), Pareto, Exponential. Weibull distribution is characterized by: A shape parameter (A), a scale (B) parameter, and a location parameter.
  17. 17. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim WEIBULL DISTRIBUTION 0 50 100 150 200 250 300 350 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 x 10 4 FIGURE: A=1
  18. 18. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim WEIBULL DISTRIBUTION 0 10 20 30 40 50 60 70 80 90 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 FIGURE: A=2
  19. 19. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim WEIBULL DISTRIBUTION 0 10 20 30 40 50 60 70 80 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 FIGURE: A=2.5
  20. 20. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim PACKET LOSS 1 (NO LOSS) 0 (LOSS) p 1-q q 1-p FIGURE:
  21. 21. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim LOSS, PAUSE AND JUMP STATE MODEL A loss, pause jump state model can be learned from a network trace analysis or the network emulation. with state transition probabilities.
  22. 22. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim LOSS, PAUSE AND JUMP STATE MODEL
  23. 23. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim LOSS, PAUSE AND JUMP STATE MODEL Conversely the state model can be used to generate realistic loss, pause and jump patterns given realistic values for mean loss, pause and jump rates. For instance: n2l = l2n × (mlr) 1−mlr . l2n = 1 − clp.
  24. 24. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim VOIP SIMULATION SYSTEM FIGURE: Simulation system for derivation of Ie,WB,eff
  25. 25. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim NETWORK TRAFFIC PARAMETERS TABLE: Various Network Traffic Parameters No. Variable Abbreviation 1 mean loss rate mlr 2 mean burst length – loss mbl_loss 3 mean pause rate mpr 4 mean burst length – pause mbl_pause 5 mean jump rate mjr 6 mean burst length – jump mbl_jump 7 mean impairment rate mir=mlr+mpr+mjr 8 mean burst length impairments mbl_impairment 9 equipment impairment factor Ie,WB 10 gradient of the Ie,WB,eff grad
  26. 26. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim Ie,WB,eff VS mir 0 5 10 15 20 25 0 10 20 30 40 50 60 70 80 MIR I e,WB,eff G.722 G.729 G.711 FIGURE: Ie,WB,eff vs mir for ITU-T G.711, G.729 and G.722
  27. 27. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim VARIOUS TEMPORAL DISCONTINUITY RATES AND THE RESPECTIVE BURST LENGTHS TABLE: Various Temporal Discontinuity Rates and The Respective Burst Lengths Temporal Discontinuity Rate Burst Length 0 0 0.005 1, 2 0.01 1, 2, 4 0.015 1, 2, 3 0.02 1, 2, 4 0.025 1, 2, 5 0.03 1, 2, 3 0.035 1, 2, 7 0.04 1, 4, 8
  28. 28. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim INTRODUCTION TO GENETIC PROGRAMMING (GP) Genetic Programming is a coarse emulation of Darwinian Evolution. The search space is composed of all the possible computer programs. GP Life Cycle: 1 Create an initial population of computer programs. 2 Evaluation. 3 Selection. 4 Reproduction. 5 Evaluation. 6 Replacement. 7 Continue from 3.
  29. 29. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim A TYPICAL GP BREEDING CYCLE FIGURE:
  30. 30. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim TABLE: Common GP Parameters among all experiments Parameter Value Initial Population Size 300 Initial Tree Depth 6 Selection LPP Tournament Size 2 Genetic Operators Crossover and Subtree Mutation Operators Probability Type Adaptive Initial Operator probabilities 0.5 each Survival Half Elitism Generation Gap 1 Function Set plus, minus, multiply, divide,sin, cos, log2, log10, loge, sqrt, power, if Terminal Set Random real-valued numbers between 0.0 and 1.0. Integers (2–10) and Network traffic parameters from Table 1.
  31. 31. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim RESULTS AND ANALYSIS TABLE: Statistical analysis of the GP experiments and derived models (a) RMSE Statistics for Best Individuals of 50 Runs for Experiments 1, 2 3 Experiment 1 Experiment 2 Experiment 3 Stats RMSEtr RMSEte Size RMSEtr RMSEte Size RMSEtr RMSEte Size Mean 5.5482 98.9506 26.2800 5.55 18.94 29.34 5.34 13.90 28.18 Std. Dev. 0.3514 152.59 11.4661 0.39 70.27 12.3612 0.4612 47.19 9.73 Max. 5.97 500 68 6.0084 494.52 73 5.89 333.60 59 Min. 4.5409 4.82 11 4.41 4.37 6 4.40 4.38 14 (b) Results of Mann-Whitney-Wilcoxon Significance Test Experiment 1 Experiment 2 Experiment 3 RMSEtr RMSEte Size RMSEtr RMSEte Size RMSEtr RMSEte Size Experiment 1 0 0 0 0 1 0 1 1 0 Experiment 2 0 1 0 0 0 0 1 0 1 Experiment 3 1 1 0 1 0 1 0 0 0
  32. 32. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim RESULTS AND ANALYSIS TABLE: Performance Statistics of the Proposed Models Training Test Model RMSEsMOS RMSEs Ie,WB,eff σ Ie,WB,eff RMSEs MOS RMSEs Ie,WB,eff σ Ie,WB,eff Equation (1) 0.1763 4.8185 0.8840 0.1759 4.8182 0.8805 Equation (2) 0.1602 4.4108 0.9038 0.1596 4.3708 0.9028 Equation (3) 0.1619 4.4021 0.9042 0.1611 4.3808 0.9023 Equation (4) 0.1692 4.6026 0.8948 0.1679 4.5460 0.8944 Equation (5) 0.1764 4.8644 0.8816 0.1948 5.4231 0.8781
  33. 33. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim THE PROPOSED MODELS Ie,WB,eff = (1) mir × cos(Ie,WB) + mbl_imp Ie,WB − mir 1/4 − mir ×(−163.87) − 9.35 Ie,WB,eff = (2)    log10(grad) mir + 9 +  (sin(mir) + mir) log10(grad) 4 − √ mpr mbl_loss+9   log10(grad)− log10(grad) mbl_jump+9    ×(−0.0933) + 87.1174
  34. 34. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim THE PROPOSED MODELS Ie,WB,eff = (3)    log10 0.54 grad + 3 × mir + log10 0.74 grad +2×mir 3 7 × log10 0.54 grad + 2 × mir + 6.56 − mbl_imp + mir    ×(270.37) + 102.40 Ie,WB,eff = (4) (sin(grad × mir)) √ 40×mbl_imp Ie,WB × 107.43 − 5.94
  35. 35. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim SCATTER PLOTS 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 Ie,WB,eff −− GP I e,WB,eff −−WB−PESQ (a) 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 Ie,WB,eff −− GP I e,WB,eff −−WB−PESQ (b) FIGURE: Ie,WB,eff predicted by equation (3) vs target Ie,WB,eff for: (a) training data (b) test data
  36. 36. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim PARAMETER SIGNIFICANCE ANALYSIS Ie,wb grad mlr mbl_loss mjr mbl_jump mpr mbl_pause mir mbl_impairment 0 10 20 30 40 50 60 70 80 90 100 Experiment 1 Experiment 2 FIGURE: Percentage of the best individuals employing various input parameters in acceptable runs of each of the two experiments.
  37. 37. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim COMPARISON WITH MULTIPLE LINEAR REGRESSION Ie,WB,eff = (5) 0.35 × Ie,WB − 0.006 × grad + 383.62 × mir − 1.18 × mbl_imp + 34.65 Has inferior performance as opposed to proposed models. Results are reported in Table 5
  38. 38. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim COMPARISON WITH E-MODEL Ie,WB,eff = Ie,WB + (129 − Ie,WB) × Ppl Ppl BurstR + Bpl (6) TABLE: Comparison between the Prediction Accuracies of the E-Model and the Proposed Model E-Model Equation (3) Codec RMSE RMSE RMSE RMSE (kbps) Bpl train test train test G.711 (64) 22.39 6.7971 6.7003 4.6748 4.5626 G.729 (8) 30.50 4.0824 3.8701 3.0513 3.1362 G.722 (64) 19.8053 8.1087 8.1510 5.6865 5.6093 Average – 6.3294 6.2405 4.4709 4.4360 % PG – – – 29.36 28.92
  39. 39. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim PERFORMANCE EVALUATION AGAINST DATA FROM AUDITORY TESTS TABLE: Target Network Impairment Conditions for the Auditory Tests Condition mlr mbl_loss mjr mbl_jump mpr mbl_pause mir mbl_impairment 1 0 0 0 0 0 0 0 0 2 0.03 1.0 0 0 0 0 0.03 1.0 3 0.03 4.0 0 0 0 0 0.03 4.0 4 0 0 0.03 1.0 0 0 0.03 1.0 5 0 0 0.03 4.0 0 0 0.03 4.0 6 0 0 0 0 0.03 1.0 0.03 1.0 7 0 0 0 0 0.03 4.0 0.03 4.0 8 0.06 1.0 0 0 0 0 0.06 1.0 9 0.06 4.0 0 0 0 0 0.06 4.0 10 0 0 0.06 1.0 0 0 0.06 1.0 11 0 0 0.06 4.0 0 0 0.06 4.0 12 0 0 0 0 0.06 1.0 0.06 1.0 13 0 0 0 0 0.06 4.0 0.06 4.0 14 0.09 1.0 0 0 0 0 0.09 1.0 15 0.09 4.0 0 0 0 0 0.09 4.0 16 0 0 0.09 1.0 0 0 0.09 1.0 17 0 0 0.09 4.0 0 0 0.09 4.0 18 0 0 0 0 0.09 1.0 0.09 1.0 19 0 0 0 0 0.09 4.0 0.09 4.0 20 0.04 4.0 0.04 4.0 0.04 4.0 0.12 12.0
  40. 40. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim PERFORMANCE EVALUATION AGAINST DATA FROM AUDITORY TESTS TABLE: Comparison Between the Results of Auditory Tests and WB-PESQ RMSE MOS σ MOS 0.4475 0.8399 TABLE: Comparison between the Prediction Accuracies of the E-Model and the Proposed Model Against Data From Auditory Tests E-Model Equation (3) Codec (kbps) Bpl Ie,WB RMSE σ RMSE σ G.711 (64) 25.1 36 18.2549 0.7827 11.8532 0.8182 G.729 (8) 19.0 47 34.9341 0.8249 9.4212 0.9309 G.722 (64) 7.1 13 46.6618 0.8124 11.8840 0.8966 Overall – – 24.8994 0.3852 11.1129 0.8758
  41. 41. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim PERFORMANCE EVALUATION AGAINST DATA FROM AUDITORY TESTS 0 0.06 0.12 0.06 0.12 0.06 0.12 −50 0 50 100 150 200 MIR I e,WB,eff Auditory E−Model ProposedG.711 G.722 G.729 FIGURE: Ie,WB,eff vs mir derived from auditory tests, E-Model and equation (3) are plotted for G.711, G.722 and G.729.
  42. 42. Background and Motivation VoIP Simulation Methodology Preparation of The Test Material Introduction to GP Experim CONCLUSIONS We have proposed a new methodology that employs GP to derive novel equipment impairment factors. The poposed models outperform the existing E-Model formulation. We have taken into account additional sources of impairments; pauses and jumps. We have also proposed a 4-state loss, pause and jump Markov model to characterize VoIP traffic. The methodology is general and may be augmented with the results of auditory tests.

×