The document describes research using genetic programming to develop models for real-time, non-intrusive evaluation of VoIP quality. Researchers collected data on VoIP traffic parameters and quality ratings to train and test models. The best models estimate quality as a function of bitrate, loss rate, frame duration, and other variables. The models perform comparably to PESQ, the standard quality estimation method, and better than prior approaches, allowing real-time quality monitoring without intrusive measurement.

1. REAL-TIME, NON-INTRUSIVE EVALUATION OF
VOIP
USING GENETIC PROGRAMMING
A. Raja1 A. Azad2 C. Flanagan1 C. Ryan2
1Wireless Access Research Centre
Department of Electronic and Computer Engineering
2Bio-Computing and Developmental Systems
Department of Computer Science and Information Sysmtems
University of Limerick, Limerick, Ireland

2. OUTLINE
1 MOTIVATION
2 TEST RESULTS
3 SIGNIFICANCE

3. OUTLINE
1 MOTIVATION
2 TEST RESULTS
3 SIGNIFICANCE

4. OUTLINE
1 MOTIVATION
2 TEST RESULTS
3 SIGNIFICANCE

5. RESEARCH GOAL
Derivation of a VoIP listening Quality estimation model as a
function of transport layer metrics.
Genetic Programming based Symbolic Regression is used
Using the ITU-T Recommendation P.862, PESQ , as the
reference system
PESQ is the de-facto standard for speech quality
estimation.

6. ON DATA COLLECTION
The VoIP traffic parameters include loss rate, burstiness
measure, frame duration, packetization interval and
bit-rate.
A total of 3360 distorted speech files were created for each
combination of network traffic parameters.
1177 35% were used for training
503 15% were used for testing
1680 50% were used for speaker independent validation

7. VOIP QUALITY MONITORING MODELS
MOS − LQOGP = −2.46 × log(cos(log(br)) + mlrVAD
×(br + fd/10)) + 3.17 (1)
MOS − LQOGP = −2.99 × cos(0.91 × sin(mlrVAD)
+mlrVAD + 8) + 4.20 (2)
Equation (1) Equation(2)
Data MSEs σ MSEs σ
Training 0.0370 0.9634 0.0520 0.9481
Testing 0.0387 0.9646 0.0541 0.9501
Validation 0.0382 0.9688 0.0541 0.9531

8. SCATTER PLOTS

9. SCATTER PLOTS
ON PERFORMANCE OF ITU-T P.563

10. SIGNIFICANCE OF MODELS
1 The model is a good approximation to PESQ.
2 Suitable for real-time and non-intrusive estimation of
speech quality whereas PESQ is NOT.
3 Simple models; depend on 3 and 1 variable respectively.
4 Performs significantly better than ITU-T P.563

11. THE HUMAN COMPETITIVE ELEMENT
1 The results are better than past approaches that include:
Various flavors of Artificial Neural Networks based models.
Non-linear least squares estimation.
Multi-state Markov Chains.
Look-up tables
2 All the above approaches (excluding ANNs) used a
restricted set of network traffic parameters.
3 By far it is the first ever application of GP on the problem of
speech quality estimation, and has produced better results.

12. THE HUMAN COMPETITIVE ELEMENT
1 The results are counter-intuitive.
2 The results solve a formidable problem and outperform
previously existing solutions.