A presentation about Modeling the Effect of Packet Loss on Speech Quality: Genetic Programming Based Symbolic Regression.

1. 1
Modeling the Effect of Packet Loss on Speech Quality:
Genetic Programming Based Symbolic Regression
Adil Raja
adil.raja@ul.ie
1

2. 2
Contents
• Packet Loss Modeling approaches: Related Research
• Current Approach.
• A Brief Intoduction to Genetic Programming (GP).
• Simulation Environment and GP Parameters.
• Analysis and Results.
• Conclusion and Future Aspirations.
2

3. 3
Packet Loss Modeling Approaches
Packet Based Approaches.
• Based on regression of packet loss parameters to MOS.
• Parameters include mean Loss rate, conditional loss probability etc.
• Some approaches include:
Markov Models [1].
Regression Using Artificial Neural Networks. [2] [3] [4]
Speech Based Approaches
• Intrusive: ITU-T Recommendation P.862 (PESQ).
• Non-intrusive: ITU-T Recommendation P.563 (PSEAM).
• Non-intrusive PESQ [5].
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 3

4. 4
Our Previous Work
ANN based Regression of network traffic metrics on speech quality.
• Useful Network loss Metrics.
• Mean Loss Rate.
• Means and Variances of Burst and Gap Length Distributions.
• Codec Type and Packetization Interval.
• Inter Loss Distance/Gap Length.
Packet loss was modeled using a Gilbert Model.
Results: -
• rtraining=0.9835;
• rvalidation=0.9821;
• rtesting=0.9763
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 4

5. 5
Packet Loss Simulation
The Gilbert-Elliot Loss Model.
p =


n−1
i=1
mi/m0

 /mi
q = 1 −


n−1
i=2
mi × (n − 1)

 /


n−1
i=1
mi × i


π1 =
p
p + q
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 5

6. 6
Packet Loss Simulation ...
Parameters of Geometrically Distributed Burst and Gap Lengths
• Mean Burst Length = 1/q
• Variance of Burst Length Distribution = (1-q)/q2
• Mean Gap Length = 1/p
• Variance of Gap Length Distribution = (1-p)/p2
The Gilbert Model:
• Packet loss can be simulated for certain values of p and q.
• During network operation bursts have to be captured for determining clp and ulp.
• The Gilbert model also models the packet loss due to jitter buffer discard/overflow.
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 6

7. 7
Current Approach and Experimental Details
• Genetic Programming has been used for mapping the effect of VoIP traffic parameters on speech
quality.
• Codecs: G.729 and G.723.1 and AMR-NB.
• Packet/frame loss simulation is done using Gilbert Model.
• Mean Loss rate (ulp) was set to 10, 20, 30 and 40 %. clp was set to 10, 60, 80 and 90 %.
• Input Variables.
Mean loss rate.
Means and variances of burst and gap length distributions (VAD).
Codec type and packetization interval.
VAD: Different packets have different importance [6].
• From a total of 1408 speech files:
35% were used for training.
15% were used for validation.
50% were used for Speaker independent testing.
• Speech activity - 70-80%
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 7

8. 8
A Brief Introduction to Genetic Programming (GP)
• GP is a Machine Learning Technique inspired by biological evolution. A branch of Evolutionary
Algorithms.
• Aimed at evolving program expressions/computer code.
• Each individual encodes a symbolic expression.
• Solution Representation.
A tree structure is the most popular representation.
Other representations include graphs and linear structures such as arrays.
• Primary application area is modelling.
Commercial Application - predicting stock index.
Scientific Application - modelling physical processes.
Engineering Application - reverse engineering, designing circuitry, regression, classifica-
tion.
Data Mining.
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 8

9. 9
A simplified GP Breeding Cycle
GP uses four steps to solve problems:
• Generate an initial population of random compositions of the functions and terminals of the
problem (computer programs).
Functions: plus, minus, times, sin, cos, mylog, mypower, divide, sqrt, mylog2, mylog10.
Terminals: Can be variables (network traffic parameters) and constants.
• Execute each program in the population and assign it a fitness value according to how well it
solves the problem.
Minimization of χ2
error.
Minimization of MSE ().
Maximization of Pearson’s product moment correlation coefficient.
• Copy the best existing programs (Selection).
Roullete Wheel Selection - Fitness Proportinate Selection.
Tournament Selection.
Lexicographic Parsimony Pressure ().
...
• Create new computer programs by mutation and crossover.
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 9

10. 10
A simplified GP Breeding Cycle: A Symbolic Representation
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 10

11. 11
Mutation
Two types of mutation are possible1
:
• A terminal replaces a terminal or a function replaces a function.
• A subtree can replace an entire subtree.
1
http://www.geneticprogramming.com/Tutorial/#anchor181526
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 11

12. 12
Crossover
Two solutions are recombined to form two new solutions or offspring.
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 12

13. 13
The GP Environment
GPLAB - A Matlab tool-box by Sara Silva (sara@dei.uc.pt).
Other GP Parameters
• Survival: Replacement (), Elitism.
• Adaptive genetic operator probabilities.
• Initial Population Size: 100.
• Generational Gap: 1.
Linear Scaling [7]
MSE (y, t) = 1/n
n
i
(ti − yi)
2
MSEs (y, t) = MSE (a + by, t) = 1/n
n
i
(ti − (a + byi))
2
a = ¯t − b ¯f(x)
b =
cov(t, y)
var(y)
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 13

14. 14
Advantages of Linear Scaling
• Bloat Control.
• Faster Training.
• Solutions better suited for real-time evaluations.
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 14

15. 15
Analysis and Results
A total of 50 runs were performed. Each run was spanned over 50 gener-
ations.
Fitness Curves:
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 15

16. 16
Analysis and Results ...
Diversity and Tree Size plots
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 16

17. 17
Analysis and Results ...
Parameter Representation in GP Environment
GP Representation Parameter
X1 codec type
X2 Packetization Interval (PI)
X3 Mean Loss Rate (mlr)
X4 Mean Burst Length (mbl)
X5 Mean Gap Length (mgl)
X6 Variance of Burst Length (vbl)
X7 Variance of gap length (vgl)
• Talkspurt based values of all the parameters are used.
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 17

18. 18
Analysis and Results ...
• Fitness= 0.0523; Test Fitness=0.0496 rtraining=0.9074;rvalidation=0.9183;
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 18

19. 19
Analysis and Results ...
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 19

20. 20
Analysis and Results
• A less bloated solution.
• Fitness= 0.0670; Test Fitness=0.0555 rtraining=0.8796rvalidation= 0.9079;
GP − MOS − LQO = −2.3843 sin
√
X3 + sin (X3) + 3.6112
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 20

21. 21
Analysis and Results ...
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 21

22. 22
Analysis and Results ...
• Fitness= 0.0678; Validation Fitness=0.0556; Test Fitness=0.0650; rtraining=0.8780rvalidation=
0.9074; rtesting=0.8881.
GP − MOS − LQO = −3.3432
√
X3 + 3.6881
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 22

23. 23
Analysis and Results ...
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 23

24. 24
Analysis and Results ...
• Fitness= 0.0678; Validation Fitness=0.0556; Test Fitness=0.0650; rtraining=0.8780rvalidation=
0.9074; rtesting=0.8881.
GP − MOS − LQO = −10.0296 X3/9 + 3.6881
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 24

25. 25
Analysis and Results ...
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 25

26. 26
Conclusions and Future Aspirations
• Results obtained by GP have advantages over other machine learning algorithms (such as
ANNs).
Simplified results: A mathematical expression.
GP searches for global minimum of the error function and is less prone to getting stuck
in local minima (due to mutation property).
The prelimanary results are not comparable to ANN based approaches but there is
room for improvement.
• Improvements: Some Speculations; The Known Knowns.
Population size should be increased to 500 for the sake of having more diverse search
space.
Selection: Lexicographic parsimony pressure vs Tournament selection.
Survival: Replacement vs some elitism criterion.
• The work can be split to two parts:
The Telecommunications intensive aspects.
The GP intensive aspects.
• Developing an in-depth understanding of GP shall be a part of future endeavors.
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 26

27. 27
Thanks!
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 27

28. 28
References
[1] A. D. Clark. Modeling the effects of burst packet loss and recency on subjective voice quality.
In 2nd IP-Telephony Workshop, Columbia University, New York, April 2001.
[2] S. Mohamed, F. Cervantes-Perez, and H. Afifi. Integrating networks measurements and speech
quality subjective scores for control purposes. In Annual Joint Conference of the IEEE Computer
and Communications Societies (INFOCOM), pages 641Ű649, 2001.
[3] S. Mohamed, G. Rubino, and M. Varela. A method for quantitative evaluation of audio quality
over packet networks and its comparison with existing techniques. In Measurement of Speech and
Audio Quality in Networks (MESAQIN), 2004.
[4] L. F. Sun and E. C. Ifeachor. Perceived speech quality prediction for voice over ip-based networks.
In IEEE International Conference on Communications (ICC), volume 4, pages 2573 -Ű 2577, 2002.
[5] A.E. Conway, Output-based method of applying PESQ to measure the perceptual quality of
framed speech signals, IEEE Communications Society, 2004.
[6] L. Sun, G.Wade, B. M. Lines, and E. C. Ifeachor. Impact of packet loss location on perceived
speech quality. In 2nd IP-Telephony Workshop, Columbia University, New York, April 2001.
[7] M. Kaijzer. Scaled Symbolic Regression. 2003.
Adil Raja, Wireless Access research Centre University of Limerick, June 2006 28