Current network services such as Voice over IP or IP Television pose new challenges to network providers. Network operators need to know if their services are being properly provided. However, current quality of service parameters commonly used in data networks (e.g. throughput, packet delays, packet losses, etc.) do not show a clear view of what the users are experiencing. Thus, it is necessary to translate such measured quality parameters into a quality of experience value. Several models are being developed to cope with this problem. For instance, some approaches have used the packet loss rate to evaluate the experienced quality of an IP television channel. Unfortunately, packet loss just explains a fraction of the quality behavior. Then, we go one step further, taking into account the different MPEG frame types that are transmitted. In this paper, we have defined a model to predict the experienced quality that is a function of the loss of the different types of MPEG frames, providing a mean opinion score of the delivered service. The final results show that our model is able to better predict the quality of experience of such video services than just using the packet loss rate.

1. Evaluating Quality of Experience in IPTV
Services Using MPEG Frame Loss Rate
Diego Hernando Loeda1, Jorge E. López de Vergara1,2,
David Madrigal1, Felipe Mata 1
<jorge@naudit.es>
1Universidad Autónoma de Madrid, Spain
2Naudit High Performance Computing and Networking, Spain
First International Workshop on Quality Monitoring,
SaCoNeT, 17th June 2013, Paris, France

2. Introduction
 ISPs and network operators providing
multimedia services
 QoS vs QoE

 Evaluation types (FR, RR, NR)
 MPEG frame types
 Resource consumption
Evaluating Quality of Experience in IPTV Services
Using MPEG Frame Loss Rate 2

3. Related Work
Original
Video
Distorted
Video
FR
RR
Information
extraction
NR
Evaluating Quality of Experience in IPTV Services
Using MPEG Frame Loss Rate 3

4. Method Description
Broadcast Replay VQM
PLR
MPEG Loss
Gilbert-Elliot Model
for packet losses
G B
r
p
1-r1-p
Evaluating Quality of Experience in IPTV Services
Using MPEG Frame Loss Rate 4

5. QoE Prediction Model (PLR)
 Based on Packet Loss Ratio
 Average of Point Clouds
MOS(PLR)=a+b·PLR
 9.4% error in MOS scale
 R² = 0.6974
Variable Coeff Stdr. Error t value p-value
a 4.9442 0.0165 299.4776 0
b -0.1642 0.0034 49.6947 0
SEE 0.3757
R² 0.6974
5

6. QoE Prediction Model (IBP)
 Types of frames in MPEG
» Intra-coded Frame
» Predictive Frame
» Bi-Predictive Frame
Evaluating Quality of Experience in IPTV Services
Using MPEG Frame Loss Rate 6

7. QoE Prediction Model (IBP)
 Initial IBP Loss Model
 Based on loss ratio of
each type of frame
MOS(I,B,P)=a+b·I+c·B+d·P
 9.5% error in MOS scale
 R² = 0.6969
Variable Coeff Stdr. Error t value p-value
a 4.9437 0.0165 298.7312 0
b -17.5488 1.5542 11.2913 0
c -0.8657 0.3376 2.4212 0.0078
d 1.6701 0.9601 1.6738 0.0472
SEE 0.3764
R² 0.6969 7

8. QoE Prediction Model (IBP)
 Importance of frame
headers in packet losses
Evaluating Quality of Experience in IPTV Services
Using MPEG Frame Loss Rate 8
Without header analysis 
 With header analysis

9. QoE Prediction Model (IBP)
 Final IBP Loss Model
 Based on loss ratio of each
type of frame…
 …But also taking into account
that a header loss implies
losing the whole frame
MOS(I,B,P)=a+b·I+c·B+d·P
 8.4% error in MOS scale
 R² = 0.7575
Variable Coeff Stdr. Error t value p-value
a 4.9030 0.0142 344.9143 0
b -1.0823 0.2935 3.6876 0.0001
c -3.2792 0.4759 6.8911 0
d -3.2323 0.2784 11.6111 0
SEE 0.3367
R² 0.7575 9

10. Efficiency
Low Quality Videos (CIF, QCIF)
 ~400 channels 10s captures can be
analyzed per second using the IBP
model
 Linear growth
High Quality Videos (1920x1080)
 ~60 channels 10s captures can be
analyzed per second using the IBP
model
 Linear growth
Evaluating Quality of Experience in IPTV Services
Using MPEG Frame Loss Rate 10

11. Conclusions & Future Work
 Conclusions
» We have obtained a new NR QoE estimation method
– Based on MPEG traffic characteristics
– No video signal is decoded
» Reasonable computational cost
– Deployable for current network services
 Future Work
» Investigate about the amount of movement
» Create a video database with different qualities and GOP
structures
» Perform a subjective evaluation for a better validation
Evaluating Quality of Experience in IPTV Services
Using MPEG Frame Loss Rate 11

12. Evaluating Quality of Experience in IPTV
Services Using MPEG Frame Loss Rate
Diego Hernando Loeda1, Jorge E. López de Vergara1,2,
David Madrigal1, Felipe Mata 1
<jorge@naudit.es>
1Universidad Autónoma de Madrid, Spain
2Naudit High Performance Computing and Networking, Spain
First International Workshop on Quality Monitoring,
SaCoNeT, 17th June 2013, Paris, France