HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential
Security Level: Internal
www.huawei.com
Speech Quality and MOS
Alex
Agenda
• Introduction
• Measurement
• Evaluation
• Optimization
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 3
MOS- Subjective Evaluation Way
MOS test is normally performed on a group of listeners.
The listeners listen to the speech material in the same
channel environment and give a score. The scores are
then averaged to obtain the MOS score. The scores vary
largely from listener to listener.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 4
MOS- Objective Evaluation Way
 Objective way compares an original signal with a degraded signal and
then outputs a MOS score. The MOS score is like the result of a subjective
listening test.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 5
PESQ(P.862)
• PESQ value is from
-0.5 to 4.5
• PESQ algorithm is
powerful enough to
test end-to-end
speech quality.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 6
MOS-LQO(P.862.1)
• MOS-LQO (Listening
Quality Objective)
value is from 1.02 to
4.56
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 7
PESQ-LQ Mapping
• This mapping is not
adopted by ITU-T
Agenda
• Introduction
• Measurement
• Evaluation
• Optimization
HUAWEI TECHNOLOGIES Co., Ltd. HUAWEI Confidential Page 9
Measurement –Nemo (MS to MS)
Nemo Audio Module:
Sending Reference
Sample
Nemo Audio Module :
Receiving Impaired
Sample
MOS
Score
HUAWEI TECHNOLOGIES Co., Ltd. HUAWEI Confidential Page 10
QVM-MOS
Measurement
QVP-Data Analysis
Measurement –QVoice (MS to PSTN)
HUAWEI TECHNOLOGIES Co., Ltd. HUAWEI Confidential Page 11
MS to PSTN Way
BTS BSC MSC PSTN
MS
Reference Sample Impaired Sample
Impaired Sample Reference Sample
Uplink MOS
Downlink MOS
 UL MOS score and DL MOS score is available.
 The configuration of Testing System is more complicated
 Required less MS when performing Benchmark
HUAWEI TECHNOLOGIES Co., Ltd. HUAWEI Confidential Page 12
 UL MOS score and DL MOS score is not available 。
 MOS test by MS to MS way is more sensitive for Um interface
 Using Local switch or TFO can improve MOS (MS to MS way)。
MS1
BTS
MSC
BSC
BSC
BTS
MS2
Reference
Sample Uplink
Impaired
Sample Downlink
MS to MS Way
Agenda
• Introduction
• Measurement
• Evaluation
• Optimization
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 14
Common Report
MOS
=Average[MOS(all samples)]
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 15
QVoice Report
Basic Report:
Advanced
Report:
DL MOS = MOS_FR × FR% + MOS_HR × HR% + MOS_EFR ×
EFR% + MOS_AFR × AFR% + MOS_AHR × AHR%
UL MOS = MOS_FR × FR% + MOS_HR × HR% + MOS_EFR ×
EFR% + MOS_AFR × AFR% + MOS_AHR × AHR%
MOS = (DL_MOS + UL_MOS) / 2
Agenda
• Introduction
• Measurement
• Evaluation
• Optimization
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 17
Page 17
Factors of MOS
RF performance(RxLev, RxQual, C/I, HO,…)
Transmission (Speech route,Tran. quality…)
BSS Features (Codec,DTX, TFO, PC…)
Hardware(BTS/FTC version,Fault…)
Factors of
MOS
 Consider comprehensively when analyzing MOS
Others(FR/HR percentage…)
Test Method (Equipment, Testing mode,Speech Sample)
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 18
 Equipment
 Different equipment has different standard (PESQ is preferred)
 Do not compare MOS value from different type of equipment.
 Testing Mode
 MS to MS
 MS to PSTN
 Speech Sample
 Speech Sample should be the same for any compare.
 English Speech Sample is preferred
Page 18
Test Method
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 19
Um interface(1)
Page 19
 SQI (Downlink Um)
Note: SQI is not equal to MOS. SQI is just a factor that can affect MOS
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 20
Um Interface(2)
Page 20
 Improve SQI
 Uplink Um Quality
Can be evaluated by single user tracing or STS, normally Uplink Um interface should
be a bit better than downlink Um interface. If not, be aware of Uplink interference,
Uplink DTX, link balance.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 21
Features(1)
Feature Recommend Effect on MOS
Speech Version
Active AMR,
EFR
The descending speech quality of 5 types of Speech
Version in GSM is as following:
AMR FR >= EFR > AMR HR >= FR > HR。
Active AMR and EFR will improve MOS remarkably.
UL or DL DTX ?
DTX will introduce clipping which is bad for MOS.
However, DTX can decrease interference which is good
for Um interface.
UL and DL PC Open Power Control will decrease network interference.
Hopping Open
Caused by frequency diversity and interference
equalization, FH is helpful for enhancing speech quality.
TFO Open
TFO can cut one encoding and decoding off a
conversation , which is helpful for enhancing MOS in MS
to MS way.
Local Switch Open
Allowing speech signal to bypass MSC and BSC, which
also can cut one encoding and decoding time.
However, if TFO is activated, local switch can only save
transmission resources.
Page 21
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 22
Feature(2)-Codec
AMR FR >= EFR > AMR HR >= FR > HR
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 23
Transmission(1)
Page 23
 Speech Signal Route
Evaluate how many times for encoding and decoding. There will be 0.2 MOS loss for one
more encoding and decoding.
 MS to PSTN
Speech Signal Route:MS- BTS-BSC-MSC-GMSC-PSTN
Be aware of transmission between MSC and GMSC:
If using IP transmission, NSS part will introduce one time encoding and decoding.
If using TDM transmission, no MOS loss
 While performing MOS benchmark, if MS to PSTN test way is adopted, you
should consider the transmission differences between MSC and GMSC.
 For example:If networks all use AMR codec, the maximum MOS value in
network with IP backbone is 3.8 to 3.9, and maximum MOS value in network with
TDM can be 4.0-4.2.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 24
Transmission(2)
Page 24
 Speech Signal Route
 MS to MS
Speech route :MS-BTS-BSC-MSC-BSC-BTS-MSC
The speech will experience 2 times encoding and decoding.
But if TFO is adopted in the network, only one time encoding and decoding, which can
provide 0.2 MOS increase.
No TFO, 2 times
encoding and decoding
TFO, 1time encoding
and decoding
 Transmission Quality
Speech quality is also strongly related to transmission quality, especially for microwave
or satellite transmission type, which are more easily affected by weather (raining,
snowing…) and cause MOS degrading.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 25
Hardware
Page 25
For BSS part, BTS and TC can affect the speech quality.
Select the recommended software version.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 26
Others…
Page 26
 FR/HR traffic distribution
Suppose score for AMR FR and AMR HR will not change, if the percentage of AFR raise
from 65% to 80%.....
MOS (3.3) = 3.5(AMR FR)* 65% + 3.0(AMR HR) * 35%
MOS (3.4) = 3.5(AMR FR)* 80% + 3.0(AMR HR) * 20%
 Vehicle Speed
 …
Thank you
www.huawei.com

Speech-Quality-and-MOS Factors, Voice Quality improvement

  • 1.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Security Level: Internal www.huawei.com Speech Quality and MOS Alex
  • 2.
  • 3.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 3 MOS- Subjective Evaluation Way MOS test is normally performed on a group of listeners. The listeners listen to the speech material in the same channel environment and give a score. The scores are then averaged to obtain the MOS score. The scores vary largely from listener to listener.
  • 4.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 4 MOS- Objective Evaluation Way  Objective way compares an original signal with a degraded signal and then outputs a MOS score. The MOS score is like the result of a subjective listening test.
  • 5.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 5 PESQ(P.862) • PESQ value is from -0.5 to 4.5 • PESQ algorithm is powerful enough to test end-to-end speech quality.
  • 6.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 6 MOS-LQO(P.862.1) • MOS-LQO (Listening Quality Objective) value is from 1.02 to 4.56
  • 7.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 7 PESQ-LQ Mapping • This mapping is not adopted by ITU-T
  • 8.
  • 9.
    HUAWEI TECHNOLOGIES Co.,Ltd. HUAWEI Confidential Page 9 Measurement –Nemo (MS to MS) Nemo Audio Module: Sending Reference Sample Nemo Audio Module : Receiving Impaired Sample MOS Score
  • 10.
    HUAWEI TECHNOLOGIES Co.,Ltd. HUAWEI Confidential Page 10 QVM-MOS Measurement QVP-Data Analysis Measurement –QVoice (MS to PSTN)
  • 11.
    HUAWEI TECHNOLOGIES Co.,Ltd. HUAWEI Confidential Page 11 MS to PSTN Way BTS BSC MSC PSTN MS Reference Sample Impaired Sample Impaired Sample Reference Sample Uplink MOS Downlink MOS  UL MOS score and DL MOS score is available.  The configuration of Testing System is more complicated  Required less MS when performing Benchmark
  • 12.
    HUAWEI TECHNOLOGIES Co.,Ltd. HUAWEI Confidential Page 12  UL MOS score and DL MOS score is not available 。  MOS test by MS to MS way is more sensitive for Um interface  Using Local switch or TFO can improve MOS (MS to MS way)。 MS1 BTS MSC BSC BSC BTS MS2 Reference Sample Uplink Impaired Sample Downlink MS to MS Way
  • 13.
  • 14.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 14 Common Report MOS =Average[MOS(all samples)]
  • 15.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 15 QVoice Report Basic Report: Advanced Report: DL MOS = MOS_FR × FR% + MOS_HR × HR% + MOS_EFR × EFR% + MOS_AFR × AFR% + MOS_AHR × AHR% UL MOS = MOS_FR × FR% + MOS_HR × HR% + MOS_EFR × EFR% + MOS_AFR × AFR% + MOS_AHR × AHR% MOS = (DL_MOS + UL_MOS) / 2
  • 16.
  • 17.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 17 Page 17 Factors of MOS RF performance(RxLev, RxQual, C/I, HO,…) Transmission (Speech route,Tran. quality…) BSS Features (Codec,DTX, TFO, PC…) Hardware(BTS/FTC version,Fault…) Factors of MOS  Consider comprehensively when analyzing MOS Others(FR/HR percentage…) Test Method (Equipment, Testing mode,Speech Sample)
  • 18.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 18  Equipment  Different equipment has different standard (PESQ is preferred)  Do not compare MOS value from different type of equipment.  Testing Mode  MS to MS  MS to PSTN  Speech Sample  Speech Sample should be the same for any compare.  English Speech Sample is preferred Page 18 Test Method
  • 19.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 19 Um interface(1) Page 19  SQI (Downlink Um) Note: SQI is not equal to MOS. SQI is just a factor that can affect MOS
  • 20.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 20 Um Interface(2) Page 20  Improve SQI  Uplink Um Quality Can be evaluated by single user tracing or STS, normally Uplink Um interface should be a bit better than downlink Um interface. If not, be aware of Uplink interference, Uplink DTX, link balance.
  • 21.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 21 Features(1) Feature Recommend Effect on MOS Speech Version Active AMR, EFR The descending speech quality of 5 types of Speech Version in GSM is as following: AMR FR >= EFR > AMR HR >= FR > HR。 Active AMR and EFR will improve MOS remarkably. UL or DL DTX ? DTX will introduce clipping which is bad for MOS. However, DTX can decrease interference which is good for Um interface. UL and DL PC Open Power Control will decrease network interference. Hopping Open Caused by frequency diversity and interference equalization, FH is helpful for enhancing speech quality. TFO Open TFO can cut one encoding and decoding off a conversation , which is helpful for enhancing MOS in MS to MS way. Local Switch Open Allowing speech signal to bypass MSC and BSC, which also can cut one encoding and decoding time. However, if TFO is activated, local switch can only save transmission resources. Page 21
  • 22.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 22 Feature(2)-Codec AMR FR >= EFR > AMR HR >= FR > HR
  • 23.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 23 Transmission(1) Page 23  Speech Signal Route Evaluate how many times for encoding and decoding. There will be 0.2 MOS loss for one more encoding and decoding.  MS to PSTN Speech Signal Route:MS- BTS-BSC-MSC-GMSC-PSTN Be aware of transmission between MSC and GMSC: If using IP transmission, NSS part will introduce one time encoding and decoding. If using TDM transmission, no MOS loss  While performing MOS benchmark, if MS to PSTN test way is adopted, you should consider the transmission differences between MSC and GMSC.  For example:If networks all use AMR codec, the maximum MOS value in network with IP backbone is 3.8 to 3.9, and maximum MOS value in network with TDM can be 4.0-4.2.
  • 24.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 24 Transmission(2) Page 24  Speech Signal Route  MS to MS Speech route :MS-BTS-BSC-MSC-BSC-BTS-MSC The speech will experience 2 times encoding and decoding. But if TFO is adopted in the network, only one time encoding and decoding, which can provide 0.2 MOS increase. No TFO, 2 times encoding and decoding TFO, 1time encoding and decoding  Transmission Quality Speech quality is also strongly related to transmission quality, especially for microwave or satellite transmission type, which are more easily affected by weather (raining, snowing…) and cause MOS degrading.
  • 25.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 25 Hardware Page 25 For BSS part, BTS and TC can affect the speech quality. Select the recommended software version.
  • 26.
    HUAWEI TECHNOLOGIES CO.,LTD. Huawei Confidential Page 26 Others… Page 26  FR/HR traffic distribution Suppose score for AMR FR and AMR HR will not change, if the percentage of AFR raise from 65% to 80%..... MOS (3.3) = 3.5(AMR FR)* 65% + 3.0(AMR HR) * 35% MOS (3.4) = 3.5(AMR FR)* 80% + 3.0(AMR HR) * 20%  Vehicle Speed  …
  • 27.

Editor's Notes