1. Drive Testing. AQuA. PVQA.
Valeri Sitnikov
Sevana Oü
Tallinn, Estonia
valeri.sitnikov@sevana.ee
+3726170331

2. End-to-End Drive Test
GSM,
WCDMA,V
oLTE,
VoWiFi,
RCS
Codec
(NB, WB)
Drive Tester
Voice Core, Packe
Core, Evolved
Packet Core
AQuA
PVQA
Drive Test
Analysis

3. AQuA Metrics
MOS score according to P.800 – reliable objective MOS scores from 1 to 5
Percentage of sound quality similarity between reference and test audio
Duration distortion
Delay or advancing of audio signal activity
Sound signal activity mistiming
Signal spectrum damages
Signal spectrum vibration
Unidirectional signal spectrum analysis: amplification or attenuation
Most affected signal spectrum: low, medium, high bands
Average pitch
Pitch deviation
Pitch frequencies distribution distortion
Quantization steps distribution distortion
Amplitude clipping level
RMS
Sound signal: Minimum Energy, Maximum Energy, Average Energy
Sound signal: Min. Sample, Max. Sample, Average Sample

4. Drive Tester
Android ported NDK library,
developed for Samsung phones
Linux, Windows compatible
application/library, outperforming
existing algorithms in performance
Any Linux-like operating system with
standard C++ compiler

5. Android Drive Tester

6. PESQ. POLQA. AQuA. Feature Comparison Table.
Feature PESQ POLQA AQuA
Measurements in wideband
audio
NO YES YES
Accurate measurements in
CDMA networks
NO YES YES
VAD detection in AMR
codec
NO YES YES
GSM/WCDMA accurate
measurements
NO YES YES
Variable delay up to 1 sec in
VoIP
NO YES YES

7. PESQ. POLQA. AQuA. Feature Comparison Table.
Feature PESQ POLQA AQuA
Strong linear distortions NO YES YES
Limitations with speech
enhancement devices (noise
reduction, gain control)
YES NO NO
Works in NB (up to 8 kHz
Sample Rate) bandwidths
NO YES YES
Works in WB (up to 16 kHz
Sample Rate) bandwidths
NO/YES YES YES
Works in SWB (above 16 kHz
Sample Rate) bandwidths
NO YES YES

8. PESQ. POLQA. AQuA. Feature Comparison Table.
Feature PESQ POLQA AQuA
Works with audio sampling rate
higher than 48kHz
NO NO YES
Works with all standard and non-
standard codecs audio (AMR-WB,
EVRC-WB, iLBC, AMB+, AAC, SLIK,
G.711, G.729)
NO YES YES
Works with all technologies
(GSM, WCDMA, LTE, VoIP, VoIP
over IMS)
NO YES YES
Can be used in testing terminals
and hand-free applications
YES YES YES
Considers codec degradations
(tandem codecs, packet loss,
frame errors, bit errors)
NO YES YES

9. PESQ. POLQA. AQuA. Feature Comparison Table.
Feature PESQ POLQA AQuA
Customizable perceptual model
and algorithm
NO NO YES
Considers network specific
degradations
NO YES YES
Copes with P.800 MOS NO YES YES
Talking quality YES NO NO/YES
MOS score for long sequences
of speech (more than 30
seconds)
NO NO YES

10. PESQ. POLQA. AQuA. Feature Comparison Table.
Feature PESQ POLQA AQuA
Noisy listening environments NO NO YES
Music clips NO NO YES
Noise suppressors NO NO YES
Non-telephony applications YES NO YES
Language independent YES NO YES

11. PESQ. POLQA. AQuA. Feature Comparison Table.
Feature PESQ POLQA AQuA
Multi-dimensional
audio(Works with Stereo)
NO NO YES
Unified perceptual model
for all types of audio
(speech, music)
NO NO YES
Able of working with long audio
(1 hour and longer)
NO NO YES
AQuA has some key features vital for sound quality evaluation during deployment and operation of
VoLTE, VoWi-Fi, RCS:
Language independent
Long test calls support
Ability to work with Stereo sound

12. PVQA Metrics
MOS score according to P.800 – reliable objective MOS scores from 1 to 5
Impairments detection (reasons for QoE drop down):
Noise
DeadAir
Amplitude clipping
VAD Clipping
Clicking
Crackling
Fuzzy voice
Echo
DTMF
By just waveform analysis PVQA is able to discover such network issues as
Packet loss (DeadAir + Crackling)
Hardware issues (Amplitude clipping, VAD clipping)

13. PVQA Analysis
• Non-intrusively calculated MOS score with standard deviation of 0.6 MOS to intrusive AQuA
test score
• CSV reports on impairments that caused sound quality degradation
Time; Average; SNR-00; DeadAir-01; DeadAir-00; AMPC-00; VADC-00; Click-00;
0.000 : 0.799; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
0.799 : 1.598; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
1.598 : 2.397; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
2.397 : 3.196; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
3.196 : 3.995; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
3.995 : 4.794; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
4.794 : 5.593; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
5.593 : 6.392; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
6.392 : 7.191; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
7.191 : 7.990; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
7.990 : 8.789; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
8.789 : 9.588; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
9.588 : 10.387; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
10.387 : 11.186; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
11.186 : 11.985; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
11.985 : 12.784; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
12.784 : 13.583; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
13.583 : 14.382; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
14.382 : 14.882; 0.013333; 0.000000; 0.080000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
Time; Average; SNR-00; DeadAir-01; DeadAir-00; AMPC-00; VADC-00; Click-00;
0.000 : 0.799; 0.014583; 0.000000; 0.087500; 0.000000; 0.000000; 0.000000; 0.000000; Ok
0.799 : 1.598; 0.018750; 0.000000; 0.112500; 0.000000; 0.000000; 0.000000; 0.000000; Poor
1.598 : 2.397; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
2.397 : 3.196; 0.085340; 0.000000; 0.425000; 0.000000; 0.000000; 0.050000; 0.037037; Poor
3.196 : 3.995; 0.039583; 0.000000; 0.225000; 0.000000; 0.000000; 0.012500; 0.000000; Poor
3.995 : 4.794; 0.018750; 0.000000; 0.100000; 0.000000; 0.000000; 0.012500; 0.000000; Poor
4.794 : 5.593; 0.034395; 0.000000; 0.162500; 0.000000; 0.000000; 0.025000; 0.018868; Poor
5.593 : 6.392; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
6.392 : 7.191; 0.055228; 0.000000; 0.300000; 0.000000; 0.000000; 0.012500; 0.018868; Poor
7.191 : 7.990; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
7.990 : 8.789; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
8.789 : 9.588; 0.043750; 0.000000; 0.225000; 0.000000; 0.000000; 0.037500; 0.000000; Poor
9.588 : 10.387; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
10.387 : 11.186; 0.114623; 0.000000; 0.612500; 0.000000; 0.000000; 0.037500; 0.037736; Poor
11.186 : 11.985; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
11.985 : 12.784; 0.014583; 0.000000; 0.075000; 0.000000; 0.000000; 0.012500; 0.000000; Ok
12.784 : 13.583; 0.004167; 0.000000; 0.025000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
13.583 : 14.382; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; 0.000000; Ok
14.382 : 14.832; 0.014815; 0.000000; 0.066667; 0.000000; 0.000000; 0.022222; 0.000000; Ok
Time; Average; SNR-00; DeadAir-01; DeadAir-00; AMPC-00; VADC-00; Click-00;
0.000 : 0.799; 0.014583; 0.000000; 0.087500; 0.000000; 0.000000; 0.000000; 0.000000; Ok
0.799 : 1.598; 0.018750; 0.000000; 0.112500; 0.000000; 0.000000; 0.000000; 0.000000; Poor
1.598 : 2.397; 0.087500; 0.000000; 0.500000; 0.000000; 0.000000; 0.025000; 0.000000; Poor
2.397 : 3.196; 0.148920; 0.000000; 0.812500; 0.000000; 0.000000; 0.062500; 0.018519; Poor
3.196 : 3.995; 0.093750; 0.000000; 0.525000; 0.000000; 0.000000; 0.037500; 0.000000; Poor
3.995 : 4.794; 0.035417; 0.000000; 0.187500; 0.000000; 0.000000; 0.025000; 0.000000; Poor
4.794 : 5.593; 0.133333; 0.000000; 0.712500; 0.000000; 0.000000; 0.087500; 0.000000; Poor
5.593 : 6.392; 0.077083; 0.000000; 0.400000; 0.000000; 0.000000; 0.062500; 0.000000; Poor
6.392 : 7.191; 0.117728; 0.000000; 0.637500; 0.000000; 0.000000; 0.050000; 0.018868; Poor
7.191 : 7.990; 0.036900; 0.000000; 0.177215; 0.000000; 0.000000; 0.025316; 0.018868; Poor
7.990 : 8.789; 0.070833; 0.000000; 0.375000; 0.000000; 0.000000; 0.050000; 0.000000; Poor
8.789 : 9.588; 0.096836; 0.000000; 0.500000; 0.000000; 0.000000; 0.062500; 0.018519; Poor
9.588 : 10.387; 0.052083; 0.000000; 0.300000; 0.000000; 0.000000; 0.012500; 0.000000; Poor
10.387 : 11.186; 0.125000; 0.000000; 0.712500; 0.000000; 0.000000; 0.037500; 0.000000; Poor
11.186 : 11.985; 0.139506; 0.000000; 0.750000; 0.000000; 0.000000; 0.050000; 0.037037; Poor
11.985 : 12.784; 0.132311; 0.000000; 0.750000; 0.000000; 0.000000; 0.025000; 0.018868; Poor
12.784 : 13.583; 0.128145; 0.000000; 0.700000; 0.000000; 0.000000; 0.050000; 0.018868; Poor
13.583 : 14.382; 0.086478; 0.000000; 0.475000; 0.000000; 0.000000; 0.025000; 0.018868; Poor
14.382 : 14.832; 0.107407; 0.000000; 0.600000; 0.000000; 0.000000; 0.044444; 0.000000; Poor

14. PVQA Analysis
MOS = 2.894737

15. PVQA Analysis

16. PVQA Root Cause Analysis
VAD clipping related to shortening of voice snippets due to
failures in voice activity detection, the so-called squelch
Amplitude clipping with noise - form of waveform
distortion that occurs when an amplifier is overdriven and
attempts to deliver an output voltage or current beyond
its maximum capability.

17. PVQA Pattern Discovery
There are two types of patterns PVQA allows to register and disover:
• Network specific impairments (prevailing of clipping, noise, or signals with
crackling)
• Combination of impairments specific for certain events, e.g. packet loss
Packet Loss pattern

18. Pitch
o AQuA offers reliable end-to-end sound quality assessment for both NB and WB audio
providing reasonable MOS scores according to P.800.
o AQuA is working on laptops, servers and Android devices providing high performance
in MOS calculation and analysis.
o AQuA can be ported to other systems that have standard C++ compiler (Linux-type
operating systems, iOS)
o PVQA provides reliable approach for non-intrusive MOS score calculation
o PVQA provides additional information on impairments that cause voice quality
degradation:
o Root Cause Analysis
o Impairments Pattern Discovery
o Packet loss detection by waveform analysis

19. THANK YOU!