Review of video over IP testing tools including: video syntax analyzer, pixel based measurement indexes like PSNR and SSIM and the tools to measure them, IP based video quality testing.

1. Video and Video over IP Testing
Methods
There are three levels of testing which apply to video quality measurements:
1) Media Stream analyzers
2) Encoder output Quality measurements
3) Client side media quality measurements
Media Analyzer
The media stream analysis is done at the output of the sender’s Encoder or multiplexer
output. The media Analyzer’s roles are:
1. Verify that the audio and video streams comply with the coding standard
2. Understand the encoding parameters including: GOP structure, encoder profile
and level, resolution, fps, VBV, use of CABAC, UEP etc.
3. Enables tweaking parameters which effects encoder latency, stream robustness
Sender’s side quality measurement
The quality measurement on the sender’s side is based on full payload quality
measurements techniques like PSNR and SSIM. Although the user perceives only the
video quality on the client side, it’s also important to check the quality on the sender’s
side. Sender’s side analysis roles include:
1) Improving video quality regardless of the network impairments.
2) Testing video quality at the encoder’s output as base level quality
3) Enabling tweaking of parameters for robustness/quality/latency tradeoffs
Receiver side quality measurement
There are two types of Receiver side testing
1) IP based testing
2) Re-synced Video comparison
IP Based testing
IP based testing uses the packet low and jitter data along with packet header information
to assess the video quality and does not use video decoding and full payload comparison
like transmitter side testing. The reasoning for using this method is that the source file is
not available, there are not enough resources to do full payload analysis or the packet loss
will cause miss-synchronization that would not enable frame-to-frame comparison and

2. will require re-synchronization.
Re-Synced video comparison
Re-synced video methods uses image code embedded on each frame understand the
frame number and resync the destination video with the original video.
Testing diagram
The testing of the system is done according to this the following diagram:
ReceiverTransmitter
Video Encoder
Audio Encoder
Stream Analyzer
Mux /
Packetizer
Video Quality testing
Network
Emulator
IP Based video
quality probe
Testing Tools
This section reviews testing tools for each of the testing scenarios described above
Methods Tool
Media Analyzers Interra H264 Analyzer (preferred) – good (recommended)
Tektronics E4A - good
Mindego H264 Analyzer - average
CodecVisa - crash, can’t find headers
DVBSnoop – free text based tool used by WiFiDi.
H264 Analyzer – text based hard to work with. for scripting
Video Quality MSU VQMT – free for SD
Interra H264 Analyzer YUV tool – included in interra
YUVTools – free for eval. Provides PSNR/SSIM
Receiver side probes Interra – Orion – does not support Promiscuous mode
Telchemy – VQCapture (recommended)
Tektroniks VQS – does not support elementary streams only M2TS
HW based solutions

3. Operating Testing Equipment
Media Analyzers -Interra
Interra – open the raw H.264 video file review if error exists, check Qp per frame graph
average bitrate etc.
PSNR Measurements – MSU VQMT
Operation:
1. Select a compressed file with high quality
2. Extract the YUV of the file using VLC command:
vlc <Source File Name>
:sout=#transcode{venc=ffmpeg{keyint=1},vcodec=I420}:std{access=file,mux=ra
w,dst=<Source File Name>}
3. Play and transmit the file while adding a "Sync Frame" in the form of a
Black/White marker on the frame every two seconds.
4. Record the video output of the encoder on the source
5. Convert the video output to YUV using the above command
6. Use the MSU VQMT software to compare the files as described in the image
below:
7. The results for PSNR are shown below, SSIM results looks usually the same.

4. 8. This diagram is for Quality/Frame. To compare with other videos/ configuration it
is convertible to a candle stick diagram with min/max and mean quality
Receiver Side IP Based testing - VQCapture
The testing configuration of IP Based testing is charted below:
Network
EmulatorVideo/Audio
over RTP ES
VQCapture
File Analysis
The packet recording is saved by wireshark on the PC after the network emulator
recording. The VQCapture provides:
1. network level statistics
2. Video Quality measurement VMOS
3. Audio Quality measurement

5. Receiver Side testing
There are two main measurements for receiver side testing:
Full payload testing after resynchronization
E2E latency and jitter
ReceiverTransmitter
Video
Encoder
Audio
Encoder
Stream Analyzer
Mux /
Packetizer
Video Quality
testing
Network
Emulator
Re-Synced Video
comparison
E2E latency &
jitter