Video quality testing


Published on

Review of video over IP testing tools including: video syntax analyzer, pixel based measurement indexes like PSNR and SSIM and the tools to measure them, IP based video quality testing.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Video quality testing

  1. 1. Video and Video over IP TestingMethodsThere are three levels of testing which apply to video quality measurements:1) Media Stream analyzers2) Encoder output Quality measurements3) Client side media quality measurementsMedia AnalyzerThe media stream analysis is done at the output of the sender’s Encoder or multiplexeroutput. The media Analyzer’s roles are:1. Verify that the audio and video streams comply with the coding standard2. Understand the encoding parameters including: GOP structure, encoder profileand level, resolution, fps, VBV, use of CABAC, UEP etc.3. Enables tweaking parameters which effects encoder latency, stream robustnessSender’s side quality measurementThe quality measurement on the sender’s side is based on full payload qualitymeasurements techniques like PSNR and SSIM. Although the user perceives only thevideo quality on the client side, it’s also important to check the quality on the sender’sside. Sender’s side analysis roles include:1) Improving video quality regardless of the network impairments.2) Testing video quality at the encoder’s output as base level quality3) Enabling tweaking of parameters for robustness/quality/latency tradeoffsReceiver side quality measurementThere are two types of Receiver side testing1) IP based testing2) Re-synced Video comparisonIP Based testingIP based testing uses the packet low and jitter data along with packet header informationto assess the video quality and does not use video decoding and full payload comparisonlike transmitter side testing. The reasoning for using this method is that the source file isnot available, there are not enough resources to do full payload analysis or the packet losswill cause miss-synchronization that would not enable frame-to-frame comparison and
  2. 2. will require re-synchronization.Re-Synced video comparisonRe-synced video methods uses image code embedded on each frame understand theframe number and resync the destination video with the original video.Testing diagramThe testing of the system is done according to this the following diagram:ReceiverTransmitterVideo EncoderAudio EncoderStream AnalyzerMux /PacketizerVideo Quality testingNetworkEmulatorIP Based videoquality probeTesting ToolsThis section reviews testing tools for each of the testing scenarios described aboveMethods ToolMedia Analyzers Interra H264 Analyzer (preferred) – good (recommended)Tektronics E4A - goodMindego H264 Analyzer - averageCodecVisa - crash, can’t find headersDVBSnoop – free text based tool used by WiFiDi.H264 Analyzer – text based hard to work with. for scriptingVideo Quality MSU VQMT – free for SDInterra H264 Analyzer YUV tool – included in interraYUVTools – free for eval. Provides PSNR/SSIMReceiver side probes Interra – Orion – does not support Promiscuous modeTelchemy – VQCapture (recommended)Tektroniks VQS – does not support elementary streams only M2TSHW based solutions
  3. 3. Operating Testing EquipmentMedia Analyzers -InterraInterra – open the raw H.264 video file review if error exists, check Qp per frame graphaverage bitrate etc.PSNR Measurements – MSU VQMTOperation:1. Select a compressed file with high quality2. Extract the YUV of the file using VLC command:vlc <Source File Name>:sout=#transcode{venc=ffmpeg{keyint=1},vcodec=I420}:std{access=file,mux=raw,dst=<Source File Name>}3. Play and transmit the file while adding a "Sync Frame" in the form of aBlack/White marker on the frame every two seconds.4. Record the video output of the encoder on the source5. Convert the video output to YUV using the above command6. Use the MSU VQMT software to compare the files as described in the imagebelow:7. The results for PSNR are shown below, SSIM results looks usually the same.
  4. 4. 8. This diagram is for Quality/Frame. To compare with other videos/ configuration itis convertible to a candle stick diagram with min/max and mean qualityReceiver Side IP Based testing - VQCaptureThe testing configuration of IP Based testing is charted below:NetworkEmulatorVideo/Audioover RTP ESVQCaptureFile AnalysisThe packet recording is saved by wireshark on the PC after the network emulatorrecording. The VQCapture provides:1. network level statistics2. Video Quality measurement VMOS3. Audio Quality measurement
  5. 5. Receiver Side testingThere are two main measurements for receiver side testing:Full payload testing after resynchronizationE2E latency and jitterReceiverTransmitterVideoEncoderAudioEncoderStream AnalyzerMux /PacketizerVideo QualitytestingNetworkEmulatorRe-Synced VideocomparisonE2E latency &jitter