The document discusses quality control for high definition television in a file-based workflow. It covers audiovisual quality control, technical quality control of file containers and metadata, and automated quality control. It also addresses the proliferation of raster formats and codecs with the emergence of HDTV, and how this impacts quality evaluation and expectations of viewers. Simulation of production and broadcast chains is discussed as well as various evaluation methods like objective metrics, subjective testing, and expert review.

1. Visual and technical quality control
for high definition television
16/1
Tom Beckers
Wim Ermens

2. Overview
Video quality control in a filebased world
Impact of HDTV emergence on quality control
Video quality evaluation
Production chain simulation
Broadcast chain simulation
VRT-medialab: onderzoek en innovatie

3. Video quality control in a filebased world
AudioVisual Quality Control
! Audio
! Video
! Interaction between audio and video : e.g. lipsync
File based AV Quality Control
! Part of an automated or manual workflow
– Diagnose
– Repair / Redo
VRT-medialab: onderzoek en innovatie

4. VRT-medialab: onderzoek en innovatie

5. Video quality control in a filebased world
Technical quality control
! Container
! Metadata
! Interaction between container, audio and video : e.g. duration of tracks
File based technical quality control
! Part of an automated or manual workflow
! Application specifications
VRT-medialab: onderzoek en innovatie

6. VRT-medialab: onderzoek en innovatie

7. Video quality control in a filebased world
Automated QC
VRT-medialab: onderzoek en innovatie

8. Video quality control in a filebased world
Application Specs DV-25 (News) D-10 (APP) HD
Contribution
MXF analysis & application Production
specification validation Play-out
Archive
Application
Specification
IRT MXF
Analyzer
MXF MXF
file Validator
analysis
Ardendo
CheckMXF
MXF
analysis
report
VRT-medialab: onderzoek en innovatie

9. Video quality control in a filebased world
Application specification
validation report
VRT-medialab: onderzoek en innovatie

10. Video quality control in a filebased world
Safeguarding audiovisual quality
! Maintaining quality throughout the production chain
– Choose material as close to source as possible
– prevent unneeded multi-generation
– Try to produce with the shortest/’most apt’ chain
– Prevent unneeded multi-generation
– Prevent transcoding 720p
720p 720p ! 1080p !
1080i !
– Check quality
1080p Black Box 720p
1080i
! Carefully design the production chain (in production/broadcast
– Choose the right codecs 1080i 1080i ! chain) 1080p!
720p ! 1080i
– Choose the right equipment 1080p
720p 1080i
VRT-medialab: onderzoek en innovatie

11. Impact of HDTV emergence on quality control
Proliferation of raster formats
SD TV HD TV
1080p25
PAL
576i50 720p50 1080i50
1080p50
VRT-medialab: onderzoek en innovatie

12. Impact of HDTV emergence on quality control
Proliferation of codecs
HDV 50 Mbps
ProRes Dirac
JPEG2K
10 bit sampling
4:2:2
4:4:4
YUV HDCAM-SR XDCAM HD42
DVCProHD
180Mbps
440Mbps 100Mbps
DNxHD
HDCAM RGB
4:2:0 / 4:1:1
SVC
AVC-Intra 8 bit sampling XDCAM HD
AVCHD
VRT-medialab: onderzoek en innovatie

13. Impact of HDTV emergence on quality control
Viewer quality expectation
! Consumer expects ROI from flat panel investment
! Depends on content type (drama, news, sports, soap)
What codec to choose?
! Picture quality ! quality evaluation needed
! Bit rate
! Performance
! Support
! Scalability
VRT-medialab: onderzoek en innovatie

14. Video quality evaluation
VRT-medialab: onderzoek en innovatie

15. Video quality evaluation
Video sequence used for testing
(source EBU) :
! High quality source material
! 10-bit YCBCR samples
! Chroma subsampling 4:2:2
! HD-SDI compatible
! 720p50
VRT-medialab: onderzoek en innovatie

16. Note on viewing conditions
Expert evaluation :
! Class 1 reference monitor @ 1H, 3H
! High quality consumer device (FullHD, Plasma) @ 1H, 3H
Present demo setup
! Overhead projector (internal processing?)
! Not sufficiently dark room
! Sub ideal circumstances
Interested parties : ask for demo in better circumstances...
VRT-medialab: onderzoek en innovatie

17. Video quality in production chain
Simulation chain
without
pixelshift with
pixelshift (+2H, +6V)
Camera Post production encoding
encoding (4 generations)
VRT-medialab: onderzoek en innovatie

18. Video quality in production chain
Codecs tested :
! Camera : AVC-I, DVCProHD, XDCAM HD422
! PostProduction : DNxHD, AVC-I, DVCProHD, XDCAM HD422
Codec properties
Codec Bitrate Chroma Sample GOP Spatial subsampling
(Mbps) subsampling size
(bits)
AVC-I 100 4:2:2 10 I frames only None
DVCProHD 100 4:2:2 8 I frames only 3:4 horizontal :
960 luma samples/line
DNxHD 115 4:2:2 8 I frames only None
XDCAM 50 4:2:2 8 LongGOP None
HD422 (n=12 @ 720p)
VRT-medialab: onderzoek en innovatie

19. Video quality in production chain
Test setup
Sony PDW-HD1500* Panasonic AJ-HPM110** Avid Airspeed + DNXchange
XDCAM HD422 DVCProHD DNxHD
AVC-I
* test model provided by Sony Belgium
** test model provided by Panasonic Belgium
VRT-medialab: onderzoek en innovatie

20. Video quality in production chain
Test setup
Encode Decode
HD-SDI HD-SDI
Uncompr. Gen 0 Gen 1 Gen 2 Gen 3 Gen 3 Gen 4
source (Cam) (PP1) (PP2) (PP3) shifted (PP4)
Uncompressed YCbCr storage
HD-SDI ingest & playout
VRT-medialab: onderzoek en innovatie

21. Video quality evaluation
Evaluation methods :
! Objective tests
! Subjective tests
! Automated subjective tests
! Expert viewing
VRT-medialab: onderzoek en innovatie

22. Video quality evaluation
Objective tests : PSNR, SSIM, ...
1 mquot;1 n quot;1 MAX R
MSE frame = !! R(i, j ) quot; I (i, j )
2
PSNR frame = 20 ! log10 (
mn i =0 j =0 MSE frame
Reference Impaired Squared difference
VRT-medialab: onderzoek en innovatie

23. Video quality evaluation
Objective tests : PSNR, SSIM, ...
Advantages Disadvantages
objective 3 resultsets per sequence (Y, Cb,
Cr)
reproducible
unsensitive to grouping of pixel
automated test
differences
VRT-medialab: onderzoek en innovatie

24. Video quality evaluation
Subjective tests : DSCQS, TSCES, ...
VRT-medialab: onderzoek en innovatie

25. Video quality evaluation
Upper anchor
Source material
Material to be
evaluated
Lower anchor
Worst case
(D)MOS scores
VRT-medialab: onderzoek en innovatie

26. Video quality evaluation
Subjective tests :
DSCQS, TSCES, ...
Advantages Disadvantages
closest to real life experience difficult to test many sequences
timeconsuming to setup
difficult to get sufficient statistics
worst case reference definition
VRT-medialab: onderzoek en innovatie

27. Video quality evaluation
Automated subjective tests : Tektronix PQA-500, Opticom PEVQ Analyzer, ...
! Aims to provide (D)MOS score
! Simulate :
– Monitor setup and characteristics
– Viewing setup and conditions
– Viewer setup and characterisics
– Human visual system (= tricky)
Advantages Disadvantages
automated test good correlation to subjective test
results to be proven
closer correlation to subjective test
sults than PSNR typically slower than objective tests
no worst case reference mandatory
VRT-medialab: onderzoek en innovatie

28. Video quality evaluation
Expert viewing
! Aim :
– Visually evaluate video quality
– Determine typical artefacts
– Compare source with test material
– Compare test material with other test material
! Method :
– Create splitscreen images
– Evaluate on (reference) monitor
– Create ranking from intercomparison
VRT-medialab: onderzoek en innovatie

29. Video quality in production chain
Expert viewing (“The proof of the pudding is in the eating”)
VRT-medialab: onderzoek en innovatie

30. Video quality in production chain
Uncompressed vs Compressed video material
VRT-medialab: onderzoek en innovatie

31. Video quality in production chain
Compressed vs Compressed video material
VRT-medialab: onderzoek en innovatie

32. Video quality due to de-interlacing
source de-interlaced
De-interlacing: difficult to perform
Good quality -converters exist (price!)
720p DNxHD (Gen0) 1080i - DNxHD (Gen0) – de-interlace – 720p
VRT-medialab: onderzoek en innovatie

33. Video quality in distribution chain
VRT-medialab: onderzoek en innovatie

34. Distribution encoder
Distribution encoder = device that compresses the broadcaster’s signal into
a format suitable for transmission/distribution
Codecs: MPEG-2 or MPEG-4 H.264 (tested)
Interfaces: HD-SDI quot; ASI
Typical bitrate in/out:
! From 50-200 Mbit/s quot; 10 Mbit/s (or less)
Parameters:
! Encoding delay
! Encoder bit-rate
! Profile/Noise reduction/GOP size/…
VRT-medialab: onderzoek en innovatie

35. Distribution encoder: bit-rate
AVC/DNxHD – 16Mbit/s H.264
Which bit-rate to choose?
Uncompressed
! Distribution channel ‘defines’ available bit-rate
! MPEG-4 does a fine job
! Motion in video is important
AVC/DNxHD – 8Mbit/s H.264
AVC/DNxHD – 10Mbit/s H.264
VRT-medialab: onderzoek en innovatie

36. Distribution encoder and production codec
Uncompressed HD ~ 1 500 Mbit/s
Production bitrate ~ 100 Mbit/s
Distribution encoder ~ 10 Mbit/s
Do we actually see the influence of the production codec?
AVC/AVC + 8 Mbit/s H.264 DVC/DVC + 8 Mbit/s H.264
VRT-medialab: onderzoek en innovatie

37. Distribution: the end of the chain
Receiver (set-top-box)
! Interlacing – de-interlacing
! Scaling
Television
! A lot of choice
! De-interlace
! Scale
! Process (sharpen – noise reduction)
Make sure there’s no unnecessary signal conversion
VRT-medialab: onderzoek en innovatie