Media File Formats for the Broadcaster
Unraveling the mystery
Tom Beckers Matthias De Geyter
Wim Ermens Luk Overmeire
VRT Medialab - 16/10/2008

Agenda
Part One - Luk Overmeire & Matthias De Geyter
Introduction
Image and advanced compression formats for HD
File formats for HD production: MXF and AAF
--- pause ---
Part Two - Tom Beckers & Wim Ermens
Visual and technical quality control for HD
Demos
2 VRT-medialab: onderzoek en innovatie

VRT Medialab
4 research domains
broadcaster customer
Information management Information front-end
Research Archiving User generated
Personalization
content
Product
Development
Manufacturing Distribution User interaction Social platforms
Engineering Automation
information
essence
Production infrastructure Distribution
Editing Mastering
Wireless and
Internet
wired networks
Media
Ingest Asset Mgnt Playout
Mobile Satellite
Infrastructure
(storage and network)
3 VRT-medialab: onderzoek en innovatie

VRT Medialab
4 research domains
Information management Information front-end
Research Archiving User generated
Personalization
content
Product
Development
Manufacturing Distribution User interaction Social platforms
Engineering Automation
Production infrastructure Distribution
Editing Mastering
Wireless and
Internet
wired networks
Media
Ingest Asset Mgnt Playout
Mobile Satellite
Infrastructure
(storage and network)
4 VRT-medialab: onderzoek en innovatie

Context: tape-based media
production
5 VRT-medialab: onderzoek en innovatie

Context: file-based media
production with central warehouse
information
information
essence
essence
6 VRT-medialab: onderzoek en innovatie

File-based media production
Concurrent
engineering
Browse Clients Processing
during
ingest/transfer
Draft POST Craft
editors
Processing Production
Media
Analysis
Asset Mgmt Subtitling
INGEST IP network PLAY-OUT
Central storage
Annotation Central Archiving
Storage
Post- Follow-up
sonorisation continuity Editor-in-Chief
7 VRT-medialab: onderzoek en innovatie

Media production formats
information
essence =
image formats
! resolution, frame rate, raster
essence
compression formats
! high & low resolution
+
file/wrapper formats information
8 VRT-medialab: onderzoek en innovatie

Typical broadcast chain
post P
File-based camera I production
N L
G A Web
contribution Y
E
central O
distribution linear
S Blu-ray DVD
T warehous U
Studio acquisition e T
production
archive
9 VRT-medialab: onderzoek en innovatie

Typical broadcast chain
Formats for Standard Definition
576 lines, interlaced
25 frames/sec
DV25
MXF MXF/AAF post P
I production
File-based camera
L H.263/VC-1
N IMX-30
G A Web
MPEG-2 MPEG-2
contribution Y
long GOP
E IMX-50
central O
distribution
long GOP linear
S 4:2:0
4:2:2 warehous U DVD
T MXF OP1a 8-10 Mbps
10-25 Mpbs
Studio acquisition e T MPEG-2
MPEG-1
VC-1
production
DV25
archive IMX-30
long GOP IMX-50
10 VRT-medialab: onderzoek en innovatie

Typical broadcast chain
Formats for High Definition
720p50, 1080i25,
1080p25, 1080p50
MXF/AAF,
Quicktime postXDCAM HD,
MXF
File-based camera I AVC-Intra, P
production H.264/VC-1
N DNxHD, L
G ProRes, A H.264 4:2:0
Web
MPEG-2 ->H.264
contribution JPEG2000 Y
4:2:0, 4:2:2
E
O
distribution
bit rate ? linear
SMXF OP1acentral Blu-ray DVD
bit rate ? T warehous U
Studio acquisition e T H.264/VC-1
H.264 (1),
production (2),
H.264
…
uncompressed, XDCAM HD,
lossless compressed ? AVC-Intra,
archive DNxHD,
ProRes,
long GOP JPEG2000
11 VRT-medialab: onderzoek en innovatie

Proliferation of media formats
HD image formats
! 720p50,1080i25, 1080p25
=> 1080p50
HD compression formats
! legacy versus new formats
! contribution, camera, production, archive, distribution
proxy formats (low resolution)
wrapper formats
! MXF flavors, Quicktime
12 VRT-medialab: onderzoek en innovatie

The HDTV format jungle
HDTV increases
variety of formats
Abtastformat
Abtastformat
Lossy Abtastformat
Kompressionsformat HDTV increases
conversions Abtastformat
Sampling formats
Kompressionsformat variety of options
(1080i/25, 720p/50, 1920/1440 pixels,
Kompressionsformat
IT-basiert: File Format
4:2:0, 4:4:4, ... )
Kompressionsformat HDTV increases need
Compression formats Format
IT-basiert: File
for efficient wrapping
(MPEG-2,IT-basiert: File Format
H.264, JPEG2000, ..) Transport
Physikalischer
IT-basiert: File Format
Physikalischer Transport
File Formats Further formats (e.g. consumer)
Presentation (Displays)
(MXF, QT, ..) Transport
Physikalischer
Physikalischer Transport
Physical Transport
Contribution („live“ and/or „File “)
Distribution
(HD-SDI, IP, Tape, ..)
Production
(more granular variants for news, features, etc.)
„A format“
13 VRT-medialab: onderzoek en innovatie

Production areas
high-end production
! landmarks, drama
mainstream production
! soap series, documentaries, shows
Mainstream HDTV platform
\" single or multiple scanning formats ?
News and sports \" single or multiple compression formats ?
\" 720p/50 a/o 1080i/25
video journalism \" 4:2:2 a/o further
subsampling
! consumer cameras \" 10 Bit a/o 8 Bit
\" I-frame only a/o long GOP
\" Headroom a/o low data rate
\"… a/o …
14 VRT-medialab: onderzoek en innovatie

Image formats for HD
(and beyond)

Today’s and future image formats
Super Hi-Vision (U-HDTV)
resolution 7680 x 4320 24 Gbit/s
Far future 60 frames/sec, progressive
audio: 22.2 channels
?
System 4 – 1080p50 3 Gbit/s
Future resolution 1920 x 1080
50 frames/sec
progressive
Today 1,5 Gbit/s
System 1 – 720p50
resolution 1280 x 720
50 frames/sec
progressive
EBU
System 2 – 1080p25 System 3 – 1080i25
resolution 1920 x 1080
Tech 3299 resolution 1920 x 1080
25 frames/sec HDTV System 1-4 50 fields/sec
progressive interlaced
16 VRT-medialab: onderzoek en innovatie

EBU – Tech 3299
HDTV
Horizontal Active Frame Sub-sampling / Net image Bit Interface Bit
Systems samples lines rate Quantisation [Bit] Rate [Gbit/s] Rate [Gbit/s]
EBU 3299
System 1
1280 720 50 4:2:2 10 0,9216 1,5
720p/50
System 2
1920 1080 25 4:2:2 10 1,0368 1,5
1080i/25
System 3
1920 1080 25 4:2:2 10 1,0368 1,5
1080p/25
System 4
1920 1080 50 4:2:2 10 2,0736 3,0
1080p/50
SDTV 25 0,27
720 576 4:2:2 10 0,207
576i/25
17 VRT-medialab: onderzoek en innovatie

The resolution revolution
now
+ 5y
Maximal viewing distance:
SD: d < 5.h
HD: d < 3.h
U-HDTV: d < 0.75.h
d = viewing distance
h = display height
+ 15y?
18 VRT-medialab: onderzoek en innovatie

Good HDTV – More than a
numbers game
1080 (i) > 720 (p) => “More is better” ?
19 VRT-medialab: onderzoek en innovatie

How can smaller be better ? (1)
interlace factor
! interlaced scanning: 50 fields/sec (1 field = ! frame)
! 540 lines < dynamic vertical resolution < 1080 lines
Studies (Bell Laboratories, NHK): interlace factor ! 0.6 => 648 lines < 720 lines
20 VRT-medialab: onderzoek en innovatie

How can smaller be better ? (2)
interlace factor
interlace artefacts (motion rendition)
! e.g. serrated edges
21 VRT-medialab: onderzoek en innovatie

How can smaller be better ? (3)
interlace factor
interlace artefacts
resolution capacity of human eye (\" 1 arc minute)
Required horizontal sampling for different screen sizes, viewed at 2.7m
720x576
22 VRT-medialab: onderzoek en innovatie

How can smaller be better?
interlace factor
interlace artefacts
resolution capacity of human eye
compression artefacts
! better compression performance for progressive
! compression of interlaced media generally involves de-
interlacing
23 VRT-medialab: onderzoek en innovatie

Progressive versus interlaced
far better suited for moving pictures (e.g.
Pro
legacy format
sports)
! as of today, generally better
easier to encode supported throughout the workflow
plasma, LCD are progressive displays – camera, set-top-box, post
=> end-to-end progressive workflow is possible production, central MAM, …
same resolution visibility from “viewing ! existing program material
distance” ! current offering is predominantly
rescaling is easier than deinterlacing 1080i25 content
the future is progressive => evolution to
1080p50 is more straightforward
compatible with 1080p25 through 1080psf
Contra
full support in end-to-end workflows to be higher complexity
carefully tested
! down conversion requires de- and re-
! camera (quality), central MAM, post interlacing
production, Dolby E surround sound ! de-interlacing is very hard to do well
! time labeling ! (editing) legacy format - “wet van de remmende voorsprong”
! distribution (Telenet, Belgacom, …) less suited for slow motion and chroma keying
! contribution (productiehuizen)
24 VRT-medialab: onderzoek en innovatie

Different scenarios for image
formats
Production Distribution
Acquisition format format
Distribution Full HD
Production
Display
1080i
Scenario 1 1080i 1080i 1080i 1080i to 1080p
Best supported
1080p
1080i 720p
Scenario 2 720p 1080p
1080i 1080i 1080i to to
720p 1080pWhat is effect
of conversion
1080i 720p 720p on quality?
720p 720p 1080p
Scenario 3 1080i to to
720p 1080p
720p
Scenario 4 720p 720p 720p 720p 1080p
to Best quality
1080p
Deinterlacing Rescaling
converter converter
25 VRT-medialab: onderzoek en innovatie

Broadcast production chain –
Points of attention for 720p50 workflows
Motion compensation algorithm
Today’s content is mostly 1080i
EICTA label - HDTV
2 9
1 Set-top-box
Cross-convertor Cross-convertor
EBU Joint Taskforce on
8 Timing & Synchronisation
Real-time infrastructure
p50 timing
Post production
Studio camera Support (legacy systems)
Surround sound
5 7 Dolby E compatibility with p50
MXF 6 TC support (legacy formats)
3 4 Full tested support (essence, TC, …)
Light sensitivity, S/N ratio File-based production environment
Central MAM
End-to-end workflow should be carefully designed !
26 VRT-medialab: onderzoek en innovatie

EBU/SMPTE Joint Taskforce on
Time Labelling and Synchronization
twofold goal
! new time labelling method allowing higher frame rates (50 fps
and higher)
! new synchronization mechanism
! current standards (SMPTE 12M, colour black) are 30 years old
=> workable but limited
status
! Request For Technology (RFT) – February 2008
– responses from Sony, Harris, TGV, …
! draft Request For Standardisation (RFS)
– time-related labeling
27 VRT-medialab: onderzoek en innovatie

Scenarios for 1080p25
psf = progressive with segmented frames
! frames are segmented into two interlaced fields with equal time
index => compatible with 1080i distribution
! easy deinterlacer (simple “weave”)
Acquisition Distribution Full HD
Production
Display
1080p25 1080psf 1080psf
Scenario 1 1080p25 1080p25 1080p25 to 1080p
to (1080i)
1080psf 1080p
1080p25 720p
Scenario 2 720p 1080p
1080p25 1080p25 1080p25 to to
720p 1080p
Rescaling Rescaling Simple
converter converter reordening
with loss of
resolution
28 VRT-medialab: onderzoek en innovatie

What about 1080p50?
no one already broadcasts in 1080p50
! but some have plans …
1080p50 in production
! not around the corner, but plausible within 3-5 years
! very good master for both 720p50 and 1080i25
! better quality (less artifacts due to smaller block sizes and
higher coding efficiency)
1080p50 in distribution
! commercial factors will likely be the driving factor
29 VRT-medialab: onderzoek en innovatie

Compression
formats for HD

Basics of video coding
What is video compression?
! Video compression is a technique that reduces the bit rate to
represent video images by exploiting signal redundancies while
preserving the original quality as much as possible
basic types of redundancies
! statistical redundancies
– spatial and temporal interpixel redundancies
– coding redundancy
! psycho-visual redundancies
– human visual system is less sensitive to higher frequencies
and color
31 VRT-medialab: onderzoek en innovatie

Common techniques for video
compression
rate control
CBR/VBR
lossless
vs lossy
input source re- entropy compressed
video encoder quantization encoding video
subsampling prediction transform
horizontal Intra-frame DCT
subsampling
color Interframe wavelet
subsampling
32 VRT-medialab: onderzoek en innovatie

Color subsampling
rate control
CBR/VBR
lossless
vs lossy
input source entropy compressed
video quantization video
encoder encoding
subsampling prediction transform
horizontal Intra-frame DCT
subsampling
color
Interframe wavelet
color
subsampling
subsampling
33 VRT-medialab: onderzoek en innovatie

Typical compression ratios
Reference = HD-SDI 720p50/1080i25 (10 bit)
Coding technique Factor Video bit rate
HD-SDI (4:2:2 subsampling) 1 1,5 Gbps
(net \" 1 Gbps)
Lossless encoding 1/2 500 Mbps
Intra-frame compression vs 1/5-1/20 50-200 Mbps
Interframe compression 1/20-1/30 35-50 Mbps
horizontal subsampling 3/4
color subsampling 4:2:0 3/4
Quantization = 8 bits 4/5
34 VRT-medialab: onderzoek en innovatie

Mainstream HD production
platform – codec dilemmas
horizontal subsampling?
color subsampling: 4:2:2 or 4:2:0?
8 or 10 bit quantization?
intra-frame or interframe coding?
scalable or single-layer compression?
single or multiple compression formats?
required bit rate?
720p or 1080i?
35 VRT-medialab: onderzoek en innovatie

Legacy HD production formats
Compression Bit rate (Mbps) Subsampling Intra/interframe
algorithm coding
Quantization
HDCAM SR 440 4:2:2 or 4:4:4 Intra
10 bit
HDCAM 140 (*) 4:2:2; 8 bit Intra
Horizontal
subsampling
DVCProHD 100 4:2:2; 8 bit Intra
Horizontal
subsampling
XDCAM HD 35 35 4:2:0; 8 bit Inter
Horizontal
subsampling
HDV 20/25 4:2:0; 8 bit Inter
Horizontal
subsampling
(*): net video rate = 117Mbps
36 VRT-medialab: onderzoek en innovatie

Advanced HD compression formats for
mainstream production
Camera formats (optimized for storage and power
consumption)
MPEG-2 Long-GOP
4:2:2, 8 Bit, data rate 50 Mbps, VBR
XDCAM HD
AVC Intra
4:2:2/4:2:0, 10/8 Bit, data rate 100 / 50 Mbps, CBR
P2 HD
DNxHD
4:2:2, 8 Bit, Data rate 120 Mbit/s (185 Mbps), CBR
EditcamHD
JPEG2000
4:2:2, 10 Bit, Data rate 100 / 75 / 50
Infinity Mbit/s
37 VRT-medialab: onderzoek en innovatie

Advanced HD compression formats for
mainstream production
post-production formats (optimized for editing)
DNxHD \" ProRes
Standard SMPTE VC-3 Standard ---
Quantisation [Bit] 8 / 10 Bit Quantisation [Bit] 8 / 10 Bit
Colour sub-sampling 4:2:2 Colour sub-sampling 4:2:2
Net video data rate 185 / 120 Net video data rate 185 / 120
[Mbit/s] [Mbit/s]
“intermediate codecs”
38 VRT-medialab: onderzoek en innovatie

MPEG-2 long GOP
aka XDCAM HD422
advantages
! proven and optimized codec
! same bit rate as for Standard Definition (50Mbps)
! in general: good quality for first generation
! early support for end-to-end broadcast chain
disadvantages
! interframe is less suited for editing purposes
– smart rendering techniques necessary
– fade-to-black, dissolve effects
– artefacts affect multiple frames
! quality loss in multi-generation scenarios
successor = H.264 long GOP?
39 VRT-medialab: onderzoek en innovatie

H.264 - AVC Intra
H.264 = MPEG-4 AVC = MPEG-4 Part 10
! joint project of ITU and MPEG
addition of High Profiles for HD resolution
! FRExt (Fidelity Range Extension)
– higher bit depths (10), 4:2:2/4:4:4 colour formats
! High 10 Intra (50 Mbps) and High 4:2:2 Intra (100 Mbps)
advanced compression tools
! Intra prediction: use of correlation between adjacent pixels
! entropy encoding improvement
– CABAC: Context Adaptive Binary Arithmetic Code
point of attention
! higher computational complexity
40 VRT-medialab: onderzoek en innovatie

JPEG 2000
available bit rates: 50/75/100 Mbps
advantages
! scalable codec: no need for proxy transcoding
! wavelet-based coding: no blocking artefacts
! industry standard for Digital Cinema
! low latency
! license free
disadvantages
! not widely supported in end-to-end broadcast chain at this
moment
41 VRT-medialab: onderzoek en innovatie

Dirac codec family
developed by BBC R&D (2000-now)
! license-free, open source/well documented
wavelet-based codec
wide range of applications
! Dirac (long GOP): low bit rate distribution (mobile, Web)
! Dirac Pro (intra-frame): low latency, high-quality production
(HD, DC, Super Hi-Vision)
– SMPTE VC-2
– transport mapping to SDI (720p50/1080i25) and HD-SDI
(1080p50)
! high flexibility: lossy/lossless, variable bit depths and colour
sample schemes, …
not yet supported, less mature than other HD codecs
42 VRT-medialab: onderzoek en innovatie

Scalable video coding (SVC)
extension of H.264
! recently standardized (2007)
! base layer (H.264) + enhancement layers
scalability options
! spatial, temporal, SNR/quality, combined
possible applications
! alternative for simulcast: SD/HD
! HD TV distribution for 720p@50fps or 1080i@25fps with
1080p@50fps enhancement
! broadcast production for different resolutions
targeted profiles for broadcasting
43 VRT-medialab: onderzoek en innovatie

Scalable video coding (SVC)
extension of H.264
! recently standardized (2007)
! base layer (H.264) + enhancement layers
scalability options
! spatial, temporal, SNR/quality, combined
possible applications
! alternative for simulcast: SD/HD
! HD TV distribution for 720p@50fps or 1080i@25fps with
1080p@50fps enhancement
! broadcast production for different resolutions
targeted profiles for broadcasting
44 VRT-medialab: onderzoek en innovatie

Choice of HD compression
format
key parameters
! quality (desired, floor level)
! storage and bandwidth requirements
! workflow
– recording medium
– editing requirements: layers, editing easiness, …
– support by different systems in HD production architecture
– end-to-end performance (transcoding, rendering, …)
! when?
45 VRT-medialab: onderzoek en innovatie

Mainstream HD production
platform – codec dilemmas
horizontal subsampling? no
color subsampling: 4:2:2 or 4:2:0? 4:2:2
8 or 10 bit quantization? 8/10
intra-frame or interframe coding?
scalable or single-layer compression?
single or multiple compression formats?
required bit rate? 50/100 Mbps
720p or 1080i?
46 VRT-medialab: onderzoek en innovatie