Battle of the Codecs In Digital Film and TV James Uren Independent Training Consultant, Senior Lecturer Broadcasting, Creative Director 3D Stories Optimisation and Compression Ravensbourne College of Design & Communication 24 November 2011
Demistifying Codecs The technologies of digital ﬁlm So many codecs... The technology behind compression The future... 2
Digital Images Deﬁned in Pixels Standard-Deﬁnition (SDTV) Europe: 720 x 576 pixels/frame. US/Japan: 720 x 480 pixels/frame High-Deﬁnition (HDTV) 1920 x 1080 pixels/frame (Also called CIF (Common Image Format)). 1280 x 720 pixels/frame Digital Film and D-cinema Full-aperture 2K is 2048 x 1536 (gives a 1.33:1 (4:3) image) Academy 2K is 1828 x 1080 (For a 1.85:1 image) Full-aperture 4K is 4096 x 3072 pixels/frame 4
Digital Image Pixel Resolutions720 x 576 1280 x 720 1920 x 1080 2048 x 1536 4096 x 3072 5
Film?• What resolution is ﬁlm? Grain...
Film?• So resolution unknown - is 4K enough?• So in ﬁlm we have: • Digital Intermediate (DI) for post-production • Digital cinema projection• Digital Capture - the last piece of the puzzle.
HD Capture CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Silicon) sensor options: CMOS now lower noise than CCDs CMOS allows A/D & processing on-chip CMOS can have >100 million transistors... ARRIFLEX D21 Examples: The ARRIFLEX D21 uses a 35mm 6 megapixel (2.9K) CMOS imager The SI 2K has a 2/3” 2.5 megapixel (2K) CMOS imager SI-2K 10
Inter-Frame Compression• MPEG is an inter-frame codec and forms the backbone of many codecs (e.g. H.264)• Exploits commonality between frames• Sends one full ‘I Frame’ then predicts through the Group of Pictures (GOP)
Intra-Frame Compression• Compressing each frame independently• Also known as ‘I Frame only’• Examples include ProRes, DNxHD• Best for ofﬂine editing as no GOP reconstruction required
So many codecs...
Terminology• Resolution (e.g. 4K)• Anamorphic pixels• Raw data (e.g. RAW / R3D)• Bit depth (e.g. 12-bit)• Chroma sampling (e.g. 4:4:4)• Bandwidth / data rate / bit rate (e.g. 100Mbit/s)• Frame rate / scan type (e.g. 50i / 25PsF*)• Audio sampling frequency (e.g. 48kHz)• Wrapper
Example Codec• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz
Example• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz Codec
Example• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz Frame rate and scan type
Example• Apple ProRes 422 (HQ) 1440x1080 25p 48kHz Audio sampling frequency
Interlaced or Progressive? Interlace Each image (frame) comprises two interlaced ﬁelds Any movement between the ﬁelds causes artefacts Introduced for TV because it halves transmission bandwidth Used in 50i (Europe), 59.94i (USA/Japan) SDTV/HDTV Progressive Each image is complete & contains all the pixels Can produce judder motion artefact at 24p or 25p 50p gives best motion portrayal Higher frame rates would be better still (See BBC research into 150 fps) 6
Interlaced or Progressive? Not straightforward: Imagers do not scan the image, neither do ﬂat panel displays Interlace introduces artefacts and reduces vertical resolution What the experts say: “All data reduction systems (such as MPEG) give better results at lower bitrates with progressive sources” “Progressive scanning gives improved motion portrayal” “1080i offers better static resolution, 720p better motion portrayal” (EBU The Review No.301) 7
Picture ImpairmentsToo much compression can introduce impairments: Blocking Mosquito Noise / Wings Contouring Smearing Freezes Breathing 42
But...• ...this is only if you compress to much.• ProRes 4444 at 250Mbit/s is indistinguishable from uncompressed...• ...and much less data
D-CinemaThe DCI (Digital Cinema Initiatives) was set up in 2002Aim was to set standards for a viable D-Cinema systemMembers include Disney, Fox, MGM, Paramount, Sony Pictures Entertainment (Columbia), Universal & Warner Bros.Adopted JPEG2000 compression 36
D-CinemaJPEG2000 uses wavelet compressionJPEG2000 does not divide the picture into discrete blocksD-cinema uses a mild level of compression to avoid artefacts (~250Mbit/s) 37
JPEG2000 Examples 39
The Future• Requirements on compression reduces with Moore’s Law• But this is countered by increasing resolutions and frame rates• Compression is here to stay, but ofﬂine and online continue to blur
Round-Up• A complete digital pipeline from capture to display• Don’t fear compression• Great pictures mean a high bitrate end-to-end, but too high and your pipeline starts to get expensive... But not for long.
Battle of the Codecs In Digital Film and TV James Uren Independent Training Consultant, Senior Lecturer Broadcasting, Creative Director 3D Stories Ravensbourne College of Design & Communication