Video Compression Demystified Intro to MPEG, WMV, H.264, AVI

Why compression? Bandwidth constraint Space constraint I don’t know what else?

Bandwidth constraint

Space constraint

I don’t know what else?

Basic idea Correlation (Entropy) Space, Time, Bits Human eye models Low-pass filter Brain ‘fills’ in for sharpness Frugality

Correlation (Entropy)

Space, Time, Bits

Human eye models

Low-pass filter

Brain ‘fills’ in for sharpness

Frugality

Pixels and frames Each picture is a set of pixels in two dimensions, third being time. Finite sharpness, resolution issue. Each group of 8 pixels = block 2x2 blocks = Macroblock

Each picture is a set of pixels in two dimensions, third being time.

Finite sharpness, resolution issue.

Each group of 8 pixels = block

2x2 blocks = Macroblock

Correlation in space Space is ‘gravitated’ towards being continuous. How do you exploit it? DCT inside a block AC-DC prediction across blocks

Space is ‘gravitated’ towards being continuous.

How do you exploit it?

DCT inside a block

AC-DC prediction across blocks

Correlation in Time The WHOLE image does not change. Do NOT repeat information. Motion estimation Locate where my marker is moving towards. Send only the coded difference. Bi-predict pictures.

The WHOLE image does not change.

Do NOT repeat information.

Motion estimation

Locate where my marker is moving towards.

Send only the coded difference.

Bi-predict pictures.

Correlation in bits Run-level-last coding for zeroes. Subsequent bits still have correlation in them. Huffman coding (VLC) Arithmetic coding (CABAC)

Run-level-last coding for zeroes.

Subsequent bits still have correlation in them.

Huffman coding (VLC)

Arithmetic coding (CABAC)

Human eye Eye is not responsive to every detail Quantization Smoothening filters Eye is more responsive to brightness Less effort of chrominance Brain fills in crucial details.

Eye is not responsive to every detail

Quantization

Smoothening filters

Eye is more responsive to brightness

Less effort of chrominance

Brain fills in crucial details.

DCT and quantization Random in Time domain is high freq. Spatial redundancy reduced to a few significant values. Zigzag coding + quantization Focus on lower freq and suppress high freq.

Random in Time domain is high freq.

Spatial redundancy reduced to a few significant values.

Zigzag coding + quantization

Focus on lower freq and suppress high freq.

Motion Estimation Find the best match around – ME Diamond, Hierarchical, Logarithmic Generally over a block. Half-pel and quarter pel motion estimation using filters. Better match in sub-pel level.

Find the best match around – ME

Diamond, Hierarchical, Logarithmic

Generally over a block.

Half-pel and quarter pel motion estimation using filters.

Better match in sub-pel level.

RLL and Huffman Long sequence of zeroes helps reducing to smaller chunks of data. This chunk also has a probabilistic models. Exploit probability of symbols and be frugal in bits. Context adaptive arithmetic coding.

Long sequence of zeroes helps reducing to smaller chunks of data.

This chunk also has a probabilistic models.

Exploit probability of symbols and be frugal in bits.

Context adaptive arithmetic coding.

MPEG MPEG-1: DCT Pixel level ME AC-DC prediction P pictures Huffman coding

MPEG-1:

DCT

Pixel level ME

AC-DC prediction

P pictures

Huffman coding

MPEG-4 DCT Quarter-pel ME B pictures AC-DC prediction Data partitioning Objects – layers, groups, streams

DCT

Quarter-pel ME

B pictures

AC-DC prediction

Data partitioning

Objects – layers, groups, streams

H.264 Integer transform on 4x4 block Quarter pel ME on 4x4 In-loop de-blocking Multiple reference pictures CABAC – 2 times frugal Bit-plane coding

Integer transform on 4x4 block

Quarter pel ME on 4x4

In-loop de-blocking

Multiple reference pictures

CABAC – 2 times frugal

Bit-plane coding

AVI Audio Video Interleave format One frame of video with equivalent audio frame(s) Necessary information Display info Shit

Audio Video Interleave format

One frame of video with equivalent audio frame(s)

Necessary information

Display info

Shit