Introduction To Video Compression

Introduction
to
Video Compression

Fast Forward Your Development www.dsp-ip.com

Compression
p
• Compression – Representing information by less
bit than the original information
• Lossless Compression – Original information
and compressed information are identical
identical.
example LZ, TAR and other compression
techniques.
techniques
• Lossy Compression – Compressed info is not the
same as uncompressed info E ample MP3
ncompressed info. Example: MP3,
JPEG etc
•LLossy compression is often MODEL B
i i ft Based d
Compression


Compression terms
p
• Encoder – Module which compress the
information
• Decoder – Module which decompress the
information
• CODEC – (en)CODer / DEcoder
• Channel – the medium which the information is
passed through for example ADSL line or disk

Channel Decoder
Encoder

Disk


Sampling
p g
• Digitization of analog signal
• Voice - Sampling in time only The number of
only.
samples per second is the Frequency – F. in
telephony,
telephony voice sampling frequency is 8KHz to
transfer 4KHz of sound.
•IImage - SSampling in 2D space - F F O thi
li i Fx, Fy. On this
screen the images are sampled 1024x768
• Vid - S
Video Sampling i 2D space and ti
li in D d time – F F
Fx, Fy,
Fn. On TV Fn is 25 or 30 fps (frames per second)


Quantization
• Lowering the number of bits needed to describe
an object
object.
• For example I scored 66 in the test an object with
100 steps need 7 description bits
bits.
• How is she looking ? 10 ! Needs only 5 bits for
description.
d i ti
• Could we lower the number of bits more? What
is th
i the minimum?
i i ?
• Quantization is not always linear - ADPCM


MODEL BASED COMPRESSION

Voice
i
Audio
Video


Model Base Transform - example
• Transforming the information from a
representation of similar importance
to all variables to a representation of
varying importance
• Th “poor sheriff” problem
The “ h iff” bl
▫ The sheriff needs to telegram the
dangers criminal profile t
d i i l fil to
headquarters.
▫ A telegram costs a cent for each
bit.
▫ Sending it bit by bit would finish
the poor sheriff salary.


Solution
• The sheriff and headquarters
agree on a b
base that would
h ld
describe images according to a
base known to both sides.
• The transformation base is:
▫ Mouth
▫ Nose
▫ Eyes
▫ Ears
• The values are the sizes of each
item quantized to millimeters
• Mad Max will be transmitted as:
8,14,12,17
8 14 12 17


Model Based Compression (Multimedia)

Pre- Model Based Quantize / Entropy
Processing Transform Prioritize Reorder Codingg


Human Visual System
• The human eye has two basic light receptors:
▫ Rods – Light Intensity receptors
▫ Cons – Colored light receptors
•


The Human Eye
• Rods Concentration >> Cons Concentration
• Green Discrimination << Red, Blue
Red
Discrimination
•LLow Frequency > High Frequency
F Hi h F


Model Based transformations
• RGB (3 equally quantized colors) ->
>
• YUV (Light Intensity + two color channels)
• Pi l based d
Pixel b d domain -> F
i > Frequency d domain
i


Speech coding
• In speech coding, the vocal tract is used as a
model:


Audio / Music Coding
• In general Audio Coding, the ear is used as a
model:
• Frequencies -> Frequency bands
• Masking and Temporal Masking are used


IMAGE COMPRESSION


What is a digital image?

• Audio PCM
▫ One 1-D array of
sample
• BMP Image
▫ Three 2-D arrays of
2D
numbers
representing Red,
Green and Blue
values


Image Compression? Why?

• Image size = 720*580
720 580
• 3 Image Layers RGB =720*580*3
• 8 Bits per pixel 720*580*3*8
720 580 3 8
= 10022400 bits
• L t of bit f one L
Lots f bits for Lena


Color matching
• How can we compare
colors so that the
content creators and
consumers know what
kno hat
they are seeing?
•MMany diff
different ways
t
including CIE
chromacity di
h it diagram


Video Color Transforms
• Largely derived from older analog methods of coding
color for TV. Luminance is separated from color
TV
information.
• YIQ is used to transmit TV signals in North America and
Japan.This coding also makes its way into VHS video
tape coding in these countries since video tape
technologies also use YIQ.
• In Europe, video tape uses the PAL or SECAM codings,
which are based on TV that uses a matrix transform
hi h b d h i f
called YUV.
• Finally digital video mostly uses a matrix transform
Finally,
called YCbCr that is closely related to YUV


YUV Separation


YUV Color Model

•YUV codes a luminance signal (for gamma-corrected
signals) equal to Y , the “luma".
•Chrominance refers to the difference between a color
and a reference white at the same luminance. (U and V)

The transform is:


RGB->YUV Color Transform
G
G
B B

Y
U

V

R
R


YIQ Color Model

YIQ is used in NTSC color TV broadcasting.
broadcasting
Again, gray pixels generate zero (I;Q)
chrominance signal.
g
I and Q are a rotated version of U and V .

The transform is:
Th t f i


YCbCr Color Model

1. The Rec. 601 standard for digital video uses
another color space YCbCr which closely
related to the YUV transform.
2. The YCbCr transform is used in JPEG image
compression and MPEG video compression.

For 8-bit coding:


Quantization
• Lowering the number of bits needed to describe
an object
object.
• For example I scored 66 in the test an object with
100 steps need 7 description bits
bits.
• How is she looking ? 10 ! Needs only 5 bits for
description.
d i ti
• Could we lower the number of bits more? What
is th
i the minimum?
i i ?
• How to describe an object with one bit?


Color based decimation
• Our eyes have better resolution and scaling for
luminance then for color
color.
• Compress color by using 4:2:0 method


Counting the bits
• How much can we save by color compression?
▫ 3*Image size in RGB 24 bit color representation
representation.
▫ 1 + 2*1/4 Image size in 4:2:0 YUV representation.
▫ Compression ratio is 2 !!
• Actual saving is bigger due to different Y and UV
quantization.
quantization


Linear Transform
• If the signal is formatted as a
g • Energy compaction p p y
gy p property:
vector, a linear transform can The transformed signal vector
be formulated as a matrix- has few, large coefficients and
vector product that transform many nearly zero small
the signal into a different coefficients. These few large
domain. coefficients can be encoded
• Examples: efficiently with few bits while
▫ K-L Transform retaining the majority of
▫ Discrete Fourier Transform energy of the original signal.
▫ Discrete cosine transform
▫ Discrete wavelet transform


Block-based Image Coding

• Block-based image
Block based • Advantages:
coding scheme: ▫ Parallel processing can
partitions the entire be applied to p
pp process
image into 8 by 8 or individual blocks in
parallel.
16 by 16 (or other
▫ Redundant information
size) blocks. in close proximity (like
• The coding algorithm cache)
is applied to
individual blocks
independently.


Transform - DCT
• The DCT transform the data from pixel intensity
to frequency intensity
intensity.
• Low frequency are important high frequency less

1 7 7 (2m  1)u (2n  1)v
• (You’ll F (u, v) cos even if you don’t remember the
 4 get launch
 u 0 v 0 16
cos
16
m  n  0;
f (m, n)   7 7
IDCT1f 
formula above) 1)u (2n  1)v
l b (2m)
 8 
F (u , v) cos cos 0  m, n  7; m  n  0.
 u 0 v 0 16 16


DCT Coefficients Quantization


AC Coefficients
• AC coefficients are first
weighted with a quantization
i h d ih i i 1 2 6 7 15 16 28 29

matrix: 3 5 8 14 17 27 30 43

C(i,j)/q(i,j)
C(i j)/q(i j) = Cq(i j)
(i,j) 4 9 13 18 26 31 42 44

Then quantized. 10 12 19 25 32 41 45 54

• Then they are scanned in a 11 20 24 33 40 46 53 55

zig-zag order into a 1D 21 23 34 39 47 52 56 61

sequence to be subject to AC 22 35 38 48 51 57 60 62
Huffman encoding.
ff di
36 37 49 50 58 59 63 64

• Question: Given a 8 by 8
array,
array how to convert it into a
Zig-Zag scan order
vector according to the zig-
zag scan order? What is the
g
algorithm?

DCT Basis Functions


DCT compression Example
Original Image


DCT 1 coefficient


DCT 6 coefficients


DCT 20 coefficient


JPEG Image Coding Algorithms

Q
Quantization DC
8x8 Matrix DC DPCM Huffman
block
DCT Q
Zig Zag AC
AC Scan Huffman
Code books

JPEG Encoding Process


Generalization of JPEG Coding

Transform Entropy
Color,
Color Frequency Quantize Reorder Coding

JPEG Encoding Process


Video Coding Basics


Video Coding
• Video coding is often implemented as encoding
a sequence of i
f images.Motion compensation
i i
is used to exploit temporal redundancy between
successive f
i frames.
• Examples: MPEG-I, MPEG-II, MPEG-IV,
H.263, H.263+, H264
• Existing video coding standards are based on
JPEG image compression as well as motion
compensation.


Video Coding Standardization
Scope
• Only restrictions on the Bitstream, Syntax, and
Decoder are standardized:
▫ Permits the optimization of encoding
▫ Permits complexity reduction
p y
▫ Provides no guarantees on quality


Video Encoding

Buffer control
Current
frame x(t) r Bit stream
+ DCT Q VLC Buffer

Q-1 This is a simplified block
diagram where the
encoding of intra coded
IDCT frames is not shown.

Xp(t): predicted ^ reconstructed residue
r(t):
frame
+
^(t)
x(t): reconstructed
t t d
Motion ^ current frame
x(t) x(t-1) Frame
Estimation &
Compensation Buffer
Motion vectors

Video Encoding
Color Frequency
Transform Buffer control
Transform

+ Q Reorder Entropy

Q-1 This is a simplified block
diagram where the
encoding of intra coded
Tf-1 frames is not shown.

Xp(t): predicted ^ reconstructed residue
r(t):
frame
+
^(t)
x(t): reconstructed
t t d
Motion ^ current frame
x(t) x(t-1) Frame
Estimation &
Compensation Buffer
Motion vectors

Forward Motion Estimation

1 2 3 4 1 2 4
3
5 6 7 8 5 7 8
6
9 10 11 12 9 11 12
10
0
13 15 16
13 14 15 16 14

Current frame constructed From
different parts of reference frame Reference frame


Video sequence : Tennis frame 0, 1

previous frame current frame

50 50

100 100

150 150

200 200

50 100 150 200 250 300 350 50 100 150 200 250 300 350


Frame Difference

Frame Difference :frame 0 and 1


What is motion estimation?
Motion Vector Field of frame 1
50

0

-50

-100

-150

-200

-250
0 50 100 150 200 250 300 350 400


What is motion compensation ?
Motion compensated frame

50

100

150

200

50 100 150 200 250 300 350


Motion Compensated Frame Difference
p

Motion Compensated Frame Difference :frame 0 and 1
Frame Difference :frame 0 and 1


Frame Types
• Three types of frames:
yp
▫ Intra (I): the frame is coded as if it is an image
▫ Predicted (P): predicted from an I or P frame
▫ Bi-directional (B): forward and backward predicted
from a pair of I or P frames.
• A typical f
i l frame arrangement i is:
I1 B1 B2 P1 B3 B4 P2 B5 B6 I2
• P1, P2 are b h f
both forward-predicted f
d di d from I1. B1, B2 are
interpolated from I1 and P1, B3, B4 are interpolated
from P1, P2, and B5, B6 are interpolated from P2, I2.
• New Coding standards added other frame types: SP,
SI, D


Macro-blocks and Blocks

Y(16x16) Cr (8x8)
RGB

Cb (8x8)

16x16x3


Introduction To Video Compression

More Related Content

What's hot

Viewers also liked

Similar to Introduction To Video Compression

Recently uploaded

Introduction To Video Compression