The document discusses image compression fundamentals including different types of data redundancy that can be exploited for compression. It provides examples to illustrate coding redundancy and how Huffman coding assigns variable length codes to symbols based on their probabilities to reduce this redundancy. Specifically, it shows how Huffman coding assigns shorter codes to more probable symbols, resulting in an average code length shorter than a fixed length code and thus higher compression.

1. FOLLOWING SLIDES ARE PRESENTED TO YOU
BY
CONCEPTSINDEPTHANDEASY !
HAPPY LEARNING : )
https://www.youtube.com/channel/UCJPZjYLPApIZokATpjj5HZg

2. IMAGE
COMPRESSIO
N

3. FUNDAMENTALS
 Data compression: it refers to the process of reducing the
amount of data required to represent a given quantity of
information.
Data vs information
 Data and information are not synonymous terms!
 Data is the means by which information is conveyed.

4. DATA VS INFORMATION (CONT’D)
 The same information can be represented by different amount
of data – for example:
Example 1: your brother, rohit, will meet you at IGI airport in new
delhi at 5 minutes past 6:00 pm tomorrow night
Example 2: your brother will meet you at IGI airport at 5 minutes
past 6:00 pm tomorrow night
Example 3: rohit will meet you at IGI at 6:00 pm tomorrow night

5. DATA ≠ INFORMATION
 Data compression aims to reduce the amount of data
while preserving as much information as possible.
 Because various amount of data can be used to
represent the same amount of information,
representations that contain irrelevant or repeated
information are said to contain redundant data.

6. INTRODUCTION
To understand the need for image compression, consider the following
scenario:
Person ABC wish to downloaded the x-men movie in 720𝑝 (1280 × 720)
format. The movie is of 2 hours duration and was recorded at 60 𝑓𝑝𝑠 (frames
per second).
1. Calculate the amount of storage it would require to store this movie?
2. How many dual layer blu-ray disk would be required to store this movie??
3. How much time would it take to transmit this video over broadband
connection having bandwidth capacity of 100 Mbps??

7. INTRODUCTION
A VIDEO IS A SEQUENCE OF VIDEO FRAMES WHERE EACH FRAME IS A FULL
COLOR STILL IMAGE.
STORAGE REQUIREMENT OF 1SEC OF VIDEO:
1 FRAME IS ONE RGB IMAGE  3 B
VIDEO FRAME  1280 X 720 X 3 B
60FPS × 1280 × 720 PPF × 3B ≈ 158.2 MB
FPS – FRAMES PER SECOND,
PPF – PIXELS PER FRAME,
B – BYTES PER PIXEL
STORAGE REQUIREMENT OF COMPLETE 2 HOUR MOVIE:
158.2 × 60 × 60 × 2 ≈ 1113GB

8. INTRODUCTION
Number of blu-ray disks required:
1113/50 = 23
Time required to transmit complete 2 hour movie:
100 Mbps  100 Mb in 1 sec
100 x 1024 x 1024 bits  1 sec
1113 x 1024 x 1024 x 1024 x 8 bits 
1113×1024×1024×1024×8
100×1024x1024
≈ 25.33 hours

9. This explains the power and need of image compression.
We need to do Image Compression, so that
• No. of discs rq  less (STORAGE)
• Transmission time rq  less

10. INTRODUCTION
Web page images & high-resolution digital camera photos are
also compressed to save storage space & reduce transmission time.
In addition to these applications, image compression plays an
important role in many other areas, including
Tele-video conferencing,
Remote sensing,
Document and medical imaging,
Fax transmission

11. INTRODUCTION

12. INTRODUCTION
 Image compression: it is the art & science of reducing
the amount of data required to represent an image.
 The number of images that are compressed and
decompressed daily is staggering, and the compressions
and decompressions themselves are virtually invisible
to the user.
 Anyone who owns a digital camera, surfs the web, or
watches the latest hollywood movies (on dvds or yify
torrents) benefits from the compression algorithms

13. DATA VS INFORMATION EXAMPLE
 In terms of an image, information corresponds to the object captured in the
image, their shape, their color etc.
 The different ways to represent this information can be either:
Through pixel intensities (spatial domain) (basic), or
Through extracting features of that image (higher level of
information e.g. edge image.. Edge out of prev image), or
Through simply noting the frequencies of the intensity values
existing in the image along with their location (frequency domain),
etc.

14. COMPRESSION RATIO
compression
Let 𝑛1 and 𝑛2 denote the number of bits in
two representations of the same information

15. COMPRESSION RATIO
RELATIVE DATA REDUNDANCY OF THE
REPRESENTATION WITH 𝑛1 BITS
𝑅 = 1 −
1
𝐶
COMPRESSION RATIO
𝐶 =
𝑛1
𝑛2

16. COMPRESSION RATIO
 Image compression involves reducing the size of image data files,
while retaining necessary information
 Retaining necessary information depends upon the application
 The reduced file created by the compression process is called the
compressed file and is used to reconstruct the image, resulting in
the decompressed image
 The original image, before any compression is performed, is called
the uncompressed image file
 The ratio of the original, uncompressed image file and the
compressed file is referred to as the compression ratio

17. TYPES OF DATA REDUNDANCY
 Three principal types of data redundancies that can be
identified and exploited in digital images
1. Coding redundancy
2. Spatial or temporal (interpixel) redundancy
3. Psychovisual redundancy (irrelevant information)
 Data compression attempts to reduce one or more of
these redundancy types.

18. CODING REDUNDANCY
The 8-bit codes that are used
to represent the intensities in
most 2-D intensity arrays
contain more bits than are
needed to represent the
intensities.
Because, we go for uniform coding,
in general.
We can solve the problem by variable
length coding.

19. SPATIAL OR TEMPORAL (INTERPIXEL)
REDUNDANCY
Related to 2D Sxy (image)
Consider the following image of size 256 × 256. In the corresponding 2-D
image:
256, 255
256, 0
256, 1
Interpixel redundancy implies that pixel values are correlated
(i.e., A pixel value can be reasonably predicted by its neighbors).
Spatial
Related to 3D (videos)
Tempora
l
Y
X t (Z)
frame is moving w.r.t time
(t)
refresh rate = 60 fps
At same coordinate (x,y)
and diff times say t1, t2,
t3,…. that intensity remains
same
t1 t2

20. PSYCHOVISUAL REDUNDANCY
(irrelevant information)
Most images contain information that is
ignored by the human visual system and/or
irrelevant to the intended use of the image. It
is redundant in the sense that it is not used.

21. CODING - DEFINITIONS
 Code: a list of symbols (letters, numbers, bits etc.)
 Code word: a sequence of symbols used to represent some
information (e.g., Gray levels).
 Code word length: number of symbols in a code word.
CODE CODE WORD

22. CODING REDUNDANCY
 𝑟𝑘  Input Intensity Value e.g. 0 – 255 for grayscale image
 𝑙 𝑟𝑘  No. of bits used to represent 𝑟𝑘
 Then average no of bits required to represent each pixel is
𝐿𝑎𝑣𝑔 =
𝑘=0
𝐿−1
𝑙(𝑟𝑘) 𝑝𝑟(𝑟𝑘)
Say, uniform width str. (all pixels are using 8 bits to represent intensity)
 equal length code (fixed length code)
 m-bit fixed length code (8 bit fixed length code in our case)

23. CODING REDUNDANCY
 In this case, 𝑙 𝑟𝑘 = 8 [each rq. 8 bits in fixed length]
𝐿𝑎𝑣𝑔 = 8
𝑘=0
𝐿−1
𝑝𝑟(𝑟𝑘) = 8 ∗ 1 = 8
 Coding redundancy tries to reduce 𝐿𝑎𝑣𝑔
 Thus, total no. of bits required to represent an 𝑀 × 𝑁 image is 𝑀𝑁𝐿𝑎𝑣𝑔
 The code can be an equal length code, or variable length code.

24. CODING REDUNDANCY
Code 1  equal length code
Code 2  variable length
code
(obt by Huffman Coding)
Min bits  higher
probability

25. CODING REDUNDANCY
 For code 1, 𝐿𝑎𝑣𝑔 = 8
 On the other hand, using code 2, the average length of the
encoded pixels is
𝐿𝑎𝑣𝑔 = 0.25 2 + 0.47 1 + 0.25 3 + 0.03 3 = 1.81 bits
 The resulting compression and corresponding relative
redundancy are
𝐶 =
256 × 256 × 8
256 × 256 × 1.81
≈ 4.42
𝑅 = 1 −
1
4.42
= 0.774
Thus, 77.4% of the data
in the original 8-bit 2-D
intensity array is
redundant.

26. CODING REDUNDANCY
 Coding redundancy is present when the codes assigned to
intensity values do not take full advantage of their
probabilities of occurrence.
 The natural consequence is that, for most images, certain
intensities are more probable than others.
 A natural binary encoding assigns the same number of bits
to both the most and least probable values, failing to
minimize and resulting in coding redundancy.

27. MEASURING INFORMATION
A key question in image compression is:
• “What is the minimum amount of data that is sufficient to describe completely an image without
loss of information?”
• How do we measure the information content of an image?
• Lavg
• Entropy
• Measure of information
• Measure of amount of uncertainty
• More uncertainty
• More information
• More entropy

28. MEASURING INFORMATION (CONT’D)
• Information generation is a probabilistic process.
• A random event E with probability 𝑃(𝐸) contains:
𝐼 𝐸 = log
1
𝑃(𝐸)
= − log 𝑃 𝐸 units of information
Note: 𝐼(𝐸) = 0 when 𝑃(𝐸) = 1

29. HOW MUCH INFORMATION DOES A PIXEL CONTAIN?
• Suppose that gray level values are generated by a random process, then
𝑟𝑘 contains:
𝐼 𝑟𝑘 = − log 𝑃 𝑟𝑘 units of information

30. HOW MUCH INFORMATION DOES AN IMAGE
CONTAIN?
Average information content of an image:
units/pixel
(e.g., bits/pixel)
using
Entropy:
It is not possible to code the intensity values with fewer
than 𝐻 bits/pixel.

31. EXAMPLE OF COIN
• CASE 1
• Entropy = - (1/2) * log(1/2) - (1/2) * log(1/2)
= 1
• CASE 2
• Entropy = - (1) * log(1)
= 0
COIN
H T
H H
CASE 1
CASE 2
NOTE
Take log base
2

32. ENTROPY
• This defines the average amount of information
obtained by observing a single source output.
• As its magnitude increases, more uncertainty and thus
more information is associated with the source.
• When the probability of the occurrence of a symbol is
less, we require more information about the symbol.

33. REDUNDANCY - REVISITED
• REDUNDANCY:
where:
Note: if 𝐿𝑎𝑣𝑔 = 𝐻, then 𝑅 = 0 (no redundancy)
𝑅 =
𝐿𝑎𝑣𝑔 − 𝐻
𝐿𝑎𝑣𝑔

34. ENTROPY ESTIMATION EXAMPLE
Image

35. COMPRESSION
METHODS

36. HUFFMAN CODING
• A measure to reduce coding redundancy
• Most popular coding redundancy technique
• Variable length code
• Min length code is assigned to one with highest probability

37. Symbols
(like intensity
levels)
Probabilities
(sorted)
Source Reduction (do till two values are left)
(Maintain in sorted order here as well)
1 2 3 4
a2 0.4 0.4 0.4 0.4 0.6
a6 0.3 0.3 0.3 0.3 0.4
a1 0.1 0.1 0.2 0.3
a4 0.1 0.1 0.1
a3 0.06 0.1
a5 0.04
HUFFMAN CODING
Say, Image size: 10 x 10 (5 bit image)
Frequency:
a2 = 40 a6 = 30 a1 = 10 a4 = 10 a3 = 6 a5 = 4
P(a2) = 40/100 = 0.4

38. Symbols
(like intensity
levels)
Probabilities
(sorted)
Source Reduction (do till two values are left)
(Maintain in sorted order here as well)
1 2 3 4
a2 0.4 1 0.4 0.4 0.4 0.6 0
a6 0.3 00 0.3 0.3 0.3 00 0.4 1
a1 0.1 011 0.1 0.2 010 0.3 01
a4 0.1 0100 0.1 0100 0.1 011
a3 0.06 01010 0.1 0101
a5 0.04 01011
Encoded String: 010100111100
Decoding : a3 a1a2 a2 a6
Parameters:
1. Average length of code
Lavg = 0.4 * 1 + 0.3 * 2 + 0.1 * 3 + 0.1 * 4 + 0.06 * 5 + 0.04 * 5 = 2.2 bits/symbol
2. Total no. of bits to be transmitted
10 * 10 * 2.2 = 220 bits
3. Entropy = 2.1396/symbol
4. How much you saved = 10 * 10 * 5 – 10 * 10 * 2.2 = 0.56 = 56%
10 * 10 * 5

39. HUFFMAN CODING
• Let an 8-level image has the gray-level distribution as
𝒓𝒌 𝒑 𝒓𝒌 𝑪𝒐𝒅𝒆 𝟏 𝑪𝒐𝒅𝒆 𝟐
0 0.19 000
1 0.25 001
2 0.21 010
3 0.16 011
4 0.08 100
5 0.06 101
6 0.03 110
7 0.02 111

40. WHY CAN’T WE ASSIGN RANDOM CODES?
• Decoding (examine string left to right)

41. HUFFMAN CODING : COMPLEX EXAMPLE
IMAGE
SYMBOL FREQUENCY PROBABILITY
(FREQUENCY/2
5)
0 2 0.08
1 4 0.16
2 3 0.12
3 6 0.24
4 3 0.12
5 2 0.08
6 2 0.08
7 3 0.12

43. SYMBOL CODE PROBAB
0 111 0.08
1 000 0.16
2 100 0.12
3 01 0.24
4 101 0.12
5 0010 0.08
6 0011 0.08
7 110 0.12
Lavg = 2.92 bits/symbol
Entropy (H) = 2.892878689 bits/symbol

44. IF YOU GENUINELY FIND THESE VIDEOS
HELPFUL
KINDLY LIKE, SHARE, SUBSCRIBE
HAPPY LEARNING : )
CONCEPTSINDEPTHANDEASY !
https://www.youtube.com/channel/UCJPZjYLPApIZokATpjj5HZg