Rate distortion theory is a major branch of information theory that provides the theoretical foundations for lossy data compression schemes. It gives an analytical expression for how much compression can be achieved using lossy compression methods by establishing a tradeoff between the rate of compression and the resulting distortion. The theory aims to minimize the amount of distortion for the lowest possible compression rate given a source distribution and distortion measure. The rate distortion function specifies the lowest rate at which an output can be encoded while keeping distortion below a given level.

1. RATE DISTORTION THEORY
INFORMATION THEORY
1
GROUP NO:21 Batch:T5
1)Rutuja R. Pawar(2013BIT006)
2)Samruddhi S. Kanhere(2013BIT018)
3)Pranali P. Rasal(2013BIT026)

2. INTRODUCTION
 Major Branch Of Information Theory
 There are 2 types of compression schemes.
1. Lossless compression
2. Lossy compression
 Provides the theoretical foundations for lossy data compression schemes
 Gives an analytical expression for how much compression can be achieved using
lossy compression methods
Information Theory
2
25-Apr-16

3. FOUNDER
CLAUDE SHANNON
Information Theory
3
25-Apr-16

4. BLOCK DIAGRAM OF GENERIC
COMPRESSION SCHEME
4
 USER
SOURCE
SOURCE
ENCODER
SOURCE
DECODER
CHANNEL
25-Apr-16 Information Theory

5. RATE AND DISTORTION
 RATE
The number of bits to be stored or transmitted per data sample.
 Distortion
Measure of difference between original and the reconstructed data.
 Extreme Conditions
1. No information is transmitted
2. No distortion
RDT is nothing but the trade-off between rate and compression in lossy
compression scheme.
5
25-Apr-16 Information Theory

6. GOAL
MINIMIZE
AMOUNT
OF DISTORTION
LOWEST RATE
POSSIBLE
6
25-Apr-16 Information Theory

7. MEASURING CLOSENESS FOR
RECONSTRUCTED DATA
7
1) Humans:
• Accurate
• Not practical
• Not useful in mathematical design approaches
2) Quantitative Metrics:
• Mathematical design approach
• Not accurate
25-Apr-16
25-Apr-16 Information Theory

8. THEORY
 Given a source distribution and the distortion measure,
 What is the minimum expected distortion achievable at
particular rate?
OR
 What is the minimum rate required to achieve particular
distortion?
8
25-Apr-16 Information Theory

9. FINDING DISTORTION
1)Squared Measure
d(x,y) = (x-y)^2
2)Absolute Difference
d(x,y) = |x-y|
We use average measures for calculations.
1)Mean squared error
2)Avg. of absolute difference(Image compression algo.)
9
25-Apr-16 Information Theory

10. RATE DISTORTION FUNCTION
10
R(D) specifies the lowest rate at which the o/p of source can be encoded while keeping the distortion less than
or equal to D.
How to compute R(D)?
1) Computational approach
2) Find a lower bound for the average mutual information & then show that we can achieve this bound.
25-Apr-16 Information Theory

11. RATE DISTORTION FUNCTION
 For a given maximum average distortion D, the rate distortion function R(D*) is
lower bound for the transmission bit rate.
 R(D*) is measured in Bits.
 It is different for different channels.
11
25-Apr-16 Information Theory

12. THANK YOU!!!
12