Huffman coding is a lossless data compression algorithm that uses variable-length codes to represent characters. It assigns shorter codes to more frequent characters and longer codes to less frequent characters, resulting in an average compressed file size that is typically 20-90% smaller than the original file. The algorithm works by building a Huffman tree from the character frequencies, where each character is a leaf node and the frequency of that character determines its distance from the root. It then traverses the tree to assign binary codes to each character, where left branches are 0 and right branches are 1. The encoded file and Huffman tree are used together during decompression to reconstruct the original file losslessly.

1. Greedy Algorithms
(Huffman Coding)
Prepared by
Md. Shafiuzzaman
Lecturer, Dept. of CSE, JUST

2. David Huffman’s idea
• A Term paper at MIT
• Build the tree (code) bottom-up in a greedy fashion

3. Huffman Coding
• A technique to compress data effectively
• Usually between 20%-90% compression
• Lossless compression
• No information is lost
• When decompress, you get the original file
Original file
Compressed file
Huffman coding

4. Huffman Coding: Applications
• Saving space
• Store compressed files instead of original files
• Transmitting files or data
• Send compressed data to save transmission time and power
• Encryption and decryption
• Cannot read the compressed file without knowing the “key”
Original file
Compressed file
Huffman coding

5. Main Idea: Frequency-Based Encoding
• Assume in this file only 6 characters appear
• E, A, C, T, K, N
• The frequencies are:
Character Frequency
E 10,000
A 4,000
C 300
T 200
K 100
N 100
• Option I (No Compression)
– Each character = 1 Byte (8 bits)
– Total file size = 14,700 * 8 = 117,600 bits
• Option 2 (Fixed size compression)
– We have 6 characters, so we need
3 bits to encode them
– Total file size = 14,700 * 3 = 44,100 bits
Character Fixed Encoding
E 000
A 001
C 010
T 100
K 110
N 111

6. Main Idea: Frequency-Based Encoding
(Cont’d)
• Assume in this file only 6 characters appear
• E, A, C, T, K, N
• The frequencies are:
Character Frequency
E 10,000
A 4,000
C 300
T 200
K 100
N 100• Option 3 (Huffman compression)
– Variable-length compression
– Assign shorter codes to more frequent characters and longer
codes to less frequent characters
– Total file size:
Char. HuffmanEncoding
E 0
A 10
C 110
T 1110
K 11110
N 11111
(10,000 x 1) + (4,000 x 2) + (300 x 3) + (200 x 4) + (100 x
5) + (100 x 5) = 20,700 bits

7. Huffman Coding
• A variable-length coding for characters
• More frequent characters  shorter codes
• Less frequent characters  longer codes
• It is not like ASCII coding where all characters have the same coding
length (8 bits)
• Two main questions
• How to assign codes (Encoding process)?
• How to decode (from the compressed file, generate the original file)
(Decoding process)?

8. Decoding for fixed-length codes is much
easier
Character Fixed-length
Encoding
E 000
A 001
C 010
T 100
K 110
N 111
010001100110111000
010 001 100 110 111 000
Divide into 3’s
C A T K N E
Decode

9. Decoding for variable-length codes is not
that easy…
Character Variable-length
Encoding
E 0
A 00
C 001
… …
… …
… …
000001
It means what???
EEEC EAC AEC
Huffman encoding guarantees to avoid this
uncertainty …Always have a single decoding

10. Huffman Algorithm
• Step 1: Get Frequencies
• Scan the file to be compressed and count the occurrence of each character
• Sort the characters based on their frequency
• Step 2: Build Tree & Assign Codes
• Build a Huffman-code tree (binary tree)
• Traverse the tree to assign codes
• Step 3: Encode (Compress)
• Scan the file again and replace each character by its code
• Step 4: Decode (Decompress)
• Huffman tree is the key to decompress the file

11. Step 1: Get Frequencies
Eerie eyes seen near lake.
Char Freq. Char Freq. Char Freq.
E 1 y 1 k 1
e 8 s 2 . 1
r 2 n 2
i 1 a 2
space 4 l 1
Input File:

12. Step 2: Build Huffman Tree & Assign Codes
• It is a binary tree in which each character is a leaf node
• Initially each node is a separate root
• At each step
• Select two roots with smallest frequency and connect them to a new
parent (Break ties arbitrary) [The greedy choice]
• The parent will get the sum of frequencies of the two child nodes
• Repeat until you have one root

13. Example
Char Freq. Char Freq. Char Freq.
E 1 y 1 k 1
e 8 s 2 . 1
r 2 n 2
i 1 a 2
space 4 l 1
E
1
i
1
y
1
l
1
k
1
.
1
r
2
s
2
n
2
a
2
☐
4
e
8
Each char. has a
leaf node with its
frequency

14. Find the smallest two frequencies…Replace them with their parent
E
1
i
1
y
1
l
1
k
1
.
1
r
2
s
2
n
2
a
2
☐
4
e
8
E
1
i
1
2

15. E
1
i
1
y
1
l
1
k
1
.
1
r
2
s
2
n
2
a
2
☐
4
e
8
2
Find the smallest two frequencies…Replace them
with their parent
y
1
l
1
2

16. E
1
i
1
k
1
.
1
r
2
s
2
n
2
a
2
☐
4
e
8
2
y
1
l
1
2
Find the smallest two frequencies…Replace them
with their parent
k
1
.
1
2

17. E
1
i
1
r
2
s
2
n
2
a
2
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2
Find the smallest two frequencies…Replace them
with their parent
r
2
s
2
4

18. E
1
i
1
n
2
a
2
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
Find the smallest two frequencies…Replace them
with their parent
n
2
a
2
4

19. E
1
i
1
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
Find the smallest two frequencies…Replace them
with their parent
E
1
i
1
2
y
1
l
1
2
4

20. E
1
i
1
☐
4
e
82
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4 4
Find the smallest two frequencies…Replace them
with their parent
☐
4
k
1
.
1
2
6

21. E
1
i
1
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4 4 6
Find the smallest two frequencies…Replace them
with their parent
r
2
s
2
4
n
2
a
2
4
8

22. E
1
i
1
☐
4
e
82
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4
6 8
Find the smallest two frequencies…Replace them
with their parent
E
1
i
1
☐
4
2
y
1
l
1
2
k
1
.
1
2
4
6
10

23. E
1
i
1
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2r
2
s
2
4
n
2
a
2
4 4
6
8 10
Find the smallest two frequencies…Replace them
with their parent
e
8
r
2
s
2
4
n
2
a
2
4
8
16

24. E
1
i
1
☐
4
e
82
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4
6
8
10 16
Find the smallest two frequencies…Replace them
with their parent

25. E
1
i
1
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4
6
8
10
16
26
Now we have a single root…This is the Huffman Tree

26. Lets Analyze Huffman Tree
• All characters are at the leaf nodes
• The number at the root = # of characters in the file
• High-frequency chars (E.g., “e”) are near the root
• Low-frequency chars are far from the root
E
1
i
1
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4
6
8
10
16
26

27. Lets Assign Codes
• Traverse the tree
• Any left edge  add label 0
• As right edge  add label 1
• The code for each character is its root-to-leaf label sequence
E
1
i
1
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4
6
8
10
16
26

28. • Traverse the tree
• Any left edge  add label 0
• As right edge  add label 1
• The code for each character is its root-to-leaf label sequence
E
1
i
1
☐
4
e
8
2
y
1
l
1
2
k
1
.
1
2
r
2
s
2
4
n
2
a
2
4
4
6
8
10
16
26
0
1
0
0
0
0
0
0 0
1
1
11
1
1
1
10
001 1
Lets Assign Codes

29. • Traverse the tree
• Any left edge  add label 0
• As right edge  add label 1
• The code for each character is its root-to-leaf label sequence
Lets Assign Codes
Char Code
E 0000
i 0001
y 0010
l 0011
k 0100
. 0101
space☐ 011
e 10
r 1100
s 1101
n 1110
a 1111
Coding Table

30. Huffman Algorithm
• Step 1: Get Frequencies
• Scan the file to be compressed and count the occurrence of each character
• Sort the characters based on their frequency
• Step 2: Build Tree & Assign Codes
• Build a Huffman-code tree (binary tree)
• Traverse the tree to assign codes
• Step 3: Encode (Compress)
• Scan the file again and replace each character by its code
• Step 4: Decode (Decompress)
• Huffman tree is the key to decompess the file

31. Generate the
encoded file
Step 3: Encode (Compress) The File
Eerie eyes seen near lake.
Input File:
Char Code
E 0000
i 0001
y 0010
l 0011
k 0100
. 0101
space☐ 011
e 10
r 1100
s 1101
n 1110
a 1111
Coding Table
+
000010 1100 000110 ….
Notice that no code is prefix to any other code 
Ensures the decoding will be unique (Unlike Slide 8)

32. Step 4: Decode (Decompress)
• Must have the encoded file + the coding tree
• Scan the encoded file
• For each 0  move left in the tree
• For each 1  move right
• Until reach a leaf node  Emit that character and go back to the root
000010 1100 000110 ….
+
Eerie …
Generate the
original file

33. Huffman Algorithm
• Step 1: Get Frequencies
• Scan the file to be compressed and count the occurrence of each character
• Sort the characters based on their frequency
• Step 2: Build Tree & Assign Codes
• Build a Huffman-code tree (binary tree)
• Traverse the tree to assign codes
• Step 3: Encode (Compress)
• Scan the file again and replace each character by its code
• Step 4: Decode (Decompress)
• Huffman tree is the key to decompess the file

34. The Algorithm
• An appropriate data structure is a binary min-heap
• Rebuilding the heap is lgn and n-1 extractions are made, so the complexity is
O( nlgn)
• The encoding is NOT unique, other encoding may work just as well, but none
will work better

35. Lab Assignment
• Example Input: Huffman coding is a data compression algorithm.
• Output: