Huffman Student

1,218 views

Published on

Published in: Technology
1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
1,218
On SlideShare
0
From Embeds
0
Number of Embeds
77
Actions
Shares
0
Downloads
51
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

Huffman Student

  1. 1. CSE 326 Huffman coding Richard Anderson
  2. 2. Coding theory <ul><li>Conversion, Encryption, Compression </li></ul><ul><li>Binary coding </li></ul><ul><li>Variable length coding </li></ul>Code examples 000,001,010,011,100,101 1,01,001,0001,00001,000001 00,010,011,100,11,101 A B C D E F
  3. 3. Decode the following 11010010010101011 100100101010 Ambiguous Prefix code E 0 T 11 N 100 I 1010 S 1011 E 0 T 10 N 100 I 0111 S 1010
  4. 4. Prefix code <ul><li>No prefix of a codeword is a codeword </li></ul><ul><li>Uniquely decodable </li></ul>A 00 1 00 B 010 01 10 C 011 001 11 D 100 0001 0001 E 11 00001 11000 F 101 000001 101
  5. 5. Prefix codes and binary trees <ul><li>Tree representation of prefix codes </li></ul>A 00 B 010 C 0110 D 0111 E 10 F 11
  6. 6. Construct the tree for the following code E 0 T 11 N 100 I 1010 S 1011
  7. 7. Minimum length code <ul><li>Average cost </li></ul><ul><li>Average leaf depth </li></ul><ul><li>Huffman tree – tree with minimum weighted path length </li></ul><ul><li>C(T) – weighted path length </li></ul>
  8. 8. Compute average leaf depth A 00 1/4 B 010 1/8 C 0110 1/16 D 0111 1/16 E 1 1/2
  9. 9. Huffman code algorithm <ul><li>Derivation </li></ul><ul><ul><li>Two rarest items will have the longest codewords </li></ul></ul><ul><ul><li>Codewords for rarest items differ only in the last bit </li></ul></ul><ul><li>Idea: suppose the weights are with and the smallest weights </li></ul><ul><ul><li>Start with an optimal code for and </li></ul></ul><ul><ul><li>Extend the codeword for to get codewords for and </li></ul></ul>
  10. 10. Huffman code H = new Heap() for each w i T = new Tree(w i ) H.Insert(T) while H.Size() > 1 T 1 = H.DeleteMin() T 2 = H.DeleteMin() T 3 = Merge(T 1 , T 2 ) H.Insert(T 3 )
  11. 11. Example: Weights 4, 5, 6, 7, 11, 14, 21 4 5 6 7 11 14 21
  12. 12. Draw a Huffman tree for the following data values and show internal weights: 3, 5, 9, 14, 16, 35
  13. 13. Correctness proof <ul><li>The most amazing induction proof </li></ul><ul><li>Induction on the number of code words </li></ul><ul><li>The Huffman algorithm finds an optimal code for n = 1 </li></ul><ul><li>Suppose that the Huffman algorithm finds an optimal code for codes size n, now consider a code of size n + 1 . . . </li></ul>
  14. 14. Key lemma <ul><li>Given a tree T, we can find a tree T’, with the two minimum cost leaves as siblings, and C(T’) <= C(T) </li></ul>
  15. 15. Modify the following tree to reduce the WPL 29 10 19 6 4 13 6 10 3 5 5
  16. 16. Finish the induction proof <ul><li>T – Tree constructed by Huffman </li></ul><ul><li>X – Any code tree </li></ul><ul><li>Show C(T) <= C(X) </li></ul><ul><li>T’ and X’ – Trees from the lemma </li></ul><ul><li>C(T’) = C(T) </li></ul><ul><li>C(X’) <= C(X) </li></ul><ul><li>T’’ and X’’ – Trees with minimum cost leaves x and y removed </li></ul>
  17. 17. X : Any tree, X’: – modified, X’’ : Two smallest leaves removed <ul><li>C(X’’) = C(X’) – x – y </li></ul><ul><li>C(T’’) = C(T’) – x – y </li></ul><ul><li>C(T’’) <= C(X’’) </li></ul><ul><li>C(T) = C(T’) = C(T’’) + x + y <= C(X’’) + x + y = C(X’) <= C(X) </li></ul>

×