Upcoming SlideShare
Loading in …5
×

# Quantization

2,845 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

• Be the first to like this

No Downloads
Views
Total views
2,845
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
85
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Quantization

1. 1. Quantization Input-output characteristic of a scalar quantizer x ˆ x Q Sometimes, this Output ˆ x convention is used: ˆ xq  2M represen-tative levels x Q q ˆ xq 1 ˆ xq -1 t q+2 q Q x ˆ tq t q+1 input signal x M-1 decision thresholds Bernd Girod: EE398A Image and Video Compression Quantization no. 1
2. 2. Example of a quantized waveform Original and Quantized Signal Quantization ErrorBernd Girod: EE398A Image and Video Compression Quantization no. 2
3. 3. Lloyd-Max scalar quantizer Problem : For a signal x with given PDF f X ( x) find a quantizer with M representative levels such that d  MSE  E  X  X    min. 2 ˆ     Solution : Lloyd-Max quantizer [Lloyd,1957] [Max,1960] tq   xq 1  xq  q  1, 2,, M -1 1  M-1 decision thresholds exactly ˆ ˆ 2 half-way between representative tq1 levels.  M representative levels in the  x f X ( x)dx centroid of the PDF between two xq  q  0,1, , M -1 tq ˆ tq 1 successive decision thresholds.  f X ( x)dx  Necessary (but not sufficient) tq conditions Bernd Girod: EE398A Image and Video Compression Quantization no. 3
4. 4. Iterative Lloyd-Max quantizer design1. Guess initial set of representative levels xq q  0,1, 2,, M -1 ˆ2. Calculate decision thresholds tq   xq 1  xq  q  1, 2,, M -1 1 ˆ ˆ 23. Calculate new representative levels tq1  x f X ( x)dx xq  q  0,1,, M -1 tq ˆ tq1  tq f X ( x)dx4. Repeat 2. and 3. until no further distortion reduction Bernd Girod: EE398A Image and Video Compression Quantization no. 4
5. 5. Example of use of the Lloyd algorithm (I) X zero-mean, unit-variance Gaussian r.v. Design scalar quantizer with 4 quantization indices with minimum expected distortion D* Optimum quantizer, obtained with the Lloyd algorithm  Decision thresholds -0.98, 0, 0.98  Representative levels –1.51, -0.45, 0.45, 1.51  D*=0.12=9.30 dB Boundary Reconstruction Bernd Girod: EE398A Image and Video Compression Quantization no. 5
6. 6. Example of use of the Lloyd algorithm (II) Convergence  Initial quantizer A:  Initial quantizer B: decision thresholds –3, 0 3 decision thresholds –½, 0, ½ Quantization Function Quantization Function 6 6 4 4 2 2 0 0 -2 -2 -4 -4 -6 5 10 15 -6 5 10 15 Iteration Number Iteration Number 0.4 0.2 0.2 D 0.15 D 0 0.1 0 5 10 15 0 5 SNROUT final = 9.2978 SNROUT10 = 9.298 15 SNROUT [dB] SNROUT [dB] 10 10 final 5 8 0 6 0 5 10 15 0 5 10 15 Iteration Number Iteration Number  After 6 iterations, in both cases, (D-D*)/D*<1% Bernd Girod: EE398A Image and Video Compression Quantization no. 6
7. 7. Example of use of the Lloyd algorithm (III) X zero-mean, unit-variance Laplacian r.v. Design scalar quantizer with 4 quantization indices with minimum expected distortion D* Optimum quantizer, obtained with the Lloyd algorithm  Decision thresholds -1.13, 0, 1.13  Representative levels -1.83, -0.42, 0.42, 1.83  D*=0.18=7.54 dB Threshold Representative Bernd Girod: EE398A Image and Video Compression Quantization no. 7
8. 8. Example of use of the Lloyd algorithm (IV) Convergence  Initial quantizer A,  Initial quantizer B, decision thresholds –3, 0 3 decision thresholds –½, 0, ½ Quantization Function Quantization Function 10 10 5 5 0 0 -5 -5 -10 -10 2 4 6 8 10 12 14 16 2 4 6 8 10 12 14 16 Iteration Number Iteration Number 0.4 0.4 0.2 D 0.2 D 0 0 0 5 10 15 0 5 10 15 8 8 SNR [dB] SNR [dB] 6 6 SNRfinal = 7.5415 SNRfinal = 7.5415 4 4 0 5 10 15 0 5 Iteration Number Iteration Number10 15  After 6 iterations, in both cases, (D-D*)/D*<1% Bernd Girod: EE398A Image and Video Compression Quantization no. 8
9. 9. Lloyd algorithm with training data1. Guess initial set of representative levels xq ; q  0,1, 2,, M -1 ˆ2. ˆ Assign each sample xi in training set T to closest representative xq  Bq  x T : Q x  q  q  0,1,2,, M -13. Calculate new representative levels 1 xq  ˆ Bq x xBq q  0,1,, M -14. Repeat 2. and 3. until no further distortion reduction Bernd Girod: EE398A Image and Video Compression Quantization no. 9
10. 10. Lloyd-Max quantizer properties Zero-mean quantization error ˆ E X  X   0   Quantization error and reconstruction decorrelated  ˆ ˆ   E X X X 0 Variance subtraction property    E 2 ˆ X  X  X 2   2 X ˆ     Bernd Girod: EE398A Image and Video Compression Quantization no. 10
11. 11. High rate approximation Approximate solution of the "Max quantization problem," assuming high rate and smooth PDF [Panter, Dite, 1951] 1 x( x)  const 3 f X ( x) Distance between two Probability density successive quantizer function of x representative levels Approximation for the quantization error variance: 3   1  3 f ( x)dx   d  E X X  2 ˆ    12M 2   X   x  Number of representative levels Bernd Girod: EE398A Image and Video Compression Quantization no. 11
12. 12. High rate approximation (cont.) High-rate distortion-rate function for scalar Lloyd-Max quantizer d  R    2 X 22 R 2 3 1  3  with      f X ( x)dx  2 2 X 12  x  Some example values for  2 uniform 1 Laplacian 9 2  4.5 3 Gaussian  2.721 2 Bernd Girod: EE398A Image and Video Compression Quantization no. 12
13. 13. High rate approximation (cont.) Partial distortion theorem: each interval makes an (approximately) equal contribution to overall mean-squared error Pr ti  X  ti 1 E      X X 2 t  X t  ˆ i i 1    Pr t j  X  t j 1 E      X X 2 t  X t  ˆ j j 1   for all i, j [Panter, Dite, 1951], [Fejes Toth, 1959], [Gersho, 1979] Bernd Girod: EE398A Image and Video Compression Quantization no. 13
14. 14. Entropy-constrained scalar quantizer Lloyd-Max quantizer optimum for fixed-rate encoding, how can we do better for variable-length encoding of quantizer index? Problem : For a signal x with given pdf f X ( x) find a quantizer with rate M 1 ˆ   R  H X    pq log 2 pq q 0 such that  d  MSE  E  X  X    min. 2 ˆ     Solution: Lagrangian cost function  J  d  R  E  X  X     H  X   min. 2 ˆ ˆ     Bernd Girod: EE398A Image and Video Compression Quantization no. 14
15. 15. Iterative entropy-constrained scalar quantizer design1. Guess initial set of representative levels xq ; q  0,1, 2,, M -1 ˆ and corresponding probabilities pq2. Calculate M-1 decision thresholds xq -1  xq ˆ ˆ log 2 pq 1  log 2 pq tq =  q  1, 2,, M -1 2 2  xq -1  xq  ˆ ˆ3. Calculate M new representative levels and probabilities pq tq1  xf X ( x)dx xq  q  0,1,, M -1 tq ˆ tq1  tq f X ( x)dx4. Repeat 2. and 3. until no further reduction in Lagrangian cost Bernd Girod: EE398A Image and Video Compression Quantization no. 15
16. 16. Lloyd algorithm for entropy-constrained quantizer design based on training set1. Guess initial set of representative levels xq ; q  0,1, 2,, M -1 ˆ and corresponding probabilities pq2. Assign each sample xi in training set T to representative ˆ xq J x  q    xi  xq    log 2 pq 2 minimizing Lagrangian cost i ˆ Bq  x T : Q  x   q q  0,1, 2,, M -13. Calculate new representative levels and probabilities pq 1 xq  ˆ Bq x xBq q  0,1,, M -14. Repeat 2. and 3. until no further reduction in overall Lagrangian cost    i  2 J J  xi x Q x i   log p 2 q  xi  xi xi Bernd Girod: EE398A Image and Video Compression Quantization no. 16
17. 17. Example of the EC Lloyd algorithm (I) X zero-mean, unit-variance Gaussian r.v. Design entropy-constrained scalar quantizer with rate R≈2 bits, and minimum distortion D* Optimum quantizer, obtained with the entropy-constrained Lloyd algorithm  11 intervals (in [-6,6]), almost uniform  D*=0.09=10.53 dB, R=2.0035 bits (compare to fixed-length example) Threshold Representative level Bernd Girod: EE398A Image and Video Compression Quantization no. 17
18. 18. Example of the EC Lloyd algorithm (II) Same Lagrangian multiplier λ used in all experiments  Initial quantizer A, 15 intervals  Initial quantizer B, only 4 intervals (>11) in [-6,6], with the same length (<11) in [-6,6], with the same length Quantization Function 6 6 Quantization Function 4 4 2 2 0 0 -2 -2 -4 -4 -6 -6 5 10 15 20 25 30 35 40 5 10 15 20 Iteration Number Iteration Number 12 12 R [bit/symbol] SNR [dB] R [bit/symbol] SNR [dB] SNRfinal = 8.9452 10 10 SNRfinal = 10.5363 8 8 6 6 0 5 10 15 20 25 30 35 40 0 5 10 15 20 2.5 2.5 Rfinal = 1.7723 2 2 Rfinal = 2.0044 1.5 1.5 1 1 0 5 10 15 20 25 30 35 40 0 5 10 15 20 Iteration Number Iteration Number Bernd Girod: EE398A Image and Video Compression Quantization no. 18
19. 19. Example of the EC Lloyd algorithm (III) X zero-mean, unit-variance Laplacian r.v. Design entropy-constrained scalar quantizer with rate R≈2 bits and minimum distortion D* Optimum quantizer, obtained with the entropy-constrained Lloyd algorithm  21 intervals (in [-10,10]), almost uniform  D*= 0.07=11.38 dB, R= 2.0023 bits (compare to fixed-length example) Decision threshold Representative level Bernd Girod: EE398A Image and Video Compression Quantization no. 19
20. 20. Example of the EC Lloyd algorithm (IV) Same Lagrangian multiplier λ used in all experiments  Initial quantizer A, 25 intervals  Initial quantizer B, only 4 intervals (>21 & odd) in [-10,10], with the (<21) in [-10,10], with the same length same length Quantization Function Quantization Function 10 10 5 5 0 0 -5 -5 -10 -10 5 10 15 20 25 30 35 40 5 10 15 20 Iteration Number Iteration Number 15 15 R [bit/symbol] SNR [dB] R [bit/symbol] SNR [dB] 10 10 SNRfinal = 11.407 5 5 SNRfinal = 7.4111 0 5 10 15 20 25 30 35 40 0 5 10 15 20 3 3 Rfinal = 2.0063 2 2 Rfinal = 1.6186 1 1 0 5 10 15 20 25 30 35 40 0 5 10 15 20 Iteration Number Iteration Number  Convergence in cost faster than convergence of decision thresholds Bernd Girod: EE398A Image and Video Compression Quantization no. 20
21. 21. High-rate results for EC scalar quantizers For MSE distortion and high rates, uniform quantizers (followed by entropy coding) are optimum [Gish, Pierce, 1968] Distortion and entropy for smooth PDF and fine quantizer interval   2   H X  h  X   log 2  ˆ 2 d   d  2  12 2 Distortion-rate function 1 2 h X  2 R d  R  2 2 12 e is factor or 1.53 dB from Shannon Lower Bound 6 1 2 h X  2 R D  R  2 2 2 e Bernd Girod: EE398A Image and Video Compression Quantization no. 21
22. 22. Comparison of high-rate performance of scalar quantizers High-rate distortion-rate function d  R    2 X 22 R 2 Scaling factor 2 Shannon LowBd Lloyd-Max Entropy-coded 6 Uniform  0.703 1 1 e e 9 e2 Laplacian  0.865  4.5  1.232  2 6 3 e Gaussian 1  2.721  1.423 2 6 Bernd Girod: EE398A Image and Video Compression Quantization no. 22
23. 23. Deadzone uniform quantizer Quantizer ˆ output x   Quantizer  input xBernd Girod: EE398A Image and Video Compression Quantization no. 23
24. 24. Embedded quantizers Motivation: “scalability” – decoding of compressed bitstreams at different rates (with different qualities) Nested quantization intervals Q0 x Q1 Q2 In general, only one quantizer can be optimum (exception: uniform quantizers) Bernd Girod: EE398A Image and Video Compression Quantization no. 24
25. 25. Example: Lloyd-Max quantizers for Gaussian PDF 0 1 0 0 1 1 0 1 0 1 2-bit and 3-bit optimal -.98 .98 quantizers not embeddable 0 0 0 01 1 1 1 Performance loss for 0 0 1 10 0 1 1 embedded quantizers 0 1 0 10 1 0 1 -1.75 -1.05 -.50 .50 1.05 1.75 Bernd Girod: EE398A Image and Video Compression Quantization no. 25
26. 26. Information theoretic analysis “Successive refinement” – Embedded coding at multiple rates w/o loss relative to R-D function ˆ E  d ( X , X 1 )   D1 ˆ I ( X ; X 1 )  R( D1 )   ˆ E  d ( X , X 2 )   D2 ˆ I ( X ; X )  R( D )   2 2 “Successive refinement” with distortions D1 and D2  D1 can be achieved iff there exists a conditional distribution fX ,X ˆ ˆ X ( x1 , x2 , x)  f X ˆ ˆ ˆ X ˆ ( x2 , x) f X ˆ ˆ X2 ˆ ˆ ( x1 , x2 ) 1 2 2 1 Markov chain condition ˆ ˆ X  X 2  X1 [Equitz,Cover, 1991] Bernd Girod: EE398A Image and Video Compression Quantization no. 26
27. 27. Embedded deadzone uniform quantizers x=0 8   4Q0 x 4   2Q1Q2 2    Supported in JPEG-2000 with general  for quantization of wavelet coefficients.Bernd Girod: EE398A Image and Video Compression Quantization no. 27
28. 28. Vector quantizationBernd Girod: EE398A Image and Video Compression Quantization no. 28
29. 29. LBG algorithm Lloyd algorithm generalized for VQ [Linde, Buzo, Gray, 1980] Best representative Best partitioning vectors of training set for given partitioning for given of training set representative vectors Assumption: fixed code word length Code book unstructured: full search Bernd Girod: EE398A Image and Video Compression Quantization no. 29
30. 30. Design of entropy-coded vector quantizers Extended LBG algorithm for entropy-coded VQ [Chou, Lookabaugh, Gray, 1989] Lagrangian cost function: solve unconstrained problem rather than constrained problem J  d  R  E   ˆ   ˆ    X  X 2    H X  min. Unstructured code book: full search for J xi  q   xi  xq   log 2 pq 2 ˆ The most general coder structure: Any source coder can be interpreted as VQ with VLC! Bernd Girod: EE398A Image and Video Compression Quantization no. 30
31. 31. Lattice vector quantization • • • • • • • pdf • • • • • • • • • • • • • • • • • • Amplitude 2 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •cell • representative vector Amplitude 1Bernd Girod: EE398A Image and Video Compression Quantization no. 31
32. 32. 8D VQ of a Gauss-Markov source 18 LBG fixed CWL 12SNR [dB] E8-lattice LBG variable CWL 6 scalar 0 r  0.95 0 0.5 1.0 Rate [bit/sample] Bernd Girod: EE398A Image and Video Compression Quantization no. 32
33. 33. 8D VQ of memoryless Laplacian source 9 E8-lattice LBG variable CWL 6 scalarSNR [dB] 3 LBG fixed CWL 0 0 0.5 1.0 Rate [bit/sample] Bernd Girod: EE398A Image and Video Compression Quantization no. 33
34. 34. Reading Taubman, Marcellin, Sections 3.2, 3.4 J. Max, “Quantizing for Minimum Distortion,” IEEE Trans. Information Theory, vol. 6, no. 1, pp. 7-12, March 1960. S. P. Lloyd, “Least Squares Quantization in PCM,” IEEE Trans. Information Theory, vol. 28, no. 2, pp. 129-137, March 1982. P. A. Chou, T. Lookabaugh, R. M. Gray, “Entropy- constrained vector quantization,” IEEE Trans. Signal Processing, vol. 37, no. 1, pp. 31-42, January 1989. Bernd Girod: EE398A Image and Video Compression Quantization no. 34