- 1. QUANTIZATION FOR CLASSIFICATION ACCURACY IN HIGH-RATE QUANTIZERS Behzad M. Dogahe Manohar N. Murthi Department of Electrical and Computer Engineering IEEE DSP Workshop, January 2011
- 2. Outline • Motivation • Background • Problem Statement and Solution • Simulations • Concluding Remarks
- 3. Motivation • Quantization of signals is required for many applications • The original signal is quantized at the encoder and at the decoder side a replica that should resemble the original signal in some sense is recovered • Present quantizers make an effort to reduce the distortion of the signal in the sense of reproduction fidelity • Consider scenarios in which signals are generated from multiple classes. The encoder focuses on the task of quantization without any regards to the class of the signal • The quantized signal reaches the decoder where not only the recovery of the signal should take place but also a decision is to be made on the class of the signal based on the quantized version of the signal only
- 4. Motivation • Goal: Design of a quantizer that is optimized for the task of classification at the decoder • Application Scenarios: Want to have good sound fidelity (good voice/audio quality) but also want to be able to perform speaker recognition Sensor network where the sensors have low complexity, simple quantizers, but the decoder/sensor sink node does more sophisticated processing (so the raw signal value is needed, but we also want to be able to classify the sensed signal)
- 5. Background Quantizerx )(ˆ xQx x xˆ x )(xp x )(x x xˆ x )(xp x )(x In high-rate theory point density function represents the density of codebook points in any region for a quantizer. The design of a quantizer is equivalent to design of the optimal point density function. )(xp : Probability Density Function
- 6. Background • Design of Quantizer involves minimizing: where is Distortion Measure • Examples of Distortion Measure: MSE Log Spectral Distortion • High-Rate Theory: 2 ˆ)ˆ,( xxxxd Optimization Problem
- 7. Background • Following the steps in [Gardner and Rao] point density function will be derived as (n is the dimension of x) W.R. Gardner and B.D. Rao, “Theoretical analysis of the high-rate vector quantization of lpc parameters,” Speech and Audio Processing, IEEE Transactions on, vol. 3, no. 5, pp. 367 –381, sep 1995.
- 8. Problem Statement • We are looking for a point density function that is representative of a quantizer that performs well in the classification task • We have to select a distortion measure that is well defined for classification purposes • We chose the symmetric Kullback-Leibler divergence measure between probability of class given the signal before and after quantization
- 9. Problem Statement & Solution We assume a generative model for classifier. Hence and are known a priori. Trade-off Distortion Measure:
- 10. Simulations • Signal is from two classes with known conditional PDFs • Dashed lines represent the decision boundaries • Point density function dedicates codebook points to the boundaries
- 11. Simulations • only dedicates codebook points where the signal is concentrated • By introducing tradeoff between MSE and classification, codebook points move to the classification boundaries
- 12. Simulations 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 KL Tradeoff (a = 0.2)Tradeoff (a = 0.8) MSE 10 Bits 8 Bits 6 Bits Classification Error (%) • The higher the bit rate of quantizer the better classification accuracy • As we move from MSE to KL, the classification accuracy improves
- 13. Simulations -50 -45 -40 -35 -30 -25 -20 -15 -10 -5 0 KL Tradeoff (a = 0.2) Tradeoff (a = 0.8) MSE 10 Bits 8 Bits 6 Bits Distortion (dB) • Pure KL performs poorly as far as the distortion of the signal • However, introducing the slightest tradeoff with MSE improves distortion significantly
- 14. Concluding Remarks • A solution for quantization of signals for the purpose of obtaining a more accurate classification at the decoder was proposed • High-rate theory for quantizer design was employed • An optimal point density function was derived • The performance of this method on synthetically generated data was examined and observed to be superior in the task of classification of signals at the decoder • The tradeoff between the reproduction fidelity and classification accuracy was studied as well • In our future work, we will study the practical vector quantizer design based on the high-rate theory