This document describes a book cover recognition system that takes an image containing a book as input and outputs the book title or ID. The baseline model uses bag-of-words with dense SIFT features, visual word learning via vector quantization, spatial pooling, and linear SVM classification. On a dataset of 9 books with 15 training and 15 test images per book, the baseline achieves nearly perfect recognition results. More challenging test images with clutter and occlusion lead to poorer results. The document explores using Hough transform for automatic book cropping before classification, which improves results from 36.24% to 58.60% mean average precision.

1. 1Book Cover RecognitionSHAO-CHUAN (SHAWN) WANGCMOS E6737 Biometrics Final Projectsw2644@columbia.eduhttp://shaochuan.info/

2. Book Cover RecognitionSystem Goal:Input: A middle resolution image taken from web camera or cell phone (640 x 480) that contains a book.Output: The book title/id. 2Book CoverRecognition System“Learning from Data”

3. Baseline Model: Bag-of-wordsLow level feature extraction (Dense SIFT) (CVPR09)Visual words learning (a.k.a codebook learning) via vector quantization.Spatial pooling of local features. Linear SVM classification. (libLinear package)3

4. A Glance on Dataset (1/2)9 books. training on half (15), test on half (15):4

5. A Glance on Dataset (2/2)5

6. Baseline: ExperimentsSystem parameters:15 training images per book, test on the rest.Resize image size to 300 pixels (long side)Visual codebook size (K = 225)# of spatial pyramid level = 0,1,2Spatial pooling function = maxL2-regularized 1-norm loss linear SVM5-fold cross validationResults:Nearly perfect recognition results. (99.259%=134/135)6

7. More Challenging Testing ImagesMore clutter, more occlusion, more realistic.7

8. More Challenging: ExperimentsResults:Poor: 36.24% (MAP) 8Confusion matrixSo, Can we “detect” and “crop” books automatically?

9. Hough TransformAssumptions:Books are rectangular, some contrasts between books and clutter.No many “false” strong edges on clutter.9

10. Auto cropping results (1/2)10Easier cases:

11. Auto cropping results (2/2)Really difficult cases:11

12. More Challenging: ExperimentsResults:Without cropping: 36.24% (MAP)With autocropping: 58.60% (MAP)12