Sparse Isotropic Hashing

6,042 views

Published on

This slide was presented at the Meeting on Image Recognition and Understanding (MIRU) 2013, Tokyo, Japan. This work was awaded the MIRU Nagao prize. The authors are: I. Sato, M. Ambai, and K. Suzuki (Denso IT Laboratory, Inc.).

Published in: Technology, Education

Sparse Isotropic Hashing

  1. 1. Sparse Isotropic Hashing Ikuro Sato, Mitsuru Ambai, Koichiro Suzuki Denso IT Laboratory, Inc. {isato, manbai, ksuzuki}@d-itlab.co.jp 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 1/28 Presented at MIRU 2013, Japan. Peer reviewed paper available at http://www.am.sanken.osaka-u.ac.jp/CVA/
  2. 2. • Introduction • Proposed method • Experiment 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 2/36
  3. 3. Practical issues of large-scale image retrieval • ex) descriptor-matching approach millions of sums-of-product / query ? slow query image query image DB: ~108 images 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 3/28
  4. 4. Potential solution: descriptor binarization computational time of similarity real 512 bit 256 bit 128 bit 64 bit 32 bit binary codes1 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 4/28
  5. 5. Binarization by hash functions 1. supervised – uses known point-to-point correspondences • ex) Ambai et al, 2012. 2. unsupervised – intends to preserve similarities among the original real vectors 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 5/28
  6. 6. Popular hash function ex) Random Proj. (Goemans et al, 1995) Very Sparse Rand. Proj. (Li et al, 2006) Sequential Proj. (Wang et al, 2010) 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. Iterative Quantization (Gong et al, 2011) Isotropic Hashing (Kong et al, 2012) this work state-of-the-art 6/28
  7. 7. Most related work: Isotropic Hashing (Kong et al, 2012) 1. orthonormality 2. isotropic variance 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 7/28
  8. 8. Most related work: Isotropic Hashing (Kong et al, 2012) 1. orthonormality 2. isotropic variance Robust to noise from spherically symmetric distribution. 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 8/28
  9. 9. Learning of Isotropic Hashing • Lift and Projection (LP) algorithm isotropic orthogonal Gradient Flow algorithm omitted. intersection 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 9/28 1) PCA:
  10. 10. Under-constrained problem It’s more natural to impose additional conditions to make the problem over-constrained. our motivation 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 10/28
  11. 11. Our contribution 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 11/28
  12. 12. • Introduction • Proposed method • Experiment 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 12/36
  13. 13. Problem setup 1. rotational matrix 2. isotropic variance 3. sparsity 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 13/28
  14. 14. Condition-1: Special orthogonal group -1 1 0 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 14/28
  15. 15. Condition-2: Cost function for isotropic variance Exact solutions exist according to the Schur-Horn Theorem (AJM1954). 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 15/28
  16. 16. 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 16/28
  17. 17. Our optimization problem 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 17/28
  18. 18. Algorithm Sparse Isotropic Hashing (SIH) • Repeat until convergence. endfor notations simplified 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 18/28
  19. 19. Illustration of the optimization process 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 19/28
  20. 20. • Introduction • Proposed method • Experiment 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 20/36
  21. 21. Dataset etc. * M. Ambai and I. Sato, “Fast binary coding of local descriptors based on supervised learning” (MIRU2012). descriptor query set (u=1) training set (u=2, 3, 4) test set (u=5, 6) CARD (Ambai et al, 2011) without binarization 12896 50053 25238 # local descriptors 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 21/28
  22. 22. Evaluation criterion • Mean Average Precision (mAP) – expected value of area under Precision-Recall curve 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. precision recall 1.0 Average Precision 22/28
  23. 23. Methods compared All methods use 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 23/28 state-of-the-art
  24. 24. mAP for CARD 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 24/28
  25. 25. mAP for CARD 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 25/28
  26. 26. mAP for CARD almost on top 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 26/28
  27. 27. mAP for CARD 10% drop in mAP, 20x faster coding env.: VS2010, C program 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 27/28
  28. 28. Conclusion Isotropic Hashing (Kong et al, 2012): highly under-constrained 8/1/2013 Copyright (C) 2013 DENSO IT LABORATORY, INC. All Rights Reserved. 28/28

×