Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Graph Regularised Hashing
Sean Moran and Victor Lavrenko
Institute of Language, Cognition and Computation
School of Inform...
Graph Regularised Hashing (GRH)
Overview
GRH
Evaluation
Conclusion
Graph Regularised Hashing (GRH)
Overview
GRH
Evaluation
Conclusion
Locality Sensitive Hashing
H DATABASE
Locality Sensitive Hashing
110101
010111
H
010101
111101
.....
DATABASE
HASH TABLE
Locality Sensitive Hashing
110101
010111
H
H
QUERY
DATABASE
HASH TABLE
010101
111101
.....
Locality Sensitive Hashing
110101
010111
H
H
COMPUTE
SIMILARITY
NEAREST
NEIGHBOURS
QUERY
010101
111101
.....
H DATABASE
QU...
Locality Sensitive Hashing
110101
010111
111101
H
H
Content Based IR
Image: Imense Ltd
Image: Doersch et al.
Image: Xu et ...
Previous work
Data-independent: Locality Sensitive Hashing (LSH) [Indyk.
98]
Data-dependent (unsupervised): Anchor Graph H...
Previous work
Method Data-Dependent Supervised Scalable Effectiveness
LSH Low
SH Low
STH Medium
BRE Medium
ITQ+CCA Medium
K...
Graph Regularised Hashing (GRH)
Overview
GRH
Evaluation
Conclusion
Graph Regularised Hashing (GRH)
Two step iterative hashing model:
Step A: Graph Regularisation
Lm ← sgn α SD−1
Lm−1 + (1−α...
Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07][1]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjac...
Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacenc...
Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacenc...
Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacenc...
Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacenc...
Graph Regularised Hashing (GRH)
-1 1 1
-1 -1 -1
ba
c
1 1 1


S a b c
a 1 1 0
b 1 1 1
c 0 1 1




D−1
a b c
a 0.5 0 0...
Graph Regularised Hashing (GRH)
-1 1 1
-1 -1 -1
ba
c
1 1 1
L1 = sgn





−1 0 0
−0.33 0.33 0.33
0 1 1


...
Graph Regularised Hashing (GRH)
-1 1 1
-1 1 1
ba
c
1 1 1
L1 =


b1 b2 b3
a −1 1 1
b −1 1 1
c 1 1 1


Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi...
Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi...
Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi...
Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi...
Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi...
Graph Regularised Hashing (GRH)
ba
c
e
f
g
h
d
Graph Regularised Hashing (GRH)
ba
c
e
f
g
h
d
Graph Regularised Hashing (GRH)
-1 1 1
1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
1 -1 -1
1 -1 -1
1 1 1
Graph Regularised Hashing (GRH)
1 1 1
-1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
1 1 1
1 -1 -1
1 -1 -1
Graph Regularised Hashing (GRH)
-1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
-1 1 1
1 1 1
First bit
flipped
1 1 -1
Second ...
Graph Regularised Hashing (GRH)
-1 1 1
1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
h1 . x−b1=0
h1
Negative
(-1)
half space...
Graph Regularised Hashing (GRH)
-1 1 1
1 1 1
-1 -1 -1
1 1 -11 -1 -1
ba
c
e
f
g
h
1 1 -1
d
-1 1 1
-1 1 1
h2
Positive
(+1)
h...
Evaluation
Overview
GRH
Evaluation
Conclusion
Datasets/Features
Standard evaluation datasets [Liu et al. ’12], [Gong and
Lazebnik ’11]:
CIFAR-10: 60K images, GIST descr...
Evaluation Metrics
Hamming ranking evaluation paradigm [Liu et al. ’12], [Gong
and Lazebnik ’11]
Standard evaluation metri...
GRH vs Literature (CIFAR-10 @ 32 bits)
LSH BRE STH KSH GRH (Linear) GRH (RBF)
0.10
0.15
0.20
0.25
0.30
0.35
mAP
Linear
GRH...
GRH vs Literature (CIFAR-10 @ 32 bits)
LSH BRE STH KSH GRH (Linear)GRH (RBF)
0.10
0.15
0.20
0.25
0.30
0.35
mAP
GRH's strai...
GRH vs Literature (CIFAR-10)
16 24 32 40 48 56 64
0.10
0.15
0.20
0.25
0.30
0.35
0.40
LSH
BRE
KSH
GRH
# Bits
mAP
GRH vs Literature (CIFAR-10)
Small amount of
supervision
required
16 24 32 40 48 56 64
0.10
0.15
0.20
0.25
0.30
0.35
0.40
...
GRH vs. Initialisation Strategy (CIFAR-10 @ 32 bits)
GRH (Linear) GRH (RBF)
0.00
0.05
0.10
0.15
0.20
0.25
0.30 LSH
ITQ+CCA...
GRH vs. Initialisation Strategy (CIFAR-10 @ 32 bits)
GRH (Linear) GRH (RBF)
0.00
0.05
0.10
0.15
0.20
0.25
0.30 LSH
ITQ+CCA...
GRH vs # Supervisory Data-Points (CIFAR-10)
Linear, T=1K Linear, T=2K RBF, T=1K RBF, T=2K
0.00
0.05
0.10
0.15
0.20
0.25
0....
GRH vs # Supervisory Data-Points (CIFAR-10)
Linear, T=1K Linear, T=2K RBF, T=1K RBF, T=2K
0.00
0.05
0.10
0.15
0.20
0.25
0....
GRH Timing (CIFAR-10 @ 32 bits)
Timings (s)
Method Train Test Total
GRH 42.68 0.613 43.29
KSH [1] 81.17 0.103 82.27
BRE [2...
Conclusion
Overview
GRH
Evaluation
Conclusion
Conclusions and Future Work
Supervised hashing model that is both accurate and easily
scalable
Take-home messages:
Regular...
Thank you for your attention
Sean Moran
Code and datasets available at:
sean.moran@ed.ac.uk
www.seanjmoran.com
Upcoming SlideShare
Loading in …5
×

Graph Regularised Hashing (ECIR'15 Talk)

768 views

Published on

Hashing has witnessed an increase in popularity over the
past few years due to the promise of compact encoding and fast query time. In order to be effective hashing methods must maximally preserve the similarity between the data points in the underlying binary representation.
The current best performing hashing techniques have utilised supervision. In this paper we propose a two-step iterative scheme, Graph Regularised Hashing (GRH), for incrementally adjusting the positioning of the hashing hypersurfaces to better conform to the supervisory signal:
in the first step the binary bits are regularised using a data similarity graph so that similar data points receive similar bits. In the second step the regularised hashcodes form targets for a set of binary classifiers which shift the position of each hypersurface so as to separate opposite bits with maximum margin. GRH exhibits superior retrieval accuracy to competing hashing methods.

Published in: Engineering
  • Login to see the comments

Graph Regularised Hashing (ECIR'15 Talk)

  1. 1. Graph Regularised Hashing Sean Moran and Victor Lavrenko Institute of Language, Cognition and Computation School of Informatics University of Edinburgh ECIR’15 Vienna, March 2015
  2. 2. Graph Regularised Hashing (GRH) Overview GRH Evaluation Conclusion
  3. 3. Graph Regularised Hashing (GRH) Overview GRH Evaluation Conclusion
  4. 4. Locality Sensitive Hashing H DATABASE
  5. 5. Locality Sensitive Hashing 110101 010111 H 010101 111101 ..... DATABASE HASH TABLE
  6. 6. Locality Sensitive Hashing 110101 010111 H H QUERY DATABASE HASH TABLE 010101 111101 .....
  7. 7. Locality Sensitive Hashing 110101 010111 H H COMPUTE SIMILARITY NEAREST NEIGHBOURS QUERY 010101 111101 ..... H DATABASE QUERY HASH TABLE
  8. 8. Locality Sensitive Hashing 110101 010111 111101 H H Content Based IR Image: Imense Ltd Image: Doersch et al. Image: Xu et al. Location Recognition Near duplicate detection 010101 111101 ..... H QUERY DATABASE QUERY NEAREST NEIGHBOURS HASH TABLE COMPUTE SIMILARITY
  9. 9. Previous work Data-independent: Locality Sensitive Hashing (LSH) [Indyk. 98] Data-dependent (unsupervised): Anchor Graph Hashing (AGH) [Liu et al. ’11], Spectral Hashing (SH) [Weiss ’08] Data-dependent (supervised): Self Taught Hashing (STH) [Zhang ’10], Supervised Hashing with Kernels (KSH) [Liu et al. ’12], ITQ + CCA [Gong and Lazebnik ’11], Binary Reconstructive Embedding (BRE) [Kulis and Darrell. ’09]
  10. 10. Previous work Method Data-Dependent Supervised Scalable Effectiveness LSH Low SH Low STH Medium BRE Medium ITQ+CCA Medium KSH High GRH High
  11. 11. Graph Regularised Hashing (GRH) Overview GRH Evaluation Conclusion
  12. 12. Graph Regularised Hashing (GRH) Two step iterative hashing model: Step A: Graph Regularisation Lm ← sgn α SD−1 Lm−1 + (1−α)L0 Step B: Data-Space Partitioning for k = 1. . .K : min ||hk ||2 + C N i=1 ξik s.t. Lik (hk xi + bk ) ≥ 1 − ξik for i = 1. . .N Repeat for a set number of iterations (M)
  13. 13. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07][1] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1) [1] Diaz, F.: Regularizing query-based retrieval scores. In: IR (2007)
  14. 14. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1)
  15. 15. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1)
  16. 16. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1)
  17. 17. Graph Regularised Hashing (GRH) Step A: Graph Regularisation [Diaz ’07] Lm ← sgn α SD−1 Lm−1 + (1−α)L0 S: Affinity (adjacency) matrix D: Diagonal degree matrix L: Binary bits at specified iteration α: Interpolation parameter (0 ≤ α ≤ 1)
  18. 18. Graph Regularised Hashing (GRH) -1 1 1 -1 -1 -1 ba c 1 1 1   S a b c a 1 1 0 b 1 1 1 c 0 1 1     D−1 a b c a 0.5 0 0 b 0 0.33 0 c 0 0 0.5     L0 b1 b2 b3 a −1 −1 −1 b −1 1 1 c 1 1 1  
  19. 19. Graph Regularised Hashing (GRH) -1 1 1 -1 -1 -1 ba c 1 1 1 L1 = sgn      −1 0 0 −0.33 0.33 0.33 0 1 1     
  20. 20. Graph Regularised Hashing (GRH) -1 1 1 -1 1 1 ba c 1 1 1 L1 =   b1 b2 b3 a −1 1 1 b −1 1 1 c 1 1 1  
  21. 21. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
  22. 22. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
  23. 23. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
  24. 24. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
  25. 25. Graph Regularised Hashing (GRH) Step B: Data-Space Partitioning for k = 1. . .K : min ||hk||2 + C N i=1 ξik s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N hk: Hyperplane k bk: bias of hyperplane k xi : data-point i Lik: bit k of data-point i ξik: slack variable ij K: # bits N: # data-points
  26. 26. Graph Regularised Hashing (GRH) ba c e f g h d
  27. 27. Graph Regularised Hashing (GRH) ba c e f g h d
  28. 28. Graph Regularised Hashing (GRH) -1 1 1 1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 1 -1 -1 1 -1 -1 1 1 1
  29. 29. Graph Regularised Hashing (GRH) 1 1 1 -1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 1 1 1 1 -1 -1 1 -1 -1
  30. 30. Graph Regularised Hashing (GRH) -1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 -1 1 1 1 1 1 First bit flipped 1 1 -1 Second bit flipped 1 -1 -1
  31. 31. Graph Regularised Hashing (GRH) -1 1 1 1 1 1 -1 -1 -1 1 1 -1 ba c e f g h d -1 1 1 h1 . x−b1=0 h1 Negative (-1) half space Positive (+1) half space 1 1 -1 1 -1 -1 -1 1 1
  32. 32. Graph Regularised Hashing (GRH) -1 1 1 1 1 1 -1 -1 -1 1 1 -11 -1 -1 ba c e f g h 1 1 -1 d -1 1 1 -1 1 1 h2 Positive (+1) half space h2 . x−b2=0 Negative (-1) half space
  33. 33. Evaluation Overview GRH Evaluation Conclusion
  34. 34. Datasets/Features Standard evaluation datasets [Liu et al. ’12], [Gong and Lazebnik ’11]: CIFAR-10: 60K images, GIST descriptors, 10 classes1 MNIST: 70K images, grayscale pixels, 10 classes2 NUSWIDE: 270K images, GIST descriptors, 21 classes3 True NNs: images that share at least one class in common [Liu et al. ’12] 1 http://www.cs.toronto.edu/~kriz/cifar.html 2 http://yann.lecun.com/exdb/mnist/ 3 http://lms.comp.nus.edu.sg/research/NUS-WIDE.htm
  35. 35. Evaluation Metrics Hamming ranking evaluation paradigm [Liu et al. ’12], [Gong and Lazebnik ’11] Standard evaluation metrics [Liu et al. ’12], [Gong and Lazebnik ’11]: Mean average precison (mAP) Precision at Hamming radius 2 (P@R2)
  36. 36. GRH vs Literature (CIFAR-10 @ 32 bits) LSH BRE STH KSH GRH (Linear) GRH (RBF) 0.10 0.15 0.20 0.25 0.30 0.35 mAP Linear GRH Non-linear GRH
  37. 37. GRH vs Literature (CIFAR-10 @ 32 bits) LSH BRE STH KSH GRH (Linear)GRH (RBF) 0.10 0.15 0.20 0.25 0.30 0.35 mAP GRH's straightforward objective outperforms more complex objectives
  38. 38. GRH vs Literature (CIFAR-10) 16 24 32 40 48 56 64 0.10 0.15 0.20 0.25 0.30 0.35 0.40 LSH BRE KSH GRH # Bits mAP
  39. 39. GRH vs Literature (CIFAR-10) Small amount of supervision required 16 24 32 40 48 56 64 0.10 0.15 0.20 0.25 0.30 0.35 0.40 LSH BRE KSH GRH # Bits mAP +25-30%
  40. 40. GRH vs. Initialisation Strategy (CIFAR-10 @ 32 bits) GRH (Linear) GRH (RBF) 0.00 0.05 0.10 0.15 0.20 0.25 0.30 LSH ITQ+CCA mAP Linear GRH Non-Linear GRH
  41. 41. GRH vs. Initialisation Strategy (CIFAR-10 @ 32 bits) GRH (Linear) GRH (RBF) 0.00 0.05 0.10 0.15 0.20 0.25 0.30 LSH ITQ+CCA mAP Eigendecomposition not necessary - saves O(d^3)
  42. 42. GRH vs # Supervisory Data-Points (CIFAR-10) Linear, T=1K Linear, T=2K RBF, T=1K RBF, T=2K 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 mAP Linear GRH Non-linear GRH Linear GRH Non-linear GRH
  43. 43. GRH vs # Supervisory Data-Points (CIFAR-10) Linear, T=1K Linear, T=2K RBF, T=1K RBF, T=2K 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 mAP Small amount of supervision required
  44. 44. GRH Timing (CIFAR-10 @ 32 bits) Timings (s) Method Train Test Total GRH 42.68 0.613 43.29 KSH [1] 81.17 0.103 82.27 BRE [2] 231.1 0.370 231.4 [1] Liu, W.: Supervised Hashing with Kernels. In: CVPR (2012) [2] Kulis, B.: Binary Reconstructive Embedding. In: NIPS (2009)
  45. 45. Conclusion Overview GRH Evaluation Conclusion
  46. 46. Conclusions and Future Work Supervised hashing model that is both accurate and easily scalable Take-home messages: Regularising bits over a graph is effective (and efficient) for hashcode learning An intermediate eigendecomposition step is not necessary Hyperplanes (linear hypersurfaces) can achieve a very good retrieval accuracy Future work: extend to the cross-modal hashing scenario (e.g. Image ↔ Text, English ↔ Spanish)
  47. 47. Thank you for your attention Sean Moran Code and datasets available at: sean.moran@ed.ac.uk www.seanjmoran.com

×