Hashing has witnessed an increase in popularity over the past few years due to the promise of compact encoding and fast query time. In order to be effective hashing methods must maximally preserve the similarity between the data points in the underlying binary representation. The current best performing hashing techniques have utilised supervision. In this paper we propose a two-step iterative scheme, Graph Regularised Hashing (GRH), for incrementally adjusting the positioning of the hashing hypersurfaces to better conform to the supervisory signal: in the first step the binary bits are regularised using a data similarity graph so that similar data points receive similar bits. In the second step the regularised hashcodes form targets for a set of binary classifiers which shift the position of each hypersurface so as to separate opposite bits with maximum margin. GRH exhibits superior retrieval accuracy to competing hashing methods.

1. Graph Regularised Hashing
Sean Moran and Victor Lavrenko
Institute of Language, Cognition and Computation
School of Informatics
University of Edinburgh
ECIR’15 Vienna, March 2015

2. Graph Regularised Hashing (GRH)
Overview
GRH
Evaluation
Conclusion

3. Graph Regularised Hashing (GRH)
Overview
GRH
Evaluation
Conclusion

4. Locality Sensitive Hashing
H DATABASE

5. Locality Sensitive Hashing
110101
010111
H
010101
111101
.....
DATABASE
HASH TABLE

6. Locality Sensitive Hashing
110101
010111
H
H
QUERY
DATABASE
HASH TABLE
010101
111101
.....

7. Locality Sensitive Hashing
110101
010111
H
H
COMPUTE
SIMILARITY
NEAREST
NEIGHBOURS
QUERY
010101
111101
.....
H DATABASE
QUERY
HASH TABLE

8. Locality Sensitive Hashing
110101
010111
111101
H
H
Content Based IR
Image: Imense Ltd
Image: Doersch et al.
Image: Xu et al.
Location Recognition
Near duplicate detection
010101
111101
.....
H
QUERY
DATABASE
QUERY
NEAREST
NEIGHBOURS
HASH TABLE
COMPUTE
SIMILARITY

9. Previous work
Data-independent: Locality Sensitive Hashing (LSH) [Indyk.
98]
Data-dependent (unsupervised): Anchor Graph Hashing
(AGH) [Liu et al. ’11], Spectral Hashing (SH) [Weiss ’08]
Data-dependent (supervised): Self Taught Hashing (STH)
[Zhang ’10], Supervised Hashing with Kernels (KSH) [Liu et
al. ’12], ITQ + CCA [Gong and Lazebnik ’11], Binary
Reconstructive Embedding (BRE) [Kulis and Darrell. ’09]

10. Previous work
Method Data-Dependent Supervised Scalable Effectiveness
LSH Low
SH Low
STH Medium
BRE Medium
ITQ+CCA Medium
KSH High
GRH High

11. Graph Regularised Hashing (GRH)
Overview
GRH
Evaluation
Conclusion

12. Graph Regularised Hashing (GRH)
Two step iterative hashing model:
Step A: Graph Regularisation
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk ||2
+ C
N
i=1 ξik
s.t. Lik (hk xi + bk ) ≥ 1 − ξik for i = 1. . .N
Repeat for a set number of iterations (M)

13. Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07][1]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)
[1] Diaz, F.: Regularizing query-based retrieval scores. In: IR
(2007)

14. Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)

15. Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)

16. Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)

17. Graph Regularised Hashing (GRH)
Step A: Graph Regularisation [Diaz ’07]
Lm ← sgn α SD−1
Lm−1 + (1−α)L0
S: Affinity (adjacency) matrix
D: Diagonal degree matrix
L: Binary bits at specified iteration
α: Interpolation parameter (0 ≤ α ≤ 1)

18. Graph Regularised Hashing (GRH)
-1 1 1
-1 -1 -1
ba
c
1 1 1


S a b c
a 1 1 0
b 1 1 1
c 0 1 1




D−1
a b c
a 0.5 0 0
b 0 0.33 0
c 0 0 0.5




L0 b1 b2 b3
a −1 −1 −1
b −1 1 1
c 1 1 1



19. Graph Regularised Hashing (GRH)
-1 1 1
-1 -1 -1
ba
c
1 1 1
L1 = sgn





−1 0 0
−0.33 0.33 0.33
0 1 1






20. Graph Regularised Hashing (GRH)
-1 1 1
-1 1 1
ba
c
1 1 1
L1 =


b1 b2 b3
a −1 1 1
b −1 1 1
c 1 1 1



21. Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N
hk: Hyperplane k
bk: bias of hyperplane k
xi : data-point i
Lik: bit k of data-point i
ξik: slack variable ij
K: # bits
N: # data-points

22. Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N
hk: Hyperplane k
bk: bias of hyperplane k
xi : data-point i
Lik: bit k of data-point i
ξik: slack variable ij
K: # bits
N: # data-points

23. Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N
hk: Hyperplane k
bk: bias of hyperplane k
xi : data-point i
Lik: bit k of data-point i
ξik: slack variable ij
K: # bits
N: # data-points

24. Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N
hk: Hyperplane k
bk: bias of hyperplane k
xi : data-point i
Lik: bit k of data-point i
ξik: slack variable ij
K: # bits
N: # data-points

25. Graph Regularised Hashing (GRH)
Step B: Data-Space Partitioning
for k = 1. . .K : min ||hk||2 + C N
i=1 ξik
s.t. Lik(hk xi + bk) ≥ 1 − ξik for i = 1. . .N
hk: Hyperplane k
bk: bias of hyperplane k
xi : data-point i
Lik: bit k of data-point i
ξik: slack variable ij
K: # bits
N: # data-points

26. Graph Regularised Hashing (GRH)
ba
c
e
f
g
h
d

27. Graph Regularised Hashing (GRH)
ba
c
e
f
g
h
d

28. Graph Regularised Hashing (GRH)
-1 1 1
1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
1 -1 -1
1 -1 -1
1 1 1

29. Graph Regularised Hashing (GRH)
1 1 1
-1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
1 1 1
1 -1 -1
1 -1 -1

30. Graph Regularised Hashing (GRH)
-1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
-1 1 1
1 1 1
First bit
flipped
1 1 -1
Second bit
flipped
1 -1 -1

31. Graph Regularised Hashing (GRH)
-1 1 1
1 1 1
-1 -1 -1
1 1 -1
ba
c
e
f
g
h
d
-1 1 1
h1 . x−b1=0
h1
Negative
(-1)
half space
Positive
(+1)
half space
1 1 -1
1 -1 -1
-1 1 1

32. Graph Regularised Hashing (GRH)
-1 1 1
1 1 1
-1 -1 -1
1 1 -11 -1 -1
ba
c
e
f
g
h
1 1 -1
d
-1 1 1
-1 1 1
h2
Positive
(+1)
half space
h2 . x−b2=0
Negative
(-1)
half space

33. Evaluation
Overview
GRH
Evaluation
Conclusion

34. Datasets/Features
Standard evaluation datasets [Liu et al. ’12], [Gong and
Lazebnik ’11]:
CIFAR-10: 60K images, GIST descriptors, 10 classes1
MNIST: 70K images, grayscale pixels, 10 classes2
NUSWIDE: 270K images, GIST descriptors, 21 classes3
True NNs: images that share at least one class in common
[Liu et al. ’12]
1
http://www.cs.toronto.edu/~kriz/cifar.html
2
http://yann.lecun.com/exdb/mnist/
3
http://lms.comp.nus.edu.sg/research/NUS-WIDE.htm

35. Evaluation Metrics
Hamming ranking evaluation paradigm [Liu et al. ’12], [Gong
and Lazebnik ’11]
Standard evaluation metrics [Liu et al. ’12], [Gong and
Lazebnik ’11]:
Mean average precison (mAP)
Precision at Hamming radius 2 (P@R2)

36. GRH vs Literature (CIFAR-10 @ 32 bits)
LSH BRE STH KSH GRH (Linear) GRH (RBF)
0.10
0.15
0.20
0.25
0.30
0.35
mAP
Linear
GRH
Non-linear
GRH

37. GRH vs Literature (CIFAR-10 @ 32 bits)
LSH BRE STH KSH GRH (Linear)GRH (RBF)
0.10
0.15
0.20
0.25
0.30
0.35
mAP
GRH's straightforward
objective outperforms
more complex
objectives

38. GRH vs Literature (CIFAR-10)
16 24 32 40 48 56 64
0.10
0.15
0.20
0.25
0.30
0.35
0.40
LSH
BRE
KSH
GRH
# Bits
mAP

39. GRH vs Literature (CIFAR-10)
Small amount of
supervision
required
16 24 32 40 48 56 64
0.10
0.15
0.20
0.25
0.30
0.35
0.40
LSH
BRE
KSH
GRH
# Bits
mAP
+25-30%

40. GRH vs. Initialisation Strategy (CIFAR-10 @ 32 bits)
GRH (Linear) GRH (RBF)
0.00
0.05
0.10
0.15
0.20
0.25
0.30 LSH
ITQ+CCA
mAP
Linear
GRH
Non-Linear
GRH

41. GRH vs. Initialisation Strategy (CIFAR-10 @ 32 bits)
GRH (Linear) GRH (RBF)
0.00
0.05
0.10
0.15
0.20
0.25
0.30 LSH
ITQ+CCA
mAP
Eigendecomposition
not necessary -
saves O(d^3)

42. GRH vs # Supervisory Data-Points (CIFAR-10)
Linear, T=1K Linear, T=2K RBF, T=1K RBF, T=2K
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
mAP
Linear
GRH
Non-linear
GRH
Linear
GRH
Non-linear
GRH

43. GRH vs # Supervisory Data-Points (CIFAR-10)
Linear, T=1K Linear, T=2K RBF, T=1K RBF, T=2K
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
mAP
Small amount of
supervision
required

44. GRH Timing (CIFAR-10 @ 32 bits)
Timings (s)
Method Train Test Total
GRH 42.68 0.613 43.29
KSH [1] 81.17 0.103 82.27
BRE [2] 231.1 0.370 231.4
[1] Liu, W.: Supervised Hashing with Kernels. In: CVPR (2012)
[2] Kulis, B.: Binary Reconstructive Embedding. In: NIPS (2009)

45. Conclusion
Overview
GRH
Evaluation
Conclusion

46. Conclusions and Future Work
Supervised hashing model that is both accurate and easily
scalable
Take-home messages:
Regularising bits over a graph is effective (and efficient) for
hashcode learning
An intermediate eigendecomposition step is not necessary
Hyperplanes (linear hypersurfaces) can achieve a very good
retrieval accuracy
Future work: extend to the cross-modal hashing scenario (e.g.
Image ↔ Text, English ↔ Spanish)

47. Thank you for your attention
Sean Moran
Code and datasets available at:
sean.moran@ed.ac.uk
www.seanjmoran.com