The document discusses max-kernel search as a method for efficiently finding similar objects based on various similarity functions, highlighting its applications in fields such as machine learning and computer vision. It outlines key concepts including the importance of kernel functions, metrics used, and the efficiency improvements gained through indexing techniques. The document emphasizes the relevance of search in machine learning and provides resources for further exploration of fast max-kernel search algorithms.

1. Max-kernel search
How to search for just about anything?
Parikshit Ram

2. Similarity search
q
● Set of objects
● Query
R ● Similarity function
1

3. Finding similar images
2

4. Drug discovery
3
http://fineartamerica.com

5. Movie recommendations
4

6. Similarity search is ubiquitous
● Machine learning
● Computer vision
● Theory
● Databases
● Information retrieval
● Web application
● Collaborative filtering
● Scientific computing
5

7. Search-based classification
6

8. Search-based classification
6
?

9. Search-based classification
6
k-nearest-neighbor classification/regression

10. Search-based classification
7
“RomCom fan”

11. Search-based classification
7
“Kids movie fanatic”

12. Search-based outlier detection
8

13. 9

14. Search-based ML
Advantage
● nonparametric - lets the data speak
● no need to train complex models
Key ingredient
● notion of similarity (domain/data-specific)
Main challenge: efficiency
● Sheer size of the data
● Varied data types
10

15. Properties of similarity functions
11
● symmetry
OR

16. 11
3
1
The dissimilarity is the size of the set-theoretic difference

17. Properties of similarity functions
11
● symmetry
● self-similarity
OR
OR

18. 11
We do not really care about this.

19. Properties of similarity functions
11
● symmetry
● self-similarity
OR
OR

20. 12

21. 12

22. 12
Metrics
used everywhere

23. 12
Metrics
used everywhere

24. 12
Bregman
divergences
widely used for
distributions
Mercer kernels
widely used in
ML for variety of
objects and
problems
???
not quite
explored in
search or ML
Metrics
used everywhere

25. Breadth of Kernel Functions
Objects Kernel Functions
Images linear, polynomial, Gaussian, Pyramid match
Documents cosine
Sequences p-spectrum kernel, alignment score
Trees subtree, syntactic, partial tree
Graphs random walk
Time series cross-correlation, dynamic time-warping
Natural Lang. convolution, decomposition, lexical semantic
13

26. What is a Kernel Function?
In words
A pairwise symmetric function
● Correlation in a richer but hidden feature space
● Cannot access the hidden space
Object space
Hidden space
Hidden mapping
14

27. Max-kernel Search
Find the object in R most similar to q
with respect to a kernel
15

28. Existing methods
● Brute-force (parallel/distributed)
○ Domain-specific optimizations
● Coerce data to use metrics
○ Only approximate
No standard search tools!
16

29. Understanding kernels
If two objects equally similar to each other
then they are equally similar to the query q
17

30. IF
17
Understanding kernels
THEN

31. 18
Indexing our collection

32. 18
Indexing our collection

33. Multi-resolution index in O( n log n ) time
p
18
Indexing our collection
Cover Tree (BKL 2006)

34. How to Search with this Index?
19
q
p

35. How to Search with this Index?
19
q
p
p'
p''

36. How to Search with this Index?
q
p
p''
p'
19

37. How to Search with this Index?
q
p
p''
p'
19

38. How to Search with this Index?
q
p
p''
p'
Safely ignore
a large chunk
(potentially millions)
19

39. Results: Efficiency
Improvement
20

40. Results: Efficiency
10000x
● Widely applicable algorithm
● Performance data/kernel-dependent
10x
Improvement
20

41. Results: Sublinear Query Time
Improvement
Object set size
Bigger data implies bigger efficiency gains
21

42. Can We Prove it?
What Makes Search Hard?
Thm.
For a set R of n objects, the query time is
● expansion constant
○ the distribution of the data
● directional concentration constant
○ the distribution of a kernel-induced transformation
of the data
22

43. Endnote
● Search is an essential tool for ML
● Exploring different types of similarity functions
increases the applicability and quality of search
● Kernels are widely applicable similarity functions
○ now we have provably fast max kernel search
Code/tutorial for Fast Exact Max-Kernel Search
23
version 1.0.5
http://www.mlpack.org Ryan R. Curtin
Email: pari@skytree.net