Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method (MIRAGE 2009)
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method (MIRAGE 2009)

on

  • 216 views

Slides for the paper presented at MIRAGE 2009

Slides for the paper presented at MIRAGE 2009
http://link.springer.com/chapter/10.1007%2F978-3-642-01811-4_38?LI=true

Statistics

Views

Total Views
216
Views on SlideShare
216
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method (MIRAGE 2009) Presentation Transcript

  • 1. K-nearest neighbor search GPU-based brute-force method Experiments Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method Vincent Garcia and Frank Nielsen ´ Ecole Polytechnique, Palaiseau, France Mirage 2009 Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 2. K-nearest neighbor search GPU-based brute-force method Experiments K-nearest neighbor search problem (kNN) Input data R = {r1 , r2 , · · · , rm }: set of m reference points in Rd Q = {q1 , q2 , · · · , qn }: set of n query points in Rd Problem: For a given query point q, find the k-nearest neighbors of q in R. Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 3. K-nearest neighbor search GPU-based brute-force method Experiments Applications Image processing Tracking Texture synthesis Image retrieval Statistics Shannon entropy Kullback-Leibler divergence Other applications Physics Biology Finance Multimedia Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 4. K-nearest neighbor search GPU-based brute-force method Experiments State of the art Brute-force method Compute the distances between q and the reference points Sort the distances and deduce the k-nearest neighbors Space-partitionning based methods Partition the reference points (kd-tree) Fast kNN search using this structure Locality sensitive hashing (LSH) Apply a set of hashing functions to the reference points (buckets) Apply the same hashing functions to q Compute the distances between q and the references points belonging to the same bucket Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 5. K-nearest neighbor search GPU-based brute-force method Experiments GPGPU GPU: Graphics Processing Unit Set of graphic processors (up to 240 at 1.5 GHz) Dedicated graphics rendering device Specialized in highly parallel process GPGPU: General-Purpose computing on GPU Technique of using a GPU to perform computation in applications traditionally handled by the CPU NVIDIA CUDA (since Feb. 2007) OpenCL (specifications in Dec. 2008) Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 6. K-nearest neighbor search GPU-based brute-force method Experiments GPU-based brute-force method Brute-force kNN search is highly parallelizable Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 7. K-nearest neighbor search GPU-based brute-force method Experiments GPU-based brute-force method Brute-force kNN search is highly parallelizable Idea Propose a GPU implementation of the brute-force kNN search using NVIDIA CUDA Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 8. K-nearest neighbor search GPU-based brute-force method Experiments GPU-based brute-force method Brute-force kNN search is highly parallelizable Idea Propose a GPU implementation of the brute-force kNN search using NVIDIA CUDA Input data: m reference points and n query points Kernels 1 Compute the m × n distances in parallel 2 Sort the n sets of distances in parallel Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 9. K-nearest neighbor search GPU-based brute-force method Experiments GPU-based brute-force method: sorting algorithm We have to perform n sorts of m distances Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 10. K-nearest neighbor search GPU-based brute-force method Experiments GPU-based brute-force method: sorting algorithm We have to perform n sorts of m distances Sorting algorithms: Quicksort (recursive algorithm) Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 11. K-nearest neighbor search GPU-based brute-force method Experiments GPU-based brute-force method: sorting algorithm We have to perform n sorts of m distances Sorting algorithms: Quicksort (recursive algorithm) Comb sort Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 12. K-nearest neighbor search GPU-based brute-force method Experiments GPU-based brute-force method: sorting algorithm We have to perform n sorts of m distances Sorting algorithms: Quicksort (recursive algorithm) Comb sort Modified insertion sort Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 13. K-nearest neighbor search GPU-based brute-force method Experiments GPU-based brute-force method: sorting algorithm We have to perform n sorts of m distances Sorting algorithms: Quicksort (recursive algorithm) Comb sort Modified insertion sort Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 14. K-nearest neighbor search GPU-based brute-force method Experiments Experiments Goal Decrease the computation time of the kNN search Compare our GPU-based implementation with a similar CPU-based approach Compare our GPU-based implementation with fast CPU-based approach Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 15. K-nearest neighbor search GPU-based brute-force method Experiments Experiments Goal Decrease the computation time of the kNN search Compare our GPU-based implementation with a similar CPU-based approach Compare our GPU-based implementation with fast CPU-based approach Setup Pentium 4 @ 3.4 GHz, 2GB of DDR2 PC2-5300 memory NVIDIA GeForce 8800 GTX, 768 MB of DDR3 memory, 128 graphic processors @ 1350 MHz Last graphic cards: NVIDIA GeForce GTX 285, 1024 MB of DDR3 memory, 240 graphic processors @ 1476 MHz Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 16. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Synthetic data Parameters n: number of reference points and query points d: dimension of points k: number of neighbors Data: points randomly drawn from U(0, 1). Methods BF-Matlab: Brute-force method implemented in Matlab BF-C: Brute-force method implemented in C BF-CUDA: Brute-force method implemented in CUDA ANN-C++: ANN method implemented in C++ Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 17. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Synthetic data d=8 d=64 d=96 Methods BF-Matlab BF-C ANN-C++ BF-CUDA BF-Matlab BF-C ANN-C++ BF-CUDA BF-Matlab BF-C ANN-C++ BF-CUDA n=1200 0.51 0.13 0.13 0.01 2.24 1.71 0.78 0.02 3.30 2.54 1.20 0.02 n=2400 1.69 0.49 0.33 0.02 9.37 7.28 3.56 0.04 13.89 10.56 4.96 0.05 n=4800 7.84 1.90 0.81 0.04 38.16 26.11 14.66 0.11 55.77 39.26 19.68 0.15 n=9600 35.08 7.53 2.43 0.13 149.76 111.91 59.28 0.40 231.69 168.58 82.45 0.57 Maximum speed-up of BF-CUDA 407 X faster than BF-Matlab 295 X faster than BF-C 148 X faster than ANN-C++ Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 18. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Synthetic data Influence of the parameter k d = 32, n = 4800 Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 19. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Synthetic data Influence of the parameter d k = 20, n = 4800 Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 20. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Synthetic data Influence of the parameter n k = 20, d = 32 Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 21. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Finding similar patches in images Goal Define manually the first patch Find the k most similar patches Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 22. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Finding similar patches in images Goal Define manually the first patch Find the k most similar patches A color patch of size 21 × 21 = point in R1323 Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 23. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Finding similar patches in images Goal Define manually the first patch Find the k most similar patches A color patch of size 21 × 21 = point in R1323 For a color image of size 128 × 128, the problem is equivalent to find the kNN among 16383 points. Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 24. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Finding similar patches in images Data Image size: 128 × 128 = 16384 pixels Patch size: 21 × 21 k = 10 Gray level image (dimension=441) ANN-C++: 3550 ms BF-CUDA: 60 ms Speed-up = 60X Color image (dimension=1323) ANN-C++: 11.03 s BF-CUDA: 0.15 s Speed-up = 75X Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 25. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Texture synthesis Texture synthesis Efros and Leung, ICIP’99 Source Image Is Target Image It Scanline s 2s + 1 L-shape window Principle Fill the target image by random-colored pixels For a given pixel, find the closest L-shape in the source image Assign the value of the L-shape center to the current pixel The L-shape corresponding to a square window of size (2s + 1) × (2s + 1) contains 2s 2 + 2s pixels Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 26. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Texture synthesis Efros and Leung Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 27. K-nearest neighbor search GPU-based brute-force method Experiments Experiments: Texture synthesis Data Source image: 64 × 64 = 4096 pixels Target image: 128 × 128 = 16384 pixels Gray level image (dimension=220) ANN-C++: 720 ms BF-CUDA: 18 ms Speed-up = 40X Color image (dimension=660) ANN-C++: 2 s BF-CUDA: 40 ms Speed-up = 50X Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 28. K-nearest neighbor search GPU-based brute-force method Experiments Conclusion We proposed a GPU-based brute-force kNN search implementation Using GPU, a simple but highly parallelizable method can be faster than a smart algorithm Speed-up obtained on synthetic data up to 400 X faster than a Matlab implementation up to 300 X faster than a C implementation up to 150 X faster than the ANN C++ library Speed-up obtained on real image processing applications Finding similar patches: 75 X Texture synthesis: 50 X Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method
  • 29. K-nearest neighbor search GPU-based brute-force method Experiments Questions Thanks for your attention Garcia, Nielsen ´ Ecole Polytechnique, Palaiseau, France Searching high-dimensional neighbors: CPU-based tailored data-structures versus GPU-based brute-force method