Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Bundling interest points for object classification

749 views

Published on

BSc thesis by Jordi Sánchez Escué.
ETSETB UPC (25/07/2014)

More details:

Published in: Technology
  • Login to see the comments

  • Be the first to like this

Bundling interest points for object classification

  1. 1. Bundling interest points for object classification Jordi Sánchez Escué Supervised by Xavier Giró i Nieto Carles Ventura Royo
  2. 2. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 1
  3. 3. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 2
  4. 4. Introduction ● Does this image contain a plane? 3
  5. 5. Introduction ● Does this image contain a plane? ● Which type of flower is it? 4
  6. 6. Introduction ● Mobile Visual Search ○ Generalist: Google Goggles ○ Leaf-based: Leafsnap ● Fine-grained classification ○ Mushrooms ○ Flowers 5
  7. 7. Introduction ● Textures around some interest points 6
  8. 8. Introduction ● Features based on regions 7
  9. 9. Introduction ● Explore combination: points & regions 8
  10. 10. Introduction ● Project Requirements and Goals ○ Comparative study bundling interest points 9
  11. 11. Introduction ● Project Requirements and Goals ○ Software Development 10
  12. 12. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 11
  13. 13. State of the art ● In Defense of Nearest-Neighbor Based Image Classification, Oren Boiman 12
  14. 14. State of the art ● Building contextual visual vocabulary for large-scale image applications, S. Zhang 13
  15. 15. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 14
  16. 16. System Architecture ● Interest points and feature extraction ○ Sparse extraction 15
  17. 17. System Architecture ● Interest points and feature extraction ○ Interest Points: SURF 16
  18. 18. System Architecture ● Binary Partition Tree (BPT) ○ Partition: 20 reg. SLIC 17
  19. 19. ● Binary Partition Tree ○ A scale is chosen (ex, N = 3) System Architecture 18
  20. 20. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 19
  21. 21. System Architecture ● Classification: Training 20 Trainer 1 2 3 4
  22. 22. System Architecture CLASSIFIER 1-NN, euclidean distance 1 3 4 2 ● Classification: Detection 21
  23. 23. Target image System Architecture Query image 22
  24. 24. System Architecture Target image Query image 23
  25. 25. System Architecture Query image Target image 24
  26. 26. System Architecture Query image Target image 25
  27. 27. System Architecture Query image Target image 26
  28. 28. System Architecture Query image Target image 27
  29. 29. System Architecture Query image Nearest Target image 11 28
  30. 30. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 29
  31. 31. System Architecture ● Evaluation ○ Development of an evaluation tool 30
  32. 32. Tools System Architecture ● Software development 31 Trainer Detector Evaluation SVM adapted to a flexible architecture New tool for evaluation Can be adapted to any classifier or descriptor
  33. 33. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 32
  34. 34. M-E. Nilsback & A. Zisserman, «A Visual Vocabulary for Flower Classification» Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2006. http://www.robots.ox.ac.uk/~vgg/data/flowers/17/ 33
  35. 35. 0.591769 0.381372 0.463660 Experiments: basic approach ● Results 34
  36. 36. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 35
  37. 37. Experiments: Class aggregation ● Aggregation of the interest points of all the images of the same class to do the matching 36
  38. 38. Experiments: Class aggregation ● Results 37 0.59 0.38 0.46 0.78 0.43 0.56
  39. 39. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 38
  40. 40. ● Region restriction Experiments: Bundling interest points 39
  41. 41. Experiments: Bundling interest points 40
  42. 42. ● Why the results did not improve? ○ Image flower segmentation Experiments: Bundling interest points 41
  43. 43. ● Why the results did not improve? ○ Bad flower segmentation (N = 2) Experiments: Bundling interest points 42
  44. 44. ● Why the results did not improve? ○ Bad flower segmentation (N = 2) ● Future work to improve results ○ Using perfect manual segmentation Experiments: Bundling interest points 43
  45. 45. ● Why the results did not improve? ○ Good region matching (flower to flower) Experiments: Bundling interest points 44
  46. 46. ● Why the results did not improve? ○ Bad region matching (flower to background) Experiments: Bundling interest points 45
  47. 47. ● Why the results did not improve? ○ Bad region matching (flower to background) ● Future work to improve results ○ Avoid using edge regions ○ Using object candidates Experiments: Bundling interest points 46
  48. 48. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 47
  49. 49. Experiments: Class aggregation & Bundling ● Class aggregation with points bundled in regions 48
  50. 50. ● Comparative study Experiments: Class aggregation & Bundling 49
  51. 51. Contents ● Introduction ● State of the art ● System Architecture ○ Feature extraction ○ Classification ○ Evaluation ● Experiments ○ Class aggregation of interest points ○ Bundling interest points ○ Class aggregation & Bundling ● Conclusions & Future work 50
  52. 52. Conclusions & Future Work ● Comparative study done ○ Bundling interest points into regions worsens the F1-score between 1% and 7% ○ Class aggregation improves the F1-score by 9.2% ● State of the art comparative study ○ Pointless having bad results ● Software development ● Future Work 51
  53. 53. Bundling interest points for object classification Jordi Sánchez Escué Supervised by Xavier Giró i Nieto Carles Ventura Royo
  54. 54. System Architecture ● Classification: Training ○ Semantic annotation & Ontology ... ... ... ... ... ... 1 2 3 4
  55. 55. System Architecture ● Binary Partition Tree (BPT) ○ 20 SLIC superpixels
  56. 56. Future work ● Add new approaches ○ Class aggregation in the query ○ Bundling query image, not bundling target images (with certain spatial restriction). ● Optimize k, change classifier, more descriptors

×