Ensemble of Exemplar-SVMs for
Object Detection and Beyond
Tomasz Malisiewicz
September 20, 2011
ComputerVision Reading Gro...
Overview
• Motivation and Related Work
• Learning Exemplar-SVMs
• Results
• PASCALVOC Object Detection Results
• Transfer ...
Discriminative Object
Detectors
Dalal and Triggs 2005
Linear SVM on HOG
Hard-Negative Mining
Sliding Window Detection
DT
Discriminative Object
Detectors
Dalal and Triggs 2005
Linear SVM on HOG
Hard-Negative Mining
Sliding Window Detection
DT
,...
Discriminative Object
Detectors
Dalal and Triggs 2005
Linear SVM on HOG
Hard-Negative Mining
Sliding Window Detection
DT
,...
Nearest Neighbor
Approaches
• Non-parametric: keep all the data around
• Enables Label Transfer
• However
• No learning im...
Per-Exemplar Methods
• NN-method, where each exemplar has its
own distance “similarity” function
• Better than using a sin...
Exemplar-SVMs
• Combine
• Effectiveness of discriminatively-trained object
detectors
• Explicit correspondence of Nearest ...
Exemplar-SVMs
• Learn a separate linear SVM for each instance
(exemplar) in the dataset (PASCALVOC)
Exemplar-SVMs
• Learn a separate linear SVM for each instance
(exemplar) in the dataset (PASCALVOC)
• Each Exemplar-SVM is...
Exemplar-SVMs
• Learn a separate linear SVM for each instance
(exemplar) in the dataset (PASCALVOC)
• Each Exemplar-SVM is...
Exemplar-SVMs
• Because each Exemplar-SVM is defined by a single
positive instance, we can use different features for
each ...
Exemplar-SVMs
• Because each Exemplar-SVM is defined by a single
positive instance, we can use different features for
each ...
Exemplar-SVMs
Exemplar E’s Objective Function:
h(x) = max(1-x,0) “hinge-loss”
Exemplar-SVMs
Exemplar represented by ~100
HOG Cells (~3,100 features)
Exemplar E’s Objective Function:
h(x) = max(1-x,0) ...
Exemplar-SVMs
Windows from images not
containing any in-class instances
(~2,000 images x ~10,000
windows/image = ~2M negat...
Large-scale training
• Each exemplar performs its
own hard negative mining
• Solve many convex learning
problems
• Paralle...
Exemplar-SVM Calibration
Exemplar-SVM Calibration
1) Apply
ExemplarSVM to
held-out negative
images and all
positive images
Exemplar-SVM Calibration
1) Apply
ExemplarSVM to
held-out negative
images and all
positive images
2) Fit sigmoid to
respon...
Exemplar-SVM Calibration
1) Apply
ExemplarSVM to
held-out negative
images and all
positive images
2) Fit sigmoid to
respon...
Ensemble of Exemplar-SVMs
Exemplars
Image + Detections
Ensemble of Exemplar-SVMs
Learn an exemplar co-occurence matrix
Exemplars
Image + Detections
Qualitative Results
• Let’s take a look at some Exemplar-SVM
results in PASCALVOC dataset
Exemplar w Averaged Detections
Exemplar w Averaged Detections
Exemplar w Averaged Detections
Average of first
10
detections
Average of first
20
detections
Evaluating
Exemplar-SVMs
• Nearest Neighbor
• No Learning
• Per-Exemplar Distance Functions
• Learning in distance-to-exem...
Comparison of 3
methods
!"#$%&'()*+,--
!"#$%&'( ! ./%0102#3#435/6708(/$0.#737#3
9:;<
*Learned Distance Function
*
Comparison of 3
methods
!"#$%&'()*+,--
!"#$%&'( ! ./%0102#3#435/6708(/$0.#737#3
9:;<
*Learned Distance Function
*
Comparison of 3
methods
!"#$%&'()*+,--
!"#$%&'( ! ./%0102#3#435/6708(/$0.#737#3
9:;<
*Learned Distance Function
*
Comparison of 3
methods
!"#$%&'()*+,--
!"#$%&'( ! ./%0102#3#435/6708(/$0.#737#3
9:;<
*Learned Distance Function
*
Quantitative: PASCAL
VOC 2007 dataset
• A standard computer vision object
detection benchmark
• 20 object categories
• Mac...
PASCALVOC 2007 Object
Category Detection Results
Object Category
Detection
NN + Cal 0.110
DFUN + Cal 0.155
Exemplar-SVMs + Cal 0.198
Exemplar-SVMs + Co-occ 0.227
DT* 0.097...
Beyond Object
Category Detection
• Based on the idea of label transfer,
ExemplarSVMs can be used for tasks which
go beyond...
Task 1: Geometry
Transfer
Task 1: Evaluation on Buses
• 43.0% Hoiem et al. 2005
• 51.0% Category-SVM* + NN
• 62.3% Exemplar-SVMs
• measure pixelwise...
Detector w
Appearance
Exemplar
Task II: Person Prediction
Detector w
Appearance
Exemplar
Meta-data
Person
Task II: Person Prediction
Detector w
Appearance
Exemplar
Person
Meta-data
Person
Task II: Person Prediction
Task II: Evaluation
More Transfer Examples
3D Model Transfer
Manually align 3D
model from Google
3D Warehouse with
a subset of PASCAL
VOC “chair”
exemplars
Conclusion
Conclusion
• ExemplarSVMs can be used for recognition, label
transfer, and complementary object prediction
Conclusion
!"# !!!
"#$%&'()*+,-./ "#$%&'()*+,-.0 "#$%&'()*+,-.1-232'45647.+,-
• Large-scale negative mining is the key to
...
ThankYou
Questions?
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Ensemble of Exemplar-SVM for Object Detection and Beyond
Upcoming SlideShare
Loading in …5
×

Ensemble of Exemplar-SVM for Object Detection and Beyond

1,494 views
1,269 views

Published on

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,494
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
27
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Ensemble of Exemplar-SVM for Object Detection and Beyond

  1. 1. Ensemble of Exemplar-SVMs for Object Detection and Beyond Tomasz Malisiewicz September 20, 2011 ComputerVision Reading Group@MIT Tomasz Malisiewicz, Abhinav Gupta and Alexei A. Efros. “Ensemble of Exemplar-SVMs for Object Detection and Beyond.” In ICCV, 2011.
  2. 2. Overview • Motivation and Related Work • Learning Exemplar-SVMs • Results • PASCALVOC Object Detection Results • Transfer and Prediction
  3. 3. Discriminative Object Detectors Dalal and Triggs 2005 Linear SVM on HOG Hard-Negative Mining Sliding Window Detection DT
  4. 4. Discriminative Object Detectors Dalal and Triggs 2005 Linear SVM on HOG Hard-Negative Mining Sliding Window Detection DT , Felzenszwalb et al. 2010 Parts Mixtures LDPM
  5. 5. Discriminative Object Detectors Dalal and Triggs 2005 Linear SVM on HOG Hard-Negative Mining Sliding Window Detection DT , Felzenszwalb et al. 2010 Parts Mixtures LDPM Parametric: A fixed number of models per category
  6. 6. Nearest Neighbor Approaches • Non-parametric: keep all the data around • Enables Label Transfer • However • No learning implies results depend on features and distance metric • Not shown to compete with discriminatively-trained LDPM on Pascal
  7. 7. Per-Exemplar Methods • NN-method, where each exemplar has its own distance “similarity” function • Better than using a single similarity measure across all exemplars Frome et al. 2007, Malisiewicz et al. 2008
  8. 8. Exemplar-SVMs • Combine • Effectiveness of discriminatively-trained object detectors • Explicit correspondence of Nearest Neighbor approaches
  9. 9. Exemplar-SVMs • Learn a separate linear SVM for each instance (exemplar) in the dataset (PASCALVOC)
  10. 10. Exemplar-SVMs • Learn a separate linear SVM for each instance (exemplar) in the dataset (PASCALVOC) • Each Exemplar-SVM is trained with a single positive instance
  11. 11. Exemplar-SVMs • Learn a separate linear SVM for each instance (exemplar) in the dataset (PASCALVOC) • Each Exemplar-SVM is trained with a single positive instance • Each Exemplar-SVM is more defined by “what it is not” vs.“what it is similar to”
  12. 12. Exemplar-SVMs • Because each Exemplar-SVM is defined by a single positive instance, we can use different features for each exemplar
  13. 13. Exemplar-SVMs • Because each Exemplar-SVM is defined by a single positive instance, we can use different features for each exemplar 7x4 HOG 4x8 HOG • Adapt features to each exemplar’s aspect ratio
  14. 14. Exemplar-SVMs Exemplar E’s Objective Function: h(x) = max(1-x,0) “hinge-loss”
  15. 15. Exemplar-SVMs Exemplar represented by ~100 HOG Cells (~3,100 features) Exemplar E’s Objective Function: h(x) = max(1-x,0) “hinge-loss”
  16. 16. Exemplar-SVMs Windows from images not containing any in-class instances (~2,000 images x ~10,000 windows/image = ~2M negatives ) Exemplar represented by ~100 HOG Cells (~3,100 features) Exemplar E’s Objective Function: h(x) = max(1-x,0) “hinge-loss”
  17. 17. Large-scale training • Each exemplar performs its own hard negative mining • Solve many convex learning problems • Parallel training on cluster CPU1 CPU2 CPUN Ex1 Ex2 ExN ...
  18. 18. Exemplar-SVM Calibration
  19. 19. Exemplar-SVM Calibration 1) Apply ExemplarSVM to held-out negative images and all positive images
  20. 20. Exemplar-SVM Calibration 1) Apply ExemplarSVM to held-out negative images and all positive images 2) Fit sigmoid to responses [Platt 1999]
  21. 21. Exemplar-SVM Calibration 1) Apply ExemplarSVM to held-out negative images and all positive images 2) Fit sigmoid to responses [Platt 1999]
  22. 22. Ensemble of Exemplar-SVMs Exemplars Image + Detections
  23. 23. Ensemble of Exemplar-SVMs Learn an exemplar co-occurence matrix Exemplars Image + Detections
  24. 24. Qualitative Results • Let’s take a look at some Exemplar-SVM results in PASCALVOC dataset
  25. 25. Exemplar w Averaged Detections
  26. 26. Exemplar w Averaged Detections
  27. 27. Exemplar w Averaged Detections Average of first 10 detections Average of first 20 detections
  28. 28. Evaluating Exemplar-SVMs • Nearest Neighbor • No Learning • Per-Exemplar Distance Functions • Learning in distance-to-exemplar space [Malisiewicz et al. 2008] • Exemplar-SVMs
  29. 29. Comparison of 3 methods !"#$%&'()*+,-- !"#$%&'( ! ./%0102#3#435/6708(/$0.#737#3 9:;< *Learned Distance Function *
  30. 30. Comparison of 3 methods !"#$%&'()*+,-- !"#$%&'( ! ./%0102#3#435/6708(/$0.#737#3 9:;< *Learned Distance Function *
  31. 31. Comparison of 3 methods !"#$%&'()*+,-- !"#$%&'( ! ./%0102#3#435/6708(/$0.#737#3 9:;< *Learned Distance Function *
  32. 32. Comparison of 3 methods !"#$%&'()*+,-- !"#$%&'( ! ./%0102#3#435/6708(/$0.#737#3 9:;< *Learned Distance Function *
  33. 33. Quantitative: PASCAL VOC 2007 dataset • A standard computer vision object detection benchmark • 20 object categories • Machine performance is far below human
  34. 34. PASCALVOC 2007 Object Category Detection Results
  35. 35. Object Category Detection NN + Cal 0.110 DFUN + Cal 0.155 Exemplar-SVMs + Cal 0.198 Exemplar-SVMs + Co-occ 0.227 DT* 0.097 LDPM** 0.266 mAP on PASCALVOC 2007 detection task *Dalal et al. 2005 **Felzenszwalb et al. 2010
  36. 36. Beyond Object Category Detection • Based on the idea of label transfer, ExemplarSVMs can be used for tasks which go beyond object category detection
  37. 37. Task 1: Geometry Transfer
  38. 38. Task 1: Evaluation on Buses • 43.0% Hoiem et al. 2005 • 51.0% Category-SVM* + NN • 62.3% Exemplar-SVMs • measure pixelwise accuracy on the 3-class geometric-labeling problem: “left,” “front,” “right”-facing *Felzenszwalb et al. 2010
  39. 39. Detector w Appearance Exemplar Task II: Person Prediction
  40. 40. Detector w Appearance Exemplar Meta-data Person Task II: Person Prediction
  41. 41. Detector w Appearance Exemplar Person Meta-data Person Task II: Person Prediction
  42. 42. Task II: Evaluation
  43. 43. More Transfer Examples
  44. 44. 3D Model Transfer Manually align 3D model from Google 3D Warehouse with a subset of PASCAL VOC “chair” exemplars
  45. 45. Conclusion
  46. 46. Conclusion • ExemplarSVMs can be used for recognition, label transfer, and complementary object prediction
  47. 47. Conclusion !"# !!! "#$%&'()*+,-./ "#$%&'()*+,-.0 "#$%&'()*+,-.1-232'45647.+,- • Large-scale negative mining is the key to learning a good ExemplarSVM • ExemplarSVMs can be used for recognition, label transfer, and complementary object prediction
  48. 48. ThankYou Questions?

×