1. Human-in-the-Loop Fine-Grained Visual
Categorization using Visipedia
or
"The Revolution will be Curated"
Serge Belongie
UC San Diego
Steve Branson
Catherine Wah Peter Welinder
Boris Babenko Pietro Perona
Florian Schroff

2. 2

3. Birds-200 Dataset
6033 images over 200 bird species

4. 4

5. MTurker Label Certainty
5

6. Visual 20 Questions
• “Computer Vision” module = Vedaldi’s VLFeat
• VQ Geometric Blur, color/gray SIFT spatial pyramid
• Multiple Kernel Learning
• Per-Class 1-vs-All SVM
• 15 training examples per bird species
• Choose question to maximize expected Information Gain
6

7. 7

8. General Observations
• User Responses are Stochastic
• Computer Vision Reduces Manual Labor
• User Responses Drive Up Performance
• Computer Vision Improves Overall
Performance
• Different Questions are Asked w/ and w/o
Computer Vision
• Recognition is not Always Successful
8

9. w/o Computer Vision
• User Responses are Stochastic
9

10. w/ Computer Vision
• Computer Vision Reduces Manual Labor
10

11. w/ Computer Vision (cont’d)
• User Responses Drive Up Performance
11

12. • Computer Vision Improves Overall Performance
• Different Questions are Asked w/ and w/o
Computer Vision

13. • Recognition is not Always Successful

14. Indigo Bunting Blue Grosbeak