Crowdsourced object
segmentation with a game

Vincent Charvillat
Axel Carlier

Amaia Salvador Aguilera
Xavier Giró i Nieto...
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing work

2
Motivation

?

3
Motivation

?

4
Semi-Supervised object segmentation

Rough segmentation

• B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman. La...
Semi-Supervised object segmentation

•

•

P. Arbelaez and L. Cohen. Constrained image segmentation from hierarchical boun...
Semi-Supervised object segmentation

Boring task for users!

7
8
Games with a purpose

• J. Steggink and C. Snoek. Adding semantics to image-region annotations with the name-itgame. Multi...
Ask’nSeek

A. Carlier, O. Marques, and V. Charvillat. Ask'nseek: A new game for object detection and labeling. In
ECCV'12 ...
Motivation
?

11
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Next steps

12
Constrained parametric min-cuts for
automatic object segmentation (CPMC)

J. Carreira and C. Sminchisescu. Constrained par...
Constrained parametric min-cuts for
automatic object segmentation

14
Motivation
CPMC

15
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing Work

16
Experiments
How many clicks do we need to achieve a certain quality in the
segmentation?

Test the algorithm for a large i...
Pascal VOC2010

1928 images divided in:
Train (964)
Validation (964)

18
Problem

Simulator

19
Simulator
• The simulator generates points using the ground truth of the image.

20
Simulator: Location of clicks

S. Goferman, L. Zelnik-Manor, and
A. Tal. Context-aware saliency
detection. PAMI, 2012.

21
Simulator: Foreground/Background ratio

22
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing Work

23
Jaccard index

Measure of similarity between the segmentation result and the ground truth mask
24
Results

Using Pascal
VOC2010
(Validation)

25
Results

Using Pascal
VOC2010
(Validation)

26
Outline
• Motivation
• Object Segmentation
• Experiments
• Results
• Conclusions
• Ongoing Work

27
Conclusions
• Realistic simulator to process large amounts of data.
• Estimation of the expected AVERAGE Jaccard index by ...
Ongoing Work

29
Ongoing Work
• Image segmentation
• CPMC candidates

●

●

30

Label propagation through
hierarchical partitions (eg. UCM,...
Ongoing Work
• Data collection
• Awarded with $250 in CrowdMM Competition (ACM MM Barcelona 2013).

• Already more than 15...
Questions, suggestions…
Thank you for your attention

• Motivation
• Object Segmentation
• Experiments
• Results
• Conclus...
Upcoming SlideShare
Loading in …5
×

Crowdsourced Object Segmentation with a Game

1,099 views
884 views

Published on

We introduce a new algorithm for image segmentation based on crowdsourcing through a game : Ask'nSeek. The game provides information on the objects of an image, under
the form of clicks that are either on the object, or on the background. These logs are then used in order to determine the best segmentation for an object among a set of candidates generated by the state-of-the-art CPMC algorithm. We also introduce a simulator that allows the generation of game logs and therefore gives insight about the number of games needed on an image to perform acceptable segmentation.

Presented by Amaia Salvador in CrowdMM 2013 (http://crowdmm.org/).

More info:
https://imatge.upc.edu/web/publications/crowdsourced-object-segmentation-game-0

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,099
On SlideShare
0
From Embeds
0
Number of Embeds
310
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Crowdsourced Object Segmentation with a Game

  1. 1. Crowdsourced object segmentation with a game Vincent Charvillat Axel Carlier Amaia Salvador Aguilera Xavier Giró i Nieto Oge Marques
  2. 2. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing work 2
  3. 3. Motivation ? 3
  4. 4. Motivation ? 4
  5. 5. Semi-Supervised object segmentation Rough segmentation • B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman. Labelme: A database and web-based tool for image annotation. IJCV, 2008 5
  6. 6. Semi-Supervised object segmentation • • P. Arbelaez and L. Cohen. Constrained image segmentation from hierarchical boundaries. In CVPR'08, 2008. 2) K. McGuinness and N. E. O'Connor. A comparative evaluation of interactive segmentation algorithms. 6
  7. 7. Semi-Supervised object segmentation Boring task for users! 7
  8. 8. 8
  9. 9. Games with a purpose • J. Steggink and C. Snoek. Adding semantics to image-region annotations with the name-itgame. Multimedia Systems, 2011. • L. von Ahn, R. Liu, and M. Blum. Peekaboom: a game for locating objects in images. In CHI'06, 2006. 9
  10. 10. Ask’nSeek A. Carlier, O. Marques, and V. Charvillat. Ask'nseek: A new game for object detection and labeling. In ECCV'12 Workshops 2012. 10
  11. 11. Motivation ? 11
  12. 12. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Next steps 12
  13. 13. Constrained parametric min-cuts for automatic object segmentation (CPMC) J. Carreira and C. Sminchisescu. Constrained parametric min-cuts for automatic object segmentation. In CVPR'10, 2010. 13
  14. 14. Constrained parametric min-cuts for automatic object segmentation 14
  15. 15. Motivation CPMC 15
  16. 16. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing Work 16
  17. 17. Experiments How many clicks do we need to achieve a certain quality in the segmentation? Test the algorithm for a large image dataset 17
  18. 18. Pascal VOC2010 1928 images divided in: Train (964) Validation (964) 18
  19. 19. Problem Simulator 19
  20. 20. Simulator • The simulator generates points using the ground truth of the image. 20
  21. 21. Simulator: Location of clicks S. Goferman, L. Zelnik-Manor, and A. Tal. Context-aware saliency detection. PAMI, 2012. 21
  22. 22. Simulator: Foreground/Background ratio 22
  23. 23. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing Work 23
  24. 24. Jaccard index Measure of similarity between the segmentation result and the ground truth mask 24
  25. 25. Results Using Pascal VOC2010 (Validation) 25
  26. 26. Results Using Pascal VOC2010 (Validation) 26
  27. 27. Outline • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing Work 27
  28. 28. Conclusions • Realistic simulator to process large amounts of data. • Estimation of the expected AVERAGE Jaccard index by clicks. • Inter-class variance of results. 28
  29. 29. Ongoing Work 29
  30. 30. Ongoing Work • Image segmentation • CPMC candidates ● ● 30 Label propagation through hierarchical partitions (eg. UCM, BPT…) Grabcut + Superpixels
  31. 31. Ongoing Work • Data collection • Awarded with $250 in CrowdMM Competition (ACM MM Barcelona 2013). • Already more than 1500 games collected with 100 users More on that in our poster! 31
  32. 32. Questions, suggestions… Thank you for your attention • Motivation • Object Segmentation • Experiments • Results • Conclusions • Ongoing Work 32

×