Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Weakly supervised semantic segmentation

109 views

Published on

1. Simple Does It: Weakly Supervised Instance and Semantic Segmentation (CVPR 2017) Weak
2. Colorful Image Colorization (ECCV 2016 oral) Self

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Weakly supervised semantic segmentation

  1. 1. Semantic Segmentation with Limited Annotation Zhedong Zheng 24 Feb 2018 1
  2. 2. What can we learn from (from Stephen Chow’s film) 2
  3. 3. 3
  4. 4. 1. Simple Does It: Weakly Supervised Instance and Semantic Segmentation (CVPR 2017) Weak 2. Colorful Image Colorization (ECCV 2016 oral) Self Related Works 4
  5. 5. 1. Simple Does It: Weakly Supervised Instance and Semantic Segmentation (CVPR 2017) Weak 2. Colorful Image Colorization (ECCV 2016 oral) Self Related Works 5
  6. 6. What 6
  7. 7. How Start from object bounding box annotations 7
  8. 8. Recall Several Rules 1. Background : No bounding box -> background 2. Object Extent : Bboxes are instance-level, provide information 3. Objectness : Spatial Continuity / Contrasting boundary 8
  9. 9. How to begin? If two boxes overlap, we assume the smaller one is in front. 9
  10. 10. How to begin? 10
  11. 11. Post-Process • Any pixel outside bbox is discard. • If IoU<50%, re-inital • DenseCRF 11
  12. 12. Result Naïve is without post-processing. 12
  13. 13. Result 13
  14. 14. Result 14
  15. 15. 1. Simple Does It: Weakly Supervised Instance and Semantic Segmentation (CVPR 2017) Weak 2. Colorful Image Colorization (ECCV 2016 oral) Self Related Works 15
  16. 16. 16
  17. 17. Grayscale image: L channel Color information: ab channels abL 17
  18. 18. abL Concatenate (L,ab)Grayscale image: L channel “Free” supervisory signal Semantics? Higher-level abstraction? 18
  19. 19. Inherent Ambiguity Grayscale 19
  20. 20. Inherent Ambiguity Our Output Ground Truth 20
  21. 21. Colors in ab space (continuous)Better Loss Function • Regression with L2 loss inadequate • Use multinomial classification • Class rebalancing to encourage learning of rare colors 21
  22. 22. Better Loss Function Colors in ab space (discrete) • Regression with L2 loss inadequate • Use multinomial classification • Class rebalancing to encourage learning of rare colors 22
  23. 23. Failure Cases 23
  24. 24. Biases 24
  25. 25. Evaluation Visual Quality Representation Learning Quantitative Per-pixel accuracy Perceptual realism Semantic interpretability Task generalization ImageNet classification Task & dataset generalization PASCAL classification, detection, segmentation Qualitative Low-level stimuli Legacy grayscale photos Hidden unit activations 25
  26. 26. faces dog faces flowers Hidden Unit (conv5) Activations 26
  27. 27. Dataset & Task Generalization on PASCAL VOC %fromGaussianto ImageNetlabels Classification Detection Segmentation Gaussian Initialization ImageNet Labels 100% 0% Pathak et al. Donahue et al. Doersch et al.Krähenbühl et al. Ours Autoencoder Wang & Gupta Agrawal et al. 27
  28. 28. Amateur Family Photo, 1956. 28
  29. 29. Amateur Family Photo, 1956. 29
  30. 30. Henri Cartier-Bresson, Sunday on the Banks of the River Seine, 1938. 30
  31. 31. Henri Cartier-Bresson, Sunday on the Banks of the River Seine, 1938. 31

×