Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Instance Segmentation - Míriam Bellver - UPC Barcelona 2018

117 views

Published on

https://telecombcn-dl.github.io/2018-dlcv/

Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.

Published in: Data & Analytics
  • Be the first to comment

Instance Segmentation - Míriam Bellver - UPC Barcelona 2018

  1. 1. Míriam Bellver miriam.bellver@bsc.edu PhD Candidate Barcelona Supercomputing Center Instance Segmentation Day 2 Lecture 4 #DLUPC http://bit.ly/dlcv2018
  2. 2. Semantic Segmentation Label every pixel! Don’t differentiate instances (cows) Classic computer vision problem Slide Credit: CS231n 2
  3. 3. Instance Segmentation Detect instances, give category, label pixels “simultaneous detection and segmentation” (SDS) Label are class-aware and instance-aware Slide Credit: CS231n 3
  4. 4. Outline Instance Segmentation Methods ● Proposal-Based ● Recurrent ● Instance Embedding 4
  5. 5. Proposal-based Slide Credit: CS231nHariharan et al. Simultaneous Detection and Segmentation. ECCV 2014 External Segment proposals Mask out background with mean image Similar to R-CNN, but with segment proposals 5
  6. 6. He et al. Mask R-CNN. ICCV 2017 Proposal-based Instance Segmentation: Mask R-CNN Faster R-CNN for Pixel Level Segmentation as a parallel prediction of masks and class labels 6
  7. 7. He et al. Mask R-CNN. ICCV 2017 Mask R-CNN ● Classification & box detection losses are identical to those in Faster R-CNN ● Addition of a new loss term for mask prediction: The network outputs a K x m x m volume for mask prediction, where K is the number of categories and m is the size of the mask (square) 7
  8. 8. Mask R-CNN He et al. Mask R-CNN. ICCV 2017
  9. 9. He et al. Mask R-CNN. ICCV 2017 Mask R-CNN: RoI Align Reminder: RoI Pool from Fast R-CNN Hi-res input image: 3 x 800 x 600 with region proposal Convolution and Pooling Hi-res conv features: C x H x W with region proposal Fully-connected layers Max-pool within each grid cell RoI conv features: C x h x w for region proposal Fully-connected layers expect low-res conv features: C x h x w x/16 & rounding → misalignment ! + not differentiable 9
  10. 10. Mask R-CNN: RoI Align 10 Roi Pooling example Figure Credit: Deepsense.aiHe et al. Mask R-CNN. ICCV 2017
  11. 11. Figure Credit: Silvio Galesso Mask R-CNN: RoI Align He et al. Mask R-CNN. ICCV 2017 use bilinear interpolation
  12. 12. 12
  13. 13. Limitations of Proposal-based models 13 1. Two objects might share the same bounding box: Only one will be kept after NMS step. 2. Choice of NMS threshold is application dependant 3. Same pixel can be assigned to multiple instances 4. Number of predictions is limited by the number of proposals.
  14. 14. Recurrent Instance Segmentation Romera-Paredes & H.S. Torr. Recurrent Instance Segmentation ECCV 2016 14 Sequential mask generation
  15. 15. Recurrent Instance Segmentation Romera-Paredes & H.S. Torr. Recurrent Instance Segmentation ECCV 2016 15
  16. 16. Recurrent Semantic Instance Segmentation Salvador, A., Bellver, Campos. V, M., Baradad, M., Marqués, F., Torres, J., & Giro-i-Nieto, X. (2018) From Pixels to Object Sequences: Recurrent Semantic Instance Segmentation.
  17. 17. Recurrent Semantic Instance Segmentation Salvador, A., Bellver, Campos. V, M., Baradad, M., Marqués, F., Torres, J., & Giro-i-Nieto, X. (2018) From Pixels to Object Sequences: Recurrent Semantic Instance Segmentation.
  18. 18. Recurrent Semantic Instance Segmentation Pascal VOC Cityscapes CVPPP Salvador, A., Bellver, Campos. V, M., Baradad, M., Marqués, F., Torres, J., & Giro-i-Nieto, X. (2018) From Pixels to Object Sequences: Recurrent Semantic Instance Segmentation.
  19. 19. Recurrent Semantic Instance Segmentation Salvador, A., Bellver, Campos. V, M., Baradad, M., Marqués, F., Torres, J., & Giro-i-Nieto, X. (2018) From Pixels to Object Sequences: Recurrent Semantic Instance Segmentation. Object discovery patterns
  20. 20. Instance Embedding Basis of Instance Embedding Segmentation ● Each pixel in the output of the network is a point in the embedding space ● Pixel belonging to same object are close in the embedding space, and pixels belonging to different objects, are distant ● Parsing the image embeddings involves some clustering embedding dimension K
  21. 21. Instance Embedding Brabandere et al. Semantic Instance Segmentation with a Discriminative Loss Function. CVPRW 2017
  22. 22. Instance Embedding 22 Mapping pixels to a N-dimensional space where pixels belonging to the same object are close to each other. Results on Cityscapes Brabandere et al. Semantic Instance Segmentation with a Discriminative Loss Function. CVPRW 2017
  23. 23. Outline Instance Segmentation Methods ● Proposal-Based ● Recurrent ● Instance Embedding 23
  24. 24. Questions? 24

×