Successfully reported this slideshow.

FCIS - Fully Instance-aware Semantic Segmentation -

2

Share

Loading in …3
×
1 of 20
1 of 20

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

FCIS - Fully Instance-aware Semantic Segmentation -

  1. 1. FCIS Fully Convolutional Instance-aware Semantic Segmentation Shingo Kitagawa University of Tokyo JSK Lab
  2. 2. What is FCIS? • Fully Convolutional Instance-aware Semantic Segmentation • Microsoft Research Asia (MSRA) • 2017/04/10 (arXiv) • CVPR2017 spotlight paper • Task:Instance Segmentation • Object Detection (Faster R-CNN) • Semantic Segmentation (FCN) • Position Sensitive ROI Pooling (ECCV2016)
  3. 3. Semantic Segmentation • FCN, SegNet, PSPNetなど https://github.com/sunshineatnoon/Paper-Collection/blob/master/FCN.md https://medium.com/@steve101777/dense-segmentation-pyramid-scene-parsing-pspnet-753b1cb6097c SegNetFCN
  4. 4. Instance Segmentation • Instance > Semantic http://blog.csdn.net/l_b_yuan/article/details/53932494
  5. 5. How to do Instance Segmentation? • Conditional random fields method • Also used in Semantic Segmentation • After FCN, SOTA is Fully convolutional method. • Fully convolutional method • Combination with object detection • Mask-RCNN is also this method (MSRA -> Facebook)
  6. 6. How to do Instance Segmentation? • Conditional random fields method • Also used in Semantic Segmentation • After FCN, SOTA is Fully convolutional method. • Fully convolutional method • Combination with object detection • Mask-RCNN is also this method (MSRA -> Facebook)
  7. 7. Object Detection • R-CNN -> Fast R-CNN -> Faster R-CNN (MSRA!) • Region Proposal Network + ROI Pooling https://andrewliao11.github.io/object_detection/faster_rcnn/ • Classification • Location Regression
  8. 8. Region Proposal Network http://mithril-ntu.github.io/L10_PAPER/ • Anchor Boxes • K types of size • Hyper parameters • Classification (cls layer) • Foreground, Background • 2 class * K anchors • Location Regression (reg layer) • Regress delta from Anchor • (dy, dx, dh, dw) * K anchors RPN predicts Region of Interest (ROI), which some object should be inside.
  9. 9. ROI Pooling https://jamiekang.github.io/2017/05/28/faster-r-cnn/ (modified) • ROIs can be various shape, so pool them to same shape. • Max Pooling
  10. 10. Faster-RCNN methods https://andrewliao11.github.io/object_detection/faster_rcnn/ http://blog.csdn.net/l_b_yuan/article/details/53932494 • Classification • Location Regression
  11. 11. FCIS methods https://andrewliao11.github.io/object_detection/faster_rcnn/ http://blog.csdn.net/l_b_yuan/article/details/53932494 • Classification • Location Regression • Segmentation
  12. 12. FCIS http://blog.csdn.net/bea_tree/article/details/73088212 Position Sensitive ROI Pooling (MSRA, ECVV2016) Faster R-CNNと同じ Inside and Outside Score Maps RPN + Position Sensitive ROI Pooling + Inside Outside Score Maps
  13. 13. Position Sensitive ROI Pooling http://blog.csdn.net/bea_tree/article/details/73088212 (modified) k: Group Size (7), C: Class (81), Pooled Size: 21 • Filters learn to extract position-sensitive features. • Average Pooling
  14. 14. Inside and Outside Score Maps http://lijiancheng0614.github.io/2016/12/01/2016_12_01_FCIS/ Position Sensitive ROI Pooling FCIS predicts inside and outside scores for all object classes. With this score maps, FCIS predicts class and instance mask simultaneously.
  15. 15. Inside and Outside Score Maps ① Detection+, Segmentation+: Inside Score Inside Bbox, Inside Mask ② Detection+, Segmentation- : Outside Score Inside Bbox, Outside Mask ③ Detection-, Segmentation-: NaN Outside Bbox, Outside Mask • Detection: ① + ② ⇔ ③ : • Score = PixelwiseMax(①, ②) • Mask: ① ⇔ ② • Score = PixelwiseSoftmax(①, ②) ① ② ③
  16. 16. http://lijiancheng0614.github.io/2016/12/01/2016_12_01_FCIS/ Red Point: Upper: Detection+, Segmentation+ Lower: Detection+, Segmentation-
  17. 17. Inference Result https://github.com/knorth55/chainer-fcis Original Chainer Implementation
  18. 18. Bounding Box Regression by FCIS
  19. 19. Instance Mask Prediction by FCIS
  20. 20. https://github.com/knorth55/chainer-fcis https://github.com/chainer/chainercv

×