The document discusses a novel recurrent neural network model for semantic instance segmentation that predicts binary masks and labels for each object without the need for post-processing. The approach aims to eliminate redundant predictions and streamline the segmentation process by allowing the model to learn when to stop predicting. It includes evaluations using datasets such as Pascal VOC and Cityscapes to demonstrate the model's effectiveness.

1. Recurrent Neural Networks for
Semantic Instance Segmentation
Amaia Salvador Jordi Torres Xavier Giró-i-Nieto Ferran MarquésManel BaradadMíriam Bellver

2. Motivation
2
Semantic Instance Segmentation
- Proposal-based solutions:
- Hundreds/Thousands of redundant predictions
- Post-processing needed (NMS)
- Holistic & class-agnostic methods
- Reduced set of predictions
- Separate network for semantics

3. Motivation
3
- Our solution:
- Recurrent model that sequentially predicts binary masks and
categorical labels for each object in an image.
- Learns to stop once all objects have been found.
- Does not need post-processing on its output.
Semantic Instance Segmentation

4. Model
4

5. Model
5

6. Pascal VOC
6
Average Precision at different IoU thresholds

7. Pascal VOC
7
Average Recall at different IoU thresholds (class-agnostic evaluation)

8. Pascal VOC
8

9. CVPPP Leaves Segmentation
9

10. Cityscapes
10

11. Object Sorting Patterns
11

12. Encoder Activations
Correlation with convolutional activations in the encoder before and after training

13. Encoder Activations
Activations at the end of the encoder before and after training