The document summarizes the faster R-CNN object detection model. It introduces the Region Proposal Network (RPN) layer that predicts bounding boxes and classifies objects in one pass of the convolutional layers, making it faster than R-CNN and fast R-CNN models. It also discusses the training procedure involving initial training of the RPN, then training the full model in stages to balance the losses. Test results show faster R-CNN achieves real-time speeds while maintaining high accuracy compared to previous models.

1. Faster R-CNN: Towards
Real-Time Object detection
Microsoft Research, NIPS2015
Presentor: Andy Tsai

2. Key Idea: Region Proposal Net (RPN) layer
Conv
Layers
Predict BBoxes Classify objects
conv5

3. Testing Time: faster R-CNN
Conv
Layers
Predict BBoxes Classify objects
conv5
1*ConvTime
1*SmallNet 1000*
FcTime

4. Testing Time: R-CNN
Conv
Layers
conv5
Object Proposal
…………….
Object Proposal
1000*ConvTime
1000*FcTime

5. Testing Time: fast R-CNN
Conv
Layers
conv5
Object Proposal
…………….
Object Proposal
1*ConvTime
1000*FcTime
ROI pooling

6. Fast, Accurate Object Detection
- fastest region proposal method: Edge Boxes [4fps, 1000 proposal]
- Testing stage
Model Time
Edge boxes + R-CNN 0.25 sec + 1000*ConvTime + 1000*FcTime
Edge boxes + fast R-CNN 0.25 sec + 1*ConvTime + 1000*FcTime
faster R-CNN 1*ConvTime + 1000*FcTime

7. RPN layer

8. RPN layer

9. RPN layer
Anchor regression

10. RPN: Loss Function
- 2 class Softmax cross entropy loss
- Discriminative training:
- pi* = 1 if IoU > 0.7
- pi* = 0 if IoU < 0.3
- otherwise, do not contribute to loss

11. RPN: Loss Function
only positive sample contribute to reg. Loss

12. How to train faster R-CNN ?
- balance sampling ( neg. vs pos. = 1:1 )
- joint training is almost impossible ( conv update alternating )
- 4 step training !

13. How to train faster R-CNN ?
1. train RPN with ImageNet pre-trained model
2. train fast R-CNN using proposal generated by 1. [ no params. sharing ]
3. use conv trained by 2. to initialize model, fix shared conv, update RPN
4. fix shared conv, fine-tune fc

14. Result - MAP
- VOC2007, ZF ConvNet

15. Result - MAP
Using VGG ConvNet, fast R-CNN vs faster R-CNN

16. Result - Time
VOC 2007, fast R-CNN vs faster R-CNN
70.0%
73.2%
59.9%

17. Official Leader Board Score

18. Code available at gitHub
Matlab(faster-rcnn) & python(py-faster-rcnn) version

19. Thank you :)