This document summarizes the SNIPER method for efficient multi-scale object detection training. SNIPER generates image chips at multiple scales to handle scale variation, selects positive chips containing objects and hard negative chips lacking objects to train the model, and skips easy negative chips to speed up training. The method enables training on large batches with fewer computations than image pyramids. Experiments show SNIPER can train Faster R-CNN on MS COCO in 14 hours on a single GPU node while improving detection performance over single-scale training.

1. visionNoob
(Jaewon Lee)
PR-199
SNIPER : Efficient Multi-Scale Training
B Singh, M Najibi, LS Davis NIPS’ 18
https://arxiv.org/abs/1805.09300

2. Main Topic
Scale Invariance in Object Recognition(Detection) Tasks
PR-199: SNIPER : Efficient Multi-Scale Training 2

3. Preliminary
- Scale Problem in Object Recognition
- Multi Scale Strategies
- SNIP(Scale Normalized Image Pyramid)
PR-199: SNIPER : Efficient Multi-Scale Training 3
PR-110: An Analysis of Scale Invariance in Object Detection – SNIP
https://youtu.be/nimHWHxjBJ8

4. Relative Scale =
!"#$(&#'& ()*'+$ )
!"#$(&#'& -.&/' )
MS COCO dataset has L
- Most small objects (Median 0.106)
- Large scale variation (20x)
- Large domain shift from pre-trained classification network4

5. PR-199: SNIPER : Efficient Multi-Scale Training 5

6. Preliminary: Multi Scale Strategies
6

7. PR-110: An Analysis of Scale Invariance in Object Detection – SNIP
https://youtu.be/nimHWHxjBJ8
PR-199: SNIPER : Efficient Multi-Scale Training 7

8. too small
small
medium
large
too large
original
Large scale variation
upscale
upscale 8

9. Normalize Scale
(Similar Scale with ImageNet)
medium
original
upscale
upscale 9

10. SNIP
Scale Normalization for Image Pyramids
10

11. Fundamental Problem of Image Pyramid?
PR-199: SNIPER : Efficient Multi-Scale Training 11

12. Train
Train
Train
Image Pyramid has to process
14 times more pixels
+ Can’t be in same batch
Original
x2
x3
PR-199: SNIPER : Efficient Multi-Scale Training 12

13. 3x Upscale
PR-199: SNIPER : Efficient Multi-Scale Training 13

14. 3x Upscale
Resampling Chips
PR-199: SNIPER : Efficient Multi-Scale Training 14

15. SNIPER
1. Chip Generation
2.Positive Chip Selection
3.Negative Chip Selection
PR-199: SNIPER : Efficient Multi-Scale Training 15

16. Chips
Positive
Chips
Easy
Negative
Chips
Hard
Negative
Chips
PR-199: SNIPER : Efficient Multi-Scale Training 16

17. Chip Generation
PR-199: SNIPER : Efficient Multi-Scale Training 17

18. Chip Generation
Chips {𝐶2
, 𝐶4
, 𝐶5
} at multiple scale {𝑠2, 𝑠4, 𝑠5}
PR-199: SNIPER : Efficient Multi-Scale Training 18

19. Chip Generation
1. 𝐶2
at scale 𝑠2 (original scale)
Resize to Width(𝑾 𝟏) and Height(𝑯 𝟏)
K
K
PR-199: SNIPER : Efficient Multi-Scale Training 19

20. Chip Generation
1. 𝐶2
at scale 𝑠2 (original scale)
Resize to Width(𝑾 𝟏) and Height(𝑯 𝟏)
Stride = 32
PR-199: SNIPER : Efficient Multi-Scale Training 20

21. Chip Generation
1. 𝐶2
at scale 𝑠2 (original scale)
Resize to Width(𝑾 𝟏) and Height(𝑯 𝟏)
PR-199: SNIPER : Efficient Multi-Scale Training 21

22. Chip Generation
2. 𝐶4
at scale 𝑠4 (2Xscale)
K
K
PR-199: SNIPER : Efficient Multi-Scale Training 22

23. Chip Generation
2. 𝐶4
at scale 𝑠4 (2Xscale)
PR-199: SNIPER : Efficient Multi-Scale Training 23

24. Chip Generation
2. 𝐶4
at scale 𝑠4 (2Xscale)
Total Chips in 𝐶4 = 49
PR-199: SNIPER : Efficient Multi-Scale Training 24

25. Positive Chip Selection
i.e. Area range : (0,802), (322, 1502), and (1202, inf)
25

26. Positive Chip Selection
PR-199: SNIPER : Efficient Multi-Scale Training 26

27. Negative Chip Selection
PR-199: SNIPER : Efficient Multi-Scale Training 27

28. Chip Generation
PR-199: SNIPER : Efficient Multi-Scale Training 28

29. Chips
PR-199: SNIPER : Efficient Multi-Scale Training 29

30. Positive Chip
Easy Negative Chip
Hard Negative Chip
No Training Chip
PR-199: SNIPER : Efficient Multi-Scale Training 30

31. 4 x 3 x 512 x 512
PR-199: SNIPER : Efficient Multi-Scale Training
=
31

32. Experiments
PR-199: SNIPER : Efficient Multi-Scale Training 32

33. Training Time (+benefit)
- Randomly sample chips from the whole dataset for batch.
MS COCO case
- 128 batch size (8 GPU)
- Area range : (0,802), (322, 1502), and (1202, inf)
- On average 5 chips (512x512) when training on scales (512/ms, 1.667, 3)
- Just 30% more than pixel than single scale training (800x1333)
- Always same input size (512x512)
- 3 scale training + large batch size (good batch normalization)
- Image resolution bottleneck is alleviated
OpenImagesV4 (1.7M)
- High resolution(1024x768) -> less important up-sampling
- Training on scales (512/ms, 1) -> total 3.5M chips (512x512)
Experiments
PR-199: SNIPER : Efficient Multi-Scale Training 33

34. Experiments
MS COCO
Training
- 14 hours to train SNIPER on single 8 GPU V100 node with Faster-RCNN with ResNet-101
Inference Time
- (480, 512), (800, 1280) and (1400, 2000)
- Soft-NSM
PR-199: SNIPER : Efficient Multi-Scale Training 34

35. SNIPER uses negative chip mining to reduce the false positive rate
while speeding up the training by skipping the easy regions inside the image.
PR-199: SNIPER : Efficient Multi-Scale Training 35

36. Experiments
PR-199: SNIPER : Efficient Multi-Scale Training 36

37. Conclusion & Future Work
Multi-scale inference?
PR-199: SNIPER : Efficient Multi-Scale Training 37