https://youtu.be/EkndN7svgUk

1. visionNoob
(Jaewon Lee)
PR-199
SNIPER : Efficient Multi-Scale Training
B Singh, M Najibi, LS Davis NIPS’ 18
https://arxiv.org/abs/1805.09300

2. Main Topic
Scale Invariance in Object Recognition(Detection) Tasks
PR-199: SNIPER : Efficient Multi-Scale Training 2

3. Preliminary
- Scale Problem in Object Recognition
- Multi Scale Strategies
- SNIP(Scale Normalized Image Pyramid)
PR-199: SNIPER : Efficient Multi-Scale Training 3
PR-110: An Analysis of Scale Invariance in Object Detection – SNIP
https://youtu.be/nimHWHxjBJ8

4. Relative Scale =
!"#$(&#'& ()*'+$ )
!"#$(&#'& -.&/' )
MS COCO dataset has L
- Most small objects (Median 0.106)
- Large scale variation (20x)
- Large domain shift from pre-trained classification network4

5. PR-199: SNIPER : Efficient Multi-Scale Training 5

6. Preliminary: Multi Scale Strategies
6

7. PR-110: An Analysis of Scale Invariance in Object Detection – SNIP
https://youtu.be/nimHWHxjBJ8
PR-199: SNIPER : Efficient Multi-Scale Training 7

8. too small
small
medium
large
too large
original
Large scale variation
upscale
upscale 8

9. Normalize Scale
(Similar Scale with ImageNet)
medium
original
upscale
upscale 9

10. SNIP
Scale Normalization for Image Pyramids
10

11. Fundamental Problem of Image Pyramid?
PR-199: SNIPER : Efficient Multi-Scale Training 11

12. Train
Train
Train
Image Pyramid has to process
14 times more pixels
+ Can’t be in same batch
Original
x2
x3
PR-199: SNIPER : Efficient Multi-Scale Training 12

13. 3x Upscale
PR-199: SNIPER : Efficient Multi-Scale Training 13

14. 3x Upscale
Resampling Chips
PR-199: SNIPER : Efficient Multi-Scale Training 14

15. SNIPER
1. Chip Generation
2.Positive Chip Selection
3.Negative Chip Selection
PR-199: SNIPER : Efficient Multi-Scale Training 15

16. Chips
Positive
Chips
Easy
Negative
Chips
Hard
Negative
Chips
PR-199: SNIPER : Efficient Multi-Scale Training 16

17. Chip Generation
PR-199: SNIPER : Efficient Multi-Scale Training 17

18. Chip Generation
Chips {𝐶2
, 𝐶4
, 𝐶5
} at multiple scale {𝑠2, 𝑠4, 𝑠5}
PR-199: SNIPER : Efficient Multi-Scale Training 18

19. Chip Generation
1. 𝐶2
at scale 𝑠2 (original scale)
Resize to Width(𝑾 𝟏) and Height(𝑯 𝟏)
K
K
PR-199: SNIPER : Efficient Multi-Scale Training 19

20. Chip Generation
1. 𝐶2
at scale 𝑠2 (original scale)
Resize to Width(𝑾 𝟏) and Height(𝑯 𝟏)
Stride = 32
PR-199: SNIPER : Efficient Multi-Scale Training 20

21. Chip Generation
1. 𝐶2
at scale 𝑠2 (original scale)
Resize to Width(𝑾 𝟏) and Height(𝑯 𝟏)
PR-199: SNIPER : Efficient Multi-Scale Training 21

22. Chip Generation
2. 𝐶4
at scale 𝑠4 (2Xscale)
K
K
PR-199: SNIPER : Efficient Multi-Scale Training 22

23. Chip Generation
2. 𝐶4
at scale 𝑠4 (2Xscale)
PR-199: SNIPER : Efficient Multi-Scale Training 23

24. Chip Generation
2. 𝐶4
at scale 𝑠4 (2Xscale)
Total Chips in 𝐶4 = 49
PR-199: SNIPER : Efficient Multi-Scale Training 24

25. Positive Chip Selection
i.e. Area range : (0,802), (322, 1502), and (1202, inf)
25

26. Positive Chip Selection
PR-199: SNIPER : Efficient Multi-Scale Training 26

27. Negative Chip Selection
PR-199: SNIPER : Efficient Multi-Scale Training 27

28. Chip Generation
PR-199: SNIPER : Efficient Multi-Scale Training 28

29. Chips
PR-199: SNIPER : Efficient Multi-Scale Training 29

30. Positive Chip
Easy Negative Chip
Hard Negative Chip
No Training Chip
PR-199: SNIPER : Efficient Multi-Scale Training 30

31. 4 x 3 x 512 x 512
PR-199: SNIPER : Efficient Multi-Scale Training
=
31

32. Experiments
PR-199: SNIPER : Efficient Multi-Scale Training 32

33. Training Time (+benefit)
- Randomly sample chips from the whole dataset for batch.
MS COCO case
- 128 batch size (8 GPU)
- Area range : (0,802), (322, 1502), and (1202, inf)
- On average 5 chips (512x512) when training on scales (512/ms, 1.667, 3)
- Just 30% more than pixel than single scale training (800x1333)
- Always same input size (512x512)
- 3 scale training + large batch size (good batch normalization)
- Image resolution bottleneck is alleviated
OpenImagesV4 (1.7M)
- High resolution(1024x768) -> less important up-sampling
- Training on scales (512/ms, 1) -> total 3.5M chips (512x512)
Experiments
PR-199: SNIPER : Efficient Multi-Scale Training 33

34. Experiments
MS COCO
Training
- 14 hours to train SNIPER on single 8 GPU V100 node with Faster-RCNN with ResNet-101
Inference Time
- (480, 512), (800, 1280) and (1400, 2000)
- Soft-NSM
PR-199: SNIPER : Efficient Multi-Scale Training 34

35. SNIPER uses negative chip mining to reduce the false positive rate
while speeding up the training by skipping the easy regions inside the image.
PR-199: SNIPER : Efficient Multi-Scale Training 35

36. Experiments
PR-199: SNIPER : Efficient Multi-Scale Training 36

37. Conclusion & Future Work
Multi-scale inference?
PR-199: SNIPER : Efficient Multi-Scale Training 37