This document discusses FastDepth, a real-time depth estimation model. It begins with prerequisites on dense prediction vs classification networks. The key points section describes depthwise separable convolutions and skip connections. Section 3 describes the FastDepth network architecture, which uses a MobileNet encoder, depthwise separable convolutions, additive skip connections, compiler optimizations, and pruning. Section 4 evaluates these components. The conclusion is that FastDepth achieves real-time performance through its lighter architecture, optimizations, and accuracy. Future work includes updating components like MobileNet and further optimization.

1. UNDERSTANDING
FASTDEPTH
2020. 7. 18.
Joondong Kim

2. 1. Prerequisite
2. Key points
3. Models
4. Evaluation
5. Conclusion
6. Future Works
Index

3. 1.PRE-REQUISITE

4. Comparison With Classification
Dense Prediction(1/2)
General Classification Network Dense Prediction
• High Resolution -> Low Resolution -> High Resolution
• Encoder + Decoder(>>)
• High Resolution -> Low Resolution -> Class
• Encoder
vs

5. Heavy Decoder on Runtime
Dense Prediction(2/2)
Dense Prediction
Heavy Decoder
• Encoder + Decoder(>>)

6. Build Lighter Encoder
MobileNet(2017)
Key Conecpts

7. 2.KEYPOINTS

8. Depthwise Separable Convolution(1/3)
Key Points; How To Make Realtime Apps

9. Depthwise Separable Convolution(2/3)
Key Points; How To Make Realtime Apps
# of ops =
out*( (in*k*k) * (w*h) )
# of ops =
in*( (k*k) * (w*h) ) + out*in*(w*h)
kernel_size = k
image width = w
image height = h

10. Depthwise Separable Convolution(3/3)
Key Points; How To Make Realtime Apps
# of ops =
out*( (in*k*k) * (w*h) )
# of ops =
in*( (k*k) * (w*h) ) + out*in*(w*h)
kernel_size = k
image width = w
image height = h
• Reduction in computations
• 1/out + 1/k^2
• EX) Using 3x3 Conv,8~9 times latency better

11. Skip-Connection
Key Points; How To Make Result Better
Skip Connection
• Reuse feature-map from encoder
• Add or Concat

12. Key Points; Optimization for Depthwise Convolution
Optimized Compilation
TVM

13. Key Points; How To Make Realtime Apps
Pruning; NetAdapt

14. 3.MODELS

15. FastDepth Network(1/4)
최종최종최종_fastdepth.png
1.MobileNet
3.Skip Connection(Add)
+ 4.Optimized Compile
+ 5.Pruning
2.NNConv5

16. Encoder
FastDepth Network(2/4)
ResNet-50
ResNet-18
MobileNet
Decoder+

17. Decoder
FastDepth Network(3/4)
Encoder +
UpProj
UpConv
DeConv5
NNConv
selected
candidates

18. FastDepth Network(1/4)
최종최종최종_fastdepth.png

19. 4.EVALUATION

20. Encoder Evaluation
selected
candidates

21. Decoder Evaluation
selected
candidates

22. Skip(Add/Concat) Evaluation
selected
candidates

23. Compile Optimization Evaluation
selected
candidates

24. Pruning Evaluation

25. Demo
• 17~18 fps in this Demo(iPhone X)
• 30 fps on 영진's iPhoneX

26. 5.CONCLUSION

27. Lighter, Compile Optimization, Better Accuracy
Conclusion
MobileNet;
Depthwise
Separable
Convolution
Encoder
NNConv5;
Depthwise
Separable
Convolution
Decoder
Additive Skip
Connection
Skip Connect
TVM
Compile Optimization
NetAdapt
Pruning

28. 6.FUTURE WORKS

29. TODO
[Implementation] MobileNet → MobilNetV2
[Implementation]TVM
[Implementation] NetAdapt
[Study] Why depthwise convolution is slower on cpu
[Study]

30. APPENDICES

31. How It Works
Deconvolution =
Transposed Convolution = Fractionally-Strided Convolution
https://simonjisu.github.io/python/2019/10/27/convtranspose2d.html
https://towardsdatascience.com/types-of-convolutions-in-deep-learning-717013397f4d