Densly Connected Convolutional Neural Network
•Download as PPTX, PDF•
3 likes•945 views
Brief review of "Densly Connected Convolutional Neural Network". This slide was presented for connectionism model seminar held in Seoul National University.
Recommended
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Densly Connected Convolutional Neural Network
- 1. Densly Connected Convolutional Neural Networks Hanock Kwak Aug. 2017 School of Computer Science and Engineering Seoul National University
- 2. THROUGHOUT THE SLIDES ◉ Lots of the slides are originated from CVPR 2017 slides presented by Gao Huang et al. ◉ Check https://www.youtube.com/watch?v=-W6y8xnd--U
- 7. THIN FEATURE MAPS k-channels k-channels k-channels k-channels • Channels are linearly increased.
- 8. BOTTLENECK LAYER BatchNorm ReLU 3x3Convolution • Reduce the number of channels to 4𝑘 before composite layer by 1x1 convolution. • It’s optional. BatchNorm ReLU 1x1Convolution 4k-channels k-channels
- 9. DENSE BLOCK • Bottleneck layers are omitted for simple illustration.
- 10. SIMPLE ILLUSTRATION OF DENSE BLOCK
- 11. DENSENET BN-ReLU-Conv BN-ReLU-Conv Pooling BN-ReLU-Conv Pooling Pooling Linear Output Dense Block 1 Dense Block 2 Dense Block 3
- 12. COMPRESSION BN-ReLU-Conv Pooling Dense Block Dense Block BatchNorm ReLU 1x1Convolution • There are 1x1 convolution between dense blocks. • Reduce the number of channels by 0 < 𝜃 ≤ 1. • 𝜃 = 0.5
- 14. ADVANTAGE 1: STRONG GRADIENT FLOW Error
- 15. ADVANTAGE 2: PARAMETER & COMPUTATIONAL EFFICIENCY Convolution Small filter size Small input channel size Small output channel size
- 16. ADVANTAGE 3: MAINTAINS LOW COMPLEXITY FEATURES LOW HIGHMIDDLE COMPLEXITY LEVEL
- 19. RESULTS ON IMAGENET Floating-point operations
- 20. COMPARISON OF DENSENET VARIATIONS • DenseNet-B : There are bottleneck layers. • DenseNet-C : Compression ratio 𝜃 is less than 1. (See the previous slide named “Compression”) • DenseNet-BC : Joint of ‘B’ and ‘C’.
- 21. Thank You!