This document discusses squeeze and excitation networks which aim to improve convolutional neural network representations by modeling interdependencies between channels. It introduces the squeeze and excitation block which squeezes global spatial information into channel descriptors using global average pooling, then captures channel dependencies through a gating mechanism to emphasize informative features. The block can be inserted into networks like VGGNet, ResNet, and Inception to boost performance on tasks like ImageNet classification, scene recognition, and object detection.

1. Squeeze and Excitation
Networks
REVIEWER. JUN YEONG PARK

2. Introduction
▪ With the goal of improving the quality of representations by
explicitly modelling the interdependencies between the
channels of its convolutional features.
▪ Learn to use global information to selectively emphasize
informative features and suppress less useful ones.

3. Related Works
▪ VGGNets
▪ Inception

4. Related Works
▪ ResNet

5. Related Works
▪ DenseNet

6. Squeeze and Excitation Block(SE block)
▪ 𝐹𝑡𝑟 : mapping 𝑋 ∈ ℝ 𝐻′×𝑊′×𝐶′
to 𝑈 ∈ ℝ 𝐻×𝑊×𝐶
▪ 𝐹𝑠𝑞 : squeeze global spatial information into a channel
descriptor
▪ 𝐹𝑒𝑥 : fully capture channel-wise dependencies
▪ 𝐹𝑠𝑐𝑎𝑙𝑒 : channel-wise multiplication

7. Squeeze
▪ Squeeze global spatial information into a channel descriptor
▪ Using Global Average Pooling(GAP)
▪ 𝑧 𝑐 = 𝐹𝑠𝑞 𝒖 𝑐 =
1
𝐻×𝑊 𝑖=1
𝐻
𝑗=1
𝑊
𝑢 𝑐(𝑖, 𝑗)

8. Excitation
▪ Fully capture channel-wise dependencies.
▪ 𝐹𝑒𝑥 𝒛, 𝑾 = 𝜎 𝑔 𝒛, 𝑾 = 𝜎 𝑾2 𝛿 𝑾1 𝒛 , 𝛿 is ReLU
▪ Why sigmoid?
- non-linear function
- ensure that multiple channels are allowed to be emphasized

9. Instantiations
▪ Networks without skip-connection(ex. VGGNet), SE block is
inserted after non-linearity following each convolution.
▪ And SE block also can be integrated into more complex
architectures(ex. ResNet, Inception)

10. Instantiations
▪ Inception

11. Instantiations
▪ ResNet

12. Computational Complexity
▪ Parameters =
2
𝑁 𝑠=1
𝑆
𝑁𝑠 ⋅ 𝐶𝑠
2, r is reduction ratio

13. Experiments - ImageNet

14. Experiments – Place365

15. Experiments - COCO

16. The role of excitation