This document discusses anti-aliasing techniques for convolutional neural networks to improve shift-invariance. It first explains the concept of shift-invariance and how aliasing can occur from operations like max pooling and strided convolutions, making networks shift-variant. It then proposes applying anti-aliasing by blurring feature maps before pooling or downsampling to remove high-frequency components and make the representations more shift-equivariant and ultimately shift-invariant. Experimental results show this anti-aliasing approach improves consistency, accuracy, and performance on image translation tasks.

1. Making Convolutional Networks
Shift-Invariant Again
Hyeongmin Lee
Image and Video Pattern Recognition LAB
Electrical and Electronic Engineering Dept, Yonsei University
4th Semester
2019.9.21

2. What is shift-invariancy??

3. What is shift-invariancy??
◆ Shift-variant??
Change in performance??

4. What is shift-invariancy??
◆ Shift-variant??

5. Aliasing

6. Aliasing
◆ Fourier Transform & Frequency Domain
𝑡 𝐹
𝑋 𝐹 = න
−∞
∞
𝑥 𝑡 𝑒−𝑗2𝜋𝐹𝑡 𝑑𝑡

7. Aliasing
◆ Down-sampling in frequency domain
𝑡 𝐹
𝑡 𝐹

8. Aliasing
◆ Discrete Signal in Frequency Domain
𝑓𝑛 2𝜋−2𝜋

9. Aliasing
◆ Discrete Down-Sampling in Frequency Domain
𝑓
𝑓
𝑛 2𝜋−2𝜋
𝑛
2𝜋−2𝜋
Aliasing Aliasing

10. Aliasing in CNN
◆ Max pooling
◆ Average pooling
◆ Strided convolution
0 0 1 1 0 0 1 1 0 1 0 1
0 1 1 0 0 1 1 0 1 1 1 1
Max-pooling
Max-pooling

11. Anti-aliasing

12. Anti-Aliasing
◆ Shift Invariancy & Shift Equivariance
• Shift Equivariance
• Shift Invariancy
Shift-Equivariant ➔ Shift-Invariant

13. Anti-Aliasing
◆ Anti-aliasing
𝑓2𝜋−2𝜋
Low Pass Filtering
(Blurring)
𝑓2𝜋−2𝜋
𝑓2𝜋−2𝜋
Sampling

14. Anti-Aliasing
◆ Anti-aliasing for max pooling
0 0 1 1 0 0 1 1 0 1 0 10 1 1 1 0 1 1 1
Max Sampling
0 1 1 0 0 1 1 0 1 1 1 11 1 1 0 1 1 1 0
Shift-Equivariant!!

15. Anti-Aliasing
◆ Anti-aliasing for max pooling
0 1 1 1 0 1 1 1 0.5 1 0.5 10.5 1 1 0.5 0.5 1 1 0.5
Blurring Sampling
1 1 1 0 1 1 1 0 1 0.5 1 0.51 1 0.5 0.5 1 1 0.5 0.5
Max

16. Results

17. Results
◆ Improvement in Consistency

18. Results
◆ Improvement in Accuracy

19. Results
◆ Improvement in Image Translation

20. Thank You!