이번 논문은 Convolutional Neural Network에서 발생하는 Aliasing 문제를 지적하고, 이를 고전적인 신호처리 기법을 이용하여 해결하는 논문입니다. Paper Link: https://arxiv.org/abs/1904.11486 Youtube Link: https://youtu.be/oTIBFH6M7YM

1. Making Convolutional Networks
Shift-Invariant Again
Hyeongmin Lee
Image and Video Pattern Recognition LAB
Electrical and Electronic Engineering Dept, Yonsei University
5th Semester
PR-252

3. What is shift-invariancy??

4. What is shift-invariancy??
 Shift-variant??
Change in performance??

5. What is shift-invariancy??
 Shift-variant??

6. Aliasing

7. Aliasing
 Fourier Transform & Frequency Domain
𝑡𝑡 Ω
𝑋𝑋 Ω = �
−∞
∞
𝑥𝑥 𝑡𝑡 𝑒𝑒−𝑗𝑗Ω𝑡𝑡 𝑑𝑑𝑑𝑑

8. Aliasing
 Down-sampling in frequency domain
𝑇𝑇 𝑥𝑥 𝑡𝑡 = �
−∞
∞
𝑥𝑥 𝑡𝑡 𝑒𝑒−𝑗𝑗Ω𝑡𝑡 𝑑𝑑𝑑𝑑 = 𝑋𝑋(Ω)
𝑇𝑇 𝑥𝑥 𝑎𝑎𝑡𝑡 = �
−∞
∞
𝑥𝑥 𝑎𝑎𝑡𝑡 𝑒𝑒−𝑗𝑗Ω𝑡𝑡
𝑑𝑑𝑑𝑑 =
1
|𝑎𝑎|
�
−∞
∞
𝑥𝑥 𝑡𝑡′ 𝑒𝑒−𝑗𝑗
Ω
𝑎𝑎
𝑡𝑡′
𝑑𝑑𝑑𝑑′
𝑇𝑇 𝑥𝑥 𝑎𝑎𝑡𝑡 =
1
|𝑎𝑎|
𝑋𝑋(Ω/𝑎𝑎)

9. Aliasing
 Down-sampling in frequency domain
𝑡𝑡 Ω
𝑡𝑡 Ω

10. Aliasing
 Discrete Signal in Frequency Domain (Discrete Time Fourier Transform)
𝜔𝜔𝑛𝑛 2𝜋𝜋−2𝜋𝜋
𝑋𝑋 𝜔𝜔 = �
𝑛𝑛=−∞
∞
𝑥𝑥 𝑛𝑛 𝑒𝑒−𝑗𝑗𝑗𝑗𝑗𝑗
𝑋𝑋 𝜔𝜔 + 2𝜋𝜋 = �
𝑛𝑛=−∞
∞
𝑥𝑥 𝑛𝑛 𝑒𝑒−𝑗𝑗𝑗𝑗𝑗𝑗 𝑒𝑒−𝑗𝑗𝑗𝑗𝑗𝑗𝑗 = �
𝑛𝑛=−∞
∞
𝑥𝑥 𝑛𝑛 𝑒𝑒−𝑗𝑗𝑗𝑗𝑗𝑗 = 𝑋𝑋(𝜔𝜔)

11. Aliasing
 Discrete Down-Sampling in Frequency Domain
𝑓𝑓
𝑓𝑓
𝑛𝑛 2𝜋𝜋−2𝜋𝜋
𝑛𝑛
2𝜋𝜋−2𝜋𝜋
Aliasing Aliasing

12. Aliasing
 Aliasing (Example)

13. Aliasing in CNN
 Max pooling
 Average pooling
 Strided convolution
0 0 1 1 0 0 1 1 0 1 0 1
0 1 1 0 0 1 1 0 1 1 1 1
Max-pooling
Max-pooling

14. Anti-aliasing

15. Anti-Aliasing
 Shift Invariancy & Shift Equivariance
• Shift Equivariance
• Shift Invariancy
Shift-Equivariant  Shift-Invariant

16. Anti-Aliasing
 Anti-aliasing
𝑓𝑓2𝜋𝜋−2𝜋𝜋
Low Pass Filtering
(Blurring)
𝑓𝑓2𝜋𝜋−2𝜋𝜋
𝑓𝑓2𝜋𝜋−2𝜋𝜋
Sampling

17. Anti-Aliasing
 Anti-aliasing for max pooling
0 0 1 1 0 0 1 1 0 1 0 10 1 1 1 0 1 1 1
Max Sampling
0 1 1 0 0 1 1 0 1 1 1 11 1 1 0 1 1 1 0
Shift-Equivariant!!

18. Anti-Aliasing
 Anti-aliasing for max pooling
0 1 1 1 0 1 1 1 0.5 1 0.5 10.5 1 1 0.5 0.5 1 1 0.5
Blurring Subsampling
1 1 1 0 1 1 1 0 1 0.5 1 0.51 1 0.5 0.5 1 1 0.5 0.5
Max

19. Anti-Aliasing
 Anti-aliasing for various sampling operations in CNN
• MaxPool
• AveragePool
• StrideConv

20. Anti-Aliasing
 Down-sampling Kernels

21. Results

22. Results
 Improvement in Consistency

23. Results
 Improvement in Accuracy

24. Results
 Improvement in Accuracy

25. Results
 Improvement in Image Translation

26. Thank You!