deep learning from scratch

1. Feedback

2. Interaction Lab. Kumoh National Institute of Technology
Deep Learning from Scratch
chapter 7. CNN
JaeYeop Jeong

3. ■Intro
■Convolution Layer
■Pooling Layer
■Implement
■Visualization
Agenda
Interaction Lab., Kumoh National Institue of Technology 3

4. ■Previous neural network
 Fully-connected : Affine layer
• Combines with all neurons in adjacent layers
Intro
Interaction Lab., Kumoh National Institue of Technology 4

5. ■Problem of previous neural network
 Data is ignored
• Image consists of width, height, and channel 3D information
• Flatten 3-D data into 1-D data
Intro
Interaction Lab., Kumoh National Institue of Technology 5

6. ■Convolution Neural Network
 Plus Convolution layer and Pooling Layer
• Use 3-D data
■ Advantage of data shape
Intro
Interaction Lab., Kumoh National Institue of Technology 6

7. ■Convolution operation
 Convolution layer
• Filter window
• Bias : 1 x 1
• Weight : filter parameter
Convolution Layer
Interaction Lab., Kumoh National Institue of Technology 7
Input Filter Bias Output

8. ■Convolution operation
Convolution Layer
Interaction Lab., Kumoh National Institue of Technology 8

9. ■Padding
 Fills around the data with specific values
 Resize the output to large
Convolution Layer
Interaction Lab., Kumoh National Institue of Technology 9
(4, 4)
Padding : 1
(3, 3)
Filter
(4, 4)
Output

10. ■Stride
 Stride : 2
 Resize the output to small
Convolution Layer
Interaction Lab., Kumoh National Institue of Technology 10

11. ■ Stride and Padding
 Input : (H, W), padding : P, stride : S, filter : (FH, FW)
• 𝑂𝐻 =
𝐻+2𝑃 −𝐹𝐻
𝑆
+ 1
• 𝑂𝑊 =
𝑊+2𝑃 −𝐹𝑊
𝑆
+ 1
 Input : (4, 4), padding : 1, stride : 1, filter : (3, 3)
• 𝑂𝐻 =
4+2 ∗1 −3
1
+ 1 = 4
• 𝑂𝑊 =
4+2 ∗1 −3
1
+ 1 = 4
 Input : (7, 7), padding : 0, stride : 2, filter : (3, 3)
• 𝑂𝐻 =
7+2 ∗0 − 3
2
+ 1 = 3
• 𝑂𝑊 =
7+2 ∗0 − 3
2
+ 1 = 3
 Input : (28, 31), padding : 2, stride : 3, filter : (5, 5)
• 𝑂𝐻 =
28 + 2 ∗ 2 − 5
5
+ 1 = 10
• 𝑂𝑊 =
31 + 2 ∗ 2 −5
5
+ 1 = 11
Convolution Layer
Interaction Lab., Kumoh National Institue of Technology 11
Value must be integer

12. ■3-D data convolution
 Plus channel
• Input channel = filter channel
Convolution Layer
Interaction Lab., Kumoh National Institue of Technology 12

13. ■3-D data convolution
 Output with multiple channels
• Use multiple filters
Convolution Layer
Interaction Lab., Kumoh National Institue of Technology 13

14. ■Batch
 3-D data -> 4-D data
• (H, W, C) -> (Data, H, W, C)
 Convolution operation on n data
Convolution Layer
Interaction Lab., Kumoh National Institue of Technology 14

15. ■Pooling layer
 Not use all data
• Pooling window = size of stride
■ 2 x 2 window 2 stride
■ 3 x 3 window 3 stride
■ 4 x 4 window 4 stride
Pooling Layer
Interaction Lab., Kumoh National Institue of Technology 15
2 x 2
window

16. ■Pooling layer
 No parameter for training
 Not change channel size
 Stable
Pooling Layer
Interaction Lab., Kumoh National Institue of Technology 16

17. ■Im2col
 Not use for loop
• Performance : numpy
Implement
Interaction Lab., Kumoh National Institue of Technology 17
input
4-D data to 2-D data
Filter application order

18. Implement
Interaction Lab., Kumoh National Institue of Technology 18

19. Q&A