Convolutional neural networks (CNNs) are a type of neural network used in image recognition and processing. CNNs use convolutional layers that apply filters to input volumes to extract features at different spatial locations. Backpropagation is used to train CNNs by propagating errors backwards. CNNs have been successfully applied to large-scale image classification tasks using datasets like ImageNet, with AlexNet achieving breakthrough results in 2012. CNNs employ several convolutional layers interspersed with activation functions to process input volumes while avoiding excessive spatial shrinking.

1. Convolutional
Neural Network
Subash Chandra Pakhrin
PhD Student
Wichita State University
Wichita, Kansas

2. Convolutional Neural Network
[1] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in
Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.

3. Back-Propagation
[2] Rumelhart, D., Hinton, G. & Williams, R. Learning representations by back-propagating errors. Nature 323,
533–536 (1986). https://doi.org/10.1038/323533a0

4. Image Net Classification with Deep
Convolutional Neural Networks
[3] Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, Image Net Classification with Deep Convolutional Neural
Networks, Advances in Neural Information Processing Systems 25 (NIPS 2012)

6. Convolution Layer

7. Convolution Layer

8. 7x7 input(spatially) assume 3x3 filter (Stride 1)
5 x 5 output
i.e. 5 spatial location horizontally,
5 spatial location vertically

9. 7x7 input(spatially) assume 3x3 filter (Stride 2)
3 x 3 output
i.e. 3 spatial location horizontally,
3 spatial location vertically

10. A closer look at spatial dimensions:
• We don’t do convolution like this because it produces asymmetric output
7x7 input (spatially) assumes 3x3 filter applied
with stride 3?
Doesn’t fit!!!
Cannot apply 3x3 filter on 7x7 input with stride
3.

11. Output Size formula

12. In practice: Common to zero pad the border
e.g. input 7x7
3x3 filter, applied with stride 1
Pad with 1 pixel border => what is the output?
Output => 7x7 !!!
(recall:)
(N-F)/stride +1

13. Example:
Input volume: 32x32x3
10, 5x5 filters with stride 1, pad 2
New N = 36 after pad 2, F = 5, stride = 1
(N-F)/stride +1
Output volume size:?
32 x 32 x 10
Number of parameters in this layer?
each filter has 5*5*3+1 = 76 parameters (+1 for bias)
=>76*10=760

14. Convolution Layer
For example, if we had 6 5x5 filters, we’ll get 6 separate activation maps.

15. ConvNet is a sequence of Convolutional Layers,
interspersed with activation functions
Caution!!!
32x32 input convolved repeatedly with 5x5 filters shrinks volumes spatially!
Shrinking too fast is not good, doesn’t work well.

17. References:
[1] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document
recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998
[2] Rumelhart, D., Hinton, G. & Williams, R. Learning representations by back-propagating errors.
Nature 323, 533–536 (1986)
[3] Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton, Image Net Classification with Deep
Convolutional Neural Networks, Advances in Neural Information Processing Systems 25 (NIPS 2012)
[4] Fei-Fei Li, Justin Johnson & Serena Yeung, Lecture 5| Convolutional Neural Networks,
https://youtu.be/bNb2fEVKeEo
[5] Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun, Faster R-CNN: Towards Real-Time Object
Detection with Region Proposal Networks, Advances in Neural Information Processing Systems 28
(NIPS 2015)
[6] Daniel Levy, Arzav Jain, Breast Mass Classification from Mammograms using Deep Convolutional
Neural Networks, Computer Vision and Pattern Recognition, NIPS 2016
[7] Jonathan Krause, Justin Johnson, Ranjay Krishna, Li Fei-Fei, A Hierarchical Approach for
Generating Descriptive Image Paragraphs The IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2017, pp. 317-325