SlideShare a Scribd company logo

Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step

•
•
Ahmed Gad
Ahmed Gad

In image analysis, #convolutional neural networks (#CNNs or #ConvNets for short) are time and memory efficient than fully connected (#FC) networks. But why? What are the advantages of ConvNets over FC networks in image analysis? How is #ConvNet derived from FC networks? Where the term #convolution in CNNs came from? These questions are to be answered in this #presentation. Image analysis has a number of challenges such as classification, object detection, recognition, description, etc. If an image classifier, for example, is to be created, it should be able to work with a high accuracy even with variations such as occlusion, illumination changes, viewing angles, and others. The traditional pipeline of image classification with its main step of feature engineering is not suitable for working in rich environments. Even experts in the field won’t be able to give a single or a group of features that are able to reach high accuracy under different variations. Motivated by this problem, the idea of feature learning came out. The suitable features to work with images are learned automatically. This is the reason why artificial neural networks (ANNs) are one of the robust ways of image analysis. Based on a learning algorithm such as gradient descent (GD), ANN learns the image features automatically. The raw image is applied to the ANN and ANN is responsible for generating the features describing it.

Data & Analytics
1 of 122
Download to read offline
Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step Ahmed Fawzy Gad ahmed.fawzy@ci.menofia.edu.eg MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION ‫المنوفية‬ ‫جامعة‬ ‫الحاسبات‬ ‫كلية‬‫والمعلومات‬ ‫المنوفية‬ ‫جامعة‬ Ahmed F. Gad 18-May-2018
15 8 9 10 17 22 20 3015 Input Image 3x3 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Weight 16 𝒘 𝟎 𝟎 𝒘 𝟏𝟓 𝟎 Total Number of Weights For First Pixel Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Weight 16 𝒘 𝟎 𝟏 𝒘 𝟏𝟓 𝟏 Total Number of Weights For Second Pixel Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 𝒘 𝟒 𝟐 𝒘 𝟓 𝟐 𝒘 𝟔 𝟐 𝒘 𝟕 𝟐 𝒘 𝟖 𝟐 𝒘 𝟗 𝟐 𝒘 𝟏𝟎 𝟐 𝒘 𝟏𝟏 𝟐 𝒘 𝟏𝟐 𝟐 𝒘 𝟏𝟑 𝟐 𝒘 𝟏𝟒 𝟐 𝒘 𝟏𝟓 𝟐 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 𝒘 𝟒 𝟑 𝒘 𝟓 𝟑 𝒘 𝟔 𝟑 𝒘 𝟕 𝟑 𝒘 𝟖 𝟑 𝒘 𝟗 𝟑 𝒘 𝟏𝟎 𝟑 𝒘 𝟏𝟏 𝟑 𝒘 𝟏𝟐 𝟑 𝒘 𝟏𝟑 𝟑 𝒘 𝟏𝟒 𝟑 𝒘 𝟏𝟓 𝟑 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 𝒘 𝟒 𝟒 𝒘 𝟓 𝟒 𝒘 𝟔 𝟒 𝒘 𝟕 𝟒 𝒘 𝟖 𝟒 𝒘 𝟗 𝟒 𝒘 𝟏𝟎 𝟒 𝒘 𝟏𝟏 𝟒 𝒘 𝟏𝟐 𝟒 𝒘 𝟏𝟑 𝟒 𝒘 𝟏𝟒 𝟒 𝒘 𝟏𝟓 𝟒 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 𝒘 𝟒 𝟓 𝒘 𝟓 𝟓 𝒘 𝟔 𝟓 𝒘 𝟕 𝟓 𝒘 𝟖 𝟓 𝒘 𝟗 𝟓 𝒘 𝟏𝟎 𝟓 𝒘 𝟏𝟏 𝟓 𝒘 𝟏𝟐 𝟓 𝒘 𝟏𝟑 𝟓 𝒘 𝟏𝟒 𝟓 𝒘 𝟏𝟓 𝟓 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 𝒘 𝟒 𝟔 𝒘 𝟓 𝟔 𝒘 𝟔 𝟔 𝒘 𝟕 𝟔 𝒘 𝟖 𝟔 𝒘 𝟗 𝟔 𝒘 𝟏𝟎 𝟔 𝒘 𝟏𝟏 𝟔 𝒘 𝟏𝟐 𝟔 𝒘 𝟏𝟑 𝟔 𝒘 𝟏𝟒 𝟔 𝒘 𝟏𝟓 𝟔 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 𝒘 𝟒 𝟕 𝒘 𝟓 𝟕 𝒘 𝟔 𝟕 𝒘 𝟕 𝟕 𝒘 𝟖 𝟕 𝒘 𝟗 𝟕 𝒘 𝟏𝟎 𝟕 𝒘 𝟏𝟏 𝟕 𝒘 𝟏𝟐 𝟕 𝒘 𝟏𝟑 𝟕 𝒘 𝟏𝟒 𝟕 𝒘 𝟏𝟓 𝟕 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 𝒘 𝟒 𝟖 𝒘 𝟓 𝟖 𝒘 𝟔 𝟖 𝒘 𝟕 𝟖 𝒘 𝟖 𝟖 𝒘 𝟗 𝟖 𝒘 𝟏𝟎 𝟖 𝒘 𝟏𝟏 𝟖 𝒘 𝟏𝟐 𝟖 𝒘 𝟏𝟑 𝟖 𝒘 𝟏𝟒 𝟖 𝒘 𝟏𝟓 𝟖 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Bias is Neglected. Ahmed F. Gad
Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500 Too Many Parameters Ahmed F. Gad
9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500+ 810+4,500=5,310= Too Many Parameters Ahmed F. Gad
Input Image 32x32 Too Many Parameters Ahmed F. Gad
Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron Input Image 32x32 Too Many Parameters Ahmed F. Gad
1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 Too Many Parameters Ahmed F. Gad
1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 CNN can create a large network but with less number of parameters than FC networks Too Many Parameters Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Number of Parameters for First Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Number of Parameters for Second Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 Continue Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 All Connections Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Hidden Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Total Parameters After Grouping Neurons 9 Pixels 4 Weight Total Parameters = 9*4=36Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% More Reduction Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons Each neuron will process an image region of a specific size. In this example, region size is 2x2.
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels 4 Weights Parameters for First Hidden Neurons Group Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels Parameters for First Hidden Neurons Group 4*4=16 Weights Parameters for Entire Network 4 Weights Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Saved Parameters 144-16=128 Reduction % (128/144)*100 =88.89% Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘 𝟑 `𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 8*𝒘` 𝟎 𝟎 +9*𝒘` 𝟏 𝟎 +17*𝒘` 𝟐 𝟎 +22𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 10*𝒘` 𝟎 𝟎 +17*𝒘` 𝟏 𝟎 +20*𝒘` 𝟐 𝟎 +15𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 17*𝒘 𝟎 𝟎 +22*𝒘 𝟏 𝟎 +15*𝒘 𝟐 𝟎 +30𝒘 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
References • Aghdam, Hamed Habibi, and Elnaz Jahani Heravi. Guide to Convolutional Neural Networks: A Practical Application to Traffic- Sign Detection and Classification. Springer, 2017. • Derivation of Convolutional Neural Network from Fully Connected Network Step-By-Step • https://www.linkedin.com/pulse/derivation-convolutional-neural-network- from-fully-connected-gad • https://www.slideshare.net/AhmedGadFCIT/derivation-of-convolutional- neural-network-convnet-from-fully-connected-network • https://www.kdnuggets.com/2018/04/derivation-convolutional-neural- network-fully-connected-step-by-step.html Ahmed F. Gad

Recommended

Roughset & it’s variants
Roughset & it’s variantsRoughset & it’s variants
Roughset & it’s variantsDr. Rajdeep Chatterjee
 
Bayesian Deep Learning
Bayesian Deep LearningBayesian Deep Learning
Bayesian Deep LearningRayKim51
 
Artificial Neural Network and its Applications
Artificial Neural Network and its ApplicationsArtificial Neural Network and its Applications
Artificial Neural Network and its Applicationsshritosh kumar
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Gaurav Mittal
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketakiKetaki Patwari
 
Fundamental, An Introduction to Neural Networks
Fundamental, An Introduction to Neural NetworksFundamental, An Introduction to Neural Networks
Fundamental, An Introduction to Neural NetworksNelson Piedra
 
Markov decision process
Markov decision processMarkov decision process
Markov decision processHamed Abdi
 
Cnn
CnnCnn
CnnNirthika Rajendran
 

Recommended

Roughset & it’s variants
Roughset & it’s variantsRoughset & it’s variants
Roughset & it’s variantsDr. Rajdeep Chatterjee
 
Bayesian Deep Learning
Bayesian Deep LearningBayesian Deep Learning
Bayesian Deep LearningRayKim51
 
Artificial Neural Network and its Applications
Artificial Neural Network and its ApplicationsArtificial Neural Network and its Applications
Artificial Neural Network and its Applicationsshritosh kumar
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Gaurav Mittal
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketakiKetaki Patwari
 
Fundamental, An Introduction to Neural Networks
Fundamental, An Introduction to Neural NetworksFundamental, An Introduction to Neural Networks
Fundamental, An Introduction to Neural NetworksNelson Piedra
 
Markov decision process
Markov decision processMarkov decision process
Markov decision processHamed Abdi
 
Cnn
CnnCnn
CnnNirthika Rajendran
 
backpropagation in neural networks
backpropagation in neural networksbackpropagation in neural networks
backpropagation in neural networksAkash Goel
 
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...Simplilearn
 
Perceptron (neural network)
Perceptron (neural network)Perceptron (neural network)
Perceptron (neural network)EdutechLearners
 
Multilayer perceptron
Multilayer perceptronMultilayer perceptron
Multilayer perceptronomaraldabash
 
Wasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 IWasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 ISungbin Lim
 
AI IEEE
AI IEEEAI IEEE
AI IEEEUtsav Yagnik
 
Hebb network
Hebb networkHebb network
Hebb networkSiksha 'O' Anusandhan (Deemed to be University )
 
Mc culloch pitts neuron
Mc culloch pitts neuronMc culloch pitts neuron
Mc culloch pitts neuronSiksha 'O' Anusandhan (Deemed to be University )
 
Euclidean Distance And Manhattan Distance
Euclidean Distance And Manhattan DistanceEuclidean Distance And Manhattan Distance
Euclidean Distance And Manhattan DistanceTharuka Vishwajith Sarathchandra
 
Kernels and Support Vector Machines
Kernels and Support Vector MachinesKernels and Support Vector Machines
Kernels and Support Vector MachinesEdgar Marca
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural NetworksDatabricks
 
Neural Networks: Principal Component Analysis (PCA)
Neural Networks: Principal Component Analysis (PCA)Neural Networks: Principal Component Analysis (PCA)
Neural Networks: Principal Component Analysis (PCA)Mostafa G. M. Mostafa
 
Truth management system
Truth management systemTruth management system
Truth management systemMohammad Kamrul Hasan
 
Perceptron & Neural Networks
Perceptron & Neural NetworksPerceptron & Neural Networks
Perceptron & Neural NetworksNAGUR SHAREEF SHAIK
 
CNN Tutorial
CNN TutorialCNN Tutorial
CNN TutorialSungjoon Choi
 
Artifical Neural Network and its applications
Artifical Neural Network and its applicationsArtifical Neural Network and its applications
Artifical Neural Network and its applicationsSangeeta Tiwari
 
K Nearest Neighbor Algorithm
K Nearest Neighbor AlgorithmK Nearest Neighbor Algorithm
K Nearest Neighbor AlgorithmTharuka Vishwajith Sarathchandra
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronMostafa G. M. Mostafa
 
Fuzzy logic - Approximate reasoning
Fuzzy logic - Approximate reasoningFuzzy logic - Approximate reasoning
Fuzzy logic - Approximate reasoningDr. C.V. Suresh Babu
 
Physics-Informed Machine Learning
Physics-Informed Machine LearningPhysics-Informed Machine Learning
Physics-Informed Machine LearningOmarYounis21
 
ICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmAhmed Gad
 
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...Ahmed Gad
 

More Related Content

What's hot

backpropagation in neural networks
backpropagation in neural networksbackpropagation in neural networks
backpropagation in neural networksAkash Goel
 
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...Simplilearn
 
Perceptron (neural network)
Perceptron (neural network)Perceptron (neural network)
Perceptron (neural network)EdutechLearners
 
Multilayer perceptron
Multilayer perceptronMultilayer perceptron
Multilayer perceptronomaraldabash
 
Wasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 IWasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 ISungbin Lim
 
Kernels and Support Vector Machines
Kernels and Support Vector MachinesKernels and Support Vector Machines
Kernels and Support Vector MachinesEdgar Marca
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural NetworksDatabricks
 
Neural Networks: Principal Component Analysis (PCA)
Neural Networks: Principal Component Analysis (PCA)Neural Networks: Principal Component Analysis (PCA)
Neural Networks: Principal Component Analysis (PCA)Mostafa G. M. Mostafa
 
Artifical Neural Network and its applications
Artifical Neural Network and its applicationsArtifical Neural Network and its applications
Artifical Neural Network and its applicationsSangeeta Tiwari
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronMostafa G. M. Mostafa
 
Fuzzy logic - Approximate reasoning
Fuzzy logic - Approximate reasoningFuzzy logic - Approximate reasoning
Fuzzy logic - Approximate reasoningDr. C.V. Suresh Babu
 
Physics-Informed Machine Learning
Physics-Informed Machine LearningPhysics-Informed Machine Learning
Physics-Informed Machine LearningOmarYounis21
 

What's hot (20)

backpropagation in neural networks
backpropagation in neural networksbackpropagation in neural networks
backpropagation in neural networks
 
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
Artificial Neural Network | Deep Neural Network Explained | Artificial Neural...
 
Perceptron (neural network)
Perceptron (neural network)Perceptron (neural network)
Perceptron (neural network)
 
Multilayer perceptron
Multilayer perceptronMultilayer perceptron
Multilayer perceptron
 
Wasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 IWasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 I
 
AI IEEE
AI IEEEAI IEEE
AI IEEE
 
Hebb network
Hebb networkHebb network
Hebb network
 
Mc culloch pitts neuron
Mc culloch pitts neuronMc culloch pitts neuron
Mc culloch pitts neuron
 
Euclidean Distance And Manhattan Distance
Euclidean Distance And Manhattan DistanceEuclidean Distance And Manhattan Distance
Euclidean Distance And Manhattan Distance
 
Kernels and Support Vector Machines
Kernels and Support Vector MachinesKernels and Support Vector Machines
Kernels and Support Vector Machines
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural Networks
 
Neural Networks: Principal Component Analysis (PCA)
Neural Networks: Principal Component Analysis (PCA)Neural Networks: Principal Component Analysis (PCA)
Neural Networks: Principal Component Analysis (PCA)
 
Truth management system
Truth management systemTruth management system
Truth management system
 
Perceptron & Neural Networks
Perceptron & Neural NetworksPerceptron & Neural Networks
Perceptron & Neural Networks
 
CNN Tutorial
CNN TutorialCNN Tutorial
CNN Tutorial
 
Artifical Neural Network and its applications
Artifical Neural Network and its applicationsArtifical Neural Network and its applications
Artifical Neural Network and its applications
 
K Nearest Neighbor Algorithm
K Nearest Neighbor AlgorithmK Nearest Neighbor Algorithm
K Nearest Neighbor Algorithm
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer Perceptron
 
Fuzzy logic - Approximate reasoning
Fuzzy logic - Approximate reasoningFuzzy logic - Approximate reasoning
Fuzzy logic - Approximate reasoning
 
Physics-Informed Machine Learning
Physics-Informed Machine LearningPhysics-Informed Machine Learning
Physics-Informed Machine Learning
 

More from Ahmed Gad

ICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmAhmed Gad
 
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...Ahmed Gad
 
Python for Computer Vision - Revision 2nd Edition
Python for Computer Vision - Revision 2nd EditionPython for Computer Vision - Revision 2nd Edition
Python for Computer Vision - Revision 2nd EditionAhmed Gad
 
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...Ahmed Gad
 
M.Sc. Thesis - Automatic People Counting in Crowded Scenes
M.Sc. Thesis - Automatic People Counting in Crowded ScenesM.Sc. Thesis - Automatic People Counting in Crowded Scenes
M.Sc. Thesis - Automatic People Counting in Crowded ScenesAhmed Gad
 
Introduction to Optimization with Genetic Algorithm (GA)
Introduction to Optimization with Genetic Algorithm (GA)Introduction to Optimization with Genetic Algorithm (GA)
Introduction to Optimization with Genetic Algorithm (GA)Ahmed Gad
 
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...Ahmed Gad
 
Avoid Overfitting with Regularization
Avoid Overfitting with RegularizationAvoid Overfitting with Regularization
Avoid Overfitting with RegularizationAhmed Gad
 
Genetic Algorithm (GA) Optimization - Step-by-Step Example
Genetic Algorithm (GA) Optimization - Step-by-Step ExampleGenetic Algorithm (GA) Optimization - Step-by-Step Example
Genetic Algorithm (GA) Optimization - Step-by-Step ExampleAhmed Gad
 
ICCES 2017 - Crowd Density Estimation Method using Regression Analysis
ICCES 2017 - Crowd Density Estimation Method using Regression AnalysisICCES 2017 - Crowd Density Estimation Method using Regression Analysis
ICCES 2017 - Crowd Density Estimation Method using Regression AnalysisAhmed Gad
 
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
Backpropagation: Understanding How to Update ANNs Weights Step-by-StepBackpropagation: Understanding How to Update ANNs Weights Step-by-Step
Backpropagation: Understanding How to Update ANNs Weights Step-by-StepAhmed Gad
 
Computer Vision: Correlation, Convolution, and Gradient
Computer Vision: Correlation, Convolution, and GradientComputer Vision: Correlation, Convolution, and Gradient
Computer Vision: Correlation, Convolution, and GradientAhmed Gad
 
Python for Computer Vision - Revision
Python for Computer Vision - RevisionPython for Computer Vision - Revision
Python for Computer Vision - RevisionAhmed Gad
 
Anime Studio Pro 10 Tutorial as Part of Multimedia Course
Anime Studio Pro 10 Tutorial as Part of Multimedia CourseAnime Studio Pro 10 Tutorial as Part of Multimedia Course
Anime Studio Pro 10 Tutorial as Part of Multimedia CourseAhmed Gad
 
Brief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNsBrief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNsAhmed Gad
 
Operations in Digital Image Processing + Convolution by Example
Operations in Digital Image Processing + Convolution by ExampleOperations in Digital Image Processing + Convolution by Example
Operations in Digital Image Processing + Convolution by ExampleAhmed Gad
 
MATLAB Code + Description : Real-Time Object Motion Detection and Tracking
MATLAB Code + Description : Real-Time Object Motion Detection and TrackingMATLAB Code + Description : Real-Time Object Motion Detection and Tracking
MATLAB Code + Description : Real-Time Object Motion Detection and TrackingAhmed Gad
 
MATLAB Code + Description : Very Simple Automatic English Optical Character R...
MATLAB Code + Description : Very Simple Automatic English Optical Character R...MATLAB Code + Description : Very Simple Automatic English Optical Character R...
MATLAB Code + Description : Very Simple Automatic English Optical Character R...Ahmed Gad
 
Graduation Project - Face Login : A Robust Face Identification System for Sec...
Graduation Project - Face Login : A Robust Face Identification System for Sec...Graduation Project - Face Login : A Robust Face Identification System for Sec...
Graduation Project - Face Login : A Robust Face Identification System for Sec...Ahmed Gad
 
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Ahmed Gad
 

More from Ahmed Gad (20)

ICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic AlgorithmICEIT'20 Cython for Speeding-up Genetic Algorithm
ICEIT'20 Cython for Speeding-up Genetic Algorithm
 
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
NumPyCNNAndroid: A Library for Straightforward Implementation of Convolutiona...
 
Python for Computer Vision - Revision 2nd Edition
Python for Computer Vision - Revision 2nd EditionPython for Computer Vision - Revision 2nd Edition
Python for Computer Vision - Revision 2nd Edition
 
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
Multi-Objective Optimization using Non-Dominated Sorting Genetic Algorithm wi...
 
M.Sc. Thesis - Automatic People Counting in Crowded Scenes
M.Sc. Thesis - Automatic People Counting in Crowded ScenesM.Sc. Thesis - Automatic People Counting in Crowded Scenes
M.Sc. Thesis - Automatic People Counting in Crowded Scenes
 
Introduction to Optimization with Genetic Algorithm (GA)
Introduction to Optimization with Genetic Algorithm (GA)Introduction to Optimization with Genetic Algorithm (GA)
Introduction to Optimization with Genetic Algorithm (GA)
 
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
Derivation of Convolutional Neural Network (ConvNet) from Fully Connected Net...
 
Avoid Overfitting with Regularization
Avoid Overfitting with RegularizationAvoid Overfitting with Regularization
Avoid Overfitting with Regularization
 
Genetic Algorithm (GA) Optimization - Step-by-Step Example
Genetic Algorithm (GA) Optimization - Step-by-Step ExampleGenetic Algorithm (GA) Optimization - Step-by-Step Example
Genetic Algorithm (GA) Optimization - Step-by-Step Example
 
ICCES 2017 - Crowd Density Estimation Method using Regression Analysis
ICCES 2017 - Crowd Density Estimation Method using Regression AnalysisICCES 2017 - Crowd Density Estimation Method using Regression Analysis
ICCES 2017 - Crowd Density Estimation Method using Regression Analysis
 
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
Backpropagation: Understanding How to Update ANNs Weights Step-by-StepBackpropagation: Understanding How to Update ANNs Weights Step-by-Step
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
 
Computer Vision: Correlation, Convolution, and Gradient
Computer Vision: Correlation, Convolution, and GradientComputer Vision: Correlation, Convolution, and Gradient
Computer Vision: Correlation, Convolution, and Gradient
 
Python for Computer Vision - Revision
Python for Computer Vision - RevisionPython for Computer Vision - Revision
Python for Computer Vision - Revision
 
Anime Studio Pro 10 Tutorial as Part of Multimedia Course
Anime Studio Pro 10 Tutorial as Part of Multimedia CourseAnime Studio Pro 10 Tutorial as Part of Multimedia Course
Anime Studio Pro 10 Tutorial as Part of Multimedia Course
 
Brief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNsBrief Introduction to Deep Learning + Solving XOR using ANNs
Brief Introduction to Deep Learning + Solving XOR using ANNs
 
Operations in Digital Image Processing + Convolution by Example
Operations in Digital Image Processing + Convolution by ExampleOperations in Digital Image Processing + Convolution by Example
Operations in Digital Image Processing + Convolution by Example
 
MATLAB Code + Description : Real-Time Object Motion Detection and Tracking
MATLAB Code + Description : Real-Time Object Motion Detection and TrackingMATLAB Code + Description : Real-Time Object Motion Detection and Tracking
MATLAB Code + Description : Real-Time Object Motion Detection and Tracking
 
MATLAB Code + Description : Very Simple Automatic English Optical Character R...
MATLAB Code + Description : Very Simple Automatic English Optical Character R...MATLAB Code + Description : Very Simple Automatic English Optical Character R...
MATLAB Code + Description : Very Simple Automatic English Optical Character R...
 
Graduation Project - Face Login : A Robust Face Identification System for Sec...
Graduation Project - Face Login : A Robust Face Identification System for Sec...Graduation Project - Face Login : A Robust Face Identification System for Sec...
Graduation Project - Face Login : A Robust Face Identification System for Sec...
 
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
Introduction to MATrices LABoratory (MATLAB) as Part of Digital Signal Proces...
 

Recently uploaded

Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Boston Institute of Analytics
 
Semantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxSemantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxMike Bennett
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfblazblazml
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBoston Institute of Analytics
 
Networking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxNetworking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxHimangsuNath
 
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxThe Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxTasha Penwell
 
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024Susanna-Assunta Sansone
 
Cyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataCyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataTecnoIncentive
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...Jack Cole
 
Principles and Practices of Data Visualization
Principles and Practices of Data VisualizationPrinciples and Practices of Data Visualization
Principles and Practices of Data VisualizationKianJazayeri1
 
Decoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectDecoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectBoston Institute of Analytics
 
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfWorld Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfsimulationsindia
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelBoston Institute of Analytics
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksdeepakthakur548787
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxaleedritatuxx
 
SMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxSMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxHaritikaChhatwal1
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaManalVerma4
 
Real-Time AI Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max PrincetonReal-Time AI Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max PrincetonTimothy Spann
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our WorldEduminds Learning
 

Recently uploaded (20)

Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
 
Semantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptxSemantic Shed - Squashing and Squeezing.pptx
Semantic Shed - Squashing and Squeezing.pptx
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
 
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis ProjectBank Loan Approval Analysis: A Comprehensive Data Analysis Project
Bank Loan Approval Analysis: A Comprehensive Data Analysis Project
 
Networking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxNetworking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptx
 
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxThe Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
 
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
 
Cyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded dataCyber awareness ppt on the recorded data
Cyber awareness ppt on the recorded data
 
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
why-transparency-and-traceability-are-essential-for-sustainable-supply-chains...
 
Principles and Practices of Data Visualization
Principles and Practices of Data VisualizationPrinciples and Practices of Data Visualization
Principles and Practices of Data Visualization
 
Decoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis ProjectDecoding Patterns: Customer Churn Prediction Data Analysis Project
Decoding Patterns: Customer Churn Prediction Data Analysis Project
 
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdfWorld Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
World Economic Forum Metaverse Ecosystem By Utpal Chakraborty.pdf
 
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis modelDecoding Movie Sentiments: Analyzing Reviews with Data Analysis model
Decoding Movie Sentiments: Analyzing Reviews with Data Analysis model
 
Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing works
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
 
SMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxSMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptx
 
IBEF report on the Insurance market in India
IBEF report on the Insurance market in IndiaIBEF report on the Insurance market in India
IBEF report on the Insurance market in India
 
Real-Time AI Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max PrincetonReal-Time AI Streaming - AI Max Princeton
Real-Time AI Streaming - AI Max Princeton
 
Insurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis ProjectInsurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis Project
 
Learn How Data Science Changes Our World
Learn How Data Science Changes Our WorldLearn How Data Science Changes Our World
Learn How Data Science Changes Our World
 

Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step

  • 1. Derivation of Convolutional Neural Network from Fully Connected Network Step-by-Step Ahmed Fawzy Gad ahmed.fawzy@ci.menofia.edu.eg MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION ‫المنوفية‬ ‫جامعة‬ ‫الحاسبات‬ ‫كلية‬‫والمعلومات‬ ‫المنوفية‬ ‫جامعة‬ Ahmed F. Gad 18-May-2018
  • 2. 15 8 9 10 17 22 20 3015 Input Image 3x3 Ahmed F. Gad
  • 3. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 4. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 5. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 6. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 7. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 8. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 9. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 10. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 11. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 12. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 13. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 Ahmed F. Gad
  • 14. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer Ahmed F. Gad
  • 15. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
  • 16. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Fully Connected Network Ahmed F. Gad
  • 17. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
  • 18. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 Fully Connected Network Ahmed F. Gad
  • 19. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 Fully Connected Network Ahmed F. Gad
  • 20. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 Fully Connected Network Ahmed F. Gad
  • 21. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 𝒘 𝒚 𝒙 𝒙: Pixel Index 𝒚: Hidden Neuron Index Fully Connected Network Ahmed F. Gad
  • 22. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Weight 16 𝒘 𝟎 𝟎 𝒘 𝟏𝟓 𝟎 Total Number of Weights For First Pixel Ahmed F. Gad
  • 23. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 24. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Ahmed F. Gad
  • 25. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 𝒘 𝟒 𝟏 𝒘 𝟓 𝟏 𝒘 𝟔 𝟏 𝒘 𝟕 𝟏 𝒘 𝟖 𝟏 𝒘 𝟗 𝟏 𝒘 𝟏𝟎 𝟏 𝒘 𝟏𝟏 𝟏 𝒘 𝟏𝟐 𝟏 𝒘 𝟏𝟑 𝟏 𝒘 𝟏𝟒 𝟏 𝒘 𝟏𝟓 𝟏 Weight 16 𝒘 𝟎 𝟏 𝒘 𝟏𝟓 𝟏 Total Number of Weights For Second Pixel Ahmed F. Gad
  • 26. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 𝒘 𝟒 𝟐 𝒘 𝟓 𝟐 𝒘 𝟔 𝟐 𝒘 𝟕 𝟐 𝒘 𝟖 𝟐 𝒘 𝟗 𝟐 𝒘 𝟏𝟎 𝟐 𝒘 𝟏𝟏 𝟐 𝒘 𝟏𝟐 𝟐 𝒘 𝟏𝟑 𝟐 𝒘 𝟏𝟒 𝟐 𝒘 𝟏𝟓 𝟐 Ahmed F. Gad
  • 27. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 𝒘 𝟒 𝟑 𝒘 𝟓 𝟑 𝒘 𝟔 𝟑 𝒘 𝟕 𝟑 𝒘 𝟖 𝟑 𝒘 𝟗 𝟑 𝒘 𝟏𝟎 𝟑 𝒘 𝟏𝟏 𝟑 𝒘 𝟏𝟐 𝟑 𝒘 𝟏𝟑 𝟑 𝒘 𝟏𝟒 𝟑 𝒘 𝟏𝟓 𝟑 Ahmed F. Gad
  • 28. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 𝒘 𝟒 𝟒 𝒘 𝟓 𝟒 𝒘 𝟔 𝟒 𝒘 𝟕 𝟒 𝒘 𝟖 𝟒 𝒘 𝟗 𝟒 𝒘 𝟏𝟎 𝟒 𝒘 𝟏𝟏 𝟒 𝒘 𝟏𝟐 𝟒 𝒘 𝟏𝟑 𝟒 𝒘 𝟏𝟒 𝟒 𝒘 𝟏𝟓 𝟒 Ahmed F. Gad
  • 29. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 𝒘 𝟒 𝟓 𝒘 𝟓 𝟓 𝒘 𝟔 𝟓 𝒘 𝟕 𝟓 𝒘 𝟖 𝟓 𝒘 𝟗 𝟓 𝒘 𝟏𝟎 𝟓 𝒘 𝟏𝟏 𝟓 𝒘 𝟏𝟐 𝟓 𝒘 𝟏𝟑 𝟓 𝒘 𝟏𝟒 𝟓 𝒘 𝟏𝟓 𝟓 Ahmed F. Gad
  • 30. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 𝒘 𝟒 𝟔 𝒘 𝟓 𝟔 𝒘 𝟔 𝟔 𝒘 𝟕 𝟔 𝒘 𝟖 𝟔 𝒘 𝟗 𝟔 𝒘 𝟏𝟎 𝟔 𝒘 𝟏𝟏 𝟔 𝒘 𝟏𝟐 𝟔 𝒘 𝟏𝟑 𝟔 𝒘 𝟏𝟒 𝟔 𝒘 𝟏𝟓 𝟔 Ahmed F. Gad
  • 31. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 𝒘 𝟒 𝟕 𝒘 𝟓 𝟕 𝒘 𝟔 𝟕 𝒘 𝟕 𝟕 𝒘 𝟖 𝟕 𝒘 𝟗 𝟕 𝒘 𝟏𝟎 𝟕 𝒘 𝟏𝟏 𝟕 𝒘 𝟏𝟐 𝟕 𝒘 𝟏𝟑 𝟕 𝒘 𝟏𝟒 𝟕 𝒘 𝟏𝟓 𝟕 Ahmed F. Gad
  • 32. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 𝒘 𝟒 𝟖 𝒘 𝟓 𝟖 𝒘 𝟔 𝟖 𝒘 𝟕 𝟖 𝒘 𝟖 𝟖 𝒘 𝟗 𝟖 𝒘 𝟏𝟎 𝟖 𝒘 𝟏𝟏 𝟖 𝒘 𝟏𝟐 𝟖 𝒘 𝟏𝟑 𝟖 𝒘 𝟏𝟒 𝟖 𝒘 𝟏𝟓 𝟖 Ahmed F. Gad
  • 33. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 34. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Ahmed F. Gad
  • 35. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
  • 36. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Number of Parameters For the Entire Network 9 Pixels 16 Weight Total Parameters = 9*16=144 Bias is Neglected. Ahmed F. Gad
  • 37. Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
  • 38. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron Too Many Parameters Ahmed F. Gad
  • 39. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500 Too Many Parameters Ahmed F. Gad
  • 40. 9*90=810 Hidden Layer 1 90 Neuron Input Layer 9 Neuron Hidden Layer 2 50 Neuron 90*50=4,500+ 810+4,500=5,310= Too Many Parameters Ahmed F. Gad
  • 41. Input Image 32x32 Too Many Parameters Ahmed F. Gad
  • 42. Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron Input Image 32x32 Too Many Parameters Ahmed F. Gad
  • 43. 1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 Too Many Parameters Ahmed F. Gad
  • 44. 1,024*500 Hidden Layer 1 500 Neuron Input Layer 1,024 Neuron 512,000= Input Image 32x32 CNN can create a large network but with less number of parameters than FC networks Too Many Parameters Ahmed F. Gad
  • 45. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 46. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 47. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 48. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟒 𝟎 𝒘 𝟓 𝟎 𝒘 𝟔 𝟎 𝒘 𝟕 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 49. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟖 𝟎 𝒘 𝟗 𝟎 𝒘 𝟏𝟎 𝟎 𝒘 𝟏𝟏 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 50. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟏𝟐 𝟎 𝒘 𝟏𝟑 𝟎 𝒘 𝟏𝟒 𝟎 𝒘 𝟏𝟓 𝟎 Ahmed F. Gad
  • 51. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  • 52. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  • 53. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Ahmed F. Gad
  • 54. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟎 𝒘 𝟏 𝟎 𝒘 𝟐 𝟎 𝒘 𝟑 𝟎 Number of Parameters for First Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
  • 55. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Ahmed F. Gad
  • 56. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟏 𝒘 𝟏 𝟏 𝒘 𝟐 𝟏 𝒘 𝟑 𝟏 Number of Parameters for Second Pixel After Grouping Neurons 16/4=4 Weights Ahmed F. Gad
  • 57. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟐 𝒘 𝟏 𝟐 𝒘 𝟐 𝟐 𝒘 𝟑 𝟐 Continue Ahmed F. Gad
  • 58. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟑 𝒘 𝟏 𝟑 𝒘 𝟐 𝟑 𝒘 𝟑 𝟑 Continue Ahmed F. Gad
  • 59. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟒 𝒘 𝟏 𝟒 𝒘 𝟐 𝟒 𝒘 𝟑 𝟒 Continue Ahmed F. Gad
  • 60. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟓 𝒘 𝟏 𝟓 𝒘 𝟐 𝟓 𝒘 𝟑 𝟓 Continue Ahmed F. Gad
  • 61. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟔 𝒘 𝟏 𝟔 𝒘 𝟐 𝟔 𝒘 𝟑 𝟔 Continue Ahmed F. Gad
  • 62. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟕 𝒘 𝟏 𝟕 𝒘 𝟐 𝟕 𝒘 𝟑 𝟕 Continue Ahmed F. Gad
  • 63. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘 𝟎 𝟖 𝒘 𝟏 𝟖 𝒘 𝟐 𝟖 𝒘 𝟑 𝟖 Continue Ahmed F. Gad
  • 64. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 All Connections Ahmed F. Gad
  • 65. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Ahmed F. Gad
  • 66. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Hidden Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Ahmed F. Gad
  • 67. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Total Parameters Before Grouping Neurons 9 Pixels 16 Weight Total Parameters = 9*16=144 Total Parameters After Grouping Neurons 9 Pixels 4 Weight Total Parameters = 9*4=36Ahmed F. Gad
  • 68. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Ahmed F. Gad
  • 69. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% Ahmed F. Gad
  • 70. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 36 Saved Parameters 144-36=108 Reduction % (108/144)*100 =75% More Reduction Ahmed F. Gad
  • 71. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 72. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 73. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 74. In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
  • 75. In image analysis, each pixel is highly correlated to pixels surrounding it (i.e. neighbors) Ahmed F. Gad
  • 79. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad
  • 80. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons
  • 81. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Hidden Layer 4 Groups 4 Filters 4 Feature Maps Ahmed F. Gad 4 Neurons Each neuron will process an image region of a specific size. In this example, region size is 2x2.
  • 82. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 83. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 84. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 85. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 86. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 87. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
  • 88. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad 𝒘` 𝒚 𝒙 𝒙: Group Index 𝒚: Index of Group Input
  • 89. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 Ahmed F. Gad
  • 90. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 91. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 92. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 93. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 94. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 95. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 96. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 97. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 98. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 99. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 100. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 101. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 102. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 103. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 104. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 105. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 106. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 107. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 108. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 109. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Ahmed F. Gad
  • 110. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels 4 Weights Parameters for First Hidden Neurons Group Ahmed F. Gad
  • 111. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Correlated Pixels Parameters for First Hidden Neurons Group 4*4=16 Weights Parameters for Entire Network 4 Weights Ahmed F. Gad
  • 112. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Ahmed F. Gad
  • 113. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Before 144 After 16 Saved Parameters 144-16=128 Reduction % (128/144)*100 =88.89% Ahmed F. Gad
  • 114. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 115. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 116. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 15*𝒘` 𝟎 𝟎 +8*𝒘` 𝟏 𝟎 +10*𝒘` 𝟐 𝟎 +17𝒘 𝟑 `𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟎 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 𝒘` 𝟏 𝟎 Ahmed F. Gad
  • 117. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 8*𝒘` 𝟎 𝟎 +9*𝒘` 𝟏 𝟎 +17*𝒘` 𝟐 𝟎 +22𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 118. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 10*𝒘` 𝟎 𝟎 +17*𝒘` 𝟏 𝟎 +20*𝒘` 𝟐 𝟎 +15𝒘` 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 119. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 15 8 9 10 17 22 20 3015 17*𝒘 𝟎 𝟎 +22*𝒘 𝟏 𝟎 +15*𝒘 𝟐 𝟎 +30𝒘 𝟑 𝟎 * 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎 Ahmed F. Gad
  • 120. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
  • 121. 15 8 9 10 17 22 20 3015 15 8 9 10 17 22 20 30 15 Input Image 3x3 Vector 9x1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* 15 8 9 10 17 22 20 3015 𝒘` 𝟎 𝟎 𝒘` 𝟏 𝟎 𝒘` 𝟐 𝟎 𝒘` 𝟑 𝟎* Ahmed F. Gad
  • 122. References • Aghdam, Hamed Habibi, and Elnaz Jahani Heravi. Guide to Convolutional Neural Networks: A Practical Application to Traffic- Sign Detection and Classification. Springer, 2017. • Derivation of Convolutional Neural Network from Fully Connected Network Step-By-Step • https://www.linkedin.com/pulse/derivation-convolutional-neural-network- from-fully-connected-gad • https://www.slideshare.net/AhmedGadFCIT/derivation-of-convolutional- neural-network-convnet-from-fully-connected-network • https://www.kdnuggets.com/2018/04/derivation-convolutional-neural- network-fully-connected-step-by-step.html Ahmed F. Gad