2. History:
In 1959, two neurophysiologists David Hubel and Torsten Wiesel experimented and later published
their paper ,“Receptive fields of single neurons in cat’s striate cortex”, described that the neurons
inside the brain of a cat are organized in layered form.
In 1980, inspired by this work, K. Fukusima proposed Neocognitron. This network is considered as
the first theoretical model of CNN.
In 1990, LeCun et al. Developed the modern framework of CNN called LeNet to recognize
handwritten digits.
• The CNN architecture become popular after AlexNet (designed by Krizhevsky et al ) in year 2012.
ILSVRC winner in that year.
ImageNet Large Scale Visual Recognition Challenge.
3. Introduction:
• CNN is a part or subset of ML(Machine learning).
• It is one of the various types of artificial neural network which are used for image
recognition, computer vision (CV) , facial recognition and also for self-driving car.
• CNN is a supervised type of Deep learning.
4. Architecture of CNN
(i) Images and types of images.
Digital image: An Image is a visual representation of something, while a digital image is a binary representation of
visual data.
Types of image: Generally there are 3-types of digital images and these are –
(a) Binary image(b) Grayscale image and (c) RGB image.
Image
Binary image Grayscale image RGB image
(1 bit image) (8 bit image) (24 bit/3 byte image)
5. 1.2 Padding: If we apply convolutional operation on the original image, then we loss the border’s
information of the image.
This is called zero padding.
6. 1.3 Polling: The polling is used to decrease the dimension of it used to shrinks the dim. of
feature maps..
Max, Min , Average polling etc ( Max polling is most popular).
1.3.1 Max polling:
7. 1.3.2 Min polling: It is mostly used when the
image has a light background since min pooling will
select darker pixels.
1.3.3 Avg polling:
Min polling Avg. polling
8. 1.4 Activation function: A.F. decides whether a neuron should be activate for next feature map or not. i.e.
whether the participation of that neuron is important or not in the process of prediction.
Step/Binary, Sigmoid function, Tanh function, ReLu function and Leaky ReLu function etc.
1.4.1 Step fiunction: Binary step function is a threshold-based activation function which means after a certain
threshold neuron is activated and below it threshold neuron is deactivated.
1.4.2 Sigmoid fiunction: The sigmoid function take real numbers as it’s input & bind the output in the range of [0,1].
9. 1.4.3 ReLu fiunction: The rectified linear activation function or ReLU for short is a piecewise linear function that will output
the input directly if it is positive, otherwise, it will output zero.
There is problem with ReLu Function and that is, it totally ignoring the negative(-ve) values by putting zero which creates problem in
backpropagation. So these neurons are not participating anymore for the prediction of output. And this is called Dying ReLu problem.
1.4.3 Leaky ReLu fiunction: Leaky Rectified Linear Unit, or Leaky ReLU, is a type of activation function based on a
ReLU, but it has a small slope for negative values.
𝛼 is very small.
Like 𝛼=0.0001
This is most popular Activation
function. It helps to get better result
by backpropagation.
10. Mostly using programming languages: Python, R, Java, C++ etc. But mostly Python is used to to
trained a data model just because of the language’s simplicity.
11. Applications.
• X-ray image analysis: CNNs have been used for medical imaging to identify tumors or other abnormalities in X-ray
images. CNN models can take an image of a human body part, such as the knee, and determine where within that image there
might be a tumor other abnormalities such as fractured bones based on previous similar images processed by CNN networks.
• Cancer detection: CNNs have been used to detect cancer in medical images such as mammograms and CT scans. CNN
models are able to produce highly accurate results .
• Understanding Climate: CNNs can be used to play a major role in the fight against climate change, especially in
understanding the reasons why we see such drastic changes and how we could experiment in curbing the effect.
• Self-driving or autonomous cars: CNN has been used within the context of automated vehicles to enable them to
detect obstacles or interpret street signs.
12. References:
[1] Robert J. Schalkoff, “Artificial Neural Network”, MIT Press and The McGraw-Hill Companies, Inc.
Indian edition.1997.
[2] Anirudha Ghosh, A Sufian, Farhana Sultana, Amlan Chakrabarti, Debashis de, “Fundamental
Concepts of Convolutional Neural Network” DOI: 10.1007/978-3-030-32644-9_36, pages 1-41,
January 2020.
