An introduction to artificial neural networks. Discusses how the biological neuron and the artificial neural networks are related and how backpropogation is used to train a network.

1. @tdevinda
Artificial Neural Networks
An Introduction

2. @tdevinda

3. @tdevinda
Anatomy of the biological neuron

4. @tdevinda
Biological neuron: Synapse

5. @tdevinda
Anatomy of an artificial neuron
Sometimes this is referred to as a
“Perceptron” as well.
A perceptron is a machine which
perceives the input and produces
an output

6. @tdevinda
Activation, Weight
and Bias
Activation denotes the amount of
activity of the neuron. (A measure of
how ‘charged’ the neuron is)
This activation output is passed over
to the next neuron as inputs for that
neuron
Activations from inputs is weighted.
Each input has its own weight (Wi)
Each neuron has an internal “Bias”
Weighted sum and the bias makes up
the activation of a neuron
W0
W1
W2
b

7. @tdevinda
Calculation of the activation
● Activation is the weighted sum of all inputs
Weighted Sum =
● In order to make the output more generalized, we use a function on this sum.
This function is called the activation function.
● An activation function for example would generalize the output to a value
between 0 and 1
W0
W1
W2
b
x0
x1
x2

8. @tdevinda
Calculation of the activation
● There are many activation functions available
○ Sigmoid
○ ReLU
○ Tanh
● Now the output becomes
W0
W1
W2
b
x0
x1
x2

9. @tdevinda
Calculation of the activation - The use of a “bias”
● When the activation function is used without any alterations, we cannot
control the output with input parameters
● The addition of a “BIAS” helps to move the function left-right.
● Bias tells when the neuron gets to be active
● Now the output is

10. @tdevinda
A complete ANN
● The combined product of these neurons
becomes an ‘Artificial Neural Network’
● Neurons getting triggered together work as
a ‘layer’
● There is an ‘INPUT’ layer and an ‘OUTPUT’
layer.
● There can be ‘HIDDEN’ layers in between.
● At the output, many neurons would be
‘active’ or ‘inactive’
● If this is a network which does a
classification, we’d want the right neurons
to be ‘active’

11. @tdevinda
Definition of cost
● When the neural network produces the output, it would naturally vary from the
desired output.
● This difference is the ‘cost’ of the result
● We obtain the cost of a network, by running all training samples through and
averaging the result.
● What we need to do is to minimize this ‘cost’ so that the right results are
produced every time we input data

12. @tdevinda
Reduction of the cost
● Getting a zero cost is the ultimate goal.
● But we’d never get to zero
● We will alter the parameters to get to a minimum possible value
○ Alterations to weights
○ Alterations to biases
● We try to find a minimum point in a multi-dimensional plane
● Good old partial differentials come into play
=

13. @tdevinda
Gradient descent
● What we actually do is to reach a point in the n-dimensional plane, where a
minima exists.
● Gets simple when you look at 1 or 2 dimensional inputs

14. @tdevinda
Learning - Back propagation
● When we have the cost, we devise the
methodology to change the weights and
biases of each neuron depending on the
gradients
● This methodology is called back propagation
● Our calculation of a back propagation function
tells us how much to change the weights and
biases
● After each change, we run the samples again,
and repeat, until we get to a minimum cost

15. @tdevinda
Many Types of ANNs
● Many types of ANNs exist
● What we tried is a multi-layered perceptron
● Different types are used for different
applications
● Convolutional Neural Networks (CNNs):
○ Used for image classifications
○ Uses filters to detect certain features of images
● Recurrent Neural Network (RNNs):
○ Used for speech recognition
○ Contains feedback points from next layers or current
layer
Many more...

16. @tdevinda
Artificial Neural Networks
An Introduction