Artificial neural networks are modeled after the human brain and nervous system. They contain interconnected nodes that mimic biological neurons. Information is passed through the network via weighted connections. The network learns by adjusting the weights based on examples provided in a training process. Common network architectures include single and multi-layer feedforward networks and recurrent networks. Backpropagation is a widely used training algorithm that propagates error backwards from output to input nodes to update weights. Neural networks can learn complex patterns from large datasets and are used for applications like classification, prediction, and data processing.

1. Artificial Neural Network
1

2. Source: Neural Networks by Simon Haykin
Neuron Structure

3. Block Diagram representation of
Nervous System

4. Structural Organization of levels in
the brain
Source: Neural Networks by Simon Haykin

5. Cytoarchitectural map of the
cerebral cortex
Source: Neural Networks by Simon
Haykin

6. Cont…
Warren Sturgis McCulloch and Walter Pitts
 A Logical Calculus of the Ideas Immanent in Nervous
Activity(1943).
 "How We Know Universals: The Perception of
Auditory and Visual Forms" (1947).
(both published in the Bulletin of Mathematical
Biophysics)

7. Non Linear Model of a Neuron
Source: Neural Networks by Simon Haykin

8. Neuron Model
Three basic elements of Neuronal model are:
A set of Synapses or Connecting Links: Each of which
is characterized by weight or strength of its own.
 An Adder for summing the input signals, weighted by
the respective synapses of the neuron
 An activation function for limiting the amplitude of
the output of the neuron

9. Cont..
In Mathematical terms we may describe neuron k by writing the
following pair of equations:

10. Cont..

11. Affine Transformation produced by
the Bias
Source: Neural Networks by Simon Haykin

12. Activation Function
Threshold function
Piece wise Linear function
Sigmoid function

13. Activation Function

14. Single Layer Feed Forward Network
Source: Neural Networks by Simon Haykin

15. Multi Layer Feed Forward Network
Source: Neural Networks by Simon Haykin

16. Recurrent Neural Network
Source: Neural Networks by Simon Haykin

17. Benefits of Neural Networks
Non Linearity
 Input Output Mapping
 Adaptivity
Evidential Response
Contextual Information
Fault Tolerance
 VLSI Implementability

18. Cont..
Uniformity of Analysis and Design
 Biological Analogy

19. Multilayer Perceptron

20. Cont…
 A multilayer perceptron is a neural network
connecting multiple layers in a directed graph,
which means that the signal path through the
nodes only goes one way.
 Each node, apart from the input nodes, has a
nonlinear activation function.
 An MLP uses back propagation as a supervised
learning technique.

21. Back Propagation Algorithm
 Back-propagation is the essence of neural network training.
 It is the method of fine-tuning the weights of a neural net based
on the error rate obtained in the previous epoch (i.e., iteration).
 Proper tuning of the weights allows us to reduce error rates and
to make the model reliable by increasing its generalization.
 Back propagation is a short form for "backward propagation of
errors." It is a standard method of training artificial neural
networks.
 This method helps to calculate the gradient of a loss function
with respects to all the weights in the network.
21
Shri Ramswaroop Memorial College of Engg. and
Management, Lucknow

22. Cont…

23. Two basic method of error correlation
learning
 Batch Learning
 In this method adjustment to the synaptic weights of the
MLP are performed after the presentation of all the N
examples in the training sample that constitute one epoch of
the training.
Cost function to be minimized here is defined over Average
Error Energy.
 On-line Learning
In this method adjustment to the synaptic weights of the
MLP are performed on an example by example basis.
 Cost function to be minimized here is defined over Instantaneous
Error Energy.

24. Cont…
Advantages of Batch Learning are:
• Accurate estimation of gradient vector (i.e. derivative of
the cost function with respect to weight vector w)
• Parallalization of the learning Process.
• Suited for solving non-linear regression problem.
Advantages of On Learning are:
• Online learning is simple to implement
• It provide effective solution to large scale and difficult
pattern classification Problem.

25. References
1. https://www.nobelprize.org/prizes/medicine/1906/s
peedread/
2. S. Haykin, Neural Network and Learning Machines,
3rd edition, PHI, 2012.
25
Shri Ramswaroop Memorial College of Engg. and
Management, Lucknow

26. Thank You!
Shri Ramswaroop Memorial College of Engg. and
Management, Lucknow

27. Q/A?
Shri Ramswaroop Memorial College of Engg. and
Management, Lucknow