The document discusses neural networks and their biological inspiration. It defines an artificial neural network as an information processing system modeled after the human brain. Neural networks can extract patterns from complex data, operate in parallel, and learn from experience. The document then covers biological neurons, characteristics of neural networks, popular neural network models, learning rules, and different types of learning.

1. UNIT I
 Introduction To Neural Networks
 Biological Neural Networks
 Characteristics of Neural Networks
 Models of Neurons

2. What is Neural Network?
 An Artificial Neural Network (ANN) is an
information processing paradigm that is
inspired by the way biological nervous
systems, such as the brain, to process
information.
 An artificial neural network is a system
based on the operation of biological
neural networks, in other words, is an
emulation of biological neural system.

3. Why learn Neural Networks?
 Neural networks, with their remarkable ability
to derive meaning from complicated or
imprecise data, can be used to extract
patterns and detect trends that are too
complex to be noticed by either humans or
other computer techniques.
 A trained neural network can be thought of
as an "expert" in the category of information
it has been given to analyze.

4. Advantages
 Adaptive learning: An ability to learn how to do tasks
based on the data given for training or initial
experience.
 Self-Organization: An ANN can create its own
organization or representation of the information it
receives during learning time.
 Real Time Operation: ANN computations may be
carried out in parallel, and special hardware devices
are being designed and manufactured which take
advantage of this capability.
 Fault Tolerance via Redundant Information Coding:
Partial destruction of a network leads to the
corresponding degradation of performance. However,
some network capabilities may be retained even with
major network damage.

5. Neural networks versus conventional
computers
 Conventional computers use an algorithmic approach
i.e. the computer follows a set of instructions in order to
solve a problem.
 Neural networks process information in a similar way
the human brain does. The network is composed of a
large number of highly interconnected processing
elements (neurons) working in parallel to solve a
specific problem. Neural networks learn by
example. They cannot be programmed to perform a
specific task.
 Neural networks and conventional algorithmic
computers are not in competition but complement
each other.

6. Biological Neural Networks
 A neural network is mans crude way
of trying to simulate the brain
electronically. So to understand how
a neural net works we first must
have a look at how the human brain
works.

7. Neuron
 The human brain contains about 10 billion
nerve cells, or neurons.
 Each neuron is connected to thousands of
other neurons and communicates with them
via electrochemical signals.

 Signals coming into the neuron are received
via junctions called synapses, these in turn
are located at the end of branches of the
neuron cell called dendrites.

10. Characteristics of Neural Network
 parallel, distributed information
processing
 high degree of connectivity among basic
units
 connections are modifiable based on
experience
 learning is a constant process, and
usually unsupervised
 learning is based only on local information
 performance degrades gracefully if some
units are removed

11. Summary
 Neural Networks exhibits brain like
behaviors like:
 Learning
 Association
 Categorization
 Generalization
 Feature Extraction
 Optimization
 Noise Immunity

12. Models of Neuron
 An artificial neuron is simply an
electronically modeled biological neuron.
How many neurons are used depends on
the task at hand.

13. Mathematical Model

14. McCulloch Pitts Model
 The early model of an artificial
neuron is introduced by Warren
McCulloch a neuroscientist and
Walter Pitts a logician in 1943.
 The McCulloch-Pitts neural model is
also known as linear threshold gate.

15. An Example
Object Purple? Round? Eat?
Blueberry 1 1 1
Golf Ball 0 1 0
Banana 0 0 0

17. Problems
 Set the threshold so that the bird
eats any object that is round or
purple or both.
 Set the threshold so that the bird
eats all the objects.

18. Excitatory & Inhibitory
 The signals are called excitatory
because they excite the neuron toward
possibly sending its own signal.
 So as the neuron receives more and
more excitatory signals, it gets more
and more excited, until the threshold
is reached, and the neuron sends out
its own signal.

19.  The Inhibitory signals have the effect of
inhibiting the neuron from sending a signal.
 When a neuron receives an inhibitory signal, it
becomes less excited, and so it takes more
excitatory signals to reach the neuron's
threshold.
 In effect, inhibitory signals subtract from the
total of the excitatory signals, making the
neuron more relaxed, and moving the neuron
away from its threshold.

20. Perceptron
 A perceptron which was introduced by
Frank Rosenblatt in 1958.
 Essentially the perceptron is an MCP
neuron where the inputs are first
passed through some "preprocessors"
which are called association units.
 These association units detect the
presence of certain specific features in
the inputs.

21. Perceptron
 In essence an association unit is
also an MCP neuron which is 1 if a
single specific pattern of inputs is
received, and it is 0 for all other
possible patterns of inputs.

22. Adaline (Adaptive linear element)
 An important generalization of the perceptron
training algorithm.
 It was presented by Widrow and Hoff as the
'least mean square' (LMS) learning
procedure, also known as the delta rule.
 The main functional difference with the
Perceptron training rule is the way the output
of the system is used in the learning rule.

23. Neural Network Topologies
 The arrangements of the processing
units, connections, and pattern input /
output is referred to as topology.
 Connections can be made in two ways
Interlayer
Intralayer

24. Feedforward & Feedback
 Feed-forward ANNs allow signals to travel
one way only; from input to output.
 There is no feedback (loops) i.e. the output
of any layer does not affect that same layer.
 They are extensively used in pattern
recognition. This type of organization is also
referred to as bottom-up or top-down.

25. Feedforward Network Diagram

26. Basic Learning Rules
 Learning: The ability of the neural network
(NN) to learn from its environment and to
improve its performance through learning.
 Learning is a process by which the free
parameters of a neural network are adapted
through a process of stimulation by the
environment in which the network is
embedded.
 The type of the learning is determined by the
manner in which the parameter changes
take place.

27. Learning Rules
 Hebb’s Law
 Perceptron Learning Law
 Delta Learning Law
 Widrow and Hoff LMS Learning Law
 Correlation Learning Law
 Instar ( Winner – take – all)
 Outstar Learning Law

28. Supervised Learning
 During the training session of neural
network, an input stimulus is applied
that results in output response.
 This response is compared with an
priori desired output signal, the target
response.
 If the actual response differs from the
target response, the neural network
generates an error signal.

29.  This error signal is used to calculate the
adjustments that should be made to
network’s synaptic weights so that the
actual o/p matches the target output.
 This error minimization process requires a
special circuit known as supervisor or
teacher, hence the name Supervised
learning.
 A neural net is said to learn supervised, if
the desired output is already known.

30. Unsupervised Learning
 Unsupervised learning does not require a
teacher; that is there is no target output.
 During the training session, the neural net
receives at its input many different
excitations or input patterns.
 The neural net arbitrarily organizes the
patterns into categories.

31.  When the stimulus is applied later, the neural
net provides an output response indicating
the class to which the stimulus belongs.
 If a class cannot be found for the input
stimulus, a new class is generated.
 Even though unsupervised learning does not
require a teacher, it requires guidelines to
determine how it will form groups.

32. Reinforced Learning
 Reinforced learning requires one or more
neurons at the output layer and a teacher.
 Unlike supervised learning, teacher does not
indicate how close the actual output is to the
target output but whether the actual output
is the same with target output or not.
 The teacher does not present the target
output to the network, but presents only a
pass/fail indication.

33. Contd..
 The error signal generated during the
training session is binary: pass or fail.
 If the teachers indication is bad, the
network readjusts its parameters and tries
again and again until it gets its output
response right.
 There is no clear indication if the output
response is moving in the right direction or
how close it is to the correct response it is.

34. Competitive Learning
 It is another form of supervised learning
that is distinctive because of its
characteristics operation and
architecture.
 Several neurons are at the output layer.
When an input stimulus is applied, each
output neuron competes with the others
to produce the closest output signal to
the target.

35. Contd..
 This output becomes the dominant one,
and the other outputs cease producing an
output signal for that stimulus.
 For another stimulus, another output
neuron becomes the dominant one.
 Thus each output neuron is trained to
respond to a different input stimulus.