This document summarizes different types of learning methods in neural networks. It discusses multilayer feedforward neural networks, which have an input layer, hidden layer, and output layer connected by weights. The document then categorizes common neural network learning algorithms into supervised learning (including gradient descent and stochastic learning), unsupervised learning (such as Hebbian and competitive learning), and reinforced learning. Specific learning methods like backpropagation, Hebbian rule, and Boltzmann learning are also overviewed.

1. BY
S.RAGAVI
M.SC(COMP.SCI)
NADAR SARASWATHI COLLEGE OF ARTS AND SCIENCE

2. CONTENT
•Multilayer Feedforward Neural Network
•Learning Methods in Neural Network

3. MULTILAYER FEEDFORWARD
NEURAL NETWORK
*Consist of multiple layers.
*Besides input and output layer there is an intermediary layer
called hidden layer.
*Unit of hidden layer is called hidden unit or hidden neuron.
*Hidden layer performs intermediary operations.
*Input layer neuron are linked to hidden layer neurons and
weight on these links are referred as input hidden layer
weights.
*Hidden layer neuron are linked to output layer neurons and
weight on these links are referred as hidden output layer
weights.

4. MULTILAYER FEEDFORWARD
NETWORK DIAGRAM

5. LEARNING METHODS IN NEURAL
NETWORKS
*Supervised Learning.
*Unsupervised Learning.
*Reinforced Learning.
*Hebbian Learning.
*Gradient descent Learning.
*Competitive Learning.
*Stochastic Learning.

6. Classifications of Neural Network Learning
Algorithms
1. Supervised Learning
.Gradient descent Learning
.Stochastic Learning
2. Unsupervised Learning
. Hebbian Learning
.Competitive Learning
3. Reinforced Learning

7. Supervised Learning
*Every input pattern used to train the network is associated
with an output pattern or target pattern.
*A teacher is assumed to be present during the learning
process.
*A comparison is made between the network computed
output and correct expected output, to determine the error.
*Error can be used to change the parameter , which result in
an improvement in performance.

8. Unsupervised Learning
*Target output is not presented to the network.
*No teacher is there to present the desired pattern.
*So, the system learns of its own by discovering and adapting
to structural features in the input patterns.

9. Reinforced learning
*Though teacher is available in this learning method , does
not present the expected answer but only indicates if the
computed output is correct or incorrect.
*The information provided help the network in its learning
process.
*A reward is given for correct computed answer.
*A penalty is given for wrong answer.

10. Hebbian Learning
* Rule was proposed by Hebb(1949).
*It is based on correlative weight adjustment.
*It is the oldest method of learning in neural network.
*A synapse between two neurons is strengthened when the
neurons on either side of the synapse (input and output) have
highly correlated outputs.

11. Gradient descent Learning
*This is based on the minimization of error E defined in
terms of weights and activation function of network.
*Activation function employed by the network is
differentiable , as the weight update is dependent on gradient
of the error E.
*∆Wij=ƞ δE/δWij
*Ƞ is the learning rate parameter.
*δE/δWij is the error gradient with reference to the weight
Wij.
*Hoffs Delta rule and Backpropagation rule are the examples
of this type of learning mechanism.

12. Competitive Learning
*In this method , Neuron strongly respond to the input stimuli
and have their weights updated.
*When the input pattern is presented , all the neuron in the
layer compete and the winning neuron undergoes weight
adjustment.
*Hence , it is called a “winner-takes-all strategy.

13. Stochastic Learning
*Stochastic neural networks are a type of artificial neural
networks built by introducing random variations into
the network, either by giving
the network's neurons stochastic transfer functions, or by
giving them stochastic weights.
*In this method , weights are adjusted in probabilistic fashion.
*This learning mechanism is employed by Boltzmann and
Cauchy machines.

14. Conclusion
*In conclusion to the learning rules in Neural Network, we
can say that the most promising feature of the Artificial
Neural Network is its ability to learn.
* The learning process of brain alters its neural structure. The
increasing or decreasing the strength of its synaptic
connections depending on their activity.
* The more relevant information has a stronger synaptic
connection.