This document summarizes different types of supervised learning networks including perceptrons, Adaline networks, and backpropagation networks. It provides details on their architectures, training algorithms, and how they work. The key points are: - Perceptrons are single layer feedforward networks that use the perceptron learning rule to update weights and converge on the correct output. - Adaline networks have a single linear output unit and use the delta rule to minimize mean squared error and adjust weights between the input and output units. - Backpropagation networks can have multiple hidden layers and use gradient descent to calculate and propagate error back through the network to update weights between layers and improve classification of input patterns.

1. Module 3
SUPERVISED LEARNING

2. • Architecture
• Flowchart Adaline
• Training Algorithm Madaline
• Testing Algorithm Back Propogation
• Radial Basis

3. Supervised learning network
(feed forward networks)
• Simple perceptron  perceptron learning rule
• Adaline  delta rule
• Single layer feed forward network
• Multilayer feed forward – back propogation

4. Perceptron
• Perceptron designed by - Rosenblatt, Minsky –Papert, Block
• Simple perceptrons – Single layer feed-forward networks.
• 3 units
• --Sensory Unit(input unit)
• ---associator unit (Hidden unit)
• ---response unit (output unit)
•

5. Features of Perceptron networks
1.Sensoy units connected to associator units with fixed weights(1,0,-1)
2.Binary activation function is used in sensory unit and associatory unit
3.Output is given by

6. Perceptron learning rule

8. • 1) Sensory unit – 2 dimensional matrix of 400 photodetectors upon
which a lighted picture with geometric black and white patterns
impinges.
• Provides binary 0 electrical signals if input exceeds threshold
• Connected randomly with associator unit .
• 2) Associatory unit
• Consists feature predicates  set of subcircuits->hardwired-detects
specific features
• Results are 0 or 1

9. • 3) response unit
• Consists of Pattern recognition or perceptrons
• Weights are trainable(not fixed).

10. Perceptron learning rule.
• Learning signal = diff between desired and actual response .
• “y” is obtained from the net input calculated and activation function
applied on net input

11. Perceptron rule convergence theorem
• If there is weight vector W such that
f(x(n)W)= t(n), for all n, then
for any starting vector w1, the perceptron learning rule will converge
to a weight vector that gives correct response for all training patterns ,
in finite number of steps.

12. Architecture of perceptron network
• Classifies input pattern as a member or not a member of a particular
CLASS

13. Flow chart –Perceptron network

14. Perceptron training algorithm for single class

15. Perceptron Training Algorithm for multiple output classes
Step 0: Initialize weights, bias and learning rate suitably
Step 1:Check for stopping condition , if false , perform step 2 to 6.
Step 2: Perform steps 3-5 for each bipolar or binary training vector
pair s:t
Step 3:set activation input for each input unit i=1 to n. Xi=Si
Step 4: Calculate output response of each output unit j=1 to m ;the net
input is calculated as

16. Perceptron Training Algorithm for multiple output classes
• Activations applied over the net input are:
• Step 5: make adjustments in weights and
Bias for j=1 to m and i=1 to n.

17. Perceptron Training Algorithm for multiple output classes
• Step 6 : test for stopping condition , if no change in weights then stop
the training process else start again from step 2.

18. Adaptive Linear Neuron (Adaline)
• A network with single linear unit is called Adaline
• Units with linear activation functions are called linear units.
• Input output relation is linear.
• Uses Bipolar activation.
• Adaline network has only one output unit
• Weights and Bias(activation =1) between input and output are
adjustable.
• Adaline uses delta rule/least mean square rule/ Widrow-Hoff rule.
• delta rule Minimizes mean squared error (activation and target value)

19. Delta rule
• Derived from gradient descent method.
• Delta rule updates the weights between the connections so as to
minimize the difference between the net input to the output unit and
the target value .
• Minimize the errors of the training patterns by reducing for each
pattern one at a time.
= learning rate
• X= vector of activation of input unit
• Yin =net input to output unit
= weight change

20. Architecture of ADALINE
1. Adaline is a single unit neuron
2. Receives input from several units and one from bias
3. Consists of trainable weights
4. Inputs have 2 values either +1 or -1 and weights have signs(+ or -).
5. Initially Random weights are assigned
6. Net input calculated is applied to quantizer transfer function(AF)---
Restores output to + 1 or -1.
7. Compares actual output with target output
8. Weights are adjusted based on the training algorithm.

21. Architecture of ADALINE

22. Adaline network training algorithm

23. Flow chart for adaline

24. Back propagation Network
1. This learning algo is applied to multilayer feed forward networks .
2. It consists of processing elements with continuous differentiable
activation functions .
3. Classifies input patters correctly.
4. Weight update concept is based on gradient descent.
5.Error is propagated back to the hidden layers.
6. Aim – memorization and generalization

25. Training of BPN
1. General difficulty of multiple hidden layer is calculation of weights
of hidden layer
2. Training of BPN – 3 stages
 feed forward of input training patterns
Calculation and back propagation of the error
Updating of weights
3 . Testing –computation of feed forward phase

26. Architecture of BPN

27. Allgorithm