ANN

1. Artificial Neural Networks
An Introduction

2. What is a Neural Network?
 A human Brain
 A porpoise brain
 The brain in a living creature
 A computer program
 Simulates (at a very rudimentary level)
a biological brain
 Limited connections

3. Artificial Neural Networks
 Artificial neural networks are information technology
inspired by studies of the brain and nervous system
 ANNs are used to simulate the massively parallel
processes that are effectively used in the brain for
learning, and storing information and knowledge

4. Biological Neuron
 Dendrites
 Axon
 Soma
 Membrane
 Synapse
 Neurotransmitter
 Spikes
Dendrites
Synapse
Axon

5. Simple Neuron Configuration
Summation
(weighted)
Transfer
Output (Y)
Inputs Weights
W1
W2
W3
W4
X1
X2
X3
X4

6. Threshold Logic Units
 Outputs are 0 or 1
 If the activation
(accumulated weighted
input) is larger than
threshold the unit
generates a signal
1
0

7. Sigmoidal Transfer function
Outputs are in the range
from 0 to 1
y=1/(1+exp(-a))
Is differentiable

8. Neural Network Architecture
 In feedforward NN, neurons are grouped into layers
 The neurons on each layer are the same type
 There are different types of layers
 Input layer: receive input from external sources
 Output layer: communicate to user
 Hidden layer(s): neurons communicate only with
other layers

9. Sample Network Configuration
Input
layer
Hidden
layer
Output
layer

10. Some Characteristics of ANN
 Tolerance to noise;
 Reliability;
 Two layer networks are restricted to linearly
separable problems;
 Additional layers can solve more complicated
problems;
 “Black Box”. Why? Non-linearity;
 Logic hidden in weights;
 Universal approximators.

11. Learning Methods
 Supervised
 Error Backpropagation
 Counter-Propagation
 Unsupervised
 Hebb’s rule
 Competitive Learning
 Reinforcement

12. Error Backpropagation Algorithm
 Generalized Delta Rule;
 Allowed training multi-layer ANN;
 Revived interest in ANN;
 Error terms are propagated back through
the network;
 The weight coefficients are updated
iteratively;

13. Error Backpropagation Algorithm:
Drawbacks
 Local Minima;
 Biologically implausible;
 Possibility of “network paralysis”;
 Slowness;
 Oscillations.

14. Problems solved by ANN
 Classification
 Cluster Analysis
 Approximation
 Forecasting
 Association
 Data compression

15. Benefits of ANN
 Parallelism
 Learning
 Generalization
 NN can learn the characteristics of a general category
of objects on specific examples from that category
 Robustness (reliability)
 Tolerance to noise
 Performance does not degrade appreciably if some of
its neurons or interconnections are lost (Distributed
memory)

16. Limitations of ANN
 Two-layer NN limited to linearly separable problems
 Local minima & oscillations
 Number of hidden layers/units hard to determine
 Lack of transparency (perspicuity)

17. Sample of Applications
 Business
 Credit scoring
 Bankruptcy prediction
 Bond rating
 Security trading
 Technological processes
 Robotics
 Consumer electronics