The document provides an overview of neural networks, including their architecture, models of artificial neurons, and learning methods. It describes different types of neural networks such as single-layer feedforward networks and discusses their applications in various fields like character recognition and stock market prediction. References for further reading are also included.

1. NEURAL NETWORKS
PRIAYABRATA SATAPATHY
1st SEMESTER CSE
MCS12121

2. 20 March 2013
CONTENTS
 Introduction.
 Artificial Neural Networks.
 Model of Artificial Neurons.
 Neural Network Architecture.
 Single Layer Feed Forward Networks.
 Learning of ANN.
 Applications of ANN.
 References.

3. 20 March 2013
3
INTRODUCTION
 Neural networks are the simplified models of the biological
neuron systems.
 Neural networks are typically organized in layers. Layers
are made up of a number of interconnected 'nodes' .which
contain an 'activation function'.
 Patterns are presented to the network via the 'input layer',
which communicates to one or more 'hidden layers' where
the actual processing is done via a system of weighted
'connections'.
 The hidden layers then link to an 'output layer' where the
answer is output

4. 20 March 2013
ARTIFICIAL NEURAL NETWORKS
Output
Inputs
An artificial neural network is composed of many artificial neurons that
are linked together according to a specific network architecture. The
objective of the neural network is to transform the inputs into meaningful
outputs.

5. 20 March 2013
5
MODEL OF ARTIFICIAL NEURON
 An appropriate model/simulation of the nervous system should be able
to produce similar responses and behaviours in artificial systems.
 The nervous system is build by relatively simple units, the neurons, so
copying their behaviour and functionality should be the solution.

6. 20 March 2013
MODEL OF ARTIFICIAL NEURON
Neuron consists of three basic components weights, thresholds and a
single activation function
A set of synapses, or connection link: each of which is characterized
by a weight or strength of its own wkj. Specifically, a signal xj at the
input synapse „j‟ connected to neuron „k‟ is multiplied by the
synaptic wkj
An adder: For summing the input signals, weighted by respective
synaptic strengths of the neuron in a linear operation.
I w1 x1 w2 x2 ....... wn xn
n
wi xi
i 1

7. 20 March 2013
MODEL OF ARTIFICIAL NEURON
Threshold for a Neuron:-
The total input for each neuron is the sum of the weighted inputs to
the neuron minus its threshold value. This is then passed through
the sigmoid function. The equation for the transition in a neuron is :
a = 1/(1 + exp(- x)) where
x = ai wi - Q
a is the activation for the neuron
ai is the activation for neuron i
wi is the weight
Q is the threshold subtracted

8. 20 March 2013
MODEL OF ARTIFICIAL NEURON
Activation function: An activation function f performs a mathematical
operation on the signal output. The most common activation
functions are:
- Linear Function,
- Threshold Function,
- Sigmoidal (S shaped) function,
The activation functions are chosen depending upon the type of
problem to be solved by the network.

9. 20 March 2013
MODEL OF ARTIFICIAL NEURON
Activation Functions f – Types:-
Threshold Function
A threshold (hard-limiter) activation function is either a binary type or a
bipolar type.
Output of a binary threshold function produces :
1 if I 0 1 if the weighted sum of the inputs is
Y f (I )
+ve, 0 if 0
0 if the weighted sum of the inputs is
–ve.
1if I 0
Output(I) a bipolar threshold function produces :
Y f of
1if I 0
1 if the weighted sum of the inputs is
+ve,
-1 if the weighted sum of the inputs
is –ve.

10. 20 March 2013
MODEL OF ARTIFICIAL NEURON
Activation Functions f – Types:-
Sigmoidal Function (S-shape function):-
The nonlinear curved S-shape function is called the sigmoid function.
This is most common type of activation used to construct the neural
networks. It is mathematically well behaved, differentiable and strictly
increasing function.
1
Y f (I) I
,0 f ( I ) 1
1 e
1 /(1 exp( I )), 0 f ( I ) 1
This is explained as
≈ 0 for large -ve input values,
1 for large +ve values, with a smooth transition between the two.
α is slope parameter also called shape parameter symbol the λ is also
used to represented this parameter.

11. 20 March 2013
11
NEURAL NETWORK ARCHITECTURE
 An artificial Neural Network is defined as a data processing system
consisting of a large number of interconnected processing elements or
artificial neurons.
 There are three fundamentally different classes of neural networks.
Those are.
1. Single layer feedforward Networks.
2. Multilayer feedforward Networks.
3. Recurrent Networks.
Here we have to discuss the single layer feed forward network.

12. 20 March 2013
SINGLE-LAYER FEED FORWARD NETWORK
- Input layer of source nodes that projects directly
onto an output layer of neurons.
- “Single-layer” referring to the output layer of computation nodes
(neuron).

13. 20 March 2013
SINGLE-LAYER FEED FORWARD NETWORK
Ii1 Oi1
1 W11
Io1 Yo1
Ii2 W21 1
Oi2
2
Io2 Yo2
Ii3 W31
Oi3 2
3 Iom
Yo
Iin Wn1
3 m
Oin
4
 The above figure is a single layer feed forward neural network. It consists
an input layer to receive the inputs and an output layer to output the
vectors.
 The input layer consists of „n‟ neurons, and the output layer contains „m‟
neurons .
 The weight of synapse connecting ith input neuron the jth output neuron
is Wij.

14. 20 March 2013
14
SINGLE-LAYER FEED FORWARD NETWORK
Here the inputs of the input layer and the outputs of the output layer is
given as I i1 Oo1
I i2 Oo2
I1 Oo
.. ..
I in Oom m 1
n 1
So I oj W1 j I I 1 W2 j I I 2 ...... Wnj I IN
Hence, the input to the output layer can be given as
T T
Io m1
W OI m n
W II n 1
Because OI n 1
II m 1
I F(I,W) O
The block diagram of a single layer feed forward network.

15. 20 March 2013
15
LEARNING IN ANN
Learning methods in neural networks can be broadly classified in three basic
types.
- Supervised Learning
- Unsupervised Learning
- Reinforcement Learning
Supervised Learning:-
 In supervised learning, both the inputs and the outputs are provided.
The network then processes the inputs and compares its resulting
outputs against the desired outputs
 Errors are then calculated, causing the system to adjust the weights
which control the network.
 Here a teacher is assume to be present during the learning process.

16. 20 March 2013
16
LEARNING IN ANN
Unsupervised Learning:-
 Here the target output is not presented to the network, Because there
is no teacher to present the described patterns.
 So the system learns of its own by discovering and adapting to
structural features of the input patterns.
Reinforcement Learning:-
 In this method, a teacher though available, does not present the
expected answer but only indicates if the computed output is correct or
incorrect.
 The information provided helps the network in its learning process.
 Here a reward is given for correct answer computed and a penalty for
a wrong answer.

17. 20 March 2013
17
APPLICATIONS OF NEURAL NETWORKS
 Character Recognition:- Neural networks can be used to recognize
handwritten characters.
 Image Compression:- Neural networks can receive and process vast
amounts of information at once, making them useful in image
compression.
 Stock Market Prediction:- Neural networks can examine a lot of
information quickly and sort it all out, they can be used to predict stock
prices.
 Travelling Salesman Problem:- Neural networks can solve the
traveling salesman problem, but only to a certain degree of
approximation.
 Security and Loan Applications:- With the acceptation of a neural
network that will decide whether or not to grant a loan.

18. 20 March 2013
18
REFERENCES
 Neural Networks, Fuzzy logic and Genetic Algorithms, Synthesis
and Applications‘ by S.Rajasekaran and G.A Vijayalakshmi Pai.
 Bertsekas, D.P., Tsitsiklis, J.N. (1996). Neuro-dynamic
programming. Athena Scientific.
 De Rigo, D., Castelletti, A., Rizzoli, A.E., Soncini-Sessa, R., Weber,
E. (January 2005). "A selective improvement technique for
fastening Neuro-Dynamic Programming in Water Resources
Network Management".
 Ferreira, C. (2006). "Designing Neural Networks Using Gene
Expression Programming". In A. Abraham, B. de Baets, M. Köppen,
and B. Nickolay, eds., Applied Soft Computing Technologies: The
Challenge of Complexity, pages 517–536, Springer-Verlag.

19. 20 March 2013
19
THANK YOU