It is based on a labeled training ε Class set. ε Class A The class of each B λ Class piece of data in λ Class B training set is A known. A ε Class Class labels are λ Class B pre-determined and provided in the training phase.
Task performed Task performed Classification Clustering Pattern NN Model : Recognition Self Organizing NN model : Maps Preceptron “class of data is not Feed-forward NN defined here”“class of data is defined here”
Nonlinear generalization of the McCullogh-Pitts neuron: 1 y= sigmoidal neurony = f ( x, w) T −w x−a 1+ e || x − w|| 2 − y=e 2a 2 Gaussian neuron
MLP = multi-layer perceptronPerceptron: yout = wT x x youtMLP neural network: 1 y1 = k − w1 kT x − a1 , k = 1,2,3 1+ e k y 1 = ( y1 , y 1 , y3 ) T 1 2 1 1 yk = 2 − w 2 kT y 1 − a k 2 , k = 1,2 1+ e y 2 = ( y12 , y 2 ) T 2 yout 2 x y out = ∑ wk y k = w3T y 2 3 2 k =1
• control• classification These can be reformulated in general as• prediction FUNCTION• approximation APPROXIMATION tasks.Approximation: given a set of values of a function g(x)build a neural network that approximates the g(x) valuesfor any input x.
Activation function used for curve the input data to know the variation
Sigmoidal (logistic) function-common in MLP 1 1 g (ai (t )) = = 1 + exp(−k ai (t )) 1 + e −k ai ( t ) where k is a positive constant. The sigmoidal function gives a value in range of 0 to 1. Alternatively can use tanh(ka) which is same shape but in range –1 to 1. Input-output function of a neuron (rate coding assumption)Note: when net = 0, f = 0.5
Multi-Layer Perceptron example for approximation
Algorithm (sequential) 1. Apply an input vector and calculate all activations, a and u 2. Evaluate ∆k for all output units via: ∆ (t ) =( d i (t ) − yi (t )) g ( ai (t )) i (Note similarity to perceptron learning algorithm)3. Backpropagate ∆ks to get error terms δ for hidden layers using: δ (t ) =g (ui (t ))∑ k (t ) wki i ∆ k vij (t + 1) = vij (t ) + ηδ i (t ) x j (t ) wij (t + 1) Evaluate ) + η∆i (t ) z j (t ) 4. = w (t changes using: ij
Here I have used simple identity activation functionwith an example to understand how neural network works
Once weight changes are computed for all units, weights are updated at the same time (bias included as weights here). An example: v11= -1 x1 w11= 1 y1 v21= 0 w21= -1 v12= 0 w12= 0 x2 v22= 1 y2 w22= 1 v10= 1 v20= 1 Have input [0 1] with target [1 0]. Use identity activation function (ie g(a) = a)
All biases set to 1. Will not draw them for clarity. Learning rate η = 0.1 v11= -1x1= 0 w11= 1 y1 v21= 0 w21= -1 v12= 0 w12= 0x2= 1 v22= 1 y2 w22= 1 Have input [0 1] with target [1 0].
Finally change weights:x1= 0 v11= -1 w11= 0.9 v21= 0 w21= -1.2 v12= 0.1 w12= -0.2 x2= 1 v22= 0.8 w22= 0.6Note that the weights multiplied by the zero input are unchanged as they do not contribute to the error We have also changed biases (not shown)
Now go forward again (would normally use a new input vector): v11= -1 z1 = 1.2 x1= 0 w11= 0.9 v21= 0 w21= -1.2 v12= 0.1 w12= -0.2 x2= 1 v22= 0.8 w22= 0.6 z2 = 1.6
Now go forward again (would normally use a new input vector): x1= 0 v11= -1 y1 = 1.66 w11= 0.9 v21= 0 w21= -1.2 v12= 0.1 w12= -0.2 x2= 1 v22= 0.8 w22= 0.6 y2 = 0.32 Outputs now closer to target value [1, 0]
Neural network applications Pattern Classification Applications examples• Remote Sensing and image classification• Handwritten character/digits Recognition Control, Time series, Estimation • Machine Control/Robot manipulation • Financial/Scientific/Engineering Time series Optimization forecasting. • Traveling sales personMultiprocessor scheduling and task Real World Application Examples assignment • Hospital patient stay length prediction • Natural gas price prediction
• Artificial neural networks are inspired by the learningprocesses that take place in biological systems.• Learning can be perceived as an optimisation process.• Biological neural learning happens by the modificationof the synaptic strength. Artificial neural networks learnin the same way.• The synapse strength modification rules for artificialneural networks can be derived by applyingmathematical optimisation methods.
• Learning tasks of artificial neural networks = functionapproximation tasks.• The optimisation is done with respect to the approximationerror measure.• In general it is enough to have a single hidden layer neuralnetwork (MLP, RBF or other) to learn the approximation ofa nonlinear function. In such cases general optimisation canbe applied to find the change rules for the synaptic weights.
1.artificial neural network,simon haykin2.artificial neural network , yegnanarayana3.artificial neural network , zurada4. Hornick, Stinchcombe and White’s conclusion (1989)Hornik K., Stinchcombe M. and WhiteH., “Multilayer feedforward networks are universalapproximators”, Neural Networks, vol. 2,no. 5,pp. 359–366, 19895. Kumar, P. and Walia, E., (2006), “Cash Forecasting: AnApplication of Artificial NeuralNetworks in Finance”, International Journal of ComputerScience and Applications 3 (1): 61-77.