Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

- Neural Network by Abhishek Sasan 906 views
- Neural network for protein secondar... by Jakub Paś 868 views
- Artificial intelligence and Neural ... by Abdullah Saghir A... 490 views
- Neural Network Classification and i... by Inderjeet Singh 3134 views
- Intoduction to Neural Network by Dr. Sanjay Shitole 778 views
- Data Science - Part VIII - Artific... by Derek Kane 1939 views

1,088 views

Published on

Published in:
Technology

No Downloads

Total views

1,088

On SlideShare

0

From Embeds

0

Number of Embeds

2

Shares

0

Downloads

66

Comments

0

Likes

2

No embeds

No notes for slide

- 1. Partha pratim deb Mtech(cse)-1st yearNetaji subhash engineering college
- 2. • Biological inspiration vs. artificial neural network• Why Use Neural Networks?• Neural network applications• Learning strategy & Learning techniques• Generalization types• Artificial neurons• MLP neural networks and tasks• learning mechanism used by multilayer perceptron• Activation functions• Multi-Layer Perceptron example for approximation
- 3. The McCullogh-Pittsmodel neurotransmission
- 4. Learning strategy
- 5. 1.Supervised learning2.Unsupervised learning
- 6. BA BA B A
- 7. A B A B B B A AA B B A
- 8. It is based on a labeled training ε Class set. ε Class A The class of each B λ Class piece of data in λ Class B training set is A known. A ε Class Class labels are λ Class B pre-determined and provided in the training phase.
- 9. Task performed Task performed Classification Clustering Pattern NN Model : Recognition Self Organizing NN model : Maps Preceptron “class of data is not Feed-forward NN defined here”“class of data is defined here”
- 10. 1.Linear2.Nonlinear
- 11. Nonlinear generalization of the McCullogh-Pitts neuron: 1 y= sigmoidal neurony = f ( x, w) T −w x−a 1+ e || x − w|| 2 − y=e 2a 2 Gaussian neuron
- 12. MLP = multi-layer perceptronPerceptron: yout = wT x x youtMLP neural network: 1 y1 = k − w1 kT x − a1 , k = 1,2,3 1+ e k y 1 = ( y1 , y 1 , y3 ) T 1 2 1 1 yk = 2 − w 2 kT y 1 − a k 2 , k = 1,2 1+ e y 2 = ( y12 , y 2 ) T 2 yout 2 x y out = ∑ wk y k = w3T y 2 3 2 k =1
- 13. • control• classification These can be reformulated in general as• prediction FUNCTION• approximation APPROXIMATION tasks.Approximation: given a set of values of a function g(x)build a neural network that approximates the g(x) valuesfor any input x.
- 14. Activation function used for curve the input data to know the variation
- 15. Sigmoidal (logistic) function-common in MLP 1 1 g (ai (t )) = = 1 + exp(−k ai (t )) 1 + e −k ai ( t ) where k is a positive constant. The sigmoidal function gives a value in range of 0 to 1. Alternatively can use tanh(ka) which is same shape but in range –1 to 1. Input-output function of a neuron (rate coding assumption)Note: when net = 0, f = 0.5
- 16. Multi-Layer Perceptron example for approximation
- 17. Algorithm (sequential) 1. Apply an input vector and calculate all activations, a and u 2. Evaluate ∆k for all output units via: ∆ (t ) =( d i (t ) − yi (t )) g ( ai (t )) i (Note similarity to perceptron learning algorithm)3. Backpropagate ∆ks to get error terms δ for hidden layers using: δ (t ) =g (ui (t ))∑ k (t ) wki i ∆ k vij (t + 1) = vij (t ) + ηδ i (t ) x j (t ) wij (t + 1) Evaluate ) + η∆i (t ) z j (t ) 4. = w (t changes using: ij
- 18. Here I have used simple identity activation functionwith an example to understand how neural network works
- 19. Once weight changes are computed for all units, weights are updated at the same time (bias included as weights here). An example: v11= -1 x1 w11= 1 y1 v21= 0 w21= -1 v12= 0 w12= 0 x2 v22= 1 y2 w22= 1 v10= 1 v20= 1 Have input [0 1] with target [1 0]. Use identity activation function (ie g(a) = a)
- 20. All biases set to 1. Will not draw them for clarity. Learning rate η = 0.1 v11= -1x1= 0 w11= 1 y1 v21= 0 w21= -1 v12= 0 w12= 0x2= 1 v22= 1 y2 w22= 1 Have input [0 1] with target [1 0].
- 21. Forward pass. Calculate 1st layer activations: v11= -1 u1 = 1x1 w11= 1 y1 v21= 0 w21= -1 v12= 0 w12= 0x2 v22= 1 y2 w22= 1 u2 = 2 u1 = -1x0 + 0x1 +1 = 1 u2 = 0x0 + 1x1 +1 = 2
- 22. Calculate first layer outputs by passing activations thru activation functions z1 = 1 v11= -1 x1 w11= 1 y1 v21= 0 w21= -1 v12= 0 w12= 0 x2 v22= 1 y2 w22= 1 z2 = 2 z1 = g(u1) = 1 z2 = g(u2) = 2
- 23. Calculate 2nd layer outputs (weighted sum thru activation functions): v11= -1 x1 w11= 1 y1= 2 v21= 0 w21= -1 v12= 0 w12= 0 x2 v22= 1 y2= 2 w22= 1 y1 = a1 = 1x1 + 0x2 +1 = 2 y2 = a2 = -1x1 + 1x2 +1 = 2
- 24. Backward pass: v11= -1x1 w11= 1 ∆1= -1 v21= 0 w21= -1 v12= 0 w12= 0x2 v22= 1 ∆2= -2 w22= 1 Target =[1, 0] so d1 = 1 and d2 = 0 So: ∆ 1 = (d1 - y1 )= 1 – 2 = -1 ∆ 2 = (d2 - y2 )= 0 – 2 = -2
- 25. Calculate weight changes for 1st layer (cf perceptron learning): v11= -1 z1 = 1 x1 w11= 1 ∆1 z1 =-1 v21= 0 w21= -1 ∆1 z2 =-2 v12= 0 w12= 0 x2 v22= 1 ∆2 z1 =-2 w22= 1 ∆2 z2 =-4 z2 = 2
- 26. Weight changes will be: v11= -1x1 w11= 0.9 v21= 0 w21= -1.2 v12= 0 w12= -0.2x2 v22= 1 w22= 0.6
- 27. But first must calculate δ’s: v11= -1x1 ∆ 1 w11= -1 ∆1= -1 v21= 0 ∆ 2 w21= 2 v12= 0 ∆ 1 w12= 0x2 v22= 1 ∆2= -2 ∆ 2 w22= -2
- 28. ∆’s propagate back: v11= -1 δ 1= 1x1 ∆1= -1 v21= 0 v12= 0x2 v22= 1 ∆2= -2 δ 2 = -2 δ1 = - 1 + 2 = 1 δ2 = 0 – 2 = -2
- 29. And are multiplied by inputs: v11= -1 δ 1 x1 = 0x1= 0 ∆1= -1 v21= 0 δ 1 x2 = 1 v12= 0 δ 2 x1 = 0 x2= 1 v22= 1 ∆2= -2 δ 2 x2 = -2
- 30. Finally change weights:x1= 0 v11= -1 w11= 0.9 v21= 0 w21= -1.2 v12= 0.1 w12= -0.2 x2= 1 v22= 0.8 w22= 0.6Note that the weights multiplied by the zero input are unchanged as they do not contribute to the error We have also changed biases (not shown)
- 31. Now go forward again (would normally use a new input vector): v11= -1 z1 = 1.2 x1= 0 w11= 0.9 v21= 0 w21= -1.2 v12= 0.1 w12= -0.2 x2= 1 v22= 0.8 w22= 0.6 z2 = 1.6
- 32. Now go forward again (would normally use a new input vector): x1= 0 v11= -1 y1 = 1.66 w11= 0.9 v21= 0 w21= -1.2 v12= 0.1 w12= -0.2 x2= 1 v22= 0.8 w22= 0.6 y2 = 0.32 Outputs now closer to target value [1, 0]
- 33. Neural network applications Pattern Classification Applications examples• Remote Sensing and image classification• Handwritten character/digits Recognition Control, Time series, Estimation • Machine Control/Robot manipulation • Financial/Scientific/Engineering Time series Optimization forecasting. • Traveling sales personMultiprocessor scheduling and task Real World Application Examples assignment • Hospital patient stay length prediction • Natural gas price prediction
- 34. • Artificial neural networks are inspired by the learningprocesses that take place in biological systems.• Learning can be perceived as an optimisation process.• Biological neural learning happens by the modificationof the synaptic strength. Artificial neural networks learnin the same way.• The synapse strength modification rules for artificialneural networks can be derived by applyingmathematical optimisation methods.
- 35. • Learning tasks of artificial neural networks = functionapproximation tasks.• The optimisation is done with respect to the approximationerror measure.• In general it is enough to have a single hidden layer neuralnetwork (MLP, RBF or other) to learn the approximation ofa nonlinear function. In such cases general optimisation canbe applied to find the change rules for the synaptic weights.
- 36. 1.artificial neural network,simon haykin2.artificial neural network , yegnanarayana3.artificial neural network , zurada4. Hornick, Stinchcombe and White’s conclusion (1989)Hornik K., Stinchcombe M. and WhiteH., “Multilayer feedforward networks are universalapproximators”, Neural Networks, vol. 2,no. 5,pp. 359–366, 19895. Kumar, P. and Walia, E., (2006), “Cash Forecasting: AnApplication of Artificial NeuralNetworks in Finance”, International Journal of ComputerScience and Applications 3 (1): 61-77.

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment