Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
×

# Introductions to Neural Networks,Basic concepts

1,081 views

Published on

Published in: Technology
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

• Be the first to like this

### Introductions to Neural Networks,Basic concepts

1. 1. Introductions to Neural Networks Basic concepts Jian QIN Intro Single-Layer Introductions to Neural Networks Basic concepts Multilayer Applications Next Jian QIN December 23, 2013 . . . . . .
2. 2. Introductions to Neural Networks Basic concepts Outline Jian QIN Intro Intro Single-Layer Multilayer Applications Single-Layer Next Multilayer Applications Next . . . . . .
3. 3. Introductions to Neural Networks Basic concepts Show Jian QIN Intro Single-Layer Multilayer Applications Next ./asamples.gif Get more from http://yann.lecun.com/exdb/lenet/index.html . . . . . .
4. 4. Introductions to Neural Networks Basic concepts Show cont. Jian QIN Intro Single-Layer Multilayer Applications Next Source: Learning Internal Representations by Error Propagation . . . . . .
5. 5. Introductions to Neural Networks Basic concepts Neuron Jian QIN Intro Single-Layer Multilayer Applications Next ▶ Input Vector: X = [x0 = 1, x1 , x2 , x3 …] ▶ Weights: W = [w0 = b, w1 , w2 , w3 …] ▶ Sum: X*WT ▶ Activation Function … . . . . . .
6. 6. Introductions to Neural Networks Basic concepts Activation Function Jian QIN Intro Single-Layer ▶ Threshold Multilayer Applications 1 2 3 4 5 if < threshold then 0 else 1 ▶ Next Sigmoid 1 1. y = 1+e−ax 2. y = tanh(x) . . . . . .
7. 7. Introductions to Neural Networks Basic concepts Perceptrons Jian QIN Intro ▶ ▶ Single-Layer Can perform pattern classiﬁcation only on linearly separable patterns. Multilayer Applications XOR Problem Next a 1 0 1 0 b 1 1 0 0 a XOR b 0 1 1 0 Critique from Minsky and Selfridge . . . . . .
8. 8. Introductions to Neural Networks Basic concepts Perceptrons ▶ On-Line learning (One by One) Jian QIN Intro w(0) = [0, 0, 0...] Single-Layer w(n + 1) = w(n) + η [d(n) − y(n)] x(n) ▶ Multilayer Applications Batch Learning where E is the set of misclassifed x perceptron cost function J(w) = ∑ Next (−wT x) x∈E gradient vector J(w) = ∑ (−x) x∈E algorithm w(n + 1) = w(n) − η J(w) ∑ w(n + 1) = w(n) − η x . . x∈E . . . .
9. 9. Introductions to Neural Networks Basic concepts SVM Jian QIN A kernel learning method on (sort of) single layer perceptron. Intro Single-Layer Multilayer Applications Next From: PDF materials of Neural Networks and Learning Machines . . . . . .
10. 10. Introductions to Neural Networks Basic concepts Multilayer Perceptrons Jian QIN Intro ▶ Input + Hidden Layer + Output Single-Layer Multilayer Applications Next http://www.seattlerobotics.org/encoder/ nov98/neural.html . . . . . .
11. 11. Introductions to Neural Networks Basic concepts Feed Forward Jian QIN Intro Single-Layer Multilayer Applications Next xlayer(N+1) = sgn(xlayer(N) ∗ Wlayer(N) ) . . . . . .
12. 12. Introductions to Neural Networks Basic concepts Back-Propagation Jian QIN Intro Single-Layer Multilayer Cost Function Applications 1 ∑∑ 2 = ej (n) N N ξav Next n=1 j∈C ej (n) = dj (n) − yj (n) http://galaxy.agh.edu.pl/~vlsi/AI/backp_t_en/ backprop.html . . . . . .
13. 13. Introductions to Neural Networks Basic concepts Word2Vec Jian QIN Intro Single-Layer Multilayer Applications Next Working on Java Version. 1/7 speed of original c version. Original Version: https://code.google.com/p/word2vec/ . . . . . .
14. 14. Introductions to Neural Networks Basic concepts matlab code Jian QIN Intro Single-Layer Multilayer Applications Next Hand-writing Image Processing . . . . . .
15. 15. Introductions to Neural Networks Basic concepts Personal Views Jian QIN Intro Single-Layer Multilayer Applications Next 1. Pattern Recognition: Image Processing. 2. Build Prior Information into NN: by hand. 3. Have no idea about the structure of the solution. . . . . . .
16. 16. Introductions to Neural Networks Basic concepts Next Jian QIN Intro Single-Layer Multilayer Applications Next 1. Recurrent NN 2. Applications 3. Neural Networks on NLP. . . . . . .
17. 17. Introductions to Neural Networks Basic concepts Links Jian QIN Intro Single-Layer Multilayer ▶ Machine Learning Applications Next https://www.coursera.org/course/ml ▶ Neural Networks for Machine Learning https://www.coursera.org/course/neuralnets ▶ Neural Networks and Learning Machines http://book.douban.com/subject/5952531/ . . . . . .