Introductions to Neural Networks,Basic concepts
Upcoming SlideShare
Loading in...5
×
 

Introductions to Neural Networks,Basic concepts

on

  • 604 views

 

Statistics

Views

Total Views
604
Views on SlideShare
597
Embed Views
7

Actions

Likes
0
Downloads
10
Comments
0

2 Embeds 7

http://blog.qinjian.me 5
http://feedly.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Introductions to Neural Networks,Basic concepts Introductions to Neural Networks,Basic concepts Presentation Transcript

    • Introductions to Neural Networks Basic concepts Jian QIN Intro Single-Layer Introductions to Neural Networks Basic concepts Multilayer Applications Next Jian QIN December 23, 2013 . . . . . .
    • Introductions to Neural Networks Basic concepts Outline Jian QIN Intro Intro Single-Layer Multilayer Applications Single-Layer Next Multilayer Applications Next . . . . . .
    • Introductions to Neural Networks Basic concepts Show Jian QIN Intro Single-Layer Multilayer Applications Next ./asamples.gif Get more from http://yann.lecun.com/exdb/lenet/index.html . . . . . .
    • Introductions to Neural Networks Basic concepts Show cont. Jian QIN Intro Single-Layer Multilayer Applications Next Source: Learning Internal Representations by Error Propagation . . . . . .
    • Introductions to Neural Networks Basic concepts Neuron Jian QIN Intro Single-Layer Multilayer Applications Next ▶ Input Vector: X = [x0 = 1, x1 , x2 , x3 …] ▶ Weights: W = [w0 = b, w1 , w2 , w3 …] ▶ Sum: X*WT ▶ Activation Function … . . . . . .
    • Introductions to Neural Networks Basic concepts Activation Function Jian QIN Intro Single-Layer ▶ Threshold Multilayer Applications 1 2 3 4 5 if < threshold then 0 else 1 ▶ Next Sigmoid 1 1. y = 1+e−ax 2. y = tanh(x) . . . . . .
    • Introductions to Neural Networks Basic concepts Perceptrons Jian QIN Intro ▶ ▶ Single-Layer Can perform pattern classification only on linearly separable patterns. Multilayer Applications XOR Problem Next a 1 0 1 0 b 1 1 0 0 a XOR b 0 1 1 0 Critique from Minsky and Selfridge . . . . . .
    • Introductions to Neural Networks Basic concepts Perceptrons ▶ On-Line learning (One by One) Jian QIN Intro w(0) = [0, 0, 0...] Single-Layer w(n + 1) = w(n) + η [d(n) − y(n)] x(n) ▶ Multilayer Applications Batch Learning where E is the set of misclassifed x perceptron cost function J(w) = ∑ Next (−wT x) x∈E gradient vector J(w) = ∑ (−x) x∈E algorithm w(n + 1) = w(n) − η J(w) ∑ w(n + 1) = w(n) − η x . . x∈E . . . .
    • Introductions to Neural Networks Basic concepts SVM Jian QIN A kernel learning method on (sort of) single layer perceptron. Intro Single-Layer Multilayer Applications Next From: PDF materials of Neural Networks and Learning Machines . . . . . .
    • Introductions to Neural Networks Basic concepts Multilayer Perceptrons Jian QIN Intro ▶ Input + Hidden Layer + Output Single-Layer Multilayer Applications Next http://www.seattlerobotics.org/encoder/ nov98/neural.html . . . . . .
    • Introductions to Neural Networks Basic concepts Feed Forward Jian QIN Intro Single-Layer Multilayer Applications Next xlayer(N+1) = sgn(xlayer(N) ∗ Wlayer(N) ) . . . . . .
    • Introductions to Neural Networks Basic concepts Back-Propagation Jian QIN Intro Single-Layer Multilayer Cost Function Applications 1 ∑∑ 2 = ej (n) N N ξav Next n=1 j∈C ej (n) = dj (n) − yj (n) http://galaxy.agh.edu.pl/~vlsi/AI/backp_t_en/ backprop.html . . . . . .
    • Introductions to Neural Networks Basic concepts Word2Vec Jian QIN Intro Single-Layer Multilayer Applications Next Working on Java Version. 1/7 speed of original c version. Original Version: https://code.google.com/p/word2vec/ . . . . . .
    • Introductions to Neural Networks Basic concepts matlab code Jian QIN Intro Single-Layer Multilayer Applications Next Hand-writing Image Processing . . . . . .
    • Introductions to Neural Networks Basic concepts Personal Views Jian QIN Intro Single-Layer Multilayer Applications Next 1. Pattern Recognition: Image Processing. 2. Build Prior Information into NN: by hand. 3. Have no idea about the structure of the solution. . . . . . .
    • Introductions to Neural Networks Basic concepts Next Jian QIN Intro Single-Layer Multilayer Applications Next 1. Recurrent NN 2. Applications 3. Neural Networks on NLP. . . . . . .
    • Introductions to Neural Networks Basic concepts Links Jian QIN Intro Single-Layer Multilayer ▶ Machine Learning Applications Next https://www.coursera.org/course/ml ▶ Neural Networks for Machine Learning https://www.coursera.org/course/neuralnets ▶ Neural Networks and Learning Machines http://book.douban.com/subject/5952531/ . . . . . .