Introductions to
Neural Networks
Basic concepts
Jian QIN
Intro
Single-Layer

Introductions to Neural Networks
Basic concep...
Introductions to
Neural Networks
Basic concepts

Outline

Jian QIN
Intro

Intro

Single-Layer
Multilayer
Applications

Sin...
Introductions to
Neural Networks
Basic concepts

Show

Jian QIN
Intro
Single-Layer
Multilayer
Applications
Next

./asample...
Introductions to
Neural Networks
Basic concepts

Show cont.

Jian QIN
Intro
Single-Layer
Multilayer
Applications
Next

Sou...
Introductions to
Neural Networks
Basic concepts

Neuron

Jian QIN
Intro
Single-Layer
Multilayer
Applications
Next

▶

Inpu...
Introductions to
Neural Networks
Basic concepts

Activation Function

Jian QIN
Intro
Single-Layer

▶

Threshold

Multilaye...
Introductions to
Neural Networks
Basic concepts

Perceptrons

Jian QIN
Intro

▶
▶

Single-Layer

Can perform pattern class...
Introductions to
Neural Networks
Basic concepts

Perceptrons
▶

On-Line learning (One by One)

Jian QIN
Intro

w(0) = [0, ...
Introductions to
Neural Networks
Basic concepts

SVM

Jian QIN

A kernel learning method on (sort of) single layer percept...
Introductions to
Neural Networks
Basic concepts

Multilayer Perceptrons

Jian QIN
Intro

▶

Input + Hidden Layer + Output
...
Introductions to
Neural Networks
Basic concepts

Feed Forward

Jian QIN
Intro
Single-Layer
Multilayer
Applications
Next

x...
Introductions to
Neural Networks
Basic concepts

Back-Propagation

Jian QIN
Intro
Single-Layer
Multilayer

Cost Function

...
Introductions to
Neural Networks
Basic concepts

Word2Vec

Jian QIN
Intro
Single-Layer
Multilayer
Applications
Next

Worki...
Introductions to
Neural Networks
Basic concepts

matlab code

Jian QIN
Intro
Single-Layer
Multilayer
Applications
Next

Ha...
Introductions to
Neural Networks
Basic concepts

Personal Views

Jian QIN
Intro
Single-Layer
Multilayer
Applications
Next
...
Introductions to
Neural Networks
Basic concepts

Next

Jian QIN
Intro
Single-Layer
Multilayer
Applications
Next

1. Recurr...
Introductions to
Neural Networks
Basic concepts

Links

Jian QIN
Intro
Single-Layer
Multilayer

▶

Machine Learning

Appli...
Upcoming SlideShare
Loading in...5
×

Introductions to Neural Networks,Basic concepts

745
-1

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
745
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
29
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Introductions to Neural Networks,Basic concepts

  1. 1. Introductions to Neural Networks Basic concepts Jian QIN Intro Single-Layer Introductions to Neural Networks Basic concepts Multilayer Applications Next Jian QIN December 23, 2013 . . . . . .
  2. 2. Introductions to Neural Networks Basic concepts Outline Jian QIN Intro Intro Single-Layer Multilayer Applications Single-Layer Next Multilayer Applications Next . . . . . .
  3. 3. Introductions to Neural Networks Basic concepts Show Jian QIN Intro Single-Layer Multilayer Applications Next ./asamples.gif Get more from http://yann.lecun.com/exdb/lenet/index.html . . . . . .
  4. 4. Introductions to Neural Networks Basic concepts Show cont. Jian QIN Intro Single-Layer Multilayer Applications Next Source: Learning Internal Representations by Error Propagation . . . . . .
  5. 5. Introductions to Neural Networks Basic concepts Neuron Jian QIN Intro Single-Layer Multilayer Applications Next ▶ Input Vector: X = [x0 = 1, x1 , x2 , x3 …] ▶ Weights: W = [w0 = b, w1 , w2 , w3 …] ▶ Sum: X*WT ▶ Activation Function … . . . . . .
  6. 6. Introductions to Neural Networks Basic concepts Activation Function Jian QIN Intro Single-Layer ▶ Threshold Multilayer Applications 1 2 3 4 5 if < threshold then 0 else 1 ▶ Next Sigmoid 1 1. y = 1+e−ax 2. y = tanh(x) . . . . . .
  7. 7. Introductions to Neural Networks Basic concepts Perceptrons Jian QIN Intro ▶ ▶ Single-Layer Can perform pattern classification only on linearly separable patterns. Multilayer Applications XOR Problem Next a 1 0 1 0 b 1 1 0 0 a XOR b 0 1 1 0 Critique from Minsky and Selfridge . . . . . .
  8. 8. Introductions to Neural Networks Basic concepts Perceptrons ▶ On-Line learning (One by One) Jian QIN Intro w(0) = [0, 0, 0...] Single-Layer w(n + 1) = w(n) + η [d(n) − y(n)] x(n) ▶ Multilayer Applications Batch Learning where E is the set of misclassifed x perceptron cost function J(w) = ∑ Next (−wT x) x∈E gradient vector J(w) = ∑ (−x) x∈E algorithm w(n + 1) = w(n) − η J(w) ∑ w(n + 1) = w(n) − η x . . x∈E . . . .
  9. 9. Introductions to Neural Networks Basic concepts SVM Jian QIN A kernel learning method on (sort of) single layer perceptron. Intro Single-Layer Multilayer Applications Next From: PDF materials of Neural Networks and Learning Machines . . . . . .
  10. 10. Introductions to Neural Networks Basic concepts Multilayer Perceptrons Jian QIN Intro ▶ Input + Hidden Layer + Output Single-Layer Multilayer Applications Next http://www.seattlerobotics.org/encoder/ nov98/neural.html . . . . . .
  11. 11. Introductions to Neural Networks Basic concepts Feed Forward Jian QIN Intro Single-Layer Multilayer Applications Next xlayer(N+1) = sgn(xlayer(N) ∗ Wlayer(N) ) . . . . . .
  12. 12. Introductions to Neural Networks Basic concepts Back-Propagation Jian QIN Intro Single-Layer Multilayer Cost Function Applications 1 ∑∑ 2 = ej (n) N N ξav Next n=1 j∈C ej (n) = dj (n) − yj (n) http://galaxy.agh.edu.pl/~vlsi/AI/backp_t_en/ backprop.html . . . . . .
  13. 13. Introductions to Neural Networks Basic concepts Word2Vec Jian QIN Intro Single-Layer Multilayer Applications Next Working on Java Version. 1/7 speed of original c version. Original Version: https://code.google.com/p/word2vec/ . . . . . .
  14. 14. Introductions to Neural Networks Basic concepts matlab code Jian QIN Intro Single-Layer Multilayer Applications Next Hand-writing Image Processing . . . . . .
  15. 15. Introductions to Neural Networks Basic concepts Personal Views Jian QIN Intro Single-Layer Multilayer Applications Next 1. Pattern Recognition: Image Processing. 2. Build Prior Information into NN: by hand. 3. Have no idea about the structure of the solution. . . . . . .
  16. 16. Introductions to Neural Networks Basic concepts Next Jian QIN Intro Single-Layer Multilayer Applications Next 1. Recurrent NN 2. Applications 3. Neural Networks on NLP. . . . . . .
  17. 17. Introductions to Neural Networks Basic concepts Links Jian QIN Intro Single-Layer Multilayer ▶ Machine Learning Applications Next https://www.coursera.org/course/ml ▶ Neural Networks for Machine Learning https://www.coursera.org/course/neuralnets ▶ Neural Networks and Learning Machines http://book.douban.com/subject/5952531/ . . . . . .
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×