0
Neural Networks
Neural Networks                  2
Natural Neural Networks• Signals “move” via electrochemical signals• The synapses release a chemical transmitter –  the su...
Natural Neural Networks• We are born with about 100 billion neurons• A neuron may connect to as many as 100,000  other neu...
Natural Neural Networks• Many of their ideas still used today e.g.  – many simple units, “neurons” combine to give    incr...
Modelling a Neuronini   j        Wj, iaj • aj      :Activation value of unit j               •   wj,i   :Weight on link fr...
Activation Functions• Stept(x) =   1 if x ≥ t, else 0 threshold=t• Sign(x) =    +1 if x ≥ 0, else –1• Sigmoid(x)   =      ...
Building a Neural Network1. “Select Structure”: Design the way that the    neurons are interconnected2. “Select weights” –...
Basic Neural Networks• Will first look at simplest networks• “Feed-forward”  – Signals travel in one direction through net...
The First Neural Neural Networks  X1           2               2  X2                            Y              -1  X3 Neur...
The First Neural Neural Networks  X1      2          2 X2                   Y         -1 X3 If the on weight on a path is ...
The First Neural Neural Networks  X1         2             2  X2                       Y            -1  X3The activation o...
The First Neural Neural Networks  X1            2                2  X2                               Y               -1  X...
The First Neural Neural NetworksX1          2             2X2                            Y           -1X3 Originally, all ...
The First Neural Neural Networks   X1           2                2  X2                              Y               -1  X3...
The First Neural Neural NetworksX1            2              2X2                              Y             -1X3The thresh...
Building Logic Gates• Computers are built out of “logic gates”• Use threshold (step) function for activation  function   –...
The First Neural Neural Networks                    1                              AND      X1                          Y ...
The First Neural Networks                               ORX1      2                      X1    X2   Y                    Y...
Perceptron         • Synonym for Single-Layer,           Feed-Forward Network         • First Studied in the 50’s         ...
Perceptron         • A single weight only affects           one output so we can restrict           our investigations to ...
What can perceptrons represent?          AND               XORInput 1   0     0   1   1   0     0   1   1Input 2   0     1...
What can perceptrons represent?                            1,1                                                            ...
XOR•    XOR is not “linearly separable”    – Cannot be represented by a perceptron•    What can we do instead?    1. Conve...
Single- vs. Multiple-Layers• Once we chain together the gates then we have “hidden  layers”   – layers that are “hidden” f...
Upcoming SlideShare
Loading in...5
×

Neural networks

441

Published on

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
441
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
32
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Neural networks"

  1. 1. Neural Networks
  2. 2. Neural Networks 2
  3. 3. Natural Neural Networks• Signals “move” via electrochemical signals• The synapses release a chemical transmitter – the sum of which can cause a threshold to be reached – causing the neuron to “fire”• Synapses can be inhibitory or excitatory 3
  4. 4. Natural Neural Networks• We are born with about 100 billion neurons• A neuron may connect to as many as 100,000 other neurons 4
  5. 5. Natural Neural Networks• Many of their ideas still used today e.g. – many simple units, “neurons” combine to give increased computational power – the idea of a threshold 5
  6. 6. Modelling a Neuronini j Wj, iaj • aj :Activation value of unit j • wj,i :Weight on link from unit j to unit i • ini :Weighted sum of inputs to unit i • ai :Activation value of unit i • g :Activation function 6
  7. 7. Activation Functions• Stept(x) = 1 if x ≥ t, else 0 threshold=t• Sign(x) = +1 if x ≥ 0, else –1• Sigmoid(x) = 1/(1+e-x) 7
  8. 8. Building a Neural Network1. “Select Structure”: Design the way that the neurons are interconnected2. “Select weights” – decide the strengths with which the neurons are interconnected – weights are selected so get a “good match” to a “training set” – “training set”: set of inputs and desired outputs – often use a “learning algorithm” 8
  9. 9. Basic Neural Networks• Will first look at simplest networks• “Feed-forward” – Signals travel in one direction through net – Net computes a function of the inputs 9
  10. 10. The First Neural Neural Networks X1 2 2 X2 Y -1 X3 Neurons in a McCulloch-Pitts network are connected by directed, weighted paths 10
  11. 11. The First Neural Neural Networks X1 2 2 X2 Y -1 X3 If the on weight on a path is positive the path is excitatory, otherwise it is inhibitory 11
  12. 12. The First Neural Neural Networks X1 2 2 X2 Y -1 X3The activation of a neuron is binary. That is, the neuroneither fires (activation of one) or does not fire (activation ofzero). 12
  13. 13. The First Neural Neural Networks X1 2 2 X2 Y -1 X3For the network shown here the activation function for unit Y is f(y_in) = 1, if y_in >= θ else 0where y_in is the total input signal received θ is the threshold for Y 13
  14. 14. The First Neural Neural NetworksX1 2 2X2 Y -1X3 Originally, all excitatory connections into a particular neuron have the same weight, although different weighted connections can be input to different neurons Later weights allowed to be arbitrary 14
  15. 15. The First Neural Neural Networks X1 2 2 X2 Y -1 X3Each neuron has a fixed threshold. If the net input into the neuron isgreater than or equal to the threshold, the neuron fires 15
  16. 16. The First Neural Neural NetworksX1 2 2X2 Y -1X3The threshold is set such that any non-zero inhibitory input will prevent the neuronfrom firing 16
  17. 17. Building Logic Gates• Computers are built out of “logic gates”• Use threshold (step) function for activation function – all activation values are 0 (false) or 1 (true) 17
  18. 18. The First Neural Neural Networks 1 AND X1 Y X1 X2 Y 1 1 1 X2 1 1 0 0 AND Function 0 1 0 0 0 0 Threshold(Y) = 2 18
  19. 19. The First Neural Networks ORX1 2 X1 X2 Y Y 1 1 1 1 0 1X2 2 0 1 1 ANDFunction OR Function 0 0 0 Threshold(Y) = 2 19
  20. 20. Perceptron • Synonym for Single-Layer, Feed-Forward Network • First Studied in the 50’s • Other networks were known about but the perceptron was the only one capable of learning and thus all research was concentrated in this area 20
  21. 21. Perceptron • A single weight only affects one output so we can restrict our investigations to a model as shown on the right • Notation can be simpler, i.e. O Step0 j WjIj 21
  22. 22. What can perceptrons represent? AND XORInput 1 0 0 1 1 0 0 1 1Input 2 0 1 0 1 0 1 0 1Output 0 0 0 1 0 1 1 0 22
  23. 23. What can perceptrons represent? 1,1 1,1 0,1 0,1 0,0 1,0 1,0 0,0 AND XOR• Functions which can be separated in this way are called Linearly Separable• Only linearly separable functions can be represented by a perceptron• XOR cannot be represented by a perceptron 23
  24. 24. XOR• XOR is not “linearly separable” – Cannot be represented by a perceptron• What can we do instead? 1. Convert to logic gates that can be represented by perceptrons 2. Chain together the gates 24
  25. 25. Single- vs. Multiple-Layers• Once we chain together the gates then we have “hidden layers” – layers that are “hidden” from the output lines• Have just seen that hidden layers allow us to represent XOR – Perceptron is single-layer – Multiple layers increase the representational power, so e.g. can represent XOR• Generally useful nets have multiple-layers – typically 2-4 layers 25
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×