Neural networks


Published on

Published in: Education, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Neural networks

  1. 1. Neural NetworksV.SaranyaAP/CSESri Vidya College of Engineering andTechnology,Virudhunagar
  2. 2. Neural Networks2
  3. 3. Natural Neural Networks• Signals “move” via electrochemical signals• The synapses release a chemical transmitter –the sum of which can cause a threshold to bereached – causing the neuron to “fire”• Synapses can be inhibitory or excitatory3
  4. 4. Natural Neural Networks• We are born with about 100 billion neurons• A neuron may connect to as many as 100,000other neurons4
  5. 5. Natural Neural Networks• Many of their ideas still used today e.g.– many simple units, “neurons” combine to giveincreased computational power– the idea of a threshold5
  6. 6. Modelling a Neuron• aj :Activation value of unit j• wj,i :Weight on link from unit j to unit i• ini :Weighted sum of inputs to unit i• ai :Activation value of unit i• g :Activation functionjjiji aWin ,6
  7. 7. Activation Functions• Stept(x) = 1 if x ≥ t, else 0 threshold=t• Sign(x) = +1 if x ≥ 0, else –1• Sigmoid(x) = 1/(1+e-x)7
  8. 8. Building a Neural Network1. “Select Structure”: Design the way that theneurons are interconnected2. “Select weights” – decide the strengths withwhich the neurons are interconnected– weights are selected so get a “good match” toa “training set”– “training set”: set of inputs and desiredoutputs– often use a “learning algorithm”8
  9. 9. Basic Neural Networks• Will first look at simplest networks• “Feed-forward”– Signals travel in one direction through net– Net computes a function of the inputs9
  10. 10. The First Neural Neural NetworksNeurons in a McCulloch-Pitts network are connected by directed, weightedpaths-122X1X2X3Y10
  11. 11. The First Neural Neural NetworksIf the on weight on a path is positive the path isexcitatory,otherwise it is inhibitory-122X1X2X3Y11
  12. 12. The First Neural Neural NetworksThe activation of a neuron is binary. That is, the neuroneither fires (activation of one) or does not fire (activation ofzero).-122X1X2X3Y12
  13. 13. The First Neural Neural NetworksFor the network shown here the activation function for unit Y isf(y_in) = 1, if y_in >= θ else 0where y_in is the total input signal receivedθ is the threshold for Y-122X1X2X3Y13
  14. 14. The First Neural Neural NetworksOriginally, all excitatory connections into a particular neuron have the sameweight, although different weighted connections can be input to differentneuronsLater weights allowed to be arbitrary-122X1X2X3Y14
  15. 15. The First Neural Neural NetworksEach neuron has a fixed threshold. If the net input into the neuron isgreater than or equal to the threshold, the neuron fires-122X1X2X3Y15
  16. 16. The First Neural Neural NetworksThe threshold is set such that any non-zero inhibitory input will prevent the neuronfrom firing-122X1X2X3Y16
  17. 17. Building Logic Gates• Computers are built out of “logic gates”• Use threshold (step) function for activationfunction– all activation values are 0 (false) or 1 (true)17
  18. 18. The First Neural Neural NetworksAND Function11X1X2YANDX1 X2 Y1 1 11 0 00 1 00 0 0Threshold(Y) = 218
  19. 19. The First Neural NetworksAND FunctionOR Function22X1X2YORX1 X2 Y1 1 11 0 10 1 10 0 0Threshold(Y) = 219
  20. 20. Perceptron• Synonym for Single-Layer,Feed-Forward Network• First Studied in the 50’s• Other networks were knownabout but the perceptronwas the only one capable oflearning and thus all researchwas concentrated in this area20
  21. 21. Perceptron• A single weight only affectsone output so we can restrictour investigations to a modelas shown on the right• Notation can be simpler, i.e.jWjIjStepO 021
  22. 22. What can perceptrons represent?AND XORInput 1 0 0 1 1 0 0 1 1Input 2 0 1 0 1 0 1 0 1Output 0 0 0 1 0 1 1 022
  23. 23. What can perceptrons represent?0,00,11,01,10,00,11,01,1AND XOR• Functions which can be separated in this way are called Linearly Separable• Only linearly separable functions can be represented by a perceptron• XOR cannot be represented by a perceptron23
  24. 24. XOR• XOR is not “linearly separable”– Cannot be represented by a perceptron• What can we do instead?1. Convert to logic gates that can be represented byperceptrons2. Chain together the gates24
  25. 25. Single- vs. Multiple-Layers• Once we chain together the gates then we have “hiddenlayers”– layers that are “hidden” from the output lines• Have just seen that hidden layers allow us to represent XOR– Perceptron is single-layer– Multiple layers increase the representational power, soe.g. can represent XOR• Generally useful nets have multiple-layers– typically 2-4 layers25