Your SlideShare is downloading. ×
Neural networks
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Neural networks


Published on

Published in: Education, Technology
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Neural NetworksV.SaranyaAP/CSESri Vidya College of Engineering andTechnology,Virudhunagar
  • 2. Neural Networks2
  • 3. Natural Neural Networks• Signals “move” via electrochemical signals• The synapses release a chemical transmitter –the sum of which can cause a threshold to bereached – causing the neuron to “fire”• Synapses can be inhibitory or excitatory3
  • 4. Natural Neural Networks• We are born with about 100 billion neurons• A neuron may connect to as many as 100,000other neurons4
  • 5. Natural Neural Networks• Many of their ideas still used today e.g.– many simple units, “neurons” combine to giveincreased computational power– the idea of a threshold5
  • 6. Modelling a Neuron• aj :Activation value of unit j• wj,i :Weight on link from unit j to unit i• ini :Weighted sum of inputs to unit i• ai :Activation value of unit i• g :Activation functionjjiji aWin ,6
  • 7. Activation Functions• Stept(x) = 1 if x ≥ t, else 0 threshold=t• Sign(x) = +1 if x ≥ 0, else –1• Sigmoid(x) = 1/(1+e-x)7
  • 8. Building a Neural Network1. “Select Structure”: Design the way that theneurons are interconnected2. “Select weights” – decide the strengths withwhich the neurons are interconnected– weights are selected so get a “good match” toa “training set”– “training set”: set of inputs and desiredoutputs– often use a “learning algorithm”8
  • 9. Basic Neural Networks• Will first look at simplest networks• “Feed-forward”– Signals travel in one direction through net– Net computes a function of the inputs9
  • 10. The First Neural Neural NetworksNeurons in a McCulloch-Pitts network are connected by directed, weightedpaths-122X1X2X3Y10
  • 11. The First Neural Neural NetworksIf the on weight on a path is positive the path isexcitatory,otherwise it is inhibitory-122X1X2X3Y11
  • 12. The First Neural Neural NetworksThe activation of a neuron is binary. That is, the neuroneither fires (activation of one) or does not fire (activation ofzero).-122X1X2X3Y12
  • 13. The First Neural Neural NetworksFor the network shown here the activation function for unit Y isf(y_in) = 1, if y_in >= θ else 0where y_in is the total input signal receivedθ is the threshold for Y-122X1X2X3Y13
  • 14. The First Neural Neural NetworksOriginally, all excitatory connections into a particular neuron have the sameweight, although different weighted connections can be input to differentneuronsLater weights allowed to be arbitrary-122X1X2X3Y14
  • 15. The First Neural Neural NetworksEach neuron has a fixed threshold. If the net input into the neuron isgreater than or equal to the threshold, the neuron fires-122X1X2X3Y15
  • 16. The First Neural Neural NetworksThe threshold is set such that any non-zero inhibitory input will prevent the neuronfrom firing-122X1X2X3Y16
  • 17. Building Logic Gates• Computers are built out of “logic gates”• Use threshold (step) function for activationfunction– all activation values are 0 (false) or 1 (true)17
  • 18. The First Neural Neural NetworksAND Function11X1X2YANDX1 X2 Y1 1 11 0 00 1 00 0 0Threshold(Y) = 218
  • 19. The First Neural NetworksAND FunctionOR Function22X1X2YORX1 X2 Y1 1 11 0 10 1 10 0 0Threshold(Y) = 219
  • 20. Perceptron• Synonym for Single-Layer,Feed-Forward Network• First Studied in the 50’s• Other networks were knownabout but the perceptronwas the only one capable oflearning and thus all researchwas concentrated in this area20
  • 21. Perceptron• A single weight only affectsone output so we can restrictour investigations to a modelas shown on the right• Notation can be simpler, i.e.jWjIjStepO 021
  • 22. What can perceptrons represent?AND XORInput 1 0 0 1 1 0 0 1 1Input 2 0 1 0 1 0 1 0 1Output 0 0 0 1 0 1 1 022
  • 23. What can perceptrons represent?0,00,11,01,10,00,11,01,1AND XOR• Functions which can be separated in this way are called Linearly Separable• Only linearly separable functions can be represented by a perceptron• XOR cannot be represented by a perceptron23
  • 24. XOR• XOR is not “linearly separable”– Cannot be represented by a perceptron• What can we do instead?1. Convert to logic gates that can be represented byperceptrons2. Chain together the gates24
  • 25. Single- vs. Multiple-Layers• Once we chain together the gates then we have “hiddenlayers”– layers that are “hidden” from the output lines• Have just seen that hidden layers allow us to represent XOR– Perceptron is single-layer– Multiple layers increase the representational power, soe.g. can represent XOR• Generally useful nets have multiple-layers– typically 2-4 layers25