Spiking neural network: an introduction I

Spiking Neural Network (SNN):
A Introduction I
Learning Group (29/Jun/2018)
Dalin Zhang

1
2
3
Spiking Neural Network
Recall
Outline
Leaky Integrate-and-Fire (LIF)
4 Encoding Approaches

Recall
Problem:
• Temporal Information

Recall
Problem:
• React only when receiving a pulse

Spiking Neural Network
Spiking Neural Network Architecture
• Encoding
• Build Network
• Loss computation
• Parameter update (learning)
• Decoding

Pulse: Current/Electric Charge
When Membrane Voltage
exceeds the threshold
Human Neuron

• Potassium Ion Channel
• Sodium Ion Channel
• Other Ions Channel
Hodgkin-Huxley (HH)

Leaky Integrate-and-Fire (LIF) Model
Hypothesis: The model makes use of the fact that neuronal action potentials of a given neuron
always have roughly the same form. If the shape of an action potential is always the same, then the
shape cannot be used to transmit information: rather information is contained in the presence or
absence of a spike. Therefore action potentials are reduced to ‘events’ that happen at a precise
moment in time.
No attempt is made to describe the shape of an action potential.

Integrate-and-fire models have two separate components that are both
necessary to define their dynamics:
First, an equation that describes the evolution of the membrane potential;
Second, a mechanism to generate spikes.

time constant
We suppose that, at time t = t0 the membrane potential takes a value
For t > t0, the input vanishes I(t)=0, Intuitively we expect that, if we wait long
enough, the membrane potential relaxes to its resting value
Indeed, the solution of the differential equation with initial condition
Thus, in the absence of input, the membrane potential decays exponentially to its
resting value. The membrane time constant is the characteristic time of the
decay.
Zero Input

Constant Input
Suppose that the passive membrane is stimulated by a constant input current
which starts at and ends at time , with the initial condition
The solution for
If the input current never stopped, the membrane potential would approach for
Once a steady state is reached, the charge on the capacitor no longer changes. All
input current must then flow through the resistor. The steady-state voltage at the
resistor is therefore so that the total membrane voltage is

Pulse Input
For short pulses the steady state value is never reached. At the end of the pulse,
the value of the membrane potential is
For pulse durations Δ ≪τm, we find
Thus, the voltage deflection depends linearly on the amplitude and the duration of the
pulse. As
And the duration Δ is made shorter and shorter, the total charge q delivered by the
current pulse is always the same.

Pulse Input
Interestingly, the voltage deflection at the end of the pulse remains unaltered.
What happens for times The membrane potential evolves from its new initial
value in the absence of any further input. Then we could use the zero input
function:

So the LIF model function with pulse input is
The solution of the linear differential equation is
Thus we can consider the limit of an infinitely short pulse
Pulse Input

Spike Generation
The term ‘firing time’ refers to the moment when a given neuron emits an action
potential. The firing time in the leaky integrate-and-fire model is defined by a threshold
criterion.
The firing time is noted and immediately after the potential is reset to a new value
For the dynamics is again given by
until the next threshold crossing occurs.

Encoding Approaches
Rate Encoding
The rate coding model of neuronal firing communication states that as the intensity
of a stimulus increases, the frequency or rate of action potentials, or "spike firing",
increases.
Example:
Figures with 28x28 size, greyscale ([0 255]).
Pixel value as stimuli frequency
Present for 500ms
Pixel with value 20 as 20Hz generates 10 spikes during present duration.

Encoding Approaches
Temporal Encoding
When precise spike timing are found to carry information, the neural code is often
identified as a temporal code.
Latency Encoding
Example:
Pixel with value 20 as 20ms after
stimulus. Only one spike for each.
Interspike Interval Encoding
Example:
Pixel with value 20 as 20ms
between two spikes.

Encoding Approaches
Population Encoding
Population coding is a method to represent stimuli by using the joint activities of a
number of neurons. In population coding, each neuron has a distribution of
responses over some set of inputs, and the responses of many neurons may be
combined to determine some value about the inputs.
Encode an input variable using multiple overlapping Gaussian Receptive Fields
(RF). Gaussian RF are used to generate firing times from real values.

Encoding Approaches
Population Encoding
Encode an input variable using multiple overlapping Gaussian Receptive Fields
(RF). Gaussian RF are used to generate firing times from real values. For a range
[IMax..IMin] of a variable, which is also called the coding interval, a set of m Gaussian
RF neurons are used.
The center Ci and the width σi of each RF neuron i are determined by the
following equations:
Where m is number of receptive fields in each population and γ is a constant
variable usually 1.5.

Encoding Approaches
While converting the activation values of RF into firing times, a threshold ϑ has
been imposed on the activation value. A receptive field that gives an activation
value less than this threshold will be marked as not-firing and the corresponding
input neuron will not contribute to the post-synaptic potential.

Encoding Approaches
Hough Spiker Algorithm (HSA)
Digital to Analog Analog to Digital

Spiking neural network: an introduction I

More Related Content

What's hot

Similar to Spiking neural network: an introduction I

Recently uploaded

Spiking neural network: an introduction I

Editor's Notes