Your SlideShare is downloading. ×
0
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Prapun B-Exam
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Prapun B-Exam

372

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
372
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion Capacity Analysis of Neurons with Descending Action Potential Thresholds Prapun Suksompong Electrical and Computer Engineering Cornell University, Ithaca, NY 14853 ps92@cornell.edu Final Examination for the Doctoral Degree (“B” Exam) July 24, 2008 Prapun Suksompong Capacity Analysis of Neurons
  • 2. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion Outline Introduction Sources of Variability for the ISIs and Derivation of the Threshold Information-Theoretic Analysis of IF Neuron Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 3. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Introduction Neuron Morphology Integrate-and-Fire Neurons Goal Sources of Variability for the ISIs and Derivation of the Threshold Information-Theoretic Analysis of IF Neuron Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 4. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Neuron Morphology A neuron is the basic working unit of the nervous system. Dendrite A typical neuron has three Nucleus Axon Hillock functionally distinct parts, called Axon dendrites, Axon Axon from Terminals soma, and another neuron Cell Body (Soma) Synapse axon. Myelin Sheath Node of Ranvier The junction between two Synaptic Vesicle Presynaptic neurons is called a synapse. Axom Terminal Synaptic Cleft Postsynaptic Dendrite Ion Channel Prapun Suksompong Capacity Analysis of Neurons
  • 5. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Action Potentials (Spikes) Looking at a synapse, we refer to the sending neuron as the presynaptic neuron and to the receiving neuron as the postsynaptic neuron. presynaptic postsynaptic synapse axon The neuronal signals consist of short electrical pulses called action potentials (APs) or spikes. A chain of APs emitted by a single neuron is called a spike train. Prapun Suksompong Capacity Analysis of Neurons
  • 6. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Quantal Synaptic Failure (QSF) presynaptic postsynaptic synapse axon Synaptic failure: It is possible that an AP fails to get “across” the synapse. We may model a synapse as a Z -channel. Spikes which successfully cross the synapse then propagate down to soma. Prapun Suksompong Capacity Analysis of Neurons
  • 7. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Integrate-and-Fire Neurons Assumption: ∼ 104 pre-synaptic neurons. True in cortex (higher brain functions). Prapun Suksompong Capacity Analysis of Neurons
  • 8. Introduction which are drawn here are in fact decreasing. We will return to t Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Integrate-and-Fire Neurons Descending Threshold Ascending Membrane Potential time Spike Train Spikes generated when the membrane potentials hit the time thresholds. • First jitter: Spike generation Descending thresholds. • Poisson approximation Prapun Suksompong Capacity Analysis of Neurons
  • 9. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion (Leaky) Integrate-and-Fire Model: LIF or IF Let τ1 , τ2 , τ3 , . . . be the sequence of time that the spikes arrive at the spike generating region. The membrane potential at time t is then X (t) = h (t, τm , Ym ) = Ym h(t − τm ). m m This is the “integrate” part of the integrate-and-fire neuron. X (t ) Ym is the weight for the mth spike due to propagation loss, synaptic strength, synaptic failure, etc. Yi + 2 h ( t − τ i + 2 ) Yi h ( t − τ i ) Yi +1h ( t − τ i +1 ) h is the shape function. τi τ i +1 τ i+2 time Prapun Suksompong Capacity Analysis of Neurons
  • 10. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron   Goal Conclusion Integrate-and-Fire Neurons (Con’t) Constant bombardment of spikes leads to increase in membrane potential. As soon as the membrane potential reaches a critical value or threshold, the neuron “fires” an time (t) action potential. Then, everything resets. Refractory period: The time after a AP is produced, during which it is impossible to generate another   Threshold AP. time (t) Set T (t) to be ∞ during this period. Prapun Suksompong Capacity Analysis of Neurons
  • 11. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Integrate-and-Fire Neurons (Summary) ∼ 104 pre-synaptic neurons. Descending Threshold Ascending Membrane Potential time Spike Train time • First jitter: Spike generation Descending thresholds. • Poisson approximation Prapun Suksompong Capacity Analysis of Neurons
  • 12. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Theoretical Approaches to Neuroscience We use IF model, but more biologically-realistic models exist (e.g. Hodgkin and Huxley [’52] model). 1. Too many parameters. Physical measurements “fundamentally disturb cell properties” 2. Provide less insight. Biological structures have evolved via natural selection to operate optimally. See the book Optima for Animals by R. McNeill Alexander. What is the best strength for a bone? At what speed should humans change from walking to running? Prapun Suksompong Capacity Analysis of Neurons
  • 13. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Information-Theoretic Optimization Application of information theory has already found success in many areas of neuroscience. Barlow’s “economy of impulses”[’59, ’69] Minimize redundancy. Linsker’s InfoMax principle [’88, ’89] Maximize the mutual information. Levy and Baxter’s energy-efficient coding [’96, ’02] Maximize mutual information per unit energy expended. Prapun Suksompong Capacity Analysis of Neurons
  • 14. Introduction Neuron Morphology Sources of Variability and Threshold Derivation Integrate-and-Fire Neurons Information-Theoretic Analysis of IF Neuron Goal Conclusion Motivation and Goal Integrate-and-Fire (IF) model is very popular. The threshold function is a crucial element of the IF model. Little amount of work exists on deriving the form of the threshold curve. In fact, using constant thresholding is also popular. This leads to large jitter in the spike timing and hence discourages the use of time coding. Goal: Find (1) an expression for threshold curve under biologically realistic constraints and (2) the optimal operating point of neuron under such threshold. Prapun Suksompong Capacity Analysis of Neurons
  • 15. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Introduction Sources of Variability for the ISIs and Derivation of the Threshold First Jitter: Spike generation Second and Third Jitters The Threshold Curve Information-Theoretic Analysis of IF Neuron Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 16. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Three sources of variability for inter-spike intervals 1. Spike generation 2. Spike propagation 3. Time-of-arrival estimation Prapun Suksompong Capacity Analysis of Neurons
  • 17. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion First jitter: Spike generation formula that governs how the membrane potential of this middle neuron rises given the combined incoming rate λ . λ1 λ2 ⊕ λ3 λ = λ1 + λ2 + λ3 + Now, of course, there is some jitter in the timing of the paper today. Large number of presynaptic neurons source of jitter comes from the fact that the in The first allows Poisson approximation for the superposed in fact some jitter in large. That assumption allows has process. presynaptic neurons are them. Now, you may recall th The membrane potential is governed by a filtered Poisson to a Poisson p .. the superposed spike trains … is close process. spike train coming out of a single neuron is not a Poiss a tractable formula that governs how the membrane po Prapun Suksompong given the combined incoming rate λ . Capacity Analysis of Neurons
  • 18. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Approximation for the first timing jitter For fixed λ, different realizations of the membrane potential correspond to different spiking times. Membrane potential X ( t ) Filtered Poisson σX approximation for amount of variation in vertical direction [Parzen’62]. Linear approximation Threshold T ( t ) for amount of σ time variation in horizontal Time direction. Prapun Suksompong Capacity Analysis of Neurons
  • 19. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Membrane potential X ( t ) σX Approximation for the first timing jitter (con’t) Threshold T ( t ) σ time Time H (τ ) T (τ ) c2 H2 (τ ) σtime (τ ) ≈ The figure here shows different realizations of the membrane potentials for a fixed combined incoming rate λ . Here, we see that the randomness from the Poisson arrivals T (τ ) h (τ ) − T (τ ) H (τ ) c1 H (τ ) causes fluctuation in the time that the membrane potentials hit the threshold. Under som linear approximation, we can relate the jitter in the vertical direction to the one in horizontal direction. This then gives us the formula for the magnitude of the timing jitte as a function of the spike time τ . h : shape function. Membrane potential X ( t ) Here, h is the shape function which describes how the membrane potential changes in e.g. exponential response to a single input spike. For the usual leaky integrate-and-fire model, this h star with some amplitude and then decay exponentially. σX h (t ) For conciseness, we define these two integrations which get used in the formula here. t H (t) = h (µ)dµ. 0 Threshold T ( t ) t σ time H2 (t) = h2 (µ)dµ. Time 0 Prapun Suksompong Capacity Analysis of Neurons
  • 20. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Approximation for the first timing jitter (con’t) H (τ ) T (τ ) c2 H2 (τ ) σtime (τ ) ≈ T (τ ) h (τ ) − T (τ ) H (τ ) c1 H (τ ) c1 and c2 are constants Membrane potential X ( t ) which depend on the σX distribution of the weight (Ym ) for each spike. Recall: X (t) = m Ym h(t − τm ). Threshold T ( t ) σ time Time Prapun Suksompong Capacity Analysis of Neurons
  • 21. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Jitter in rate estimation The only information contained in a Poisson process is its rate λ. Different λ’s ⇒ different spiking times τ ’s. Descending Threshold Spike times vary inversely with λ. PSP for Large λ T (τ ) λ(τ ) ≈ c1 H (τ ) PSP for Small λ Error in rate estimation: 1 T (τ ) c2 H2 (τ ) σλ (τ ) ≈ . Time c1 H (τ ) c1 H (τ ) However, this spike time has some jitter, so the $lambda$ estimation also have some error. We then go on and approximate this error: Prapun Suksompong Capacity Analysis of Neurons
  • 22. The second jitter is the randomness in the length of time a spike takes to propa Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivationanother neuron. It is on the order of 10 microseconds. The third jitt synapse on Second and Third Jitters time-of-arrival estimation error; that is, if this neuron tries to measure the inter Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion interval, it needs to find out what time a spike arrives. We borrow some formu Radar guys shown here because this is exactly the problem that they call the ra Second and Third Jitters over N0 here. Iterror depends on the signal-to-noise the shape ofis show ranging problem. The this Es also depends on the bandwidth for ratio which the acti potential. The amount of error here is about 10 microseconds as well. Propagation Time. Time-of-Arrival Estimation Error (radar ranging problem): −1 2Es 2 4π 2 f . N0 f 2 : Gabor-bandwidth. ES : Signal energy. N0 2 : Spectral height of the Noise. Small: < 10µs. Prapun Suksompong Capacity Analysis of Neurons
  • 23. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Deriving The Threshold Curves Recall: Deviation in time: H (τ ) T (τ ) c2 H2 (τ ) σtime (τ ) = . T (τ ) h (τ ) − T (τ ) H (τ ) c1 H (τ ) We consider the thresholds which 1) preserve timing jitter σtime (τ ) ≡ σtime,0 , or σtime (τ ) 2) preserve relative timing jitter τ ≡ σ%time,0 across spiking times (or spiking frequencies) of interest. Prapun Suksompong Capacity Analysis of Neurons
  • 24. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Deriving The Threshold Curves Recall: H(τ ) T (τ )c2 H2 (τ ) (a) Deviation in time: σtime (τ ) = T (τ )h(τ )−T (τ )H(τ ) c1 H(τ ) . 1 T (τ )c2 H2 (τ ) (b) Deviation in λ estimation: σλ (τ ) = c1 H(τ ) c1 H(τ ) . We consider the thresholds which 1) preserve timing jitter σtime (τ ) ≡ σtime,0 , or σtime (τ ) 2) preserve relative timing jitter τ ≡ σ%time,0 , or 3) preserve jitter in λ estimation σλ (τ ) ≡ σλ,0 , or σλ (τ ) 4) preserve relative jitter in λ estimation λ ≡ σ%λ,0 , or 5) preserve jitter in ln λ estimation across spiking times (or spiking frequencies) of interest. Prapun Suksompong Capacity Analysis of Neurons
  • 25. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Deriving The Threshold Curves (con’t) Constant timing jitter level: h (t) 1 c2 H2 (t) T (t) = T (t) − T (t) . H (t) σtime,0 c1 H (t) Constant relative-timing-jitter level: h (t) 1 c2 H2 (t) T (t) = T (t) − T (t) . H (t) tσ%time,0 c1 H (t) Preserve relative error in λ estimation: H2 (t) T (t) = c0 . H (t) Prapun Suksompong Capacity Analysis of Neurons
  • 26. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion The differential equations are Bernoulli equations of the form T (t) = T (t) P (t) − T (t)Q (t) . They can be reduced to linear equation by introducing v (t) = T (t) which gives 1 1 v (t) = P (t) v (t) − Q (t) . 2 2 Linear! The solution is t 1 v (t) = v (t0 ) φ (t, t0 ) − φ (t, τ ) Q (τ )dτ, 2 t0 t 1 2 P(τ )dτ where φ (t, s) = e s . Prapun Suksompong Capacity Analysis of Neurons
  • 27. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Comparison between derived thresholds Constant timing jitter Constant relative timing jitter Exponential Heavy-tail Linear 6 7 8 Time [ms] Prapun Suksompong Capacity Analysis of Neurons
  • 28. Introduction First Jitter: Spike generation Sources of Variability and Threshold Derivation Second and Third Jitters Information-Theoretic Analysis of IF Neuron The Threshold Curve Conclusion Summary Analyze and quantify three sources of timing jitter Predict shape of threshold curves Constant timing jitter Constant relative timing jitter Exponential Heavy-tail Linear 6 7 8 Time [ms] Prapun Suksompong Capacity Analysis of Neurons
  • 29. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Introduction Sources of Variability for the ISIs and Derivation of the Threshold Information-Theoretic Analysis of IF Neuron OPT1: Maximization of Mutual Information OPT2: Mutual Information per Unit Energy Cost Rate Matching Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 30. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Conditional density Q(t|λ) = fτ |Λ (t|λ)   We have formula(s) for the threshold curve T (t). Assumption: λ stays constant during each ISI. Given Poisson input intensity λ, can find the conditional density Q(t|λ) = fτ |Λ (t|λ). τ = g (Λ)+jitter. Λ λ  τ   τ |Λ |λ Prapun Suksompong Capacity Analysis of Neurons
  • 31. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Optimization 1: Mutual Information OPT1: sup I (Λ; τ ) where fΛ,τ (Λ, τ ) I (Λ; τ ) = E log fΛ (Λ)fτ (τ ) and the supremum is taken over all possible fΛ (λ). Blahut-Arimoto Algorithm (BAA) Prapun Suksompong Capacity Analysis of Neurons
  • 32. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Exponential “Constant-Jitter” “Constant-Relative-Jitter” Threshold Threshold Threshold -3 -3 -3 x 10 x 10 x 10 6 6 6 4 4 4 f () f () f () 2 2 2 0 0 0 0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000    0.8 0.8 0.8 0.6 0.6 0.6 f(t) f(t) f(t) 0.4 0.4 0.4 0.2 0.2 0.2 0 0 0 4 6 8 10 12 14 4 5 6 7 8 4 5 6 7 8 9 t t t C = 5.457 C = 4.931 C = 5.109 (a) (b) (c) Prapun Suksompong Capacity Analysis of Neurons
  • 33. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Capacity-achieving input densities look similar. −3 x 10 6 exponential constant jitter 5 constant relative jitter 4 fΛ(λ) 3 2 1 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 λ Prapun Suksompong Capacity Analysis of Neurons
  • 34. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation BAA does not provide any insight. Our simpler formula: σ0 (c1 H (t))3 fΛ (λ) ≈ d T (t) c2 H2 (t) t=g (λ) where g (λ) = E [τ |λ]. Prapun Suksompong Capacity Analysis of Neurons
  • 35. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation −3 BAA does not provide any 6 x 10 exponential insight. 5 constant jitter constant relative jitter approximation Our simpler formula: 4 fΛ(λ) (c1 H (t))3 3 σ0 fΛ (λ) ≈ d T (t) c2 H2 (t) 2 t=g (λ) 1 where g (λ) = E [τ |λ]. 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 λ Prapun Suksompong Capacity Analysis of Neurons
  • 36. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation BAA does not provide any exponential constant jitter insight. constant relative jitter approximation Our simpler formula: fΛ(λ) −3 10 σ0 (c1 H (t))3 fΛ (λ) ≈ d T (t) c2 H2 (t) t=g (λ) where g (λ) = E [τ |λ]. 1 10 10 2 λ 10 3 Prapun Suksompong Capacity Analysis of Neurons
  • 37. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Approximation Strategy Assume that g (Λ) is uniform and then find the corresponding fΛ . Exponential “Constant-Jitter” “Constant-Relative-Jitter” Threshold Threshold Threshold -3 -3 -3 x 10 x 10 x 10 6 6 6 4 4 4 f () f () f () 2 2 2 0 0 0 0 500 1000 1500 2000 0 500 1000 1500 2000 0 500 1000 1500 2000    0.8 0.8 0.8 0.6 0.6 0.6 f(t) f(t) f(t) 0.4 0.4 0.4 0.2 0.2 0.2 0 0 0 4 6 8 10 12 14 4 5 6 7 8 4 5 6 7 8 9 t t t C = 5.457 C = 4.931 C = 5.109 (a) (b) (c) Exponential “Constant-Jitter” “Constant-Relative-Jitter” Threshold Threshold Threshold Prapun Suksompong Capacity Analysis of Neurons
  • 38. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Approximation Strategy Assume that g (Λ) is uniform and then find the corresponding fΛ .    * Convolution For invertible function g , the pdf of Z = g (Λ) is given by d −1 1 fZ (z) = g (z) fΛ g −1 (z) = fΛ (λ) , dz |g (λ)| where z = g (λ). Prapun Suksompong Capacity Analysis of Neurons
  • 39. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion OPT2: Energy-Efficient Neuron Brains consume 20% of energy consumption for adults and 60% for infant [Laughlin and Sejnowski’03]. Suppose neuron spends 1 unit of energy per ms when it is idle, and e unit of energy per ms when AP is produced. e 1. If the time to the next spike is τ = t, the energy expended is bo (t) = 1 × (t − ∆) + e × ∆ = t + (e − 1)∆ = t + r . where ∆ is the time used to produce a spike. The value of r depends on the type of neurons under consideration. Prapun Suksompong Capacity Analysis of Neurons
  • 40. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Optimization 2: I/E OPT2: I (Λ; τ ) sup E [bo (τ )] where the supremum is taken over all possible fΛ (λ). I (Λ;τ ) Jimbo-Kunisawa algorithm (JKA) maximizes E[b(Λ)] . b is a function of input. Our bo is a function of output. We define b(λ) = E [bo (τ )|Λ = λ] and apply JKA. Because bo (τ ) = τ + r , we have b(λ) = E [τ |λ] + r = g (λ) + r . Prapun Suksompong Capacity Analysis of Neurons
  • 41. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation Can use the same technique as in OPT1 to do approximation of input-intensity density. In stead of uniform density, consider bounded exponential density of the form γ f (t; γ, α, β) = e −γt 1[α,β] (t) . e −γα − e −γβ Prapun Suksompong Capacity Analysis of Neurons
  • 42. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Input-Intensity Density Approximation Can use the same technique as in OPT1 to do approximation of input-intensity density. Result: (c1 H (t))3 fΛ (λ) ≈ σ0 f (t; γ, g (b), g (a)) , T (t) c2 H2 (t) t=g (λ) where f (t; γ, g (b), g (a)) is the bounded exponential pdf with support on the interval [g (b), g (a)] and parameter γ. Prapun Suksompong Capacity Analysis of Neurons
  • 43. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion −3 x 10 6 exponential exponential constant jitter constant jitter 5 constant relative jitter 0.5 constant relative jitter approximation 4 0.4 fΛ(λ) fτ(t) 3 0.3 2 0.2 1 0.1 0 0 0 500 1000 1500 2000 4 6 8 10 12 14 λ t [ms] exponential constant jitter constant relative jitter approximation fΛ(λ) −3 fτ(t) 10 −1 10 exponential constant jitter constant relative jitter 1 2 3 4 5 6 7 8 9 10 10 10 10 λ t [ms] Prapun Suksompong Capacity Analysis of Neurons
  • 44. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Free Parameters - Revisited Recall, for example, the differential equation that define our “constant-relative-jitter” threshold: h (t) 1 c2 H2 (t) T (t) = T (t) − T (t) . H (t) tσ%time,0 c1 H (t) There are a couple of parameters which we want to revisit. The constants c1 and c2 . Embedded in them is the effect of QSF. σtime,0 σ%time,0 = t0 . What value should we set σtime,0 to be? 10µs. Prapun Suksompong Capacity Analysis of Neurons
  • 45. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion By scaling the unit of the voltage, we can make c1 = 1. The scaling makes 1 1 c2 ∝ = psuccess 1 − pfailure where pfailure is the QSF probability. pfailure depends on the type of neurons under consideration. Let σtime,0 = σ1 and play with it. Prapun Suksompong Capacity Analysis of Neurons
  • 46. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Average Rates From our optimization (either OPT1 or OPT2), we get the optimal input-intensity density fΛ (λ) and spiking-time density fτ (t) for each value of σ1 . ¯ Each value of σ1 gives average arrival rate λin and the average ¯ out . spiking rate λ ¯ λout = 1 . E[τ ] ¯ λin = E [Λ]? Prapun Suksompong Capacity Analysis of Neurons
  • 47. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Average Afferent Rate High value of Λ corresponds to low value of τ . Neuron experiences large Λ value for short amount of time. ¯ λin should be lower than E [Λ]. Let (Λi , τi ) be the pair of input-intensity and the length of the ISI associated with the ith spike. The number of input spikes during the ith ISI is a Poisson random variable Ni with mean Λi τi . The long-term average input rate is then k 1 k i=1 Ni k i=1 Ni E [Λ1 τ1 ] k = 1 k → . i=1 τi i=1 τi E [τ1 ] k ¯ E[Λτ ] λin = E[τ ] . Prapun Suksompong Capacity Analysis of Neurons
  • 48. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Rate Matching On average, our neuron in consideration is bombarded with an ¯ average input-intensity λin . Suppose our neuron is receiving input from n other neurons. Then, on average, each of the sending neurons fire at a rate 1¯ of n λin . Our neuron should also generate spikes at this rate. Therefore, we must have 1¯ ¯ λin = λout . n Prapun Suksompong Capacity Analysis of Neurons
  • 49. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Rate Matching (Con’t) Average spiking rate [spikes/s] 200 input 150 output 100 50 0 10 20 30 40 50 60 70 80 90 100 σ1 [microsec] 5.4 Capacity [bits] 5.2 5 4.8 10 20 30 40 50 60 70 80 90 100 σ1 [microsec] Prapun Suksompong Capacity Analysis of Neurons
  • 50. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Rate Matching (Con’t) Capacity value increases as we increase the noise level Average spiking rate [spikes/s] 200 σ1 . 150 input output 100 Larger value of σ1 implies 50 threshold decays slower This 0 10 15 20 25 30 35 40 45 50 55 60 σ1 [microsec] gives larger support for the spiking time. 5.8 Capacity [bits] 5.6 Noise is small. Effect of 5.4 increasing the support of the 5.2 10 15 20 25 30 35 40 45 50 55 60 output is stronger than the σ1 [microsec] effect of increased noise. Prapun Suksompong Capacity Analysis of Neurons
  • 51. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion 60 55 (c) max I/E, r = 100  50 Optimal rate [spikes per second] 45 40 (b) max I/E, r = 400  35 30 25 exponential constant relative jitter (a) max I  20 15 1 1.5 2 2.5 3 3.5 4 4.5 5 c2   Prapun Suksompong Capacity Analysis of Neurons
  • 52. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Rate Matching (Con’t) 60 55 (c) max I/E, r = 100  Firing rate decreases as the 50 Optimal rate [spikes per second] failure probability increases. 45 Smaller value of r 40 (b) max I/E, r = 400  35 corresponds to higher rate. 30 As r → ∞, the rate 25 exponential constant relative jitter (a) max I  converges to the max-I case. 20 15 1 1.5 2 2.5 3 3.5 4 4.5 5 c2   Prapun Suksompong Capacity Analysis of Neurons
  • 53. Introduction OPT1: Maximization of Mutual Information Sources of Variability and Threshold Derivation OPT2: Mutual Information per Unit Energy Cost Information-Theoretic Analysis of IF Neuron Rate Matching Conclusion Optimal Threshold: c2 = 5 and r = 400 800 700 exponential constant relative timing jitter 600 500 400 300 200 100 0 0 20 40 60 80 100 120 140 160 180 200 Time [ms] Prapun Suksompong Capacity Analysis of Neurons
  • 54. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion Introduction Sources of Variability for the ISIs and Derivation of the Threshold Information-Theoretic Analysis of IF Neuron Conclusion Prapun Suksompong Capacity Analysis of Neurons
  • 55. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion Contributions 1. Quantify the amount of timing jitter in neuron. 2. Construct threshold functions. 3. Provide optimal operating points for neurons which are close to experimentally observed values. Formulas to approximate the optimal input-intensity densities. Prapun Suksompong Capacity Analysis of Neurons
  • 56. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion References I P. Suksompong and T. Berger. Jitter Analysis of Timing Codes for Neurons with Descending Action Potential Thresholds. ISIT, 2006. P. Suksompong and T. Berger. Capacity Analysis of Neurons with Descending Action Potential Thresholds. In preparation for special issue of IEEE Tran. on Info. Theory, 2009. Prapun Suksompong Capacity Analysis of Neurons
  • 57. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion References II P. Suksompong and T. Berger. Energy-Efficient Neurons with Descending Action Potential Thresholds. In preparation for Journal of Comp. Neuroscience, 2009. T. Berger and W.B. Levy. Encoding of excitation via dynamic thresholding. Society for Neuroscience, 2004. W.B. Levy and R.A. Baxter. Energy-Efficient Neuronal Computation via Quantal Synaptic Failures. Journal of Neuroscience, 2002. Prapun Suksompong Capacity Analysis of Neurons
  • 58. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion References III W.B. Levy and R.A. Baxter. Energy Efficient Neural Codes Neural Computation, 1996. Patrick Crotty and William Levy. Biophysical limits on axonal transmission rates in axons. CNS, 2005. E. Parzen. Stochastic Processes. Holden Day, 1962. H. Vincent Poor. An introduction to signal detection and estimation (2nd ed.). Springer-Verlag New York, Inc., New York, NY, USA, 1994. Prapun Suksompong Capacity Analysis of Neurons
  • 59. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion References IV Dale Purve et al. Neuroscience (3rd ed.). Sinauer Associates Inc., Sunderland, MA USA, 1997. Prapun Suksompong Capacity Analysis of Neurons
  • 60. Introduction Sources of Variability and Threshold Derivation Information-Theoretic Analysis of IF Neuron Conclusion THE END Prapun Suksompong Capacity Analysis of Neurons

×