MACHINE LEARNING ON
NEUROIMAGING DATA
Ilya Kuzovkin
AACIMP, August 2014
LECTURE 1: NEUROIMAGING TECHNIQUES
THE COURSE
NEUROIMAGING
DATA
L1
P1
THE COURSE
NEUROIMAGING
DATA
MACHINE
LEARNING
L1
P1
L2
P2
+
THE COURSE
NEUROIMAGING
DATA
MACHINE
LEARNING
BRAIN-COMPUTER
INTERFACE
L1
P1
L2
P2 P3
+ =
PART I
INTRACORTICAL
NEURONS
http://biomedicalengineering.yolasite.com
NEURONS
http://biomedicalengineering.yolasite.com
SPIKE
http://faculty.tcc.edu/mmitchell/142/ap-graph1.jpg
SPIKE
http://faculty.tcc.edu/mmitchell/142/ap-graph1.jpg
How can
voltage be
negative?
http://newton.umsl.edu/tsytsarev_files/neuron_electrode.JPG
ELECTRODES
THE SIGNAL
• Local Field Potential
http://www.frontiersin.org/files/Articles/414/fnins-02-037/image_m/fnins-02-037-g001.jpg
http://lifesciences.ieee.org/images/stories/life-sciences/2012-01-spike-01_l.png
THE SIGNAL
• Multi-unit spikes
• Local Field Potential
http://www.frontiersin.org/files/Articles/414/fnins-02-037/image_m/fnins-02-037-g001.jpg
http://lifesciences.ieee.org/images/stories/life-sciences/2012-01-spike-01_l.png
THE SIGNAL
• Multi-unit spikes
• Local Field Potential
Spike sorting
http://www.frontiersin.org/files/Articles/414/fnins-02-037/image_m/fnins-02-037-g001.jpg
http://lifesciences.ieee.org/images/stories/life-sciences/2012-01-spike-01_l.png
THE SIGNAL
http://www.frontiersin.org/files/Articles/414/fnins-02-037/image_m/fnins-02-037-g001.jpg
http://lifesciences.ieee.org/images/stories/life-sciences/2012-01-spike-01_l.png
• Multi-unit spikes
• Local Field Potential
• Single-unit spikes
Spike sorting
For each measured neuron we record
the time moments when it spiked
RASTER PLOT
RASTER PLOT
What is on
the X axis?
RASTER PLOT
Time
RASTER PLOT
What is on
the Y axis?
Time
RASTER PLOT
Time
Neurons
RASTER PLOT
Time
Neurons
rastrum
PSTH
Post-Stimulus Time Histogram
PSTH
Post-Stimulus Time Histogram
NeuronsTrials
PSTH
Post-Stimulus Time Histogram
NeuronsTrials
100ms
PSTH
Post-Stimulus Time Histogram
NeuronsTrials
100ms
100ms
PSTH
Post-Stimulus Time Histogram
NeuronsTrials
100ms
100ms
100ms
100ms
PSTH
Post-Stimulus Time Histogram
NeuronsTrials
100ms
100ms
100ms
100ms
31spikes
34spikes
27spikes
218spikes
33spikes
PSTH
Time
NeuronsTrials
Post-Stimulus Time Histogram
PSTH
Time
NeuronsTrials
Post-Stimulus Time Histogram
What is on
the Y axis?
PSTH
Time
NeuronsTrials
Post-Stimulus Time Histogram
Spikesper100ms
PSTH
Time
NeuronsTrials
Post-Stimulus Time Histogram
Spikesper100ms
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
40 spikes per second (40 Hz)45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
40 spikes per second (40 Hz)
0
45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
40 spikes per second (40 Hz)
7 spikes per second (7 Hz)0
45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
40 spikes per second (40 Hz)
7 spikes per second (7 Hz)0
45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
40 spikes per second (40 Hz)
7 spikes per second (7 Hz)0
45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
40 spikes per second (40 Hz)
7 spikes per second (7 Hz)
What do you
expect to see
on this plot?
0
45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
40 spikes per second (40 Hz)
7 spikes per second (7 Hz)0
45
TUNING CURVE
http://www.beatricebiologist.com/2012/11/types-of-cells.html
40 spikes per second (40 Hz)
7 spikes per second (7 Hz)0
45
SUMMARY
• High spatial resolution
(up to single neuron)
!
• High temporal resolution
SUMMARY
• High spatial resolution
(up to single neuron)
!
• High temporal resolution
What does
“spatial
resolution” mean?
SUMMARY
• Requires surgery
!
• Localized
What does
“temporal
resolution” mean?
• High spatial resolution
(up to single neuron)
!
• High temporal resolution
• High spatial resolution
(up to single neuron)
!
• High temporal resolution
SUMMARY
• Requires surgery
!
• Localized
• High spatial resolution
(up to single neuron)
!
• High temporal resolution
SUMMARY
• Requires surgery
!
• Localized
Very informative and precise,
but not applicable to general
audience due to surgery.
PART II
FMRI
http://www.csulb.edu/~cwallis/482/fmri/fmri.h3.gif
http://neuro.mediasauce.com/images/fmri_machine.png
NUCLEAR MAGNETIC RESONANCE
Physical phenomenon in which nucleus
of an atom can absorb or reemit
electromagnetic radiation if placed in a
magnetic field
NUCLEAR MAGNETIC RESONANCE
Physical phenomenon in which nucleus
of an atom can absorb or reemit
electromagnetic radiation if placed in a
magnetic field
NUCLEAR MAGNETIC RESONANCE
Physical phenomenon in which nucleus
of an atom can absorb or reemit
electromagnetic radiation if placed in a
magnetic field
NUCLEAR MAGNETIC RESONANCE
Physical phenomenon in which nucleus
of an atom can absorb or reemit
electromagnetic radiation if placed in a
magnetic field.
SPIN
Spin-UP
+1/2
Spin-DOWN
-1/2
http://www.markusehrenfried.de/science/physics/hermes/whatisspin.html
SPIN
Spin-UP
+1/2
Spin-DOWN
-1/2
http://www.markusehrenfried.de/science/physics/hermes/whatisspin.html
Nothing actually spins in there. No
analogy in classical mechanics.
SPIN
Spin-UP
+1/2
Spin-DOWN
-1/2
Nothing actually spins in there. No
analogy in classical mechanics.
But why call it spin then?
http://www.markusehrenfried.de/science/physics/hermes/whatisspin.html
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
NOT A JOKE
Six known types of quarks and their official names.
So spin is not that bad.
MAGNETIC MOMENT
MAGNETIC MOMENT
Particle with non-zero
nuclear spin is like a
small magnet.
MAGNETIC MOMENT
or ?
Particle with non-zero
nuclear spin is like a
small magnet.
MAGNETIC MOMENT
or ?
Particle with non-zero
nuclear spin is like a
small magnet.
MAGNETIC MOMENT
Particle with non-zero
nuclear spin is like a
small magnet.
IN A MAGNETIC FIELD
What happens
to a magnet if
you put it in a
magnetic field?
IN A MAGNETIC FIELD
What happens
to a magnet if
you put it in a
magnetic field?
IN A MAGNETIC FIELD
IN A MAGNETIC FIELD
IN A MAGNETIC FIELD
IN A MAGNETIC FIELD
Stronger field — larger energy gap
Nuclear Magnetic Resonance
Nuclear Magnetic Resonance
Nuclear Magnetic Resonance
?
Nuclear Magnetic Resonance
RADIO FREQUENCY PULSE
RADIO FREQUENCY PULSE
We can give
energy to a particle
and move it to a
higher energy level
RADIO FREQUENCY PULSE
We can give
energy to a particle
and move it to a
higher energy level
To do that we send an
electromagnetic wave
RADIO FREQUENCY PULSE
We can give
energy to a particle
and move it to a
higher energy level
Nucleus will absorb energy only if
frequency of the wave is correct for
• the nucleus we work with
• strength of the magnetic field
To do that we send an
electromagnetic wave
RELAXATION
Now we stop
the pulse
RELAXATION
Now we stop
the pulse
RELAXATION
Nucleus will give out same amount of energy in
the form of electromagnetic radiation.
Now we stop
the pulse
Nuclear Magnetic Resonance Imaging
Nuclear Magnetic Resonance Imaging
Nuclear Magnetic Resonance Imaging
HYDROGEN
HYDROGEN
Possible spin states
+1/2 or -1/2
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
HYDROGEN IN A MAGNETIC FIELD
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
HYDROGEN IN A MAGNETIC FIELD
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
RF PULSE
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
RF PULSE
What will
happen?
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
RF PULSE
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
RELAXATION
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
RELAXATION
For different tissues
the time of relaxation
is different
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
NUCLEI SEND SIGNALS
One large
piece is still
missing…
GRADIENT COILS
http://www.howtolearn.com/HTL/media/Visualisations-in-Medicine-MRI-Hi-Res-Crop1.jpg
GRADIENT COILS
http://www.howtolearn.com/HTL/media/Visualisations-in-Medicine-MRI-Hi-Res-Crop1.jpg
GRADIENT COILS
http://www.howtolearn.com/HTL/media/Visualisations-in-Medicine-MRI-Hi-Res-Crop1.jpg
GRADIENT COILS
http://www.howtolearn.com/HTL/media/Visualisations-in-Medicine-MRI-Hi-Res-Crop1.jpg
GRADIENT COILS
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
GRADIENT COILS
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
GRADIENT COILS
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
GRADIENT COILS
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
GRADIENT COILS
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
GRADIENT COILS
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
FMRI
F FOR FUNCTIONAL*
FMRI
F FOR FUNCTIONAL*
*MRI signals associated with functional brain activity
BOLD
BLOOD-OXYGEN-LEVEL DEPENDENT
Active neuron Passive neuron
I need
oxygen and
glucose!
I need nothing…
FMRI
Have different magnetic resonance
Distinguishable by MRI
FMRI DATA
• High spatial resolution
!
• No surgery!
!
• Records whole brain
SUMMARY
• Low temporal resolution
!
• Cost
!
• Size
Still pretty informative and precise, does not
require a surgery, but is huge and costs a lot.
https://www.coursera.org/course/fmri
PART III
EEG
http://infantlab.psych.sc.edu/PWREEGDemoBaby
NEURONS
http://biomedicalengineering.yolasite.com
NEURONS
http://www.conncad.com/gallery/single_cells.html
NEURONS
http://en.wikipedia.org/wiki/Neural_oscillation
NEURONS
http://en.wikipedia.org/wiki/Neural_oscillation
NEURONS
http://en.wikipedia.org/wiki/Neural_oscillation
NEURONS
http://en.wikipedia.org/wiki/Neural_oscillation
What is the
frequency in
this example?
BRAINWAVES
BRAINWAVES
Delta
0-4 Hz
Theta
4-7 Hz
Alpha
7-14 Hz
Mu
8-13 Hz
Beta
15-30 Hz
Gamma
30-100 Hz
BRAINWAVES
Delta
0-4 Hz
Theta
4-7 Hz
Alpha
7-14 Hz
Mu
8-13 Hz
Beta
15-30 Hz
Gamma
30-100 Hz
slow wave sleep, babies,
lesions
children, drowsiness,
meditation, relaxed
closed eyes, relaxed
motor neuron in rest,
mirror neurons
motor activity, anxious
thinking, concentration
networking between
populations of neurons
EEG
EEG
EEG
TIME
CHANNELS
EEG
TIME
CHANNELS
EEG
Alpha
7-14 Hz
Beta
15-30 Hz
Gamma
30-100 Hz
?TIME
CHANNELS
EEG
Alpha
7-14 Hz
Beta
15-30 Hz
Gamma
30-100 Hz
?TIME
CHANNELS
Jean Baptiste Joseph Fourier
1768 — 1830
FOURIER TRANSFORM*
FOURIER TRANSFORM*
*discrete
FOURIER TRANSFORM*
*discrete
=
FOURIER TRANSFORM*
*discrete
signal at time t
frequency
complex number
=
FOURIER TRANSFORM*
*discrete
signal at time t
frequency
complex number
=
Amplitude of the
component with
frequency k
FOURIER TRANSFORM*
*discrete
signal at time t
frequency
complex number
=
Amplitude of the
component with
frequency k
EEG DATA
EEG DATA
EEG DATA
TIME-FREQUENCY DOMAIN
Are we done?
EEG DATA
Are we done?
Hint:
EEG DATA
300 MS
EEG DATA
300 MS
EEG DATA
300 MS
EEG DATA
300 MS
EEG DATA
300 MS
10 channels
50 frequencies
3 seconds of data
300 ms window
EEG DATA
300 MS
10 channels
50 frequencies
3 seconds of data
300 ms window
• How many numbers to describe
1 reading of 300 ms?
EEG DATA
300 MS
• How many numbers to describe
1 reading of 300 ms?
• How many numbers to describe
all 3 seconds of data?
10 channels
50 frequencies
3 seconds of data
300 ms window
EEG DATA
• High temporal resolution
!
• No surgery!
!
• Mobile
!
• Cheap
SUMMARY
• Low spatial resolution
Available to wide audience, but
measurements are approximate.
SUMMARY OF NEUROIMAGING TECHNIQUES
Technology Electrical Magnetic Optical
Name EEG ECoG Intracortical MEG fMRI fNIRS
Invasive
!
Portable
!
Cost From $100 to $30,000+
$1000
grid
$2000 per
array
$1 mln
$2-3
mln
$200,000
Temporal
resolution
50 ms 3 ms 3 ms 50ms 1-2 s 1 s
Spatial
resolution
1+ cm 1 mm
0.5 mm -
0.05 mm
5 mm
1 mm
voxels
5 mm
Pattern!
classification
VEP
ERD/
ERS
P300
Performance
2 class 90%
3 class 80%
4 class ?
Large
number
of targets
2 cls
90%
Large
number
of targets
8 cls
90%
High*
~ same
as EEG
based
4 cls
90%
2 cls
90%
Technology Electrical Magnetic Optical
Name EEG ECoG Intracortical MEG fMRI fNIRS
Invasive
!
Portable
!
Cost From $100 to $30,000+
$1000
grid
$2000 per
array
$1 mln
$2-3
mln
$200,000
Temporal
resolution
50 ms 3 ms 3 ms 50ms 1-2 s 1 s
Spatial
resolution
1+ cm 1 mm
0.5 mm -
0.05 mm
5 mm
1 mm
voxels
5 mm
Pattern!
classification
VEP
ERD/
ERS
P300
Performance
2 class 90%
3 class 80%
4 class ?
Large
number
of targets
2 cls
90%
Large
number
of targets
8 cls
90%
High*
~ same
as EEG
based
4 cls
90%
2 cls
90%

Neuroimaging: Intracortical, fMRI, EEG