Slides on my talk at SfN 2018 about how signal processing and machine learning can help to model and analyse neural time series. Slides are on purpose not too technical for an audience of neuroscientists

1. Alexandre Gramfort
alexandre.gramfort@inria.fr
Non-linear machine learning and
signal models reveal new insights on
neural oscillations
SfN conf. - Nov. 2018
Joint work withTom Dupré laTour, Mainak Jas,Thomas Moreau, LucilleTallot,
Laetitia Grabot,Valérie Doyère,Virginie vanWassenhove andYves Grenier

2. Non-linear Auto-Regressive Models for
Cross-Frequency Coupling (CFC) in
Neural Time Series
Signal from the striatum of a rodent
1
Code: https://pactools.github.io
T. Dupré laTour, L.Tallot, L. Grabot,V. Doyère,V. vanWassenhove,Y. Grenier,A. Gramfort,
(2017) PLOS Computational biology

3. Non-linear Auto-Regressive Models for
Cross-Frequency Coupling (CFC) in
Neural Time Series
Signal from the striatum of a rodent
1
Code: https://pactools.github.io
T. Dupré laTour, L.Tallot, L. Grabot,V. Doyère,V. vanWassenhove,Y. Grenier,A. Gramfort,
(2017) PLOS Computational biology
CFC: High frequency bursts coupled with slow waves

4. SOTA of Phase Amplitude Coupling estimation
[Dvorak & Fenton (2014).Toward a proper estimation of phase–amplitude coupling in neural oscillations]
2 Bandpass
Filters
2 Hilbert
Transform
Custom step
2 bandpass
filters
2 Hilbert
transforms
Custom
step
Comodu-
logram

5. SOTA of Phase Amplitude Coupling estimation
[Dvorak & Fenton (2014).Toward a proper estimation of phase–amplitude coupling in neural oscillations]
2 Bandpass
Filters
2 Hilbert
Transform
Custom step
2 bandpass
filters
2 Hilbert
transforms
Custom
step
Comodu-
logram
Issues:
• How do you set the filtering parameters?
• Requires Hilbert transform on broad-band signals

6. SOTA of Phase Amplitude Coupling estimation
[Dvorak & Fenton (2014).Toward a proper estimation of phase–amplitude coupling in neural oscillations]
2 Bandpass
Filters
2 Hilbert
Transform
Custom step
2 bandpass
filters
2 Hilbert
transforms
Custom
step
Comodu-
logram
Issues:
• How do you set the filtering parameters?
• Requires Hilbert transform on broad-band signals
Driven Autor-Regressive (DAR) models:
• optimize for explained variance (not CFC strength!)
• allows model selection/comparison with cross-validation
• works with shorter signals (better for time-varying CFC)
• can tell if low freq. (LF) drives high freq. (HF) or vice versa.

7. Alex Gramfort Non-linear ML for neural oscillations
Driven Auto-Regressive model
• Auto-Regressive (AR) model
4
[Parametric estimation of spectrum driven by an exogenous signal,
T. Dupré la Tour,Y. Grenier A. Gramfort, Proc. IEEE ICASSP, 2017]
[Grenier et al. IEEE Trans.Acoustics, Speech and Signal Processing, 1988]

8. Alex Gramfort Non-linear ML for neural oscillations
Driven Auto-Regressive model
• Auto-Regressive (AR) model
4
[Parametric estimation of spectrum driven by an exogenous signal,
T. Dupré la Tour,Y. Grenier A. Gramfort, Proc. IEEE ICASSP, 2017]
[Grenier et al. IEEE Trans.Acoustics, Speech and Signal Processing, 1988]
• Non-linear AR model

9. Alex Gramfort Non-linear ML for neural oscillations
Driven Auto-Regressive model
• Auto-Regressive (AR) model
4
[Parametric estimation of spectrum driven by an exogenous signal,
T. Dupré la Tour,Y. Grenier A. Gramfort, Proc. IEEE ICASSP, 2017]
[Grenier et al. IEEE Trans.Acoustics, Speech and Signal Processing, 1988]
• Non-linear AR model
AR coefficients are functions of
the amplitude of the driving signal

10. Alex Gramfort Non-linear ML for neural oscillations
Maximum Likelihood
5
regressors, it is possible to obtain an analytical expression of the pa
As the innovation "(t) is assumed to be a Gaussian white noise, t
L is obtained via:
L =
TY
t=p+1
1
q
2⇡ (t)
2
exp
"(t)
2
2 (t)
2
!
or 2 log(L) = T log(2⇡) +
TX
t=p+1
"(t)
2
(t)
2 + 2
TX
t=p+1
log(
DAR models are estimated by likelihood maximization. Here, if the
(t)
2
is considered fixed, maximizing this function boils down to min
Objective: Maximize
"(t) ⇠ N(0, (t)
2
)w.r.t. model parameters

11. Alex Gramfort Non-linear ML for neural oscillations
Maximum Likelihood
5
regressors, it is possible to obtain an analytical expression of the pa
As the innovation "(t) is assumed to be a Gaussian white noise, t
L is obtained via:
L =
TY
t=p+1
1
q
2⇡ (t)
2
exp
"(t)
2
2 (t)
2
!
or 2 log(L) = T log(2⇡) +
TX
t=p+1
"(t)
2
(t)
2 + 2
TX
t=p+1
log(
DAR models are estimated by likelihood maximization. Here, if the
(t)
2
is considered fixed, maximizing this function boils down to min
Objective: Maximize
"(t) ⇠ N(0, (t)
2
)w.r.t. model parameters
• From AR coefficients you get the power spectrum (PSD):

12. Alex Gramfort Non-linear ML for neural oscillations
Maximum Likelihood
5
regressors, it is possible to obtain an analytical expression of the pa
As the innovation "(t) is assumed to be a Gaussian white noise, t
L is obtained via:
L =
TY
t=p+1
1
q
2⇡ (t)
2
exp
"(t)
2
2 (t)
2
!
or 2 log(L) = T log(2⇡) +
TX
t=p+1
"(t)
2
(t)
2 + 2
TX
t=p+1
log(
DAR models are estimated by likelihood maximization. Here, if the
(t)
2
is considered fixed, maximizing this function boils down to min
Objective: Maximize
"(t) ⇠ N(0, (t)
2
)w.r.t. model parameters
• From AR coefficients you get the power spectrum (PSD):
PSD depends on
the driving signal

13. Alex Gramfort Non-linear ML for neural oscillations
Maximum Likelihood
5
regressors, it is possible to obtain an analytical expression of the pa
As the innovation "(t) is assumed to be a Gaussian white noise, t
L is obtained via:
L =
TY
t=p+1
1
q
2⇡ (t)
2
exp
"(t)
2
2 (t)
2
!
or 2 log(L) = T log(2⇡) +
TX
t=p+1
"(t)
2
(t)
2 + 2
TX
t=p+1
log(
DAR models are estimated by likelihood maximization. Here, if the
(t)
2
is considered fixed, maximizing this function boils down to min
Objective: Maximize
"(t) ⇠ N(0, (t)
2
)w.r.t. model parameters
• From AR coefficients you get the power spectrum (PSD):
freq
PSD
PSD depends on
the driving signal

14. Alex Gramfort Non-linear ML for neural oscillations
Maximum Likelihood
5
regressors, it is possible to obtain an analytical expression of the pa
As the innovation "(t) is assumed to be a Gaussian white noise, t
L is obtained via:
L =
TY
t=p+1
1
q
2⇡ (t)
2
exp
"(t)
2
2 (t)
2
!
or 2 log(L) = T log(2⇡) +
TX
t=p+1
"(t)
2
(t)
2 + 2
TX
t=p+1
log(
DAR models are estimated by likelihood maximization. Here, if the
(t)
2
is considered fixed, maximizing this function boils down to min
Objective: Maximize
"(t) ⇠ N(0, (t)
2
)w.r.t. model parameters
• From AR coefficients you get the power spectrum (PSD):
freq
PSD
PSD depends on
the driving signal
x1

15. Alex Gramfort Non-linear ML for neural oscillations
Maximum Likelihood
5
regressors, it is possible to obtain an analytical expression of the pa
As the innovation "(t) is assumed to be a Gaussian white noise, t
L is obtained via:
L =
TY
t=p+1
1
q
2⇡ (t)
2
exp
"(t)
2
2 (t)
2
!
or 2 log(L) = T log(2⇡) +
TX
t=p+1
"(t)
2
(t)
2 + 2
TX
t=p+1
log(
DAR models are estimated by likelihood maximization. Here, if the
(t)
2
is considered fixed, maximizing this function boils down to min
Objective: Maximize
"(t) ⇠ N(0, (t)
2
)w.r.t. model parameters
• From AR coefficients you get the power spectrum (PSD):
freq
PSD
PSD depends on
the driving signal
x0
x1

16. Alex Gramfort Non-linear ML for neural oscillations
Simulated example:
6

17. Alex Gramfort Non-linear ML for neural oscillations
Simulated example:
6
PAC

18. Alex Gramfort Non-linear ML for neural oscillations
Simulated example:
6

19. Alex Gramfort Non-linear ML for neural oscillations
Simulated example:
6
no PAC

20. Alex Gramfort Non-linear ML for neural oscillations
Driver selection
7

21. Alex Gramfort Non-linear ML for neural oscillations
Driver selection
7
Maximizing the
likelihood leads
to the best
low-frequency
signal

22. Alex Gramfort Non-linear ML for neural oscillations
Results
8

23. Alex Gramfort Non-linear ML for neural oscillations
Results
8
What does it mean for neuroscience?
• Driving signal is not that narrow band
• Driving signal has a non-symmetric spectrum
• Filtering parameters affect the neuroscientific interpretation

24. Alex Gramfort Non-linear ML for neural oscillations
Directionality and delay estimation
9
By shifting in time the driving signal one can test if
high-frequencies are preceding (“causing”) slow
waves or vice versa

25. Alex Gramfort Non-linear ML for neural oscillations
Directionality and delay estimation
9
By shifting in time the driving signal one can test if
high-frequencies are preceding (“causing”) slow
waves or vice versa
DAR models allow to ask new questions

26. Alex Gramfort Non-linear ML for neural oscillations
Robustness to short signals
10
DAR models need
shorter signals to
capture CFC

27. https://pactools.github.io

28. Convolutional Sparse Coding (CSC)
for learning the morphology of
neural signals
2
Code: https://alphacsc.github.io
Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018),
T. Dupré laTour, T. Moreau, M. Jas,A. Gramfort, Proc. NIPS Conf.
Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding,
(2017), M. Jas,T. Dupré laTour, U. Simsekli,A. Gramfort, Proc. NIPS Conf.

29. Alex Gramfort Non-linear ML for neural oscillations
Shape of brain rhythms matter
Advances in automating analysis of neural time series
13
[Cole andVoytek, 2017]
μ rhythm

30. Alex Gramfort Non-linear ML for neural oscillations
Shape of brain rhythms matter
Advances in automating analysis of neural time series
13
[Cole andVoytek, 2017]
μ rhythm
asymmetry
Problem of linear filtering:
Raw signal
Filtered signal

31. Alex Gramfort Non-linear ML for neural oscillations
Shape of brain rhythms matter
Advances in automating analysis of neural time series
13
[Cole andVoytek, 2017]
μ rhythm
asymmetry
Problem of linear filtering:
Raw signal
Filtered signal
After linear filtering everything
looks like a sinusoid!

32. Alex Gramfort Non-linear ML for neural oscillations
From ICA to CSC
14
https://pypi.python.org/pypi/python-picard/0.1
Independent Component Analysis (ICA)

33. Alex Gramfort Non-linear ML for neural oscillations
From ICA to CSC
14
https://pypi.python.org/pypi/python-picard/0.1
Independent Component Analysis (ICA)
X S

34. Alex Gramfort Non-linear ML for neural oscillations
From ICA…
15
https://pypi.python.org/pypi/python-picard/0.1
X
=
A S
= + +…
a1 aks1 sk
+ +…
https://pierreablin.github.io/picard/auto_examples/plot_ica_eeg.html
https://www.martinos.org/mne/stable/auto_tutorials/plot_artifacts_correction_ica.html

35. Alex Gramfort Non-linear ML for neural oscillations
…
… to CSC
16
https://pypi.python.org/pypi/python-picard/0.1
X
=
+
u1 v1
⇤
z1
uk vk
⇤
zk
convolution

36. Alex Gramfort Non-linear ML for neural oscillations
…
⇤
Topography waveform temporal activations
… to CSC
16
https://pypi.python.org/pypi/python-picard/0.1
X
=
+
u1 v1
⇤
z1
uk vk
⇤
zk
convolution

37. Alex Gramfort Non-linear ML for neural oscillations
…
⇤
Topography waveform temporal activations
… to CSC
16
https://pypi.python.org/pypi/python-picard/0.1
X
=
+
u1 v1
⇤
z1
uk vk
⇤
zk
convolution
CSC allows to learn jointly
• topography (ie. localization)
• signal waveform
• when waveform occurs

38. Alex Gramfort Non-linear ML for neural oscillations
Signal from the striatum of a rodent
CSC on LFP
17
https://pypi.python.org/pypi/python-picard/0.1
[Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding,
(2017), M. Jas,T. Dupré laTour, U. Simsekli,A. Gramfort, Proc. NIPS Conf.]
~80 Hz

39. Alex Gramfort Non-linear ML for neural oscillations
Signal from the striatum of a rodent
CSC on LFP
17
https://pypi.python.org/pypi/python-picard/0.1
[Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding,
(2017), M. Jas,T. Dupré laTour, U. Simsekli,A. Gramfort, Proc. NIPS Conf.]
~80 Hz
CSC reveals CFC

40. Alex Gramfort Non-linear ML for neural oscillations
CSC on MEG
18
https://pypi.python.org/pypi/python-picard/0.1
[Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018),
T. Dupré laTour, T. Moreau, M. Jas,A. Gramfort, Proc. NIPS Conf.]
•MEG vectorview
•Median nerve stim.

41. Alex Gramfort Non-linear ML for neural oscillations
CSC on MEG
18
https://pypi.python.org/pypi/python-picard/0.1
[Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018),
T. Dupré laTour, T. Moreau, M. Jas,A. Gramfort, Proc. NIPS Conf.]
•MEG vectorview
•Median nerve stim.
CSC reveals mu-
shaped waveforms

42. Alex Gramfort Non-linear ML for neural oscillations
CSC on MEG
18
https://pypi.python.org/pypi/python-picard/0.1
[Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018),
T. Dupré laTour, T. Moreau, M. Jas,A. Gramfort, Proc. NIPS Conf.]
•MEG vectorview
•Median nerve stim.
See the frequency
harmonics
CSC reveals mu-
shaped waveforms

43. https://alphacsc.github.io

44. https://pypi.python.org/pypi/python-picard/0.1
https://alphacsc.github.io/auto_examples/multicsc/plot_sample_evoked_response.html

45. https://pypi.python.org/pypi/python-picard/0.1
https://alphacsc.github.io/auto_examples/multicsc/plot_sample_evoked_response.html

46. https://pypi.python.org/pypi/python-picard/0.1
https://alphacsc.github.io/auto_examples/multicsc/plot_sample_evoked_response.html
Use permutation
statistics to know if
atoms are condition
specific and if so when

47. Conclusion
• Neuroscience signals are under exploited
• Need for better models and tools
• Open source software to replicate all slides
• Need more interdisciplinary work (CS, ML,
stats, neuro, physics…)
• If you want the maths look at papers…

48. http://www.martinos.org/mne
MNE software for processing MEG and EEG data, A. Gramfort, M. Luessi, E. Larson, D. Engemann, D.
Strohmeier, C. Brodbeck, L. Parkkonen, M. Hämäläinen, Neuroimage 2013

49. http://www.martinos.org/mnehttp://www.martinos.org/mne
Harvard
U. Montreal
MNE developer in 2010
MNE developers in 2018
Berkeley
Paris
NYU
Cambridge Aalto
Ilmenau
Julich
Sheffield
U.Wash.
UCSF
Graz
Marseille
More than 80 people have contributed to MNE

50. Thanks !
GitHub : @agramfort Twitter : @agramfort
Support ERC SLAB,ANR THALAMEEG ANR-14-NEUC-0002-01
NIH R01 MH106174, DFG HA 2899/21-1.
http://alexandre.gramfort.netContact
T. Dupré la Tour, L.Tallot, L. Grabot,V. Doyère,V. van Wassenhove,Y. Grenier,A. Gramfort,
Non-linear Auto-Regressive Models for Cross-Frequency Coupling (CFC) in Neural
Time Series, (2017) PLOS Computational biology
T. Dupré la Tour, T. Moreau, M. Jas,A. Gramfort, Multivariate Convolutional Sparse
Coding for Electromagnetic Brain Signals, (2018), Proc. NIPS Conf.
M. Jas,T. Dupré la Tour, U. Simsekli,A. Gramfort, Learning the Morphology of Brain Signals
Using Alpha-Stable Convolutional Sparse Coding, (2017), Proc. NIPS Conf.
Tom Dupré laTour
Mainak Jas
Thomas Moreau
LucilleTallot
Laetitia Grabot
Valérie Doyère
Virginie vanWassenhove
Yves Grenier
Joint work with: