EmmanuelleTognoli,
Center for Complex Systems and Brain Sciences, Florida Atlantic University
Complex Systems & Weinberg Institute Seminar,
October 27th, 2020
Computational Social Neuroscience
tognoli@ccs.fau.edu
1
Feel welcome to scribble,
interject, stamp
(Grateful if someone wipes the
slides clean)
Slides are numbered for recall,
questions and comments
during the final discussion.
2
Most illustrations from Polish
surrealist artist Igor Morski
Social
behavior
Development, Aging
Mental health
3
Interactive human-oriented
technologies
{HMI, BMI, BCI, BMI, BBI, HRI…}
Key areas
Many dynamical models have been used
in {Social} and/or {Affective} and/or {Cognitive} and/or {Neuro}science
Agent
based
model
s
Bayesian
Stochast
ic
Diffusion
Markov
Signal
flow
graphs
Lokta-
Volterra
HKB
Recurren
t
network
s
Cellular
automat
a
For our purpose:
model=recreation of a natural system to understand it
Binary garden, Marpi
dynamical
4
nonlinearly
coupled
oscillators
Igor Morski
Left-hand side: intrinsic dynamics Right-hand side: coupling function
(shown at oscillator level)
Oscillator x
Oscillator y
accelerationvelocity positionstuff that controls trajectory shape
intrinsic
frequenc
y
terms for coupling strengthwhere the social magic happens
5
important for complexity, bistable tendencies, metastability
Haken, Kelso and Bunz, Biol.
Cybern., 1985
Intrinsi
c
dynam
ic
Igor Morski
6
Collectiv
e
dynamic
Oscillations:
atoms of
dynamics
7
Many dynamical models do not embody a “self”
The system starts from one state,
…reacts when perturbed,
…returns to rest until perturbed again
Bonus 2: models are scale & level independent (can model a neuron and a finger together)
To have autonomy, anticipation, within an open system w/ nonequilibrium dynamics,
Oscillations a basis of life?
- spontaneous neural & motor activity prenatally (Khazipof & Luhmann, 2006; Robertson, 1990),
- unmasking (?) of repetitive behavior in developmental, aging disorders of the brain (Abbott et al., 2018; Brown,
2006)
- central pattern generators for subcortical control (Grillner et al., 1998; Righetti et al., 2005),
- maybe cortical self-organization (Yuste et al., 2005; Tognoli & Kelso, Neuron, 2014)
there needs to be a “self”
Bonus 1: beautiful / crucial mathematical simplifications due to symmetries
8
9
Traversing
Scales
Kelso et al., NN, 2013
Dumas et al., 2018
10
world
objects & events
machines
other peoples
organism
brain
brain areas
microcircuits columns,
laminas
neuron
dendrites
synaptic terminal
soma
ion channels
neurotransmitters
A focus on dynamics ignores the boundaries
between disciplines and scales
The complementarity of
Experime
nts
Computati
onal
models
&11
ModelExperiments: our social coordination paradigm
That?! Social???
A minimal model of
coupling between individuals
…state variables
that can be explicitly
quantified, modeled, compared
Finger “wiggling”:
the workhorse of Coordination Dynamics
12
Tognoli, in Springer 2008
Tognoli et al., Front. Hum.
Neurosci. 2020
Order Parameters
a.k.a. coordination
variables
13
Kelso, MIT Press, 1995
Tognoli et al., ICCN, 2013
Tognoli & Kelso, Frontiers Hum.
Neurosci. 2015
Tognoli et al., Frontiers Hum.
Neurosc. 2020
p
0
time
14
Quantitative framework: coordination dynamics
State transitions
Or Metastability
Here metastability
Behavi
or
evidence
Brain
Tognoli & Kelso, Prog.
Neurobiol. 2009
Tognoli & Kelso, Neuron, 2014
Tognoli & Kelso, Neurosc. Res.
2020
Tognoli et al., PNAS, 2007
Kelso et al., PLoS One,
2009;
Dumas et al., PNAS, 2014
Tognoli et al., ICCN 2017
Tognoli et al., Front. Hum.
Neurosci. 202015
behavior brain
Metastability, Bistable tendencies inphase and antiphase16
Tognoli & Kelso, Prog.
Neurobiol. 2009
Tognoli & Kelso, Neuron,
2014
Tognoli & Kelso, Neurosc.
Res. 2020
Tognoli et al., PNAS, 2007
Kelso et al., PLoS One,
2009;
Dumas et al., PNAS, 2014
Tognoli et al., ICCN 2017
Tognoli et al., Front. Hum.
Neurosci. 2020
Findings: coordination dynamics
Tognoli et al.,
Frontiers Hum.
Neurosc.
2020
O
V
E
R
V
I
E
W17
A
U
G
M
E
N
T
A
T
I
O
N
S
‘HKB’
Haken, Kelso and
Bunz,
Biol. Cybern,
198518
---HKB---
Igor Morski
SYMMETRIES
Oscillator x
Oscillator y
intrinsic
frequenc
y
terms for coupling strength
19
Evolving
dynamics
What for?We can sync together
Breaking the Symmetry
Kelso et al., in Attention
& Performance XIII,
1990
20
21
How?
• Symmetry breaking term shifts
attractors
• Unleashes metastability
Why?
• In nature, pieces hardly ever
have identical intrinsic
properties
What for?
We can sync or coordinate
even if we are not perfectly
identical
• Agility
• Lability of coordinative
organizations
Appending discreteness
Jirsa & Kelso,
J. Motor Behavior, 2005 Dumas et al, PNAS, 201422
23
How?
• Inspired by a model of neuronal
excitability
• Introduces fast and slow
variables
Why?
• Discrete dynamics is
observed
What for?
• We can take turns
• We can have roles
Social memory
(the self that slowly becomes what the interaction
made it to be)
IgorMorski
Nordham et al., Ecol. Psychol., 2018
After socially interacting, human “oscillators” do not fully
return to their initial frequency.
Not predicted by classical models of coupled oscillators.
Empirically complex: depends on coordinative stability,
initial difference, coupling strength…
Self is changed by interaction, slowly, in a manner
that is empirically verified
We now have a model in which a father’s behavior
will edify his child’s, where a teacher will leave her
mark on pupils, where teammates and spouses
will attune to each other over time, where cities
will move toward a common pace.
Parameter
24
staticdynamics
intrinsic dynamicsDumas et al., PNAS, 2014;
Nordham et al., Ecolog. Psych.
2018;
Tognoli et al., Frontiers Hum.
Neurosc. 2020
25
Dumas et al., PNAS 2014
Nordham et al., Ecolog.
Psych. 2018
How?
• Intrinsic dynamics changed at
rate e
Why?
• Adaptation, social memory are
observed
What for?
• We can track someone’s
frequency (social
adaptation)
• We can change our
internal state to
facilitate coordination
with others (atunement
to others)
Directedness
Dumas et al, PNAS, 2014
Kostrubiec et al., PLoS One, 2015
Tognoli et al., Frontiers Hum.
Neurosc. 202026
Dumas et al, PNAS, 2014
Kostrubiec et al., PLoS
One, 2015
27
How?
• Add a phase directing term to
change attractor landscape
Why?
• Coordination inphase and
antiphase easy, at other phases
-> laborious
What for?
• You can teach me
coordinative states that
are not natural to my
behavioral repertoire
(skillful joint actions)
The interplay of multiple agents
The ZhangTrilogy:
Zhang et al., PLoS one, 2018;
Zhang et al., Royal Interface,
2019
Zhang et al., Meth. Cog.
Neurosci. 2020
Tognoli et al., Frontiers Hum.
Neurosc. 202028
29
How?
• Hybrid Kuramoto and HKB
Why?
• Coordination at N>>2 is not
S of dyadic coordination ->
emergence
What for?
• Multi-agent
• Across levels
“Nesting-
doll”
strategy
and
outlook
30
Kelso et al., NN, 2013
Dumas et al., PNAS, 2014
Tognoli et al., Cog. Neurodyn.
2018
Kelso et al., NN, 2013
Dumas et al., PNAS, 2014
Tognoli et al., Cog. Neurodyn.
201831
Incremental functional space w/ deferred socio-
cognitive gratification
HKB
intrinsic
HKB
coupling
Broken
symmetry
Discreteness
AdaptationMultiagency
Directedness
32
Multiscale NeuroComputational
Modeling
Kelso et al., NN, 2013
Tognoli et al., Cog.
Neurodyn. 2018
Our neurocomputational model for social
interactions (Inspired from Dumas et al., 2012)
Dual use:
-Clamp behavioral
coordination dynamics
study resulting neural
oscillations
-Clamp neural oscillation
and study effect on
behavioral coordination
Growth potential: Add levels (genes, endocrine glands,
machine, culture…, anything whose dynamics can be
tracked and put into the model) and build useful multiscale
descriptions of common social behavior
33 Tognoli, Dumas & Kelso, Cognitive Neurodynamics, 2018
Neuromarkers
of
social
coordination
Tognoli et al., PNAS, 2007;
Tognoli & Kelso, Front. Hum.
Neurosci. 2015
Dumas et al., Cercor, 2020
Dodel et al., Front. Hum.
Neurosci. 202034
Empirically grounded in neurodynamics of social
behavior: benchmarks
Phi complex, a neuromarker of social coordination
Tognoli et al., PNAS, 2007
IgorMorski
Tognoli & Kelso, Frontiers in Human Neuroscience, 2015
Many neuromarkers for social behavior,
task-specificity to be further elucidated
35
36
Dynamical integrity of
the self and agency
Dumas et al., Cer. Cor. 2020
37 Neurobiological foundations of Agency Attribution
Dumas et
al., CerCor,
2020
Present and past sponsors Computational tool sharing
M code and tutorial: how to compute and visualize the relative
phase f (etognoli@fau.edu)
See also interactive online tool from Dumas & Drever, embodying a
{competitive or cooperative} virtual partner governed by HKB
equations of social coordination at:
http://www.morphomemetic.org/vpi/
Some contributors to the HKB
model, pre-social and social era
Haken
Schöner
Jirsa
Assisi
Stefanescu
Experiments
Zanone
Fink
Banerjee
Jantzen
Dumas
Nordham
Zhang
Beetle
Experiments
Lagarde
Oullier
Benites
Reveley
Kelso, de Guzman, Fuchs
Tightly integrated
transdisciplinary
teams
…and many others
Key concepts: nonequilibrium, intrinsic dynamics, relative phase, multiscale modelling, weak coupling, metastability
38
Phase transition, Bistability, Bifurcation
Discussion?
39
From the top down,
but Frankenstein
problem
Sociocognitive functions >
physiology
From the ground up,
functional prowess
deferred
Dynamics > neurobehavioral

Computational Social Neuroscience - E Tognoli

  • 1.
    EmmanuelleTognoli, Center for ComplexSystems and Brain Sciences, Florida Atlantic University Complex Systems & Weinberg Institute Seminar, October 27th, 2020 Computational Social Neuroscience tognoli@ccs.fau.edu 1
  • 2.
    Feel welcome toscribble, interject, stamp (Grateful if someone wipes the slides clean) Slides are numbered for recall, questions and comments during the final discussion. 2 Most illustrations from Polish surrealist artist Igor Morski
  • 3.
    Social behavior Development, Aging Mental health 3 Interactivehuman-oriented technologies {HMI, BMI, BCI, BMI, BBI, HRI…} Key areas
  • 4.
    Many dynamical modelshave been used in {Social} and/or {Affective} and/or {Cognitive} and/or {Neuro}science Agent based model s Bayesian Stochast ic Diffusion Markov Signal flow graphs Lokta- Volterra HKB Recurren t network s Cellular automat a For our purpose: model=recreation of a natural system to understand it Binary garden, Marpi dynamical 4
  • 5.
    nonlinearly coupled oscillators Igor Morski Left-hand side:intrinsic dynamics Right-hand side: coupling function (shown at oscillator level) Oscillator x Oscillator y accelerationvelocity positionstuff that controls trajectory shape intrinsic frequenc y terms for coupling strengthwhere the social magic happens 5 important for complexity, bistable tendencies, metastability Haken, Kelso and Bunz, Biol. Cybern., 1985
  • 6.
  • 7.
  • 8.
    Many dynamical modelsdo not embody a “self” The system starts from one state, …reacts when perturbed, …returns to rest until perturbed again Bonus 2: models are scale & level independent (can model a neuron and a finger together) To have autonomy, anticipation, within an open system w/ nonequilibrium dynamics, Oscillations a basis of life? - spontaneous neural & motor activity prenatally (Khazipof & Luhmann, 2006; Robertson, 1990), - unmasking (?) of repetitive behavior in developmental, aging disorders of the brain (Abbott et al., 2018; Brown, 2006) - central pattern generators for subcortical control (Grillner et al., 1998; Righetti et al., 2005), - maybe cortical self-organization (Yuste et al., 2005; Tognoli & Kelso, Neuron, 2014) there needs to be a “self” Bonus 1: beautiful / crucial mathematical simplifications due to symmetries 8
  • 9.
    9 Traversing Scales Kelso et al.,NN, 2013 Dumas et al., 2018
  • 10.
    10 world objects & events machines otherpeoples organism brain brain areas microcircuits columns, laminas neuron dendrites synaptic terminal soma ion channels neurotransmitters A focus on dynamics ignores the boundaries between disciplines and scales
  • 11.
  • 12.
    ModelExperiments: our socialcoordination paradigm That?! Social??? A minimal model of coupling between individuals …state variables that can be explicitly quantified, modeled, compared Finger “wiggling”: the workhorse of Coordination Dynamics 12 Tognoli, in Springer 2008 Tognoli et al., Front. Hum. Neurosci. 2020
  • 13.
  • 14.
    Kelso, MIT Press,1995 Tognoli et al., ICCN, 2013 Tognoli & Kelso, Frontiers Hum. Neurosci. 2015 Tognoli et al., Frontiers Hum. Neurosc. 2020 p 0 time 14 Quantitative framework: coordination dynamics State transitions Or Metastability Here metastability
  • 15.
    Behavi or evidence Brain Tognoli & Kelso,Prog. Neurobiol. 2009 Tognoli & Kelso, Neuron, 2014 Tognoli & Kelso, Neurosc. Res. 2020 Tognoli et al., PNAS, 2007 Kelso et al., PLoS One, 2009; Dumas et al., PNAS, 2014 Tognoli et al., ICCN 2017 Tognoli et al., Front. Hum. Neurosci. 202015
  • 16.
    behavior brain Metastability, Bistabletendencies inphase and antiphase16 Tognoli & Kelso, Prog. Neurobiol. 2009 Tognoli & Kelso, Neuron, 2014 Tognoli & Kelso, Neurosc. Res. 2020 Tognoli et al., PNAS, 2007 Kelso et al., PLoS One, 2009; Dumas et al., PNAS, 2014 Tognoli et al., ICCN 2017 Tognoli et al., Front. Hum. Neurosci. 2020 Findings: coordination dynamics
  • 17.
    Tognoli et al., FrontiersHum. Neurosc. 2020 O V E R V I E W17 A U G M E N T A T I O N S
  • 18.
  • 19.
    ---HKB--- Igor Morski SYMMETRIES Oscillator x Oscillatory intrinsic frequenc y terms for coupling strength 19 Evolving dynamics What for?We can sync together
  • 20.
    Breaking the Symmetry Kelsoet al., in Attention & Performance XIII, 1990 20
  • 21.
    21 How? • Symmetry breakingterm shifts attractors • Unleashes metastability Why? • In nature, pieces hardly ever have identical intrinsic properties What for? We can sync or coordinate even if we are not perfectly identical • Agility • Lability of coordinative organizations
  • 22.
    Appending discreteness Jirsa &Kelso, J. Motor Behavior, 2005 Dumas et al, PNAS, 201422
  • 23.
    23 How? • Inspired bya model of neuronal excitability • Introduces fast and slow variables Why? • Discrete dynamics is observed What for? • We can take turns • We can have roles
  • 24.
    Social memory (the selfthat slowly becomes what the interaction made it to be) IgorMorski Nordham et al., Ecol. Psychol., 2018 After socially interacting, human “oscillators” do not fully return to their initial frequency. Not predicted by classical models of coupled oscillators. Empirically complex: depends on coordinative stability, initial difference, coupling strength… Self is changed by interaction, slowly, in a manner that is empirically verified We now have a model in which a father’s behavior will edify his child’s, where a teacher will leave her mark on pupils, where teammates and spouses will attune to each other over time, where cities will move toward a common pace. Parameter 24 staticdynamics
  • 25.
    intrinsic dynamicsDumas etal., PNAS, 2014; Nordham et al., Ecolog. Psych. 2018; Tognoli et al., Frontiers Hum. Neurosc. 2020 25 Dumas et al., PNAS 2014 Nordham et al., Ecolog. Psych. 2018 How? • Intrinsic dynamics changed at rate e Why? • Adaptation, social memory are observed What for? • We can track someone’s frequency (social adaptation) • We can change our internal state to facilitate coordination with others (atunement to others)
  • 26.
    Directedness Dumas et al,PNAS, 2014 Kostrubiec et al., PLoS One, 2015 Tognoli et al., Frontiers Hum. Neurosc. 202026 Dumas et al, PNAS, 2014 Kostrubiec et al., PLoS One, 2015
  • 27.
    27 How? • Add aphase directing term to change attractor landscape Why? • Coordination inphase and antiphase easy, at other phases -> laborious What for? • You can teach me coordinative states that are not natural to my behavioral repertoire (skillful joint actions)
  • 28.
    The interplay ofmultiple agents The ZhangTrilogy: Zhang et al., PLoS one, 2018; Zhang et al., Royal Interface, 2019 Zhang et al., Meth. Cog. Neurosci. 2020 Tognoli et al., Frontiers Hum. Neurosc. 202028
  • 29.
    29 How? • Hybrid Kuramotoand HKB Why? • Coordination at N>>2 is not S of dyadic coordination -> emergence What for? • Multi-agent • Across levels
  • 30.
    “Nesting- doll” strategy and outlook 30 Kelso et al.,NN, 2013 Dumas et al., PNAS, 2014 Tognoli et al., Cog. Neurodyn. 2018
  • 31.
    Kelso et al.,NN, 2013 Dumas et al., PNAS, 2014 Tognoli et al., Cog. Neurodyn. 201831 Incremental functional space w/ deferred socio- cognitive gratification HKB intrinsic HKB coupling Broken symmetry Discreteness AdaptationMultiagency Directedness
  • 32.
    32 Multiscale NeuroComputational Modeling Kelso etal., NN, 2013 Tognoli et al., Cog. Neurodyn. 2018
  • 33.
    Our neurocomputational modelfor social interactions (Inspired from Dumas et al., 2012) Dual use: -Clamp behavioral coordination dynamics study resulting neural oscillations -Clamp neural oscillation and study effect on behavioral coordination Growth potential: Add levels (genes, endocrine glands, machine, culture…, anything whose dynamics can be tracked and put into the model) and build useful multiscale descriptions of common social behavior 33 Tognoli, Dumas & Kelso, Cognitive Neurodynamics, 2018
  • 34.
    Neuromarkers of social coordination Tognoli et al.,PNAS, 2007; Tognoli & Kelso, Front. Hum. Neurosci. 2015 Dumas et al., Cercor, 2020 Dodel et al., Front. Hum. Neurosci. 202034
  • 35.
    Empirically grounded inneurodynamics of social behavior: benchmarks Phi complex, a neuromarker of social coordination Tognoli et al., PNAS, 2007 IgorMorski Tognoli & Kelso, Frontiers in Human Neuroscience, 2015 Many neuromarkers for social behavior, task-specificity to be further elucidated 35
  • 36.
    36 Dynamical integrity of theself and agency Dumas et al., Cer. Cor. 2020
  • 37.
    37 Neurobiological foundationsof Agency Attribution Dumas et al., CerCor, 2020
  • 38.
    Present and pastsponsors Computational tool sharing M code and tutorial: how to compute and visualize the relative phase f (etognoli@fau.edu) See also interactive online tool from Dumas & Drever, embodying a {competitive or cooperative} virtual partner governed by HKB equations of social coordination at: http://www.morphomemetic.org/vpi/ Some contributors to the HKB model, pre-social and social era Haken Schöner Jirsa Assisi Stefanescu Experiments Zanone Fink Banerjee Jantzen Dumas Nordham Zhang Beetle Experiments Lagarde Oullier Benites Reveley Kelso, de Guzman, Fuchs Tightly integrated transdisciplinary teams …and many others Key concepts: nonequilibrium, intrinsic dynamics, relative phase, multiscale modelling, weak coupling, metastability 38 Phase transition, Bistability, Bifurcation Discussion?
  • 39.
    39 From the topdown, but Frankenstein problem Sociocognitive functions > physiology From the ground up, functional prowess deferred Dynamics > neurobehavioral