This document discusses the concept of speaker-listener neural coupling and its significance in successful communication, highlighting that brain responses of listeners become coupled with those of speakers during verbal exchanges. It suggests that the extent of this coupling can predict communication success and emphasizes the importance of temporal dynamics in these interactions. Additionally, the findings indicate that recording neural responses from both parties can enhance our understanding of both verbal and non-verbal communication in various contexts.

1. Do We Click? Speaker-listener
neural coupling underlies
successful communication
Lauren Silbert1, Greg Stephens2,3, Uri Hasson1
1PrincetonNeuroscience Institute and Psychology Department,
Princeton University, 2Lewis-Sigler Institute for Integrative Genomics,
and 3 Joseph Henry Laboratories of Physics, Princeton University,
Princeton, NJ

2. Verbal communication is a joint activity through
which individuals share information
Listener Speaker
(comprehension) (production)

3. Communication is integral to the functioning of
our complex societies
Globalization and the rapidly changing world
demand we increase our understanding of how
communication succeeds

4. Verbal communication is a joint activity through
which individuals share information
Listener Speaker
(comprehension) (production)

5. Neurolinguistic studies constrained by
boundaries of individual brains
Listener Speaker
(comprehension) (production)

6. Neurolinguistic studies constrained by
boundaries of individual brains
Listener Speaker
(comprehension) (production)

7. Assessing an interactive process:
laboratory verse real-world stimuli
Control simplified stimuli
red
cat
bat
Isolation from the environment

8. Assessing an interactive process:
laboratory verse real-world stimuli
Control simplified stimuli
red Complex natural stimuli
cat
bat
Two brains interacting
Isolation from the environment

9. Design
Speaker
15 min monologue
(real life unrehearsed story)

10. FOMRI™ II Dual Channel MRI Microphone System
Mic 1
Mic 2
Two orthogonal pressure gradient optical microphones
Mic 1: Signal +
noise
Mic 2: noise
Extracted
signal

11. Design
Speaker Listener
15 min monologue
(n=11)
(real life unrehearsed story)

12. Brain activity during production as a model for
comprehension
model
(n=11)

13. Brain activity during production as a model for
comprehension
model
(n=11)

14. Brain activity during production as a model for
comprehension
model Listener’s response
(n=11)

15. Brain activity during production as a model for
comprehension
• Bypass the need to specify a priori any formal model
of linguistic processes in any given brain area
• Ability to look at similarity of responses during speech
production and speech comprehension in the same area

16. Adding Temporal
Dynamics to the Model:
(n=11)

17. Adding Temporal
Dynamics to the Model:
(n=11)

18. Adding Temporal
Dynamics to the Model:
(n=11)

19. Adding Temporal
Dynamics to the Model:
(n=11)

20. Adding Temporal
Dynamics to the Model:
(n=11)

21. Adding Temporal
Dynamics to the Model:
(n=11)

22. Adding Temporal
Dynamics to the Model:
(n=11)

23. Adding Temporal
Dynamics to the Model:
(n=11)

24. Adding Temporal
Dynamics to the Model:
(n=11)

25. Conservative Statistics: Parametric and Non-parametric
Correct for
Analytic F-test applied multiple
to overall model fit comparisons
using FDR
g=0.05 1000
Actual correlation
coefficient
(n=11)

26. Predictions
1. Are neural responses during speech production
and speech comprehension coupled?
2. Can the extent of speaker-listener neural
coupling predict the success of communication?

27. Predictions
1. Are neural responses during speech production
and speech comprehension coupled?
2. Can the extent of speaker-listener neural
coupling predict the success of communication?

28. Brain responses are reliably shared across all
listeners during comprehension
response
subject 4
f MRI
subject 3
subject 2
subject 1
time

29. Speaker’s brain responses during production are
coupled to listener’s responses during comprehension

30. Areas involved in comprehension overlap with areas
coupled during communication

31. Necessary controls
I. Is the speaker-listener coupling tied to the
content of the story?
• Block Communication
• Misaligned Communication

32. Speaker-Listener neural coupling is absent in the
absence of successful communication
?
X
Russian Speaker Non-Russian speaking
Listeners

33. Speaker-Listener neural coupling is absent when
communication is incongruous
?
X
Speaker telling story 2 Listeners to story 1

34. Necessary controls
I. Is the speaker-listener coupling tied to the
content of the story?
• Block Communication
• Shuffle Communication
II. Is the speaker a listener of herself?
• Temporal dynamics

35. Brain responses among listeners are time-locked to
the moment of vocalization
Average beta weights

36. Listener’s brain responses mirror the speaker’s
brain responses with a delay
Average beta weights

37. Brain responses shared among listeners are
temporally aligned to the moment of vocalization

38. Temporal dynamics of speaker-listener neural
coupling varies across areas

39. Importance of temporal coupling results:
• Dynamics of coupling between a speaker and a
listener are fundamentally different from dynamics
shared among all listeners.
vs
• Spatial specificity of temporal coupling
demonstrates effect cannot be attributed to non-
specific, spatially global effects like arousal.

40. Predictions
1. Are neural responses during speech production
and speech comprehension coupled?
2. Can the extent of speaker-listener neural
coupling predict the success of communication?

41. Level of understanding varies across listeners, as
measured by reliable independent raters

42. The degree of neural coupling predicts the success
of communication
Speaker-listener Neural Coupling

43. Areas where the listener’s responses precede the
speaker’s show strongest correlation with behavior

44. Summary
I. In the course of communication the listener’s brain responses
become coupled with the speaker’s brain responses.

45. Summary
I. In the course of communication the listener’s brain responses
become coupled with the speaker’s brain responses.
II. The extent of speaker-listener neural coupling is indicative of the
success of the communication.

46. Summary
I. In the course of communication the listener’s brain responses
become coupled with the speaker’s brain responses.
II. The extent of speaker-listener neural coupling is indicative of the
success of the communication.
III. An ability to evoke similar brain patterns in another individual via
speech may gate our communication abilities.

47. Summary
I. In the course of communication the listener’s brain responses
become coupled with the speaker’s brain responses.
II. The extent of speaker-listener neural coupling is indicative of the
success of the communication.
III. An ability to evoke similar brain patterns in another individual via
speech may gate our communication abilities.
IV. The recording of neural responses from both the speaker brain
and the listener brain may be used to assess verbal and non-
verbal forms of interaction in both human and other model
systems.

48. Thank You
Christopher Honey David Heeger
Yulia Lerner Bruno Galantucci
Chris Thompson Simon Garrod
Mina Cikara
Alana D'Alfonso

50. mirror neurons