The document discusses the neuroscience of multisensory integration, highlighting that perception is typically multisensory rather than unimodal. It details key concepts such as cross-modal interactions, integration, and the physiological mechanisms involved, particularly focusing on the superior colliculus. Additionally, it addresses principles of multisensory integration, the binding problem, and phenomena like the McGurk effect and synaesthesia.

1. Neuroscience of Multisensory
Integration
M2 NPES 2024/2025
Quentin Moreau, PhD

2. Our brain is multisensory
Perception is almost never unimodal
Senses interact with each other
Multimodality is an evolutionary
advantage because certain senses can
compensate for the (temporary or
permanent) lack of others

3. Key concepts
• Multisensory = more than one sensory
modality is used in perception
• Cross-modal = interaction between
different sense modalities
• Integration = merging of information
from different sense modalities into a
unified percept

4. Perception enhancement
Do you see what I’m saying ? (Ross et
al., 2007, Drijvers and Holler, 2022)
Overcoming the cocktail-party effect

5. How do we integrate sensory information ?
The role of the superior colliculus
(SC)
SC in the midbrain contains
neurons which respond to visual,
auditory and tactile stimuli

6. Multisensory enhancement
Superadditivity
Stein and Stanford, 2008

7. Evidence of perception enhancement in the SC

8. Development of Multisensory Integration (in cats)

9. Development of Multisensory Integration (in cats)

10. Sensory Deprivation and MultisensoryTraining

11. Principles of Multisensory Integration
The spatial rule - multisensory integration
is more likely or stronger when the
constituent uni-sensory stimuli arise from
approximately the same location.
The temporal rule - multisensory
integration is more likely or stronger
when the constituent uni-sensory stimuli
arise at approximately the same time.
Binding Problem

12. McGurk effect
https://www.youtube.com/watch?v=2k8fHR9j
KVM&ab_channel=sixesfullofnines
The McGurk effect (1976)

13. The Ventriloquist Illusion

14. The Rubber Hand Illusion
https://www.youtube.com/watch?v=
sxwn1w7MJvk&ab_channel=BBC

15. Visual Dominance Theory
Complex models relying on the theory
that visual perception in less plastic
than auditory or other senses, leading
to modulation of other unimodal
senses

16. Is Visual Always Dominant ? Aglioti et al., 1995

17. Modality appropriateness
Auditory Dominance over Visual for
Temporal Rate perception (Welsh and
Warren, 1985)
The Ventriloquist Effect Results from
Near-Optimal Bimodal Integration -
Alais and Burr, 2004

18. Bayesian Theories of Multimodal Integration

19. Bayesian Theories of Multimodal Integration

20. Peripersonal Space
Visuo-tactile neurons in the
premotor cortex of monkeys
(Graziano and Gross, 1998)
Physiological and psychological
components of the peripersonal
space

21. Peripersonal space is plastic
Peripersonal space extend to tool
use in humans (Farnè et al., 2000)
=> integration of objects
PPS is also modulated by social
priors between people
=> integration of others

22. Synaesthesia

23. Synaesthesia
Wannerton

24. Grapheme–color synesthesia
Most common form of synesthesia
Tomson et al., 2013; Witthoft et al., 2015

25. Chromesthesia
The association of sounds with colors

26. Pruning Theory of Synaesthesia

27. Summary
Perception is multisensory
Multimodal neurons integrate
information and generate
multisensory enhancement
No global theory about how does the
brain perform binding