Presentation about a little-known kind of synaesthesia in which visual motion evokes auditory sensations. Presented at UK Synaesthesia Association 2016 by Elliot Freeman. Download to see movie demos simulating this condition, and demos of the different tasks (~17Mb).

1. Hearing through your eyes:
Modulation of visually-evoked auditory
response by transcranial electrical
stimulation
Elliot Freeman
Christopher Fassnidge
Synøve Knudsen, Anthony Tipple, Danny Ball
Cognitive Neuroscience Research Unit,
City University London

2. the ‘visually evoked auditory response’
(vEAR)

3. Questions
1. Is vEAR perceptually ‘real’?
2. How prevalent is vEAR?
3. Do visual and auditory cortices interact
differently in vEAR?

4. Is vEAR perceptually ‘real’?
• Most people better at
auditory sequence
discrimination than visual
• but people with vEAR as
almost good at visual
(Saenz & Koch, 2006)
• Do they actually hear the
flashes?
• → Measure detection of
faint sounds while
watching visual motion
Lovelace et al 2003

5. vEAR sounds are real
Better visual sequence
discrimination re: audio
harder to detect faint
sounds in presence of
visual motion
Visual-induced auditory
sensation interferes
with actual hearing
Better visual sequence discrimination re: aud →
Betterauditorydetectionwithvisionre:none→

6. Questions
• vEAR is perceptually ‘real’
• How prevalent is vEAR?
• Do visual and auditory cortices interact
differently in vEAR?

7. Participants and task
• 37 Participants
• 18 to 55 years (M24,
SD8.69)
• Included 9 self-reporting
synaesthetes
• 11 Royal College of Music
students
• Debrief questions
• ‘Did you hear faint sounds
accompanying the
flashes?’
• ‘Are you diagnosed as
synaesthete or experience
pairings, e.g. …’
Table 1
n /
37
prop.
Heard Sounds 11 30%
Musicians 11 30%
Synaesthetes 9 25%

8. Trait associations
• Musicians tend to
have vEAR
𝜒2= 5.54, p<0.02
• Musicians tend to
have synaesthesia
𝜒22 = 6.44, p<0.01
• Synaesthetes tend to
have vEAR
𝜒22 = 9.05, p<0.01
Control Musicians
Control 19 8
Syn 2 7
Control Musicians
No vEAR 18 8
vEAR 3 8
NonSyn Syn
No vEAR 23 3
vEAR 4 6

9. Questions
1. vEAR is perceptually ‘real’
2. vEAR is prevalent especially in
musicians
3. Do visual and auditory cortices interact
differently in vEAR?

10. Do vEAR brains work differently?
• alpha-frequency
Transcranial Alternating
Current Stimulation
• Temporal or occipital
sites
• Measure effect on
auditory and visual
sequence discrimination
(Saenz & Koch 2006)
➔ Does TACS effect
depend on individual
differences in vEAR?

11. Hypotheses
Cross-activation
- In vEAR, auditory cortex
is more excited by visual
inputs
- αTACS might interrupt
this crosstalk.
➔ Greater effect in vEAR
Unmasking
- In non-vEAR,
audiovisual crosstalk is
usually inhibited
- αTACS might interrupt
this inhibition.
➔ Greater effect in non-
vEAR

12. Methods: TACS
• 1000μA bilateral for 15
minutes during task
• Stimulation vs Sham
double-blinded;
counterbalanced
within-session
• Sites: occipital pole
(O1, O2) vs temporal
(T3, T4);
counterbalanced
between session

13. Results: Task x Site
— Temporal stimulation
impairs auditory
discrimination, but
improves visual
— Occipital stimulation,
impairs visual
discrimination, but
improves auditory
→ Inter-cortical
competition?
d’StimminusSham
[F(1,35) = 4.34, p<0.05]

14. Results: Task x Site x
HeardSounds
• — — Reduced stimulation
effect in participants who
reported hearing illusory
sounds accompanying the
visual flashes
→ Inter-cortical cooperation?
• Visual and auditory cortices
used together to ‘hear’
flashes and ‘see’ sounds.
• No interactions with
Musicianship or
synaesthesia
d’StimminusSham
[F(1,35) = 6.33, p<0.05]

15. Which hypothesis is right?
Cross-activation
- In vEAR, auditory cortex
is more excited by visual
inputs
- αTACS interrupts this
crosstalk.
➔ Greater effect in vEAR
Unmasking
- In non-vEAR,
audiovisual crosstalk is
usually inhibited
- αTACS might interrupt
this inhibition.
➔ Greater effect in non-
vEAR
✔

16. Discussion
• Vision and audition
normally inhibit each
other (Mattingley et al, 1997; Iurilli
et al, 2012)
• Inhibition is carried by α
frequency oscillations
(Klimesch et al, 2007)
• TACS stimulation might
interrupt the inhibitory
signals
• Competition is biased
in favour of the
unstimulated cortex.
A V
A V

17. Discussion
• In vEAR, TACS may
be less effective
because there is less
inhibition
• Synaesthetic
crosstalk is unmasked
(Grossenbacher & Lovelace, 2001;
Cohen Kadosh & Walsh, 2006)
• Visual and auditory
cortices cooperate,
not compete
A V
A V

18. Answers
1. vEAR is perceptually ‘real’
• visually-evoked sounds may mask real sounds
2. vEAR is prevalent
• more frequent in musicians and synaesthetes
• also common in controls: 3/18 = 16%
3. Visual and auditory cortices interact differently
• People who hear visual motion use both vision
and audition cooperatively, not competitively
• reduced αTACS effect in vEAR supports
‘unmasking’ theory of synaesthesia