These slides target more general audience in the study group at NTHU Linguistics.

1. Neural responses to syllable gaps
in Taiwanese Mandarin: an
event-related potential study
Presenter: Chiung-Yu Chang
Advisor: Feng-fan Hsieh
National Tsing Hua Univeristy

2. Mandarin syllables
(C) (G) V (X)
p, pʰ, m, f, t, tʰ, n, l, k, kʰ, x, ʐ,
t
͡ ɕ, t
͡ ɕʰ, ɕ, ʈ
͡ ʂ, ʈ
͡ ʂʰ, ʂ, t͡s, t
͡ sʰ, s
i, u, y /i, u, y, e, o, ɤ, a/
i, u, n, ŋ

3. Gaps in the Mandarin syllabary
Syllabary: all possible combinations of (C)(G)V(X)
According to the dictionary, 64% of the slots are empty.
Tonal gaps (TG): attested segmental combinations that
do not bear certain tone(s), e.g., [ly1] (cf. [ly4] ‘green’)
Segmental gaps (SG): unattested segmental
combinations regardless of the bearing tone, e.g., [ki]

4. TG vs. SG
Native speakers seems to process TGs and SGs in
different ways.
• Reaction time in lexical decision task: TG > SG
(Yao & Sharma, 2017).
• Wordlikeness: TG > SG
(Lai, 2003; Myers, 2002; Wang, 1998) .

5. Tones and segments in spoken Mandarin
• Most studies agree that segments are more
important/immutable than tones.
• Some studies showed that tones are processed
later than segments.
• Both types of information are integrated and
dependent.

6. Motivation #1
to investigate the time course of processing different
types of gaps, using event-related potentials (ERP), an
analysis method of encephalography (EEG) data.
→ real-time signal
→ high temporal resolution in ms
(cf. behavioral measures)

7. ERP correlates of phonotactic processing
N400: a component typically associated with lexico-
semantic integration.
Difference in the N400 amplitude?
• Rossi et al.’s (2011) on German CC cluster
legal [brop] > illegal [bzop]
• White & Chiu (2017) on English CC cluster and SSP
ill-formed [tlace] > well-formed [ltace]

8. Late positive complex (LPC): a vague term…
Difference in the LPC amplitude?
• Domahs et al.’s (2009) on German OCP
ill-formed [spɛp] > well-formed [spɛf]
The N400 effect was attributed to lexicality.
ERP correlates of phonotactic processing (cont.)

9. Phonotactic probability:
• Hunter (2013): the P2 (240-300 ms)
• Silva et al. (2019):
the N1-P2 complex (140-250 ms)
the P3 (330-450 ms)
ERP correlates of phonotactic processing (cont.)
Early effects

10. The previous results are inconsistent.
Possible explanations:
• Different tasks (passive listening vs. overt response)
• Different languages and phonotactic constraints
• Lexicality & phonological neighborhood density
(i.e., similar forms stored in the lexicon)
ERP correlates of phonotactic processing (cont.)

11. Motivation #2
to explore the task dependency of ERP correlates to
phonotactic processing.
Experiment 1: Passive listening + Go/No-go (filler task)
Experiment 2: lexical decision

12. Research questions
Q1: Are there differences in the ERP responses to TGs,
SGs, and real syllables?
→ compared the ERP amplitudes across conditions
Q2: How does task requirements influence the ERP
results?
→ the interaction between Condition and Task

13. Hypotheses & Predictions
• Function weight: segments > tone
• Probabilistic phonotactics of TC co-occurrence
→ amplitude difference between ERPs:
Δ (SG-real) > Δ (TG-real)
• The N400 is related to lexico-semantics
→ more pronounced N400 effect in lexical decision

14. Experiment 1
Go/No-go task

15. Method
• Participants: 24 right-handed native speakers of
Taiwanese Mandarin
• Material: 80 TGs, 80 SGs, and 80 real words
• Procedure: passive listening + Go/No-go (filler)

16. EEG data preprocessing
• Epochs: time-locked to the stimulus onset
from -150 to 750 ms
• Low-pass filter
• Baseline correction
• Artefact rejection
• Averaging for each condition and each subject
in ERPLAB (Lopez-Calderon & Luck 2014)

17. Statistical analysis
• Hypothesis driven
amplitudes in 3 pre-selected time windows:
75-125 ms, 150-250 ms, 300-500 ms
from 9 electrodes covering the central area
→ repeated-measure ANOVA
• Exploratory
Factorial mass univariate analysis (Fields, 2018)

18. Results: overview
N1
P2
Let’s call it “P3” tentatively…

19. Results: overview

20. Results: hypothesis driven
• Amplitude
75-125 ms: TG < SG; TG < real (at C3, CP3)
150-250 ms: no significance
300-500 ms: TG < SG; TG < real (at FC3, CP3, CPz)
• Topography
No interaction between Condition and other factors
i.e., Site (Anterior/Posterior) and Hemisphere.

21. Results: exploratory
• Significant difference between conditions:
at FC3 in 341-349 ms
at Cz at 345 ms
• Post hoc t-test:
in 341-349 ms at FC3 and Cz, TG < SG.
(mass univariate ERP toobox (Groppe et al., 2011))

22. Discussion
• Potential confounds?
Vowel onset time: SG > TG; SG > real
Proportion of voiced onsets: TG > SG; TG > real
• Past research has suggested that
earlier voicing is associated with larger N1

23. Discussion (cont.)
• However, physical or acoustic confounds seldom
have direct impact on later time window ~ 345 ms
• Effect of tone-consonant co-occurrence probability
but not categorical phonotactics (Silva et al., 2019)
• Daffner et al. (2000): familiarity → larger N2
• Decreased N2 appears as increased P3?

24. Discussion (cont.)
• Late negativity related to anticipation of the prompt
contingent negative variation (CNV) or
stimulus-preceding negativity (SPN)
• The primary task may direct the participants’
attention to certain perceptual dimensions that
reliably differentiates targets from non-targets,
such as f0 and duration → no lexical processing

25. Experiment 2
Lexical decision task

26. Method
• Participants: 11 right-handed native speakers of
Taiwanese Mandarin
• Material: same as Experiment 1 + 80 real words
• Procedure: lexical decision

27. EEG data
acquisition
preprocessing
statistical analyses
• Same as in Experiment 1
• Except that the epoch is from -200 to 800 ms

28. Behavioral results
• Accuracy: TG < SG; TG < real
• Reaction time: TG > SG; TG > real

29. ERP results: overview
N1
P2
N400
LPC
TG
SG
Real word

30. ERP results: overview

31. ERP results: hypothesis driven
• Amplitude
300-500 ms: TG > SG (at FCz, C3, Cz, C4, CP3, CPz)
no significance elsewhere
• Topography
The difference along the anterior-posterior axis
TG > SG; TG > real

32. ERP results: exploratory
• Significant difference between conditions:
at many posterior electrodes in 719-763 ms
• Post hoc t-test:
SG > TG
SG > real
Around 750 ms

33. Discussion
• The N400 component in the response to SGs
indicates that categorical phonotactic constraints
does not prevent it from entering lexical access.
• Neighborhood density
vs. phonological well-formedness?

34. Discussion (cont.)
• The LPC has been suggested to correlate with task
difficulty and reanalysis of the stimuli.
• The RTs and accuracies for SGs and real words were
actually similar in the present study
→ favor the reanalysis account
• The accuracy for real words was larger in Yao &
Sharma (2017) because they used confusable ones,
e.g., [ɕən3]

35. Discussion (cont.)
• The difference in the topography suggests that
tones and segments might involve different neural
mechanisms to integrate.
• But I cannot say too much about this because
ERP/EEG alone does not provide enough spatial
information (cf. fMRI, MEG)

36. General discussion

37. Comparison of ERP responses across tasks
Task Go/No-go Lexical decision
N1 TG < SG; TG < real topography
P2 Not significant topography
300-500 ms TG < SG; TG < real SG > TG
Exploratory TG < SG; TG < real
~ 345 ms
SG > TG; SG > real
~ 750 ms

38. • Different levels of speech perception:
sensory → sublexical → lexical → semantic
• Early stages of speech perception are subject to
top-down influence.
• Task-induced processing requirements are crucial
factors that must be taken into consideration in the
experimental design (Swaab et al., 2012)
Task dependency of auditory speech processing

39. • Systematic cross-linguistic differences in the ERP
signatures (see Bornkessel-Schlesewsky et al. (2011)
and the references therein).
• Mandarin listeners might process syllables in a
more holistic fashion (Zhao et al., 2011; Myers &
Chen, 2019; but see Malins & Joanisse (2012) for
counterevidence)?
Cross-linguistic differences

40. • Influence of Chinese characters
• The lexicality effect on wordlikeness judgment is
stronger in Mandarin than in English (Myers, 2002).
• The phonotactic effect is presumably weaker in
Mandarin spoken word recognition?
More statistical power to detect the early effect?
Cross-linguistic differences (cont.)

41. • A need for the representations of tones
• The results in Experiment 2 also suggested that
listeners are influenced by tonal neighborhood
density (e.g., [tau1], [tau3], [tau4])
→ separate representations for tones and vowels
in the lexical level
Implications for models of spoken word recognition

42. • The ERP effects may not be generalizable over all
items in the same category
• No by-item analysis,
i.e., sonorant-initial versus obstruent-initial?
→ Other advanced analysis methods, such as multiway
canonical correlation analysis (MCCA)
→ Data-driven analysis, such as hierarchical clustering
Current limitations and future directions

43. Neural oscillations
Current limitations and future directions (cont.)

44. • Yao, Y. & Sharma, B. What is in the neighborhood of a tonal syllable?
Evidence from auditory lexical decision in Mandarin Chinese. Proceedings
of the Linguistic Society of America 2, 45-41-14 (2017).
• Wang, H. S. An experimental study on the phonotactic constraints of
Mandarin Chinese. Studia Linguistica Serica, 259-268 (1998).
• Swaab, T. Y., Ledoux, K., Camblin, C. C. & Boudewyn, M. A. in The Oxford
handbook of event-related potential components. Oxford library of
psychology. 397-439 (Oxford University Press, 2012).
Selected references

45. • Myers, J. An analogical approach to the Mandarin syllabary. Chinese
Phonology 11, 163-190 (2002).
• Lai, Y.-C. A Perceptual Investigation on Mandarin Tonotactic Gaps Master
thesis, National Tsing Hua University, (2003).
• Zhao, J., Guo, J., Zhou, F. & Shu, H. Time course of Chinese monosyllabic
spoken word recognition: Evidence from ERP analyses. Neuropsychologia
49, 1761-1770 (2011).
• White, J. & Chiu, F. Disentangling phonological well-formedness and
attestedness: An ERP study of onset clusters in English. Acta Linguistica
Academica 64, 513-537 (2017).
Selected references

46. • Rossi, S. et al. Implicit processing of phonotactic cues: evidence from
electrophysiological and vascular responses. Journal of Cognitive
Neuroscience 23, 1752-1764 (2011).
• Malins, J. G. & Joanisse, M. F. Setting the tone: An ERP investigation of the
influences of phonological similarity on spoken word recognition in
Mandarin Chinese. Neuropsychologia 50, 2032-2043 (2012).
• Domahs, U., Kehrein, W., Knaus, J., Wiese, R. & Schlesewsky, M. Event-
related potentials reflecting the processing of phonological constraint
violations. Language and Speech 52, 415-435 (2009).
Selected references

47. • Bornkessel-Schlesewsky, I. et al. Think globally: Cross-linguistic variation in
electrophysiological activity during sentence comprehension. Brain and
Language 117, 133-152 (2011).
• Silva, S., Vigário, M., Fernandez, B. L., Jerónimo, R., Alter, K., & Frota, S.
(2019). The sense of sounds: Brain responses to phonotactic frequency,
phonological grammar and lexical meaning. Frontiers in psychology, 10.
• Myers, J. & Chen, T.-Y. in 27th Annual Conference of the International
Association of Chinese Linguistics (Kobe City University of Foreign Studies,
2019).
Selected references

48. The end.
Thank you very much!