This document appears to be lecture slides on speech sounds and speech production. It discusses various topics including consonants and their features, vowels and formants, larynx and phonations, and the speech chain model. It provides examples of coordinated articulatory gestures needed for speech, such as those required to produce the word "palm". It also discusses connected speech processes like contraction and provides an example of the unique rhotic sequence in the word "Saturday".

1. Speech sound Speech production Measuring speech References
探索語言
How bodies talk?
chenhaochiu@ntu.edu.tw
Oct. 29, 2020
1 / 99

2. Speech sound Speech production Measuring speech References
Today’s kick-off
Midterm last year:
2 / 99

3. Speech sound Speech production Measuring speech References
Today’s kick-off
Midterm last year:
1 True/False
2 Multiple choice
3 Short answer: Groups, tests, tree forms, reasons, etc.
3 / 99

4. Speech sound Speech production Measuring speech References
Today’s kick-off
Midterm last year:
1 True/False
2 Multiple choice
3 Short answer: Groups, tests, tree forms, reasons, etc.
Lecture
In-class exercise
4 / 99

5. Speech sound Speech production Measuring speech References
I.
Speech sound
5 / 99

6. Speech sound Speech production Measuring speech References
Consonants
Last week we talked about consonants and their IPA symbols.
There are a few criteria/features to describe consonants:
1 Place of articulation (labial, alveolar, velar, ... etc.)
2 Manner of articulation (stops, fricatives, affricates, ... etc.)
3 Voicing (voiced vs. voiceless)
4 Aspiration (aspirated vs. unaspirated)
6 / 99

7. Speech sound Speech production Measuring speech References
Consonants
Last week we talked about consonants and their IPA symbols.
There are a few criteria/features to describe consonants:
1 Place of articulation (labial, alveolar, velar, ... etc.)
2 Manner of articulation (stops, fricatives, affricates, ... etc.)
3 Voicing (voiced vs. voiceless)
4 Aspiration (aspirated vs. unaspirated)
Note that Mandarin only has aspiration contrast not voicing
contrast.
⇒ ㄅ [p]; ㄆ [ph
]
7 / 99

8. Speech sound Speech production Measuring speech References
Consonants
Last week we talked about consonants and their IPA symbols.
There are a few criteria/features to describe consonants:
1 Place of articulation (labial, alveolar, velar, ... etc.)
2 Manner of articulation (stops, fricatives, affricates, ... etc.)
3 Voicing (voiced vs. voiceless)
4 Aspiration (aspirated vs. unaspirated)
Note that Mandarin only has aspiration contrast not voicing
contrast.
⇒ ㄅ [p]; ㄆ [ph
]
In comparison, Taiwan Southern Min contrasts both aspiration
and voicing.
⇒ 爸 [p]; 打[ph
]; 肉 [b]
8 / 99

9. Speech sound Speech production Measuring speech References
Vowels
What are the (N. American) English vowels?
9 / 99

10. Speech sound Speech production Measuring speech References
Vowels
What are the (N. American) English vowels?
[i, e, E, æ, a, @, u, U, o, 2, O, A]
10 / 99

11. Speech sound Speech production Measuring speech References
Vowels
What are the (N. American) English vowels?
[i, e, E, æ, a, @, u, U, o, 2, O, A]
What are vowels?
11 / 99

12. Speech sound Speech production Measuring speech References
Vowels
What are the (N. American) English vowels?
[i, e, E, æ, a, @, u, U, o, 2, O, A]
What are vowels?
Vowels: Sounds (with vocal folds vibrating) produced with at
most only a slight narrowing somewhere in the vocal tract,
allowing air to flow freely through the oral cavity.
12 / 99

13. Speech sound Speech production Measuring speech References
About vowels
13 / 99

14. Speech sound Speech production Measuring speech References
About vowels
Traditional view of vowels
1 Height
2 Backness
14 / 99

15. Speech sound Speech production Measuring speech References
Formants
Formants: Resonant frequency
that amplifies some groups of
harmonics above others.
Appears as a dark band on a
spectrogram.
Transitions into and out of the
vowels.
15 / 99

16. Speech sound Speech production Measuring speech References
Formants
Since 60s, linguists found a strong
correlation between the tongue
position and formants.
Different vowels are characterized by
different formant profiles.
(Figure adapted from Goldstein (2010), Fig. 13.2)
16 / 99

17. Speech sound Speech production Measuring speech References
Formants
Since 60s, linguists found a strong
correlation between the tongue
position and formants.
Different vowels are characterized by
different formant profiles.
(Figure adapted from Goldstein (2010), Fig. 13.2)
By changing its shape, the vocal
tract create distinct patterns of
resonating formant frequencies for
each vowel.
⇒ Source-filter theory
In a figure of spectral slice (frequency
on the x-axis and amplitude on the y-axis),
the first peak is defined as F1, the
second peak is termed F2, and so
on.
17 / 99

18. Speech sound Speech production Measuring speech References
Tongue and formants
18 / 99

19. Speech sound Speech production Measuring speech References
Tongue and formants
19 / 99

20. Speech sound Speech production Measuring speech References
Tongue and formants
20 / 99

21. Speech sound Speech production Measuring speech References
Tongue and formants
Classical formant-cavity affiliation:
F1 – back cavity
F2 – front cavity
21 / 99

22. Speech sound Speech production Measuring speech References
Tongue and formants
English vowels (high front [i], high
back [u], and low back [A]) from
MRI data: right midsagittal view
Tongue height and tongue advancement
Tongue height: High vowels (/i/, /u/) ⇒
tongue moves out of the pharyngeal area ⇒
increase air volume in the pharyngeal area
⇒ lowers F1
Tongue advancement: Back vowels (/A/,
/u/) ⇒ far back in the oral cavity ⇒
lengthens the anterior oral cavity ⇒
lowers F2
NOTE: Lip protrusion can also increase
the anterior oral cavity!
22 / 99

23. Speech sound Speech production Measuring speech References
About vowels
Now you understand why the IPA
vowel chart is arranged this way.
23 / 99

24. Speech sound Speech production Measuring speech References
About vowels
Now you understand why the IPA
vowel chart is arranged this way.
But, some of the details remained
unexplained.
I know [i], but what is [y]?
I know [u], [U], but what is [W]?
24 / 99

25. Speech sound Speech production Measuring speech References
About vowels
Now you understand why the IPA
vowel chart is arranged this way.
But, some of the details remained
unexplained.
I know [i], but what is [y]?
I know [u], [U], but what is [W]?
This vowel chart also encodes
another vowel quality:
roundedness
25 / 99

26. Speech sound Speech production Measuring speech References
About vowels
Roundedness
[i] and [y] contrast in roundedness,
so do [u] and [W].
⇒ [i] is [-round]
⇒ [y] is [+round]
26 / 99

27. Speech sound Speech production Measuring speech References
About vowels
Roundedness
[i] and [y] contrast in roundedness,
so do [u] and [W].
⇒ [i] is [-round]
⇒ [y] is [+round]
Try this: 鮨 (鮓; 壽司)
27 / 99

28. Speech sound Speech production Measuring speech References
IPA chart
iOS app:
https://apps.apple.com/us/app/ipa-phonetics/id869642260
28 / 99

29. Speech sound Speech production Measuring speech References
Larynx and phonations
Larynx is a complicated system.
By vibrating the vocal folds (VF), we
produce (voiced) sounds.
29 / 99

30. Speech sound Speech production Measuring speech References
Larynx and phonations
Larynx is a complicated system.
By vibrating the vocal folds (VF), we
produce (voiced) sounds.
Unsurprisingly, we can change the way
we vibrate our VF to make different
sound qualities (i.e., phonations).
Modal
Breathy
Creaky
Whisper
Falsetto
30 / 99

31. Speech sound Speech production Measuring speech References
Larynx and phonations
Modal
Characterized as most common form of
voicing, with (more or less) a balance of
closed and open vocal fold frequency.
31 / 99

32. Speech sound Speech production Measuring speech References
Larynx and phonations
Modal
Characterized as most common form of
voicing, with (more or less) a balance of
closed and open vocal fold frequency.
Breathy
Combines voicing with glottal friction.
Forms a continuum with modal voice.
32 / 99

33. Speech sound Speech production Measuring speech References
Larynx and phonations
Creaky
Lower F0
Quite efficient, used when tired
33 / 99

34. Speech sound Speech production Measuring speech References
Larynx and phonations
Creaky
Lower F0
Quite efficient, used when tired
Whisper
voiceless speech, sometimes with
high 4P
Very insufficient - run out of air
quickly
34 / 99

35. Speech sound Speech production Measuring speech References
Larynx and phonations
Falsetto
Makes the vocal fold very thin and
high-frequency
High medial compression and
longitudinal tension
35 / 99

36. Speech sound Speech production Measuring speech References
II.
Speech production
36 / 99

37. Speech sound Speech production Measuring speech References
Speech production
1 Definition of speech production
(in terms of motor control)?
37 / 99

38. Speech sound Speech production Measuring speech References
Speech production
1 Definition of speech production
(in terms of motor control)?
Wiki: “Speech production is the
process by which thoughts are
translated into speech. This includes
the selection of words, the
organization of relevant grammatical
forms, and then the articulation of
the resulting sounds by the motor
system using the vocal apparatus.”
38 / 99

39. Speech sound Speech production Measuring speech References
Speech production
1 Definition of speech production
(in terms of motor control)?
Wiki: “Speech production is the
process by which thoughts are
translated into speech. This includes
the selection of words, the
organization of relevant grammatical
forms, and then the articulation of
the resulting sounds by the motor
system using the vocal apparatus.”
39 / 99

40. Speech sound Speech production Measuring speech References
Speech chain
Thought (brain) → Linguistic representation (lexeme, morpheme,
phoneme, etc.) coding → Articulation (vocal tract) → Sounds
(acoustics) → Auditory perception → Linguistic representation
decoding → Understand meaning (brain).
40 / 99

41. Speech sound Speech production Measuring speech References
Speech chain
Thought (brain) → Linguistic representation (lexeme, morpheme,
phoneme, etc.) coding → Articulation (vocal tract) → Sounds
(acoustics) → Auditory perception → Linguistic representation
decoding → Understand meaning (brain).
41 / 99

42. Speech sound Speech production Measuring speech References
Speech chain
Thought (brain) → Linguistic representation (lexeme, morpheme,
phoneme, etc.) coding → Articulation (vocal tract) → Sounds
(acoustics) → Auditory perception → Linguistic representation
decoding → Understand meaning (brain).
NB: Uni-directional arrows! All rightward! ⇒ “Feedforward”
42 / 99

43. Speech sound Speech production Measuring speech References
Speech chain
Thought (brain) → Linguistic representation (lexeme, morpheme,
phoneme, etc.) coding → Articulation (vocal tract) → Sounds
(acoustics) → Auditory perception → Linguistic representation
decoding → Understand meaning (brain).
NB: Uni-directional arrows! All rightward! ⇒ “Feedforward”
Anything missing?
43 / 99

44. Speech sound Speech production Measuring speech References
Revised speech chain
44 / 99

45. Speech sound Speech production Measuring speech References
Revised speech chain
⇒ Multi-modal speech chain.
45 / 99

46. Speech sound Speech production Measuring speech References
Speech behaviours
Speech production requires a sequence of well-coordinated
articulatory movements. For example, to produce the word palm
/ph
am/, a speaker needs to
46 / 99

47. Speech sound Speech production Measuring speech References
Speech behaviours
Speech production requires a sequence of well-coordinated
articulatory movements. For example, to produce the word palm
/ph
am/, a speaker needs to
1 shut the mouth for a bilabial closure
47 / 99

48. Speech sound Speech production Measuring speech References
Speech behaviours
Speech production requires a sequence of well-coordinated
articulatory movements. For example, to produce the word palm
/ph
am/, a speaker needs to
1 shut the mouth for a bilabial closure
2 pump up intraoral pressure,
48 / 99

49. Speech sound Speech production Measuring speech References
Speech behaviours
Speech production requires a sequence of well-coordinated
articulatory movements. For example, to produce the word palm
/ph
am/, a speaker needs to
1 shut the mouth for a bilabial closure
2 pump up intraoral pressure,
3 open the glottis; burst open the lips
49 / 99

50. Speech sound Speech production Measuring speech References
Speech behaviours
Speech production requires a sequence of well-coordinated
articulatory movements. For example, to produce the word palm
/ph
am/, a speaker needs to
1 shut the mouth for a bilabial closure
2 pump up intraoral pressure,
3 open the glottis; burst open the lips
4 make sure the tongue is at its designated position during the
bilabial burst, in this case, low-back position
50 / 99

51. Speech sound Speech production Measuring speech References
Speech behaviours
Speech production requires a sequence of well-coordinated
articulatory movements. For example, to produce the word palm
/ph
am/, a speaker needs to
1 shut the mouth for a bilabial closure
2 pump up intraoral pressure,
3 open the glottis; burst open the lips
4 make sure the tongue is at its designated position during the
bilabial burst, in this case, low-back position
5 leave the velopharyngeal port open in order to make it nasal
51 / 99

52. Speech sound Speech production Measuring speech References
Speech behaviours
Speech production requires a sequence of well-coordinated
articulatory movements. For example, to produce the word palm
/ph
am/, a speaker needs to
1 shut the mouth for a bilabial closure
2 pump up intraoral pressure,
3 open the glottis; burst open the lips
4 make sure the tongue is at its designated position during the
bilabial burst, in this case, low-back position
5 leave the velopharyngeal port open in order to make it nasal
6 close the mouth again for the /m/ closure while the vocal folds
keep vibrating
52 / 99

53. Speech sound Speech production Measuring speech References
Speech behaviours
We can visualize this overlapped yet coordinated “gestures,”
53 / 99

54. Speech sound Speech production Measuring speech References
Speech behaviours
We can visualize this overlapped yet coordinated “gestures,”
⇒ This is the essence of “Articulatory Gestures.” (Browman and
Goldstein, 1989)
⇒ Different tiers of gesture scores
54 / 99

55. Speech sound Speech production Measuring speech References
Speech behaviours
Connected speech (usually) involves
contraction (i.e., some kind of
omission).
Note: You can’t have complete
contraction. (Think about boxing
combinations.)
55 / 99

56. Speech sound Speech production Measuring speech References
Let’s talk about “Saturday”!
Saturday has a very unique
sequence: [RõR]
56 / 99

57. Speech sound Speech production Measuring speech References
Let’s talk about “Saturday”!
Saturday has a very unique
sequence: [RõR]
In English, there are two ways to
produce rhotics: tongue tip-up [õ]
and tongue tip-down [ô]. (Derrick et
al., 2015)
57 / 99

58. Speech sound Speech production Measuring speech References
Let’s talk about “Saturday”!
Saturday has a very unique
sequence: [RõR]
In English, there are two ways to
produce rhotics: tongue tip-up [õ]
and tongue tip-down [ô]. (Derrick et
al., 2015)
There are four identifiable /R/ in
English. (Derrick et al., 2015)
58 / 99

59. Speech sound Speech production Measuring speech References
Let’s talk about “Saturday”!
Saturday has a very unique
sequence: [RõR]
In English, there are two ways to
produce rhotics: tongue tip-up [õ]
and tongue tip-down [ô]. (Derrick et
al., 2015)
There are four identifiable /R/ in
English. (Derrick et al., 2015)
1 alveolar tap
2 down-flap
3 up-flap
4 postalveolar tap
59 / 99

60. Speech sound Speech production Measuring speech References
Let’s talk about “Saturday”!
Phonetically, is the retroflex produced as [õ] or as [ô]?
60 / 99

61. Speech sound Speech production Measuring speech References
Let’s talk about “Saturday”!
Phonetically, is the retroflex produced as [õ] or as [ô]?
Phonetically, is this sequence produced as [R-
õR&
] or as [Rl
õRl
]?
61 / 99

62. Speech sound Speech production Measuring speech References
Let’s talk about “Saturday”!
Phonetically, is the retroflex produced as [õ] or as [ô]?
Phonetically, is this sequence produced as [R-
õR&
] or as [Rl
õRl
]?
Derrick et al. (2015) predict an up-down sequence: [R-
õR&
]
62 / 99

63. Speech sound Speech production Measuring speech References
Ultrasound set-up for ”Saturday”
63 / 99

64. Speech sound Speech production Measuring speech References
Let’s talk about ”Saturday”
The rhotics in “Saturday” is more likely to be produced as a
tongue tip-up retroflex [õ].
Most of the first flap (T variant) is [R-
] (191/213).
Of the 193 tokens of “Saturday” produced with [õ], fully 187
ended with [R&
].
64 / 99

65. Speech sound Speech production Measuring speech References
Speech pre-plans
Two possible approaches:
1 One holistic event (i.e., [R-
õR&
])
2 Two (or multiple) independent events (i.e., [Rl
õRl
])
65 / 99

66. Speech sound Speech production Measuring speech References
Speech pre-plans
Two possible approaches:
1 One holistic event (i.e., [R-
õR&
])
2 Two (or multiple) independent events (i.e., [Rl
õRl
])
⇒ Derrick et al. (2015) use ultrasound and 3D model simulation.
They concluded: [R-
õR&
] .
⇒ There IS pre-planning!
66 / 99

67. Speech sound Speech production Measuring speech References
III.
Measuring speech
67 / 99

68. Speech sound Speech production Measuring speech References
Measuring speech
Static
1 Palatography; linguography
Dynamic
1 Electroglottography
2 Ultrasound
68 / 99

69. Speech sound Speech production Measuring speech References
Palatography; linguography
Palatography
A way to record contact between the tongue and the roof the
mouth to get articulatory records for the production of speech
sounds.
Apply marker paint on the tongue, produce the word, and see
where ink rubbed off onto the roof of the mouth.
⇒ palatograms.
Since palatography requires contact, i.e. some kind of
obstruction, for the paint to rub off, palatography is most
suitable for looking at consonants.
69 / 99

70. Speech sound Speech production Measuring speech References
Palatography; linguography
An example of palatogram:
70 / 99

71. Speech sound Speech production Measuring speech References
Palatography; linguography
Palatography
Merits:
Easy and safe; suitable for field work.
(Relative) More comprehensive view of the lingual contact.
71 / 99

72. Speech sound Speech production Measuring speech References
Palatography; linguography
Palatography
Merits:
Easy and safe; suitable for field work.
(Relative) More comprehensive view of the lingual contact.
Downsides:
No timing information.
Careful speech; does not allow errors.
Not suitable for sounds with no lingual contact.
Only captures the production of a single sound, not a sequence of
sounds.
72 / 99

73. Speech sound Speech production Measuring speech References
Palatography; linguography
Linguography
Palatography applies marker paint on the tongue
and see where the ink stays on the palate after
the contact.
Linguography applies marker paint on the
palate and see where the ink stays on the surface
of the tongue.
Everything else is identical to palatography.
Palatography focuses on where in the palate the
contact was made whereas linguography focuses
on which part of the tongue is making the
contact.
73 / 99

74. Speech sound Speech production Measuring speech References
EGG
Electroglottography (EGG)
Non-invasive measurement of laryngeal
movements.
Easy set-up.
Measuring the resistance between the two
electrodes placed outside the thyroid
notch.
Very good temporal resolution.
Bad location information ⇒ Don’t know
where the opening of contact is or the
width of the opening
Raising or lower the larynx may cause
loss of data acquisition.
74 / 99

75. Speech sound Speech production Measuring speech References
General Impression
When talking about ultrasound, we
think about this:
75 / 99

76. Speech sound Speech production Measuring speech References
General Impression
When talking about ultrasound, we
think about this:
76 / 99

77. Speech sound Speech production Measuring speech References
General Impression
When talking about ultrasound, we
think about this:
And ultrasound images should look like
this:
77 / 99

78. Speech sound Speech production Measuring speech References
General Impression
When talking about ultrasound, we
think about this:
And ultrasound images should look like
this:
78 / 99

79. Speech sound Speech production Measuring speech References
General Impression
When talking about ultrasound, we
think about this:
And ultrasound images should look like
this:
79 / 99

80. Speech sound Speech production Measuring speech References
Basics about ultrasound
The transducer produces high-frequency
sound (ultra-high, in fact!) ranging from
3 ∼ 16 MHz. (We human can only hear 20 ∼ 20
KHz sounds!)
80 / 99

81. Speech sound Speech production Measuring speech References
Basics about ultrasound
The transducer produces high-frequency
sound (ultra-high, in fact!) ranging from
3 ∼ 16 MHz. (We human can only hear 20 ∼ 20
KHz sounds!)
Some of these sound waves penetrate
skin, fat, and muscle (parts of the body that
hold a lot of water), while some reflect back
to the transducer.
81 / 99

82. Speech sound Speech production Measuring speech References
Basics about ultrasound
The transducer produces high-frequency
sound (ultra-high, in fact!) ranging from
3 ∼ 16 MHz. (We human can only hear 20 ∼ 20
KHz sounds!)
Some of these sound waves penetrate
skin, fat, and muscle (parts of the body that
hold a lot of water), while some reflect back
to the transducer.
The sound waves get absorbed by bone
and reflect sharply off of air boundaries,
so that ultrasound doesn’t image bone or
air very well.
82 / 99

83. Speech sound Speech production Measuring speech References
Basics about ultrasound
Compared to X-ray and other imaging
techniques, ultrasound machines don’t
produce ionizing radiation.
Ultrasound provides immediate images
(wonderful temporal resolution) across large
areas (fairly good spatial resolution).
Very easy to use; can be portable
⇒ Great for tongue imaging!
83 / 99

84. Speech sound Speech production Measuring speech References
B-mode; B/M-mode
B-mode
Bi-dimenstional:
84 / 99

85. Speech sound Speech production Measuring speech References
B-mode; B/M-mode
B-mode
Bi-dimenstional:
For tongue, phoneticians care
about:
85 / 99

86. Speech sound Speech production Measuring speech References
B-mode; B/M-mode
B-mode
Bi-dimenstional:
For tongue, phoneticians care
about:
- height
- backness (frontness)
86 / 99

87. Speech sound Speech production Measuring speech References
B-mode; B/M-mode
B-mode
Bi-dimenstional:
For tongue, phoneticians care
about:
- height
- backness (frontness)
B/M-mode
B-mode plus movement on ONE
dimension, along the anatomical
line
87 / 99

88. Speech sound Speech production Measuring speech References
Data
Aligning ultrasound images with sounds.
88 / 99

89. Speech sound Speech production Measuring speech References
Data processing
1 Grab still images
89 / 99

90. Speech sound Speech production Measuring speech References
Data processing
1 Grab still images 2 Tongue surface tracing
90 / 99

91. Speech sound Speech production Measuring speech References
Data processing
3 Multiple tracing
91 / 99

92. Speech sound Speech production Measuring speech References
Data processing
4 Fitting (Smoothing spline analysis of variance; SS ANOVA)
92 / 99

93. Speech sound Speech production Measuring speech References
Data processing
4 Fitting (Generalized Additive Mixed Model; GAMM)
Figure adapted from Chiu & Sun (2020)
93 / 99

94. Speech sound Speech production Measuring speech References
Data processing
4 Fitting (Generalized Additive Mixed Model; GAMM)
Figure adapted from Chiu & Sun (2020)
94 / 99

95. Speech sound Speech production Measuring speech References
Challenges
Transducer stabilization problem:
95 / 99

96. Speech sound Speech production Measuring speech References
Challenges
Transducer stabilization problem:
96 / 99

97. Speech sound Speech production Measuring speech References
Challenges
Transducer stabilization problem:
Frame rate and resolution tradeoff.
97 / 99

98. Speech sound Speech production Measuring speech References
Challenges
Transducer stabilization problem:
Frame rate and resolution tradeoff.
Some of the images may be grainy.
98 / 99

99. Speech sound Speech production Measuring speech References
References
Chiu, C., & Sun, J. T.-S. (2020). On pharyngealized vowels in
Northern Horpa: An acoustic and ultrasound study. Journal of the
Acoustical Society of America, 147(4), 2928–2946. doi:
10.1121/10.0001005
Derrick, D., Stavness, I., & Gick, B. (2015). Three speech rounds, one
motor action: Evidence for speech-motor disparity from English
flap production. Journal of the Acoustical Society of America,
137(3), 1493 - 1502. doi: 10.1121/1.4906831
99 / 99