Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Current Dev


Published on

Published in: Technology, Health & Medicine
  • Be the first to comment

  • Be the first to like this

Current Dev

  1. 1. Current developments in phonetics Louis C.W. Pols Institute of Phonetic Sciences (IFA) Amsterdam Center for Language and Communication (ACLC)
  2. 2. Overview <ul><li>My job: Aspects of Phonetics </li></ul><ul><li>Size of Phonetics Community </li></ul><ul><li>Ken Stevens and Phonetics </li></ul><ul><li>My choices of topics </li></ul><ul><li>Conclusions </li></ul>
  3. 3. Aspects of Phonetics <ul><li>Phonetics is fantastic and interdisciplinary </li></ul><ul><li>it is about the speech signal and much more: </li></ul><ul><li>spoken language, spoken communication </li></ul><ul><li>phonemes and prosody </li></ul><ul><li>speaking and listening </li></ul><ul><li>mental storage and retrieval </li></ul><ul><li>speech acquisition and speech pathology </li></ul><ul><li>speech technology, speech databases </li></ul><ul><li>languages of the world, dialects </li></ul><ul><li>and more: e.g. laboratory phonology, evaluating cochlear implants, or designing Web avatars </li></ul>
  4. 4. My choices , given other intro’s <ul><li>Phonetics as a basic science </li></ul><ul><li>Computational modeling, computational phonetics </li></ul><ul><li>Knowledge from annotated, and preferable freely accessible, speech corpora </li></ul><ul><li>Phonetics as an interdisciplinary science </li></ul>
  5. 5. Size of Phonetics Community <ul><li>1000+ participants to speech conferences like: ICSLP & Eurospeech (now Interspeech under ISCA), ICASSP, LREC, ICPhS (under IPA) </li></ul><ul><li>numerous workshops (see ISCA and FoNETiks newsletters, and News section in SpeCom) </li></ul><ul><li>IPA ~1000 members, ISCA ~1350 members </li></ul><ul><li>phonetics community at least 10 times bigger </li></ul><ul><li>books; journals; LDC & ELRA </li></ul><ul><li>ICPhS’03 Barcelona: 50 countries (USA 158; FR 81; GE 73; UK 71; JAP 46; SP 45, SW 41; NE 31; CAN 25; RU 19; IT 17; FI 14; AU 12; BRA 12, CH 12) </li></ul><ul><li>Ba-Ma system: less specialization in Phonetics </li></ul>
  6. 6. Ken Stevens and Phonetics <ul><li>ESCA medal at Eurospeech’95 in Madrid </li></ul><ul><li>on average one paper per year in JASA </li></ul><ul><li>special issue JPhon “Quantal Theory” (1989) </li></ul><ul><li>1998 master piece “Acoustic Phonetics” </li></ul><ul><li>regular keynote speaker at conferences </li></ul><ul><li>many international contacts (also Europe) </li></ul><ul><li>many good students world-wide </li></ul>
  7. 7. Banquet Eurospeech95, Madrid E’95 chairman ESCA-medalist ESCA president J.M. Pardo Ken Stevens Mrs. Pardo Louis Pols
  8. 8. Textbook Phonetics <ul><li>Summer course in English Phonetics (UCL): </li></ul><ul><ul><li>phonemic systems (vowels and consonants) </li></ul></ul><ul><ul><li>segmental analysis (allophonic processes) </li></ul></ul><ul><ul><li>word stress </li></ul></ul><ul><ul><li>weakening and coarticulation processes </li></ul></ul><ul><ul><li>sentence stress (accent, tonal stress) </li></ul></ul><ul><ul><li>intonation and meaning </li></ul></ul><ul><li>similar in most textbooks </li></ul>
  9. 9. Invariance Symp., MIT 1983 <ul><li>Invariance and variability in speech processes (Perkell & Klatt, 1986) </li></ul><ul><li>also Leitmotiv for my Amsterdam group </li></ul><ul><li>perception of dynamic speechlike sounds (vW) </li></ul><ul><li>formant dynamics (van Son) </li></ul><ul><li>appropriate context (van Son) </li></ul><ul><li>acoustic vowel reduction (van Bergem) </li></ul><ul><li>efficiency of speech (van Son) </li></ul>
  10. 10. DL for short speech-like transitions Adopted from van Wieringen & Pols (1998), Acta Acustica 84, 520-528 “ Discrimination of short and rapid speechlike transitions” complex simple short longer trans. initial final
  11. 11. Static vs. dynamic V recogn. <ul><li>see Weenink (2001) </li></ul><ul><ul><li>“ Vowel normalizations with the TIMIT acoustic phonetic speech corpus”, IFA Proc. 24, 117-123 </li></ul></ul><ul><li>438 males, both train & test sentences TIMIT </li></ul><ul><li>35,385 vowel segments, hand segmented </li></ul><ul><li>13 monophthongeal vowel categories </li></ul><ul><li>1-Bark bandfilter anal. (18), intensity normal. </li></ul><ul><li>3 frames per segment: central and 25 ms L/R </li></ul>
  12. 12. Some results <ul><li>Vowel classif. (%) with discriminant functions </li></ul>94.5 87.9 5,374 speaker normalized 90.1 78.9 5,374 438x13 V centers per speaker 69.2 62.2 35,385 speaker normalized 66.9 59.3 35,385 438x13x(1…25) Original Dynamic 3 frames Static 1 frame # Items Condition
  13. 13. Perceiving (speech) dynamics <ul><li>vowel perception w/w or w/o transitions? </li></ul><ul><li>our claims (vSon, IFA proc. 17(1993): </li></ul><ul><ul><li>only evidence for compensatory processes (i.e. perceptual-overshoot and dynamic-specification), when in an appropriate context </li></ul></ul><ul><ul><li>synthetic isolated dynamic formant tracks lead to perceptual undershoot (=averaging) </li></ul></ul><ul><ul><li>silent center studies are ambiguous </li></ul></ul><ul><li>concl.: info in formant dynamics is only used when V’s are heard in appropriate context </li></ul>
  14. 15. Vowel identification <ul><li>compare V responses for dynamic stimuli with those for static stimuli </li></ul><ul><li>calculate net shift in V responses per onglide (CV), complete (CVC), or offglide (VC) </li></ul><ul><li>result: responses average over the trailing part of the formant track </li></ul><ul><li>see Pols & vSon, “Acoustics and perception of dynamic vowel segments”, Speech Comm. </li></ul>
  15. 16. Perceptual undershoot Net shift in vowel responses to tokens with curved formant tracks vs. stationary tokens. All values significant, except small open triangles
  16. 17. Local context and C & V identification <ul><li>120 CVC fragments taken from a read text </li></ul><ul><li>various segments per CVC-fragment </li></ul><ul><li>(50ms V-kernel and beyond) </li></ul><ul><li>both accented and unaccented vowels </li></ul><ul><li>subjects identified (pre- or post-vocalic) consonant or vowel in CV-, VC-, or CVC-segments </li></ul><ul><li>vSon & Pols (1999), “Perisegmental speech improves consonant and vowel identification”, Speech Comm. 29, 1-22 </li></ul>
  17. 19. Error rates of vowel identification for the individual stimulus token types. Long-short vowel errors (/ α-a:, -o:/) are ignored c
  18. 20. <ul><li>results: </li></ul><ul><li>phoneme identification benefits from extra speech </li></ul><ul><li>left context more beneficial than right context </li></ul><ul><li>better identification in CV when also other member of pair was identified correctly (context effect) </li></ul>
  19. 21. Effect of (lack of) context <ul><li>100 Dutch listeners identifying V segments </li></ul><ul><ul><li>“ Vowel contrast reduction”, K-vBeinum (1980) </li></ul></ul>ASC = 1/n Σ |LF i - LF i | 2 (total variance), LF i = 100 10 log F i i=1 n 33.0 189 38.9 255 33.3 209 28.7 119 31.2 174 unstr., free conv. % (10) ASC 84.3 407 85.3 529 84.9 374 78.8 320 88.1 406 words % (5) ASC 89.6 480 86.4 634 88.0 447 88.9 404 95.2 433 isolated V % (3) ASC Av. F2 F1 M2 M1 3 conditions
  20. 22. Historical biases <ul><li>R. Plomp (2002) “The intelligent ear. On the nature of sound perception” </li></ul><ul><li>biases in research: </li></ul><ul><ul><li>dominance for simple stimuli (e.g., phonemes) </li></ul></ul><ul><ul><li>preference for microscopic approach (e.g., phoneme discrimination rather than intelligibility) </li></ul></ul><ul><ul><li>emphasis on psychophysical rather than cognitive aspects of hearing </li></ul></ul><ul><ul><li>use of clean signals in lab (rather than acoustic reality of outside world with its disruptive sounds) </li></ul></ul>
  21. 23. Computational Phonetics <ul><li>R. Moore (1995) 13th ICPhS, Stockholm </li></ul><ul><li>unify the emerging theoretical and practical developments in speech technology with the established knowledge and practices in phonetic sciences </li></ul><ul><li>Sagisaka et al. (1997), “Computing prosody. Computational models for processing spontaneous speech” </li></ul><ul><li>Klatt (1987), vSanten (1997), Wang (1997), duration modeling </li></ul><ul><li>vBergem (1993), Acoustic and lexical vowel reduction </li></ul><ul><li>Steeneken (1992), Speech Transmission Index </li></ul>
  22. 24. Stylized formant contour c 2 c 1 F 2 (t) = c 0 + c 1 t + c 2 t 2 (second order polynomial) F 2 (t) = F 2 (t) + α 2 p (t) + β 2 t (t) + γ 2 α (t) for @ in /p@t α/ F 2 (-1) = 1352 Hz ; F 2 (0) = 1435 Hz; F 2 (1)=1485 Hz F 2 normalized time -1 F center (c 0 ) F offset 0 1 F onset
  23. 25. Schwa realization The schwa is not just a centralized vowel but something that is completely assimilated with its phonemic context
  24. 26. Human word intelligibility vs. noise from Ph.D thesis H. Steeneken (1992) ‘ On measuring and predicting speech intelligibility’
  25. 27. Knowledge from Annotated Sp. Corp. <ul><li>knowledge casted in rules vs. knowledge derived from intelligent searches in DB ’ s </li></ul><ul><li>vSanten (1997) greedy algorithm </li></ul><ul><li>Greenberg et al. (2003) Switchboard </li></ul><ul><li>Oostdijk et al. (2002) 1000 hrs.- 10M words spoken Dutch corpus ( CGN ) </li></ul><ul><li>vSon et al. (2001) 5.5 hrs. IFA corpus </li></ul><ul><li>Intas915 project (Dutch, Finnish, Russian) </li></ul>
  26. 28. Freq. effects vs. vowel reduction Dutch Finnish Russian -0,100 -0,050 0,000 0,050 0,100 0,150 0,200 0,250 0,300 Duration F12Dist CoG Intensity Correlation Coefficient -> R read speech spontaneous speech -log 2 (word frequency) vs. acoustic vowel reduction (in terms of duration, F1F2Dist, CoG, and Intensity) for Du, Fi, Ru Dutch Finnish Russian 0,000 0,050 0,100 0,150 0,200 0,250 0,300 Duration F12Dist CoG Intensity Correlation Coefficient -> R
  27. 29. Phonetics an Interdisciplinary Science <ul><li>some examples </li></ul><ul><ul><li>phonetics is a contributor to many signal and data processing techniques as well as pattern recognition techniques </li></ul></ul><ul><ul><li>use of source-filter model to describe early speech development </li></ul></ul><ul><ul><li>laryngectomized speech, production and evaluation </li></ul></ul><ul><ul><li>turn switches in conversational dialogs </li></ul></ul><ul><ul><li>progress in vowel production in babies </li></ul></ul>
  28. 30. Early speech development vBeinum, Clement, vdDikkenberg, Developmental Sc. 4, 61-70 (2001) average onset (in weeks) Stage I Stage II Stage III Stage IV Stage V (babbling) Stage VI (‘words’) 0 6 10 20 31 40
  29. 31. Tracheoesophageal speech C. van As, Ph.D thesis (2001)
  30. 32. Turn switches in conversation <ul><li>shift in phonetics from isolated stimuli to conversational speech </li></ul><ul><li>quantitative modelling of the identification of turn-relevent places (TRP’s) </li></ul><ul><li>integration process of temporally unfolding information at different levels in speech, from conversation acts and semantics to prosody, phonetics and visual cues </li></ul><ul><li>use of laryngograph to detect preparatory glottal closure that precedes most TRP’s </li></ul><ul><li>new project Rob van Son (start Jan. 2004) </li></ul>
  31. 33. Progression in V production of babies <ul><li>especially in the first year of life </li></ul><ul><li>utterances difficult to identify as phon. seq. </li></ul><ul><li>spectro-temporal analyses difficult because of very high pitch </li></ul><ul><li>formant measurements biased by expectations </li></ul><ul><li>pitch-related bandfilter analysis (automatic) </li></ul><ul><li>5 normal-hearing and 5 hearing-impaired </li></ul><ul><li>vdStelt et al. (2003) </li></ul>
  32. 34. Spectral measurements normal hearing child 5 & 24 mo. hearing-impaired child 5 & 24 mo. i u a
  33. 35. Conclusions <ul><li>importance of dynamic information </li></ul><ul><li>implications of (lack of) (local) context </li></ul><ul><li>interdisciplinary nature of phonetics </li></ul><ul><li>need for large, annotated, and freely accessible speech corpora </li></ul><ul><li>generalization via computational phonetics </li></ul><ul><li>phonetics and phonology (Patricia Keating) </li></ul>