Word Recognition Models

10,717 views

Published on

Overview of Coltheart's Dual-Route Model and Seidenberg & McClelland's neural network models of word recognition.

Course presentation for PSYC365*, Fall 2004, Dr. Butler, Queen's University.

Images used without permission.

Published in: Technology, Education
1 Comment
1 Like
Statistics
Notes
  • hello, i need a little help please. This is my essay question...however I am a little confused about the main points i need to discuss.. can someone help me please? 'Briefly describe the main forms of acquired dyslexia. With respect to relevant
    theoretical models of dyslexia, assess critically the efficacy of using different
    types of word pools to differentiate between forms of acquired dyslexia'... thank you.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
10,717
On SlideShare
0
From Embeds
0
Number of Embeds
102
Actions
Shares
0
Downloads
148
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Word Recognition Models

  1. 1. Word Recognition Models Lucas Rizoli Thursday, September 30 PSYC 365*, Fall 2004 Queen’s University, Kingston
  2. 2. Human Word Recognition ● Text interpreted as it is perceived – Stroop test (Red, Green, Yellow) – Aware of results, not of processes ● Likely involves many areas of brain – Visual – Semantic – Phonological – Articulatory ● How can we model this?
  3. 3. Creating a Word Recognition Model ● Assumptions – Working in English – Only monosyllabic words ● FOX, CAVE, FEIGN... – Concerned only with simple word recognition ● Symbols → sounds ● Visual, articulatory systems function independently ● Context of word is irrelevant
  4. 4. Creating a Word Recognition Model ● Rules by which to recognize CAVE – C → /k/ – A → /A/ – VE → /v/ ● Describe grapheme-phoneme correspondences (GPC) – Grapheme → phoneme
  5. 5. Creating a Word Recognition Model ● Recognize HAVE – H → /h/ – A → /A/ – VE → /v/ – So HAVE → /hAv/ ? ● Rules result in incorrect pronunciation
  6. 6. Creating a Word Recognition Model ● English is quasi-regular – Can be described as systematic, but with exceptions – English has a deep orthography ● grapheme → phoneme rules inconsistent – GAVE, CAVE, SHAVE end with /Av/ – HAVE ends with /@v/
  7. 7. Creating a Word Recognition Model ● Models needs to recognize irregular words ● Check for irregular words before applying GPCs – List irregular words and their pronunciations ● HAVE → /h@v/, GONE → /gon/, ... – Have separate look-up process
  8. 8. Our Word Recognition Model From Visual System Orthographic Input Irregular GPCs Words Phonological Output To Articulatory System
  9. 9. The Dual-Route Model ● Proposed by Max Coltheart in 1978 – Supported by Pinker, Besner – Revised throughout the 80’s, 90’s, 00’s ● Context sensitive rules ● Rule frequency checks ● Lots of other complex stuff ● We’ll follow his 1993 model (DR93)
  10. 10. DR93 Examples Note: Above, /a/ should be /@/ Context-sensitive GPC
  11. 11. What’s Good About DR93 ● Regular word pronunciation – Goes well with rule-based theories ● Berko’s Wug test (This is a wug, these are two wug_) ● Childhood over-regularization ● Nonword pronunciation – NUST, FAIJE, NARF are alright
  12. 12. What’s Not Good About DR93 ● Irregular word pronunciation – GONE → /dOn/, ARE → /Ar/ ● GPCs miss subregularities – OW → /aW/, from HOW, COW, PLOW – SHOW, ROW, KNOW are exceptions ● Biological plausibility – Do humans need explicit rules in order to read?
  13. 13. The SM89 Model ● Implemented by Seidenberg and McClelland in 1989 – Response to dual-route model – Neural network/PDP model – “As little as possible of the solution built in” – “As much as possible is left to the mechanisms of learning” ● We’ll call it SM89
  14. 14. The SM89 Model Hidden Units (200 units) Orthographic Units Phonological Units (400 units) (460 units) From Visual System To Articulatory System
  15. 15. The SM89 Model ● Orthographic units are triples – Three characters – Letters or word-border Orthographic Units – CAVE (400 units) ● _CA, CAV, AVE, VE_ – Context-sensitive
  16. 16. The SM89 Model Hidden Units (200 units) ● Hidden units needed for complete neural network ● Encode information in a non-specified way ● Learning occurs by changing weights on connections to and from hidden units – Process of back-propagation
  17. 17. The SM89 Model ● Phonological units are also triples – /kAv/ ● _kA, kAv, Av_ ● Triples are generalized Phonological Units (460 units) ● [stop, vowel, fricative] ● Number of units are sufficient for English monosyllables
  18. 18. How SM89 Learns ● Orthographic units artificially stimulated ● Activation spreads to hidden, phonological units – Feedforward from ortho. to phono. units ● Model response is pattern of activation in phonological units
  19. 19. How SM89 Learns ● Difference in activation between response and the correct activation ● Error computed as the sum of difference for each unit, squared ● Weights of all connections between units adjusted
  20. 20. How SM89 Learns ● Simply, it learns to pronounce words properly – Don’t worry about the equations
  21. 21. How SM89 Learns ● Trained using a list of ~ 3000 English monosyllabic words – Includes homographs (WIND, READ) and irregulars ● Each training session called an epoch ● Words appeared somewhat proportionately to their frequency in written language
  22. 22. Practical Limits on SM89’s Training ● Activation calculated in a single step – Impossible to record how long it took to respond – Correlated error scores with latency ● Error → time ● Frequency of words was compressed – Would’ve required ~ 34 times more epochs – Saved computer time
  23. 23. How SM89 Performed
  24. 24. How SM89 Performed Human SM89
  25. 25. What’s Good About SM89 ● Regular word pronunciation ● Irregular word pronunciation ● Similar results to human studies – Word naming latencies – Priming effects ● Behaviour the result of learning – Ability increases in human fashion
  26. 26. What’s Not Good About SM89 ● Nonword pronunciation – Significantly worse than skilled readers – JINJE, FAIJE, TUNCE pronounced strangely ● Design was awkward – Triples – Feedforward network – Compressed word frequencies – Single-step computation
  27. 27. The SM94 Model ● Seidenberg, Plaut, and McClelland revise SM89 in 1994 – Response to criticism of SM89’s poor nonword performance ● We’ll call this model SM94 ● Compared humans’ nonword responses with model’s responses
  28. 28. The SM94 Model Hidden Units (100 units) Graphemic Units Phonological Units (108 units) (50 units) From Visual System To Articulatory System
  29. 29. How SM94 Differs From SM89 ● Feedback loops for hidden and phonemic units ● Weights adjusted using cross-entropy method – Complicated math, results in better learning ● Not computed in a single step ● No more triples – Graphemes for word input – Phonemes for word output – Input based on syllable structure
  30. 30. Examples of SM94’s Units
  31. 31. Nonwords ● May be similar to regular words – SMURF ← TURF ● In many cases there are many responses – BREAT ● ← EAT ? ● ← GREAT ? ● ← YEAH ?
  32. 32. Nonwords Human
  33. 33. How SM94 and DR93 Performed Note: Above, PDP is SM94; Rules is DR93
  34. 34. Comparing SM94 and DR93 ● Both perform well with list of ~ 3000 words – SM94 responds 99.7% correctly, DR93 78% ● Both do well with nonwords – SM89’s weakness caused by design issues ● SM94 avoids such issues – Neural networks equally capable for nonwords
  35. 35. Comparing SM94 and DR93 ● SM94 is a good performer – Regular, irregular words – Behaviour similar to human ● Latency effects ● Nonword pronunciation ● DR93 still has problems – Trouble with irregular words – More likely to regularize words
  36. 36. Models and Dyslexia ● Consider specific types of dyslexia – Phonological Dyslexia ● Trouble pronouncing nonwords – Surface Dyslexia ● Trouble with irregular words – Developmental Dyslexia ● Inability to read at age-appropriate level ● How can word recognition models account for dyslexic behaviour?
  37. 37. DR93 and Dyslexia ● Phonological dyslexia as damage to GPC route – Cannot compile sounds from graphemes – Relies on look-up ● Surface dyslexia as damage to look-up route – Cannot remember irregular words – Relies on GPCs ● Developmental dyslexia – Problems somewhere along either route ● Cannot form GPCs, slow look-up, for example
  38. 38. SM89 and Dyslexia ● Developmental dyslexia as damaged or missing hidden units 200 Hidden Units 100 Hidden Units
  39. 39. The 1996 Models and Dyslexia ● Plaut, McClelland, Seidenberg, and Patterson study networks and dyslexia (1996) – Variations of the SM89/SM94 models ● Feedforward ● Feedforward with actual word-frequencies ● Feedback with attractors ● Feedback with attractors and semantic processes – Compare each to case studies of dyslexics
  40. 40. Feedforward and Dyslexia Case- Studies
  41. 41. Feedback, with Attractors and Semantics, and Dyslexia Case-Studies
  42. 42. The 1996 Models and Dyslexia ● Most complex damage caused closest results – Not as simple as removing hidden units ● Severing semantics ● Distorting attractors ● Results are encouraging
  43. 43. Questions or Comments

×