Language and speech processing

1. Basic Issues in Language and
Speech Processing
Chapter 9
Lawrence Heller, Psy.D.

2. LINGUISTICS
• Linguistics (n.) The scientific study of language;
also called linguistic science. (David
Crystal:2008)

3. Psycholinguistics
• The study of the processes involved in using
language
• Psycholinguists are interested in exactly how
we execute our language abilities known as
linguistic performance
• The relatively pure knowledge of language
and its rules is called linguistic competence.

4. Study of human language

5. Anatomy of Language
• Brain
– Size
– Laterality
– Wernicke’s area
– Broca’s area
– Motor cortex
Motor cortex

6. What is language?
• Language is defined as a set of symbols (i.e.,
words) and rules or principles for the
combination (i.e., grammar) of those symbols
that allow for communication and
comprehension

7. Mental Lexicon
• We use words to communicate with a wide
variety of thousands of words to describe our
thoughts, feelings, and anything else we want
to communicate. The mental lexicon is all of
the words a person knows. This can also be
described as a mental library.

8. Semantic Memory
• The mental library is a significant dimension of
Semantic Memory.
• Each representation of the mental lexicon
contains more than just the meaning of a
word but the sound of each word, its written
form, and the roles it can take on in a
sentence. (e.g., noun, verb, etc.)
• Grammar- the rules that govern how words
may be combined.

9. Other Communication
• Human:
– Direct
• Body language (kinesics), tone of voice, personal space
(proxemics), gesture
– Indirect
• Writing, mathematics, music, painting, signs

10. Design Features of Language
• Language is distinguished by a number of design
features, which include:
• Semanticity- the symbols of language reflect
real-world objects, events, and ideas; they have
meaning
• The symbols of language exhibit arbitrariness, in
that they in no way represent the concepts to
which they refer.
• For example, there’s no reason why C-A-T should
denote the four-legged furry friends that says
meow.

11. Design Features of Language
• Language has the power to transport us in time
with the use of tenses, using combinations of
words about he future or past. Language allows
for Displacement in time.
• Prevarication- Language allows for deception; we
can lie
• Reflectiveness-we can use language to talk about
language
• Productivity-we can generate infinite
combinations from finite symbols

12. Structure of Language
• Phonology (sounds)
• Morphology (words)
• Syntax (sentence structure)
• Semantics (meaning)
• Pragmatics or grammar (rules)

13. Structure of Language - Phonology
• The study of sounds of a language.
• No human language uses all the sounds
humans can make.
• When communicating first, sounds must be
perceived.
• Second, they must activate representations of
the corresponding concepts in semantic
memory (object recognition).

14. Phonology
• Bottom up processing refers to the
identification of a pattern based on the
component data
• Speech perception involves more than just the
compilation of speech data.
• Speech signals are unclear or imperfect
• We need to rely on surrounding information
(context) or previous knowledge (i.e. top-
down processing) to make the signal clear.

15. Phones
• A phone is an acoustically distinct sound.
• For example, the /o/ in ‘boat’ as spoken by
a person from Minnesota might be physically
different from how a New Yorker might say it.
While the two sounds are phonetically
different this does not change the meaning of
the word.

16. Phonemes
• A phoneme is a category of speech sounds
that change the meaning of a word.
• The phonemes /b/ and /p/ are
phonemically different from each other and
yield quite different concepts when combined
with the segment -ig

17. Phonemes
• Vowel Phonemes involve a continuous flow of air
through the vocal tract and are a product of positions
of the tongue.
• Consonant phonemes involve some kind of obstruction
of the airflow in the vocal tract and are produced by
varying three dimensions:
– 1) Place of articulation-part of the vocal tract used to
make the sound
– 2) Manner of articulation-refers to how exactly the airflow
is obstructed
– 3) Voicing-refers to what the vocal cords do when the
airflow disruption stops

18. Phonemes
• Coarticulation- the nature of phoneme
transmission varies depending on neighboring
phonemes (what they are next to). Phonemes
are to some extent articulated together.
– For example, the /a/ in CAT sounds different than
the /a/ in BAD because is it surrounded by
different phonemes.

19. Phoneme Boundaries
• The invariance in perceiving phonemes in
spite of their different acoustical properties is
the result of categorical perception.
• Categorical perception is our tendency to not
discriminate between subtle shadings in the
way a particular phoneme sound.

20. Phoneme Boundaries
Have you ever wondered why it is hard to
understand non-native speakers of English?
• The reason is related to categorical perception-
their pronunciation of English phonemes is
sometimes phonetically different enough that it
doesn’t fit with our English phoneme categories.
• We only understand and produce the phonemes
of our native language

21. Structure of Language - Morphology
• Morphemes are the smallest units of language that
carry meaning.
• It may refer to a single word or to a prefix or suffix
that changes the precise meaning of the word
– Words (dog, cat) = free morphemes (stand alone)
– Prefixes (un-, sub-)
– Syllables (-s, -ly )
= bound morphemes

22. Producing (or “Morphing” the Spoken
Word)
• Transformation of words may rely on two
separate systems
– One for regular forms and one for irregular forms
• Around age 3 or so, children begin to make
mistakes termed over-regularizations. This is
where the child extends a grammatical rule too
far, treating an irregular form as a regular one.
– For instance, adding “ed” to “go” to come up with the
past tense “goed.”

23. Producing (or “Morphing” the Spoken
Word)
• As the child gets older, they make mistakes
they have ever made before but gradually are
able to tell when the rule applies and when it
doesn’t.

24. Perceiving the Spoken Word
• According to the Cohort Model of word
recognition:
• Spoken words are recognized by activating the
entire set of possible words based on the
word’s initial sound. With a subsequent
narrowing of the candidate set as more of the
word is perceived.

25. TRACE Model
• One of the main alternatives to the cohort
model is the TRACE model
• This model is similar to the cohort model in
that it involves a process of evidence
accumulation.

26. TRACE Model
• The TRACE model does a better job of
accounting for Top-Down Processing (previous
knowledge can help you identify a spoken
word)
– For Example, “Greg heard the meowing, and knew
it was time to get out of bed to feed the ______

27. TRACE Model
• The TRACE model assumes an interaction
between multiple levels of word
representation and better accounts for the
effects of context in word recognition, which is
affected by prosody.
• Prosody- the rhythm, stress, and intonation of
speech.

28. Say it with (opposite) feeling
• For example, you might express how upset
you have been over a difficult situation at
work and how you have no time to study for
your upcoming exam. You can say it with a
cheery disposition and big smile.

29. • How Language Shapes the Way We think

30. Bottom Up Factors
• A variety of factors for facilitating the
determination of word boundaries
• Phonotactic Knowledge
– Our sensitivity to the rules that govern phonemes
(i.e., sound) combinations in various languages.
– For example, the phonemes /t/ and /zh/ are never
combined in English.
– For example, /h/ frequenty starts a word but
never ends one.

31. Bottom Up Factors
• Metrical Segmentation
– Segmentation of words is dependent on the
phonology of a particular language.
– For instance, in English the important (content)
words in a sentence are more likely to start with a
strong syllable (one that contains a short or long
vowel sound and end with a weak syllable. (bacon,
candor)

32. Top-Down Factors
• Have you ever heard someone speak in a
foreign language and it sounds like they are
speaking a mile a minute.
• This is kind of an illusion, caused by lack of
familiarity with the language.
• Word boundaries are blurred, when people
are speaking naturally in conversations.

33. Mental Lexicon
• Word Frequency
• High frequency and non-ambiguous words
(e.g., house) are more easily accessed than
low-frequency and ambiguous words (e.g.,
bungalow).
• When ambiguous words are encountered

34. Mental Lexicon
• Lexical Ambiguity
– When a word with two possible meanings (e.g.,
bank) is encountered.
– Both meanings are temporarily activated
– The context in which the word is used leads to
disambiguation

35. Bilingual Individuals
• People that can speak two language seem to
have a common semantic memory store
underlying the two lexicons.

36. SYNTAX AND SEMANTICS
• Broca’s Aphasia
– Frontal lobe damage
– Speech is telegraphic and incorrectly structured
– Use of content words (i.e., nouns and verbs) is less
affected.

37. SYNTAX AND SEMANTICS
• Wernicke’s Aphasia
– Associated with Temporal Lobe damage
– Breakdown of the semantic aspects of language
• A person with this type of aphasia may speak with
intact sentence structures, but distorted choice of
content words
• Brocas and Wernicke’s aphasia

38. Structure of Language - Syntax
• Rules for how to put together sentences and phrases.

39. Structure of Language - Semantics
• The meaning of symbols, words, phrases, and
sentences of a language.

40. SYNTAX AND SEMANTICS
• According to linguist Noam Chomsky, we are
born with an implicit sensitivity to sentence
structure and the rules of syntax.
• Chomsky’s approach- termed
transformational grammar

41. Transformational Grammar
• A sentence exists as both a deep structure
that represents meaning and a surface
structure that conveys it.
• Sentences can have the same surface
structure but different deep structures.
• Also, different surface structures can be
associated with the same deep structures.

42. Universal Grammar
• Because Chomsky’s principles apply regardless
of language they are referred to as universal
grammar.
• The notion of universal grammar has a
challenge:
– The claim that languages are much too diverse to
support the idea that the same underlying (i.e.,
universal) grammar can account for all of them.
•

43. Phrase Structure, Surface Structure,
And Deep Structure
• Phrase Structure-this involves breaking a sentence
down into its component phrases, termed constituents
• For example, “The engaging professor entertained the
class.
• This would be broken down into two major phrases or
constituents.
• A noun phrase, “The engaging professor”
• A verb phrase, “entertained the class.
• Each of these in turn can be further broken down into
an article- “the”, an adjective-“engaging”, and an
object- “class”

44. Phrase Structure
• At times it is hard to understand the meaning
of a sentence just based on phrase structure.
– Such as the sentence-“The shooting of the
hunters was terrible.”

45. Phrase Structure, Surface Structure,
And Deep Structure
• Chomsky thought that sentences must exist at
two levels both as an idea and a concrete
representation of that idea which he described as
deep structure and surface structure.
• Deep Structure- this part of the sentence conveys
its meaning
• Surface Structure-this part of the sentence is the
particular ordering used to convey that meaning.

46. Challenges to Chomsky’s Theories
• 1) Language may not be so universal- there
are so many languages to say his theory
applies to all of them is questionable
• 2) Language may not be un-learnable- the
structural aspects of language may be
learnable.

47. The Constraint Based Approach
• Proposes that language is learnable via the same
cognitive and brain mechanisms used for other
tasks.
• Probabilistic constraints within language can be
exploited by neural networks during the
language-learning process.
• Child Directed Speech (CDS) is one rich source of
information that allows developing infants to pick
up on words, word boundaries, and syntax.
• The prosody (rhythm, stress, and intonation) of
CDS serves both a linguistic and an emotional
function.

48. Modern Technology
• Modern technologies such as TV, phones, tablets,
computers, video cameras, etc. can have a detrimental
effect on frequency of CDS (speech that is directed at the
child typically from an adult.
• If the TV is on in the background the child may not pay
attention to it. However, the adult is paying attention to the
TV(or other technology).
• Thus there is a reduction in speech directed at the child.
• The impact of not talking to your child or talking less has
long term effects on language development.
“Talk to your kids!”

49. PRAGMATICS
• Pragmatics are the practical knowledge we
need to use language effectively during
conversation.

50. Conversational Structure
• Conversations usually have a standard structure that
includes a greeting, turn-taking, and a little overlap.
• Grice (1975) identified several characteristics that
seem to define the unwritten contract of conversation.
• 1) Quantity-say as much as you need to, but not more.
• 2) Quality- don’t say things that you know to be false;
don’t say things for which you lack evidence.
• 3) Relation- Be relevant to the topic at hand.
• 4) Manner- Be clear; avoid obscurity and ambiguity.

51. EXPERIMENT
• Experiment: in conversation deliberately
violate these Gricean Maxims and see what
happens.

52. Gender and Conversation
• Lakoff (1975) observational sociological study:
– 1) Women tend to be more polite
– 2) Women tend to use more tag questions (questions at
the end of a statement.
• In a meta-analysis by Anderson and Leaper (1998), men
are more likely to interrupt than women are, but the
effect is small.
• Men make more intrusive interruptions than women.
• Women and Men seem to use about the same number
of words in everyday speech.

53. Language Production and Perception
• Stages of Speech Production
• 1) Conceptualizing what we want to say
• 2) Develop a Linguistic Plan- organizing your
thoughts in terms of language.
• 3) Articulating the Linguistic Plan.
• 4) Monitoring-keep track of what we’re saying
and whether the message and tone are as
intended.

54. Slips of the Tongue
• Shift- one speech segment disappears from its location
and reappears elsewhere.
• Exchange- two segments change places
• Anticipation-when a later segment replaces an earlier
one
• Perseveration-earlier segments replaces later segment
but does not disappear from its appropriate location.
• Addition- inserting something
• Deletion-leaving something out
• Substitution-intruder replaces an intended segment
• Blend-two words combine into one.

55. Slips of the Tongue

56. Slips of the Tongue
• Slips can be motivational as in Freudian Slips
but these are the exception, not the rule.

57. Self-Monitoring
• We often engage in self-repair- we stop
ourselves and correct what we just said.
• “Scratch that, reverse it.”
– -Charlie and the chocolate factory

58. Self-Monitoring
• Levelt (1983)- self-repairs have a consistent
structure
• 1) We interrupt ourselves when we detect an
error
• 2) We issue what might be termed an editing
expression, like “um,” “Oh wait,” or “sorry”
• 3) We “repair” what we’ve just said by saying
such things as “er….I mean.”

59. Speech Perception
• Modular View-language is made up of a
unique set of abilities and capacities that can
not be reduced to or explained solely in terms
of other cognitive processes. It is species-
specific and innate (present from birth)
• Non-Modular View-language is the joint
production of cognitive processes reviewed
thus far.

60. Motor Theory of Speech Production
• Posits a close link between the mechanisms we
use to articulate speech and our perception of
speech.
• Liberman and Whalen (2000)
– 1) Because only humans possess the mechanisms
necessary for speech, only humans are capable of
understanding speech.
– 2) Speech perception is innate; infants are born with
the representations that allow for speech perception
and production.

61. Non-Modular Theories of Speech
Perception
• Auditory Theory- speech perception is the
product of “regular” auditory perceptual
processes.
– No special mechanism necessary.
– Other species have auditory systems similar to our
own, so the ability to perceive speech sounds
should not be unique to humans.