1. The Role of Visual Social
Attention in Language
Development
Stanford Undergraduate Psychology Conference 2014
Erin Park
Tawny Tsang
Scott P. Johnson
UCLA
1
4. Eye-tracking and communication
• Joint attention
• Infant’s visual behaviors index their
understanding of other people
4
Navab et al., 2012
5. Eye-tracking and faces
5
• Visual exploration
of the face changes
in the first year
• Why? Is this related
to language
development?
6. Our Questions
• How do changes in visual exploration of the
face relate to language?
– Do specific patterns of looking result in greater
expressive language?
6
7. Methods
• Participants
– N = 26 ; 18 M
– Age Range: 5.75-11.6 months; mean = 8.76
• Developmental Assessment
– Mullen Scales of Early Learning
• Developmental assessment normed from birth to 68 months
– Verbal Scores (VDQ)
• Mean: 86.45 + 14.21
– Non-Verbal Scores (NVDQ)
• Mean: 107.75 + 18.36
7
9. How do changes in visual exploration
of the face relate to language?
9
r = 0.325
p = 0.1
10. How do changes in visual exploration
of the face relate to language?
• BUT!
• The combination of relative mouth looking
and frequency of scanning between the eyes
and the mouth highly predicts verbal scores,
controlling for gender and total viewing time.
• R2 = 0.528, p = 0.027
10
11. Do specific patterns of looking result in
greater expressive language?
• YES!
• Scanning between the eye and mouth and relative
preference for the mouth strongly predicts expressive
language (R2 = 0.269, p < 0.05) but not receptive
language (R2=0.052, p = 0.54)
11
12. Discussion
• Recap: Scanning between the eyes and the
mouth and relative fixation on the mouth
predicted verbal skills.
• This suggests that looking to a region is not
enough to support language, but looking at
the right place at the right time is.
• What about Receptive Language?
• Limitation: gender
12
Hi My name is Erin Park, and today I’ll be speaking about the role of visual social attention in language development.
I want to highlight the importance of language in our everyday interactions. Language is crucial in communication and conveying information: information about the world, and about the self. If you think about it, what other modes of communication are there? Language has a better spatial and temporal resolution than gestures do, for example. In other words, there are more mediums available to us for communication than gestures. We can communicate with others through the phone, in person, letters, online chatting, and many more. Normed alphabets and other attempts at created manners of helping us understand other languages and increase our communication worldwide exist. With gestures, we are much more limited. Without language, there is no way for us to transmit information about how we’re feeling, current emotional states, and it enables us to really achieve a deeper form of communication.
As adults, we do this very seamlessly. For example, I am using language right now to communicate research ideas. The vocabulary of an adult is much larger and more complex than that of a younger, newer human. The older humans have more experience in words and what they mean to others. A big question is: How do infants learn to do this very important skill? We know that infants begin to acquire language within the first year. One way infants can learn communicative intent is by watching other people, more specifically, through observing the faces of others for communicative cues. We will refer to this behavior as visual social attention.
Eye-tracking is a method to measure visual social attention in infants and how much they understand communicative cues. Our example here demonstrates this. In this joint attention task, a woman makes eye contact with the infant participant then shifts her gaze to one of the objects. Infants who understand the intent of the gaze shift will follow her gaze and look at the object as well. Thus the use of eye-tracking can provide insight into infant cognition. We are interested in using this method to understand how infants’ visual behaviors index their understanding of other people. (Navab et al., 2012)
Because we are also specifically interested in language, it is crucial to know and understand the changes in the way infants look at faces that occur while they are learning to acquire language. In this graph here, these researchers found that between 6-12 months, infants increasingly look towards the mouth of a speaking face. This age range coincided with when infants began to make utterances resembling speech sounds. While the authors suggested that increased mouth looking was related to understanding speech cues, they did not directly test this hypothesis. The authors hypothesized that this was due to gains in expressive language, or the ability to produce language, as well as learning to view the redundant audiovisual cues of the mouth as a means to aid with language acquisition.
This is where our study comes in. The questions we hope to answer are: How do changes in visual exploration of the face relate to language? Do specific patterns of looking result in greater expressive language?
We tested 26 infants, 18 of which were male. Their ages ranged between 6 to 12 months. Our study consisted of two parts: a behavioral task and an eye tracking task. The behavioral task was a developmental assessment aimed to measure the verbal and nonverbal skills of our infants. The Mullen Scales of Early Learning is a developmental assessment normed from birth to 68 months. It consists of five scales: gross motor, fine motor, visual reception, expressive language, and receptive language. The verbal scores were calculated from a combination of expressive and receptive language scores. It was calculated similarly to how IQ scores are. The non-verbal scores are calculated from fine motor and visual reception. We didn’t use the non-verbal scores, however, because the focus of our research was language. Here are the means.
Our eye tracking paradigm consisted of a woman speaking in infant directed speech, or exaggerated emotional speech. We tried to make it as naturalistic as possible. Here’s an example. (click button right after)
We coded areas of interest as mouth, eyes, and background. This is an example of a child’s eye tracking data. He spent a lot of time looking at the woman’s eyes. So that was our methods, and now we’re going to show you the results of our data.
As a reminder, our research question was: how do changes in visual reception relate to language? The variables that we are interested in are relative looking to the mouth. This was calculated as the difference between percent of fixations directed on the mouth versus the eyes. (click!) This is what we found--the relative fixation on the mouth in and of itself had no significant correlation to the verbal scores.
BUT! Here is where it’s very interesting and exciting. The combination of relative mouth looking and frequency of scanning between the eyes and the mouth highly predicts verbal scores, controlling for gender and total viewing time. Our R2 (how well you can predict the variable). Almost 53% of the variance accounted for in verbal scores could be explained by looking at the mouth and scanning between the eyes and the mouth.
[The scanning and percent of time looking are not related measures.] if someone asks
Receptive: r = 0.195
p = 0.34
Expressive: r = 0.336
p = 0.093
Our second question was: do specific patterns of looking result in greater expressive languages. As a reminder, our verbal score contained both expressive and receptive language. It measured how well an infant understood language and the amount of language they were producing themselves.
To recap: Scanning between the eyes and the mouth and relative fixation on the mouth predicted verbal skills. This suggests that looking to a region is not enough to support language, but looking at the right place at the right time is. And this is measured by the scanning behavior.
We had to covary for gender, because there was an unequal distribution of gender. We had more boys than girls. The means for the verbal scores that we had were a little lower than normal, because language develops earlier in girls than boys. There was also the fact that not all of the babies watched all of the videos. Some of the babies would get fussy or fidgety during different parts of the videos, so we also had to account for that by only accepting the data if they completed a certain percentage of it.
Future studies can look into whether or not the precise timing of these shifts and see whether it’s related to certain things that the women are saying. It would be interesting to see whether or not infants have an early understanding of the semantic content in speech, and what parts catch their attention. Another point to emphasis is that we found evidence of visual behaviors indexing expressive language in infants. Another direction for a follow-up study would be looking into how visual behaviors contribute to receptive language.
It would be interesting if this data was delved into a little bit more in order to create more standardized tests for verbal development.
I would like to acknowledge all of the members of the baby lab, and my graduate student mentor, Tawny Tsang. I wouldn’t have been able to have come this far without her. Thank you!
