Coordination of Gaze and Speech in Communication Between Children With Hearing Impairment and Normal-Hearing Peers

JSLHR
Research Article
Coordination of Gaze and Speech in
Communication Between Children
With Hearing Impairment and
Normal-Hearing Peers
Olof Sandgren,a
Richard Andersson,a
Joost van de Weijer,a
Kristina Hansson,a
and Birgitta Sahléna
Purpose: To investigate gaze behavior during communication
between children with hearing impairment (HI) and
normal-hearing (NH) peers.
Method: Ten HI–NH and 10 NH–NH dyads performed a
referential communication task requiring description of faces.
During task performance, eye movements and speech were
tracked. Using verbal event (questions, statements, back
channeling, and silence) as the predictor variable, group
characteristics in gaze behavior were expressed with
Kaplan-Meier survival functions (estimating time to gaze-to-
partner) and odds ratios (comparing number of verbal events
with and without gaze-to-partner). Analyses compared the
listeners in each dyad (HI: n = 10, mean age = 12;6 years,
mean better ear pure-tone average = 33.0 dB HL; NH: n = 10,
mean age = 13;7 years).
Results: Log-rank tests revealed significant group differences
in survival distributions for all verbal events, reflecting a higher
probability of gaze to the partner’s face for participants
with HI. Expressed as odds ratios (OR), participants with HI
displayed greater odds for gaze-to-partner (ORs ranging
between 1.2 and 2.1) during all verbal events.
Conclusions: The results show an increased probability for
listeners with HI to gaze at the speaker’s face in association
with verbal events. Several explanations for the finding are
possible, and implications for further research are discussed.
Key Words: referential communication, eye tracking, child
hearing impairment, gaze behavior, survival analysis
F
ace-to-face interaction consists of more than the
verbal exchanges that make up the sound stream.
The interaction is replete with gazes to the conver-
sational partner. In a previous study, Sandgren, Andersson,
van de Weijer, Hansson, and Sahlén (2012) demonstrated that
children with normal hearing displayed increased probability
of gaze-to-partner when asking questions than when mak-
ing statements, results indicating that the linguistic and prag-
matic content of the conversation influences gaze behavior.
But does a hearing impairment affect the use of gaze in
conversation additionally? In the present study, we test the
hypothesis that children with hearing impairment gaze more
to the conversational partner than normal-hearing peers by
examining gaze-to-partner during different verbal events
(questions, statements, back channeling, and silence) in a
referential communication task.
Gaze Behavior in Typical Populations
Gaze behavior during conversation has been the topic
of a number of studies, and many provide consistent results
(Bavelas, Coates, & Johnson, 2002; Kendon, 1967; Mirenda,
Donnellan, & Yoder, 1983; Turkstra, Ciccia, & Seaton, 2003).
In free and unscripted dialogue, listeners look more at the
speaker than vice versa (Kendon, 1967; Turkstra et al., 2003).
Turkstra et al. (2003), for example, in a study of interactive
behaviors in adolescents, reported that participants look at
their partners 65% of the time when listening compared with
40% of the time when speaking. Kendon (1967), reporting
ranges and providing few details on participant characteris-
tics, found that adults looked at their partners 30%–80%
of the time when listening versus 20%–65% when speaking.
Differences between the studies regarding data collection and
a
Lund University, Lund, Sweden
Correspondence to Olof Sandgren: olof.sandgren@med.lu.se
Editor: Rhea Paul
Associate Editor: Elizabeth Crais
Received October 23, 2012
Revision received April 19, 2013
Accepted September 24, 2013
DOI: 10.1044/2013_JSLHR-L-12-0333
Disclosure: The authors have declared that no competing interests existed at the
time of publication.
Journal of Speech, Language, and Hearing Research • Vol. 57 • 942–951 • June 2014 • A American Speech-Language-Hearing Association942
Downloaded From: http://jslhr.pubs.asha.org/ by a Proquest User on 08/05/2014
Terms of Use: http://pubs.asha.org/ss/Rights_and_Permissions.aspx

analysis should, however, be considered when comparing
results. Whereas both studies investigated gaze exchanges
between unacquainted interlocutors, Turkstra et al. (2003)
used video-based analysis of an adolescent population, and
Kendon (1967) studied adults using technology with lim-
ited temporal resolution, likely increasing the variability
of gaze-to-partner across individuals and obscuring rapid
gaze exchanges between participants. The present study
uses eye-tracking technology and a semistructured task to
address the need for temporal resolution and ecological
validity.
Several attempts have been made to define the com-
municative and social role played by gaze to the partner in
conversation. Kendon (1967), who used a conversation
analytic approach, found a lower rate of gaze-to-partner at
the beginning of utterances and a higher rate at the end,
causing him to suggest a relation between gaze behavior and
turn taking in conversation. This turn-regulating function has
also been described by Cummins (2012) and Bavelas et al.
(2002), who, in addition, went on to propose that gaze not
only regulates turn exchanges but also creates short time-
windows for back channeling responses (for example, “Yeah,”
“Uh-huh,” “Mhm”) within an ongoing speaking turn. Pre-
vious studies, reviewed and summarized by Mirenda et al.
(1983), have proposed additional, pragmatically related
functions of gaze, including declaring an interest in the
topic, expressing the degree of intimacy with the conver-
sational partner, providing the speaker with information on
misunderstandings and communicative breakdowns, and
expressing emotions. By analogy, Turkstra (2005) con-
cluded that an atypical gaze behavior can cause “negative
outcomes that range from missing a cue to end a conver-
sation to misperceiving the emotional communication of
one’s partner” (Turkstra, 2005, p. 1430).
Gaze Behavior in Atypical Populations
Gaze behavior is known to be affected by various
clinical disorders, including autism spectrum disorders (ASDs),
which are associated with a lower rate of gaze-to-partner
(Corden, Chilvers, & Skuse, 2008) and/or a distorted timing
of gazes (Willemsen-Swinkels, Buitelaar, Weijnen, & van
Engeland, 1998), and Williams syndrome, for which atypi-
cally high levels of gaze-to-partner are reported (Doherty-
Sneddon, Riby, & Whittle, 2012). Norbury et al. (2009), in
a study investigating the focus of gaze when viewing video
clips of emotionally engaging dialogue, found normal rates of
gaze to the characters’ eyes in teenagers with ASD but, inter-
estingly, no relation between the gaze behavior and the level
of social competence. Instead, gaze behavior was related to
communicative competence. The authors concluded that the
role of gaze behavior, and in particular gaze to the partner’s
eyes, in the social competence of the teenagers with ASD
may have been exaggerated and that linguistic ability, atten-
tion, and rigidity of behavior better predict social competence
outcome. It has also been proposed that gaze-to-partner in
ASD may reflect more of an orienting function and less of a
reciprocal social exchange (Nadig, Lee, Singh, Bosshart, &
Ozonoff, 2010). This stems from an experiment in which
children with high-functioning autism, speaking about per-
sonal interests as opposed to generic topics, exhibited more
atypical verbal production but more typical gaze behav-
ior. Nadig et al. (2010) concluded that speaking about
a personal interest made participants with ASD more stereo-
typical and monologue-like. Higher, more typical rates of
gaze-to-partner were only made possible because talking
about a highly practiced topic made cognitive resources
available to be directed at, for example, the partner.
Nadig et al.’s (2010) conclusion, which implies that
an appropriate gaze behavior requires sufficient allocation of
cognitive resources, finds support in the cognitive load hy-
pothesis proposed by Glenberg, Schroeder, and Robertson
(1998). The cognitive load hypothesis has been investigated in
association with gaze aversion, that is, the deliberate avoid-
ance of gaze-to-partner, in children with both typical and
atypical development. Doherty-Sneddon and Phelps (2005)
investigated why the recipient of a question looks away from
the partner and tested whether this behavior serves to reduce
the cognitive load or to alleviate the social stress associated
with the risk of giving the wrong answer. Using a design
comparing face-to-face and video-linked questioning, the
authors found that the largest impact on gaze aversion was
the degree of difficulty of the questions, indicating that gaze
aversion serves more to manage cognitive load than social
stress. It is possible that the same mechanisms can help ex-
plain Kendon’s (1967) finding of lower rates of gaze-to-
partner at the beginning of utterances. The linguistic planning
required to form an utterance is more easily performed when
blocking out unnecessary visual stimuli. On the other hand,
turn-regulation mechanisms could also explain gaze aversion
in the recipient of a question. By avoiding gaze-to-partner,
the recipient claims the speaking turn and signals that a
response is imminent.
Present Study
From previous research, it can be concluded that
gaze-to-partner is actively used in face-to-face conversation.
Furthermore, gaze behavior is affected by linguistic and/or
social deficits, and results have indicated a possible link
between linguistic proficiency and gaze-to-partner. The
present study addresses the paucity of research on gaze-to-
partner in participants with hearing impairment, a population
reported to use visual cues more than normal-hearing peers
(Skelt, 2006) and more often exhibiting language delay
(Gilbertson & Kamhi, 1995; Hansson, Forsberg, Löfqvist,
Mäki-Torkko, & Sahlén, 2004; Yoshinaga-Itano & Sedey,
1998). An increased use of visual cues has been suggested as a
compensatory strategy in children with hearing impairment,
aiding language processing and comprehension and com-
pensating for the degraded auditory input and restricted
ability to use incidental hearing for learning (Blamey et al.,
2001). This suggestion is further supported by findings of
improved speech perception for audiovisual speech over speech
presented through the auditory modality only (Bergeson,
Pisoni, & Davis, 2003; Garcia & Dagenais, 1998; Most,
Sandgren et al.: Coordination of Gaze and Speech 943

Rothem, & Luntz, 2009; Woodhouse, Hickson, & Dodd, 2009).
Furthermore, visual cues are used in the perspective-taking
crucial for efficient communication, as shown by children
taking into account the partner’s field of view when inter-
preting instructions (Nadig & Sedivy, 2002; Nilsen &
Graham, 2009). Together, the results illustrate the necessity
of instant integration of visual information in the percep-
tion and processing of speech and in the pragmatic under-
standing of communication.
In a detailed analysis of the gaze behavior of adults with
severe-to-profound hearing impairment while speaking to
either their audiologist or a family member, Skelt (2006)
found the participants with hearing impairment behaved
qualitatively similarly to the participants with normal hearing
in Kendon’s (1967) study, however with higher rates of gaze-
to-partner when listening and lower rates when speaking.
Skelt (2006) described how the participants with hearing im-
pairment through use of gaze initiations and gaze withdrawals
controlled the turn exchanges in the conversations. Although
gaze is only one of several tools used in turn regulation,
Skelt (2006) emphasized its role in displaying readiness to
accept or reject the speaking turn.
Participants with hearing impairment are clinically
relevant to study since this population in many countries is
educated in inclusive settings (Hyde & Power, 2003; Stinson
& Antia, 1999). Modern teaching often involves classroom
tasks requiring collaboration between individual students
or work in small groups (Toe & Paatsch, 2010), for which
questioning and responding is necessary. Consequently, the
school setting demands well-functioning interaction, verbal
as well as nonverbal. Previous studies have found students
with hearing impairment to be less prone to request clarifying
information despite misunderstanding (Marschark et al.,
2007) and to exhibit difficulties with turn taking, at least when
normal-hearing turn-taking behavior is considered the norm
(Duchan, 1988). To address these issues, the present study
reports data on participants in middle childhood, a period
of increasing demands on independence in school work and
peer interaction, and investigates gaze behavior for its role
in the buildup and management of social interaction.
The present study used a referential communication
task requiring collaboration between participants for suc-
cessful completion. In order to ensure ecological validity, a
semistructured, unscripted paradigm was used, defining the
procedure without restricting the participants’ production
or choice of conversational partner. In a previous study of
children with normal hearing, Sandgren et al. (2012) have
shown the verbal production of questions, statements, back
channeling, and silence to influence the probability of gaze-
to-partner. In the present study, we investigate the influence
of these verbal events on the probability of gaze-to-partner
in children with hearing impairment. While expecting the
verbal event to influence the probability of gaze-to-partner
for all participants, hearing impaired or not, we hypothesized
that the participants with hearing impairment would con-
sistently exhibit higher probability of gaze-to-partner than the
participants with normal hearing, possibly using gaze-to-
partner as a compensatory strategy.
Method
Participants
HI–NH dyads. Twenty children and adolescents (for
the sake of brevity, henceforth labeled children), seven girls
and 13 boys, ranging in age between 9;8 (years;months) and
15;10 (M = 12;4, SD = 1;9) were recruited to form conver-
sational pairs. Out of these, 10 participants, three girls and
seven boys, mean age = 12;6 years, SD = 2;0, labeled HI),
had documented bilateral mild-to-moderate sensorineural
hearing impairment, that is, pure-tone average (PTA) air-
conduction hearing thresholds for octave frequencies from
0.5 to 4 kHz (ISO 8253–1, 2010) between 26 and 55 dB HL
(Clark, 1981), and had received bilateral hearing aids. In
the collected sample, better ear PTA ranged between 20 and
43 dB HL (M = 33.0, SD = 7.8). All impairments were
symmetrical (mean difference = 7.1 dB, SD = 6.1). According
to medical records, mean age at identification of the hear-
ing impairment was 3;7 years (SD = 1;1) and mean age at
amplification was 5;2 years (SD = 2;7). An outlier, with 25 dB
HL, identified at 5;0 years and receiving amplification at
10;0 years, was found not to differ from the other participants
on the measures obtained in this study and was included in
the analyses. All participants with hearing impairment were
raised in oral speaking families and were educated in oral
settings, exhibited no speech impairments, and were given
no formal training in sign language, visually aided com-
munication, or speech reading.
The remaining 10 participants, four girls and six boys
(mean age = 12;3, SD = 1;7) were normal-hearing same-age
peers invited by the participant with hearing impairment
to take part in the study as conversational partners. All par-
ticipants with hearing impairment chose to bring a classmate,
thus, a partner familiar with their hearing loss, differing
maximally 1 year in age. All except three HI–NH dyads
consisted of same-sex participants.
NH–NH dyads. Twenty children and adolescents,
10 girls and 10 boys, ranging in age between 10;2 and 15;4
(M = 13;6, SD = 1;11), were recruited to form normal-hearing
control dyads. Half of the participants in the control dyads,
five girls and five boys (mean age = 13;7, SD = 1;11, labeled
NH) composed a control group, matched to the ages of the
HI group. The other half, five girls and five boys (mean age =
13;5, SD = 2;0), were classmates invited by their NH peers
to participate as conversational partners. All NH–NH dyads
consisted of same-sex participants. Group descriptives are
summarized in Table 1.
The HI and NH groups did not differ significantly
on age, t(18) = 1.281, p = .22, or receptive grammar, t(18) =
1.469, p = .159, as measured by standardized assessment with
the Test for Reception of Grammar, Version 2 (TROG-2;
Bishop, 2009). All participants had Swedish as their first
language and all had nonverbal IQs within normal limits as
measured by Raven’s Standard Progressive Matrices (Raven,
Raven, & Court, 2004). All participants had normal or
corrected to normal vision, and all normal-hearing partici-
pants passed a 20 dB pure-tone hearing screening at 0.5, 1, 2,
4, and 6 kHz before data collection. Ethical approval for the
944 Journal of Speech, Language, and Hearing Research • Vol. 57 • 942–951 • June 2014

study was granted by the Regional Ethics Review Board for
southern Sweden, approval number 2009/383.
Materials and Procedure
Experimental task. An unscripted referential commu-
nication task was used, in which the target children with
hearing impairment and normal-hearing controls acted as
the listeners, and the conversational partners acted as the
speakers. This study reports data on the listener. The task has
previously been used in several studies of conversational
strategies and interaction in children with language and/or
hearing impairment (Ibertsson, Hansson, Mäki-Torkko,
Willstedt-Svensson, & Sahlén, 2009; Sandgren, Ibertsson,
Andersson, Hansson, & Sahlén, 2011) and in a study of gaze
behavior during linguistic problem solving between chil-
dren with normal hearing (Sandgren et al., 2012). A screen
displaying 16 pictures of faces, visible only to the speaker, was
placed between the participants. The listener was provided
with 24 pictures of faces. The instructions given were for
the speaker to describe each picture and its position with as
much detail for the listener to be able to identify the correct
picture and place it in the correct position. The pictures
of faces differed only in details, and the listener was forced
to request further information when confronted with an
insufficiently detailed description. The analysis focused on
the process of task resolution, not the end result, and a ceiling
effect was expected.
Equipment and data collection. During the referential
communication task, the participants wore identical SMI
iView X HED head-mounted video-based pupil and corneal
reflex eye-tracking systems, calibrated with a 9-point cali-
bration procedure. The data from each eye-tracking system
were merged with the video of a forward-facing camera,
creating an output video showing the participant’s field of
view with a moving cursor indicating gaze position. The video
was filmed at 25 frames per second, creating an effective
sampling frequency of 25 Hz. The participants were seated
approximately 120 cm from each other, separated by the
30-cm tall picture screen. The height of the screen created
real-life-like conversational conditions by allowing eye
contact and visual cues. The dialogues were video recorded
using a fixed digital video camera capturing both participants
from a side view. For audio recording, the camera’s built-in
microphone was used. Recordings were made in a quiet
laboratory setting in the Humanities Laboratory at Lund
University.
The dialogues were transcribed orthographically by
the first author and transcriptions were exported to ELAN
(Wittenburg, Brugman, Russel, Klassmann, & Sloetjes,
2006), an open-source audio and video annotation software,
where each listener’s speech was categorized into four types of
verbal events: requests, nonrequests, back channeling, and
silence. Requests included requests for confirmation of new
information (“Has she got blue eyes?”), requests for confir-
mation of old information (“Did you say she had blue eyes?”),
and requests for elaboration (“What color are her eyes?”).
In a previous study (Sandgren et al., 2011), these types of
requests have been found to account for over 90% of requests.
Back channeling included verbal signals of comprehension
and interest (for example, “Yeah,” “Uh-huh,” “Mhm”). The
remaining speech, including, for example, statements from
the listener not directly related to the task resolution, such as
“He looks a bit like your dad,” was categorized as non-
requests in order to provide a baseline condition for com-
parison in the analysis of gaze-to-partner during requests.
Similarly, periods of silence, often constituting the partner
speaking, were categorized to provide a baseline condition in
the analysis of gaze-to-partner during back channeling.
The fourth author independently coded the verbal events in
25% of the dialogues. The interrater reliability as estimated
with Cohen’s kappa was .941. Table 2 provides examples
and group data on verbal event types.
Annotation of eye movements was made by the first
author using ELAN (Wittenburg et al., 2006). The output
videos of the eye-tracking systems were merged and syn-
chronized with the orthographic transcription, creating an
annotation file containing all verbal and gaze annotations.
Three areas of interest regarding gaze focus were specified:
task (the pictures of faces), face (the partner’s face), and
off (gaze focused elsewhere). All instances of gaze within the
specified areas of interest were recorded, providing infor-
mation on the participants’ gaze focus for the duration of the
conversation. The second author independently annotated
the eye movements in 20% of the data. Reliability was mea-
sured using the overlap calculation in ELAN (Wittenburg
et al., 2006), with a modification weighting the annotations
on their duration in time, allowing annotations of greater
Table 1. Group descriptives.
Study groups and
conversation partners
(n; sex)
Mean age
(SD)
Mean BEPTA
(SD)
Mean age at
identification
(SD)
Mean age at
amplification
(SD)
Target HI (n = 10; 3 f, 7 m) 12;6 (2;0) 33.0 (7.8) 3;7 (1;1) 5;2 (2;7)
HI partner (n = 10; 4 f, 6 m) 12;3 (1;7)
Control NH (n = 10; 5 f, 5 m) 13;7 (1;11)
NH partner (n = 10; 5 f, 5 m) 13;5 (2;0)
Note. BEPTA = better ear pure-tone average; HI = participants with hearing impairment; HI partner = normal-hearing conversational partners of
HI; NH = normal-hearing control group; NH partner = normal-hearing conversational partners of NH; f = female participants; m = male participants.

duration to affect the reliability score more than shorter
annotations. The interrater reliability was 88.5%.
Data Analysis
Gaze and verbal annotation data were extracted from
ELAN for analysis. In all, 2,946 cases of verbal events were
identified and used in the analyses. The dependent variable
(gaze to the speaker’s face) was scored binarily, on 10-ms
intervals, over a 3,000-ms time window centered at the onset
of the predictor event (the verbal events). Thus, for each
case of a verbal event, 300 measurements of the occurrence of
gaze-to-partner (1/0) were made, covering the time span
between 1,500 ms preceding and 1,500 ms following the ver-
bal event onset. The raw data were plotted in SPSS to provide
probability plots of gaze to the partner’s face for the differ-
ent verbal event types. Figure 1 shows a schematic illus-
tration of the verbal events and the gaze analysis window.
In order to analyze not only if but also when gaze
to the speaker’s face occurred, data were fitted to a survival
function estimating the probability of the target event to
occur. The survival function estimates the event time (the time
from the beginning of measurements, in this case 1,500 ms
preceding the verbal event onset) to the target event, while
statistically accommodating the influence of censored cases
(verbal events performed without gaze-to-partner within the
time window) and displaying the probability of target event
occurrence as a cumulative survival. For group comparisons of
the probability of gaze-to-partner during the different verbal
event types, Kaplan-Meier survival analysis with Mantel-Cox
log-rank tests were performed in SPSS. Three analyses were
performed, comparing (a) the probability of gaze-to-partner
during requests to the probability during a baseline of non-
requests; (b) the probability of gaze-to-partner during back
channeling to the probability during a baseline of silence; and
(c) the probability of gaze-to-partner during the two main types
of requests, that is, requests for confirmation of new infor-
mation and requests for confirmation of old information. The
raw data (883,800 rows) used for the probability plots were
aggregated in two steps to (a) display whether the target event
(gaze-to-partner) occurred within the 3,000-ms time window
and (b) establish the point in time at the target event. The
Table 2. Verbal event types, descriptions, examples, and distribution.
Verbal event type Description Example n (HI) n (NH)
Requests Questions 288 254
“Has she got blue eyes?”a
(194)a
(182)a
“Did you say she had blue eyes?”b
(54)b
(57)b
“What color are her eyes?”c
(40)c
(15)c
Nonrequests Statements “He looks a bit like your dad.” 176 309
Back channeling Feedback “Uh-huh.” “Mhm.” 269 165
Silence Partner speaking 745 740
Total 1,478 1,468
Note. n shows number of verbal events of each type.
a
Request for confirmation of new information. b
Request for confirmation of old information. c
Request for elaboration.
Figure 1. Schematic illustration of verbal and gaze data. a
Request for confirmation of new information. b
Request for confirmation of old
information. c
Request for elaboration. d
Nonrequests. e
Back channeling. f
Silence. Gaze analysis window showing 3,000-ms time frame for study
of listeners’ gaze-to-partner, centered at verbal event onset.

Kaplan-Meier survival analysis eliminates cases as they
experience the target event, expressed graphically as a declin-
ing slope ultimately showing the “survivors,” or censored cases
(that is, the cases not experiencing the target event). Quanti-
fication of group differences in survival time was expressed
as odds ratios obtained by dividing the ratio of cases expe-
riencing the target event in the two groups with the ratio of
censored cases from both groups. Odds ratios with confidence
intervals not encompassing 1 indicate significant differences
between the groups.
Results
Raw Data Examination
In an initial analysis, raw data were examined to explore
the probability of gaze-to-partner during the various verbal
events. Figure 2 provides an example of the raw data, show-
ing probability plots for gaze-to-partner during requests
(including requests for confirmation of new information,
requests for confirmation of old information, and requests
for elaboration) compared with a nonrequest baseline in a
3,000-ms analysis window centered at the request/nonrequest
onset. Raw data exploration indicates a higher probability
of gaze-to-partner after request onset for HI than for NH.
Analysis of the corresponding raw data graphs gener-
ated for gaze-to-partner during back channeling compared
with a baseline of silence indicates that back channeling onset
entailed a decreased probability of gaze-to-partner, still,
however, with higher probability in the HI group.
While indicating a higher overall probability of gaze-to-
partner for participants with HI than for participants with
NH, raw data graphs displaying gaze-to-partner during
the two types of requests for confirmation (of new and old
information, respectively) do not demonstrate clear differ-
ences between request types.
Survival Function
Data were fitted to a Kaplan-Meier survival analysis
estimating the event time from beginning of measurements to
occurrence of gaze-to-partner for the verbal events of the
participant groups and displaying the estimate as a cumula-
tive survival. Figure 3 presents survival estimates for request
and nonrequest, with data labels displaying censored data
(that is, verbal events produced without gaze-to-partner).
Mantel-Cox log-rank tests of equality of survival distributions
revealed significantly lower survival rates for the HI group
for requests (c2
(1, N = 542) = 4.826, p = .028) as well as for
nonrequests (c2
(1, N = 485) = 6.354, p = .012).
Survival estimates for back channeling and a baseline
of silence are presented in Figure 4. Again, Mantel-Cox
log-rank statistics revealed significantly lower survival rates
Figure 2. Mean probability of gaze-to-partner during requests (solid line) and nonrequests (dotted line) for participants with NH (upper panel,
n = 10) and participants with HI (lower panel, n = 10). The horizontal axis shows time, with 0 ms marking the onset of the verbal event.

Figure 4. Kaplan-Meier estimates of survival probabilities during back channeling (left panel) and silence (right panel) for participants with HI (solid
line) and participants with NH (dotted line). The horizontal axis shows time, with 0 ms marking the onset of the verbal event. Data labels present
censored data.
Figure 3. Kaplan-Meier estimates of survival probabilities during requests (left panel) and nonrequests (right panel) for participants with HI (solid
line) and participants with NH (dotted line). The horizontal axis shows time, with 0 ms marking the onset of the verbal event. Data labels present
censored data.

for participants with HI than for participants with NH,
regarding both back channeling (c2
(1, N = 434) = 11.801,
p = .001) and silence (c2
(1, N = 1485) = 26.881, p < .001).
Mantel-Cox log-rank tests of equality of survival dis-
tributions for the two types of requests for confirmation
(that is, of new and old information) revealed no significant
group differences for either request type (requests to confirm
new information: c2
(1, N= 376) = 0.879, p = .348; requests
to confirm old information: c2
(1, N= 111) = 2.024, p = .155).
Odds Ratios
Odds ratios were estimated by dividing the ratio of
cases experiencing the event with the ratio of censored cases.
Table 3 summarizes the data used for the analyses with sig-
nificant group differences in odds ratios and survival distri-
butions highlighted. A higher probability of gaze-to-partner
in association with requests, nonrequests, back channeling,
and silence is shown for the participants with hearing im-
pairment, as indicated by significant differences in Mantel-
Cox log-rank tests of equality of survival distributions and
significant differences in odds ratios, compared with the
participants with normal hearing.
Summary of Results
To summarize the results, an increased probability of
concurrent gaze-to-partner during verbal events was shown
for participants with hearing impairment compared with
participants with normal hearing. Kaplan-Meier survival
functions showed significantly reduced survival rates for
participants with HI, reflecting a higher propensity to gaze
in association with requests, nonrequests, back channeling,
and silence. Odds ratios express this finding by showing
an increase in the odds to look at the partner of 1.5 to 2.1 for
participants with HI during the same verbal events. While
displaying similar overall patterns, nonsignificant survival
distributions and odds ratios were found for the different
types of requests for confirmation.
Discussion and Conclusions
The results presented in this study support the hypoth-
esis that participants with hearing impairment do, indeed,
gaze more to their conversational partner during verbal events
than do normal-hearing peers (matched in age, nonverbal
reasoning, and receptive language skills). The findings—
gathered from a task posing demands similar to many school
tasks—provide evidence that school-age children and ado-
lescents with HI display significantly lower survival distribu-
tions and increased odds for gaze-to-partner when asking
questions, when making statements, when providing the
speaker with back-channeling responses, and during silence.
This discussion proposes directions for the next steps of
research, using these basic gaze data to delve into the reasons
behind the increased use of gaze.
First, task-dependent characteristics may have influ-
enced the gaze behavior. In this referential communication
task, the participants were seated face-to-face in a laboratory
setting, with competing auditory and visual stimuli kept to
a minimum. This may have made the participants with HI
more prone to gaze at the speaker than would otherwise have
been the case. It is, however, also easy to envision how the
laboratory setting could have resulted in the opposite be-
havior, that is, that the favorable acoustic conditions would
have made gaze-to-partner less necessary. Adequate exami-
nation of the influence of the task on the participants’ gaze
behavior requires the experiment to be replicated in a variety
of settings, the most naturalistic being a classroom environ-
ment. Future studies should also evaluate the effect of the
conversational partner, comparing gaze with known and un-
known partners. Furthermore, the referential communication
task used in this study should be scrutinized and compared
with other tasks encouraging more gaze exchanges between
Table 3. Data and result summary.
Verbal event Group
Cases with
eventa
Censored
casesb
Odds ratio
[95% CI] c2
Log rankc
p
Request HI 136 152. 1.5 [1.1, 2.1] 4.826 .028
NH 95 159.
Nonrequest HI 71 105. 1.7 [1.1, 2.5] 6.354 .012
NH 89 220.
Back channeling HI 107 162. 2.1 [1.4, 3.3] 11.801 .001
NH 39 126.
Silence HI 342 403. 1.7 [1.4, 2.] 26.881 .000
NH 242 498.
Requests for confirmation of new information HI 88 106. 1.2 [0.8, 1.9] 0.879 .348
NH 73 109.
Requests for confirmation of old information HI 23 31. 1.7 [0.8, 3.8] 2.024 .155
NH 17 40.
Note. Significant group differences are in bold.
a
Number of cases experiencing gaze-to-partner. b
Number of cases not experiencing gaze-to-partner. c
Mantel-Cox p value for test of group
difference in survival distribution between HI and NH groups.

the participants. However, gaze-to-partner during a task
requiring visual attention to be directed away from the partner,
performed with a friend with whom the child functions well,
strengthens, rather than weakens, the argument of gaze to the
conversational partner serving an important communicative
role. Whereas the present investigation provides necessary
groundwork on the gaze behavior in a semistructured task
without restrictions on verbal productions, future studies
should also use scripted utterances of predetermined duration.
By expanding the gaze analysis window, this would provide
an experimental control condition to more naturalistic tasks,
adding details on the probability of gaze-to-partner over the
course of an entire utterance.
Second, the higher probability of gaze-to-partner in
participants with HI could be interpreted as a way of com-
pensating for the degraded auditory signal. This should be
further explored in different ways. Systematic variation of the
participants’ access to visual cues would provide information
on the possible benefit of visual cues. Previous studies have
showed improved speech perception for audiovisual speech
compared with speech presented through only one modality,
in participants with hearing impairment (see, e.g., Woodhouse
et al., 2009), a finding deeply rooted in common knowledge
and practice. Future studies should further investigate individ-
ual characteristics of the participants as a factor influencing
gaze-to-partner. If future studies confirm a compensatory role
of gaze-to-partner, attempts must be made to tease apart
underlying causes that may interact with the hearing im-
pairment such as cognitive and linguistic ability. Participants
from other age groups, and other degrees and etiologies of
hearing impairment, should be studied. Although inflated by
an outlier who received auditory amplification unusually
late, the participants in this study received hearing aids at a
mean age of approximately 5 years. Today, newborn infants
in Sweden undergo otoacoustic emission screening for hearing
impairment, enabling earlier identification and amplifica-
tion. Thus, an investigation of whether earlier amplification
reduces the need for gaze-to-partner is warranted, as is a
replication of the study in participants with more severe,
and also unilateral, hearing impairments. Furthermore, a
possible increased cognitive load on individuals with hearing
impairment could be investigated using measures of gaze
aversion.
Without ruling out the possibility of the intraindividual
benefits described above, a third alternative, encompassing
interindividual benefits of gaze-to-partner during conversa-
tion, should be investigated. Instead of just a monitoring role
for gaze, the combination of gaze and speech can be used to
adjust the communicative content to reach conversational
objectives. This is similar to how, for example, number and
length of speaking turns and the use of requests can be used by
individuals with hearing impairment to control conversation
(Caissie, Dawe, Donovan, Brooks, & MacDonald, 1998).
Skelt (2006) described how participants with hearing im-
pairment, by maintaining or withholding gaze, exert control
over the turn exchanges. The conversational partners in her
study adhered to the gaze cue, allowing it to “overrule”
syntactic and prosodic cues for turn exchange. In our data,
the overall similarities in gaze behavior between the partici-
pants with and without hearing impairment can be seen as an
indication of gaze serving as a more generally applied turn-
regulating mechanism, similar for both groups. The group
differences in the probability for gaze-to-partner could, there-
fore, express the extra need for visual cues accumulated by
the hearing impairment.
If future studies show gaze-to-partner to compensate
for auditory deficits and play a role in turn regulation, it is
evident that gaze cues, and the opportunity to use them, must
be considered vital for the ability of individuals with hearing
impairment to participate in interaction on equal terms
with their normal-hearing peers. Clinical and educational
implications could include increased awareness of the use of
gaze during conversation in the child with hearing impair-
ment, family members, teachers, and friends, as well as
classroom modifications optimizing both visual and auditory
aspects of communication. With well-informed interlocutors
and well-adapted surroundings, individuals with hearing
impairment are more likely to be able to show their full
potential, using both verbal and gaze cues to participate fully
in the interaction.
Acknowledgments
We gratefully acknowledge the support of the Linnaeus
Centre Thinking in Time: Cognition, Communication and Learning,
financed by the Swedish Research Council (Grant 349-2007-8695).
We would also like to thank Jonas Brännström for valuable com-
ments in the preparation of the manuscript and express our sincer-
est gratitude to all participants.
References
Bavelas, J. B., Coates, L., & Johnson, T. (2002). Listener responses
as a collaborative process: The role of gaze. Journal of Com-
munication, 52, 566–580.
Bergeson, T. R., Pisoni, D. B., & Davis, R. A. (2003). A longitudinal
study of audiovisual speech perception by children with hear-
ing loss who have cochlear implants. The Volta Review, 103,
347–370.
Bishop, D. V. M. (2009). Test for Reception of Grammar, Version 2
(M. Garsell, Trans.). Stockholm, Sweden: Pearson.
Blamey, P. J., Sarant, J. Z., Paatsch, L. E., Barry, J. G., Bow, C. P.,
Wales, R. J., ... Toocher, R. (2001). Relationships among speech
perception, production, language, hearing loss, and age in
children with impaired hearing. Journal of Speech, Language,
and Hearing Research, 44, 264–285.
Caissie, R., Dawe, A. L., Donovan, C., Brooks, H., & MacDonald,
S. M. (1998). Conversational performance of adults with a
hearing loss. Journal of the Academy of Rehabilitative Audiology,
31, 45–67.
Clark, J. G. (1981). Uses and abuses of hearing loss classification.
ASHA, 23, 493–500.
Corden, B., Chilvers, R., & Skuse, D. (2008). Avoidance of emo-
tionally arousing stimuli predicts social-perceptual impairment
in Asperger’s syndrome. Neuropsychologia, 46, 137–147.
Cummins, F. (2012). Gaze and blinking in dyadic conversation:
A study in coordinated behaviour among individuals. Language
and Cognitive Processes, 27, 1525–1549.

Doherty-Sneddon, G., & Phelps, F. (2005). Gaze aversion: A re-
sponse to cognitive or social difficulty? Memory & Cognition, 33,
727–733.
Doherty-Sneddon, G., Riby, D. M., & Whittle, L. (2012). Gaze
aversion as a cognitive load management strategy in autism
spectrum disorder and Williams syndrome. Journal of Child
Psychology and Psychiatry, 53, 420–430.
Duchan, J. F. (1988). Assessing communication of hearing-impaired
children: Influences from pragmatics. Journal of the Academy of
Rehabilitative Audiology, 21(Mono. Suppl.), 19–40.
Garcia, J. M., & Dagenais, P. A. (1998). Dysarthric sentence intelli-
gibility: Contribution of iconic gestures and message predic-
tiveness. Journal of Speech, Language, and Hearing Research, 41,
1282–1293.
Gilbertson, M., & Kamhi, A. G. (1995). Novel word learning in
children with hearing impairment. Journal of Speech and
Hearing Research, 38, 630–642.
Glenberg, A., Schroeder, J., & Robertson, D. (1998). Averting the
gaze disengages the environment and facilitates remembering.
Memory & Cognition, 26, 651–658.
Hansson, K., Forsberg, J., Löfqvist, A., Mäki-Torkko, E., & Sahlén,
B. (2004). Working memory and novel word learning in children
with hearing impairment and children with specific language
impairment. International Journal of Language & Communica-
tion Disorders, 39, 401–422.
Hyde, M., & Power, D. (2003). Characteristics of deaf and hard-
of-hearing students in Australian regular schools: Hearing
level comparisons. Deafness and Education International, 5,
133–143.
Ibertsson, T., Hansson, K., Mäki-Torkko, E., Willstedt-Svensson, U.,
& Sahlén, B. (2009). Deaf teenagers with cochlear implants
in conversation with hearing peers. International Journal of
Language & Communication Disorders, 44, 319–337.
International Organization for Standardization. (2010). ISO 8253–1.
Acoustics—Audiometric test methods—Part 1: Pure-tone air and
bone conduction audiometry. Geneva, Switzerland: Author.
Kendon, A. (1967). Some functions of gaze-direction in social
interaction. Acta Psychologica, 26, 22–63.
Marschark, M., Convertino, C. M., Macias, G., Monikowski, C. M.,
Sapere, P., & Seewagen, R. (2007). Understanding communi-
cation among deaf students who sign and speak: A trivial
pursuit? American Annals of the Deaf, 152, 415–424.
Mirenda, P. L., Donnellan, A. M., & Yoder, D. E. (1983). Gaze
behavior: A new look at an old problem. Journal of Autism and
Developmental Disorders, 13, 397–409.
Most, T., Rothem, H., & Luntz, M. (2009). Auditory, visual, and
auditory-visual speech perception by individuals with cochlear
implants versus individuals with hearing aids. American Annals of
the Deaf, 154, 284–292.
Nadig, A., Lee, I., Singh, L., Bosshart, K., & Ozonoff, S. (2010).
How does the topic of conversation affect verbal exchange
and eye gaze? A comparison between typical development and
high-functioning autism. Neuropsychologia, 48, 2730–2739.
Nadig, A. S., & Sedivy, J. C. (2002). Evidence of perspective-taking
constraints in children’s on-line reference resolution. Psycho-
logical Science, 13, 329–336.
Nilsen, E. S., & Graham, S. A. (2009). The relations between
children’s communicative perspective-taking and executive
functioning. Cognitive Psychology, 58, 220–249.
Norbury, C. F., Brock, J., Cragg, L., Einav, S., Griffiths, H., &
Nation, K. (2009). Eye-movement patterns are associated
with communicative competence in autistic spectrum dis-
orders. Journal of Child Psychology and Psychiatry, 50,
834–842.
Raven, J., Raven, J. C., & Court, J. H. (2004). Manual for Raven’s
Progressive Matrices and Vocabulary Scales. Section 3: Stan-
dard Progressive Matrices: 2000 edition, updated 2004. San Antonio,
TX: Pearson.
Sandgren, O., Andersson, R., van de Weijer, J., Hansson, K., &
Sahlén, B. (2012). Timing of gazes in child dialogues: A time-
course analysis of requests and back channelling in referential
communication. International Journal of Language & Com-
munication Disorders, 47, 373–383.
Sandgren, O., Ibertsson, T., Andersson, R., Hansson, K., &
Sahlén, B. (2011). “You sometimes get more than you ask for”:
Responses in referential communication between children and
adolescents with cochlear implant and hearing peers. Inter-
national Journal of Language & Communication Disorders, 46,
375–385.
Skelt, L. (2006). See what I mean: Hearing loss, gaze and repair in
conversation (Unpublished doctoral dissertation). Australian
National University, Canberra.
Stinson, M., & Antia, S. (1999). Considerations in educating deaf
and hard-of-hearing students in inclusive settings. Journal of
Deaf Studies and Deaf Education, 4, 163–175.
Toe, D. M., & Paatsch, L. E. (2010). The communication skills used
by deaf children and their hearing peers in a question-and-
answer game context. Journal of Deaf Studies and Deaf
Education, 15, 228–241.
Turkstra, L. S. (2005). Looking while listening and speaking:
Eye-to-face gaze in adolescents with and without traumatic brain
injury. Journal of Speech, Language, and Hearing Research, 48,
1429–1441.
Turkstra, L. S., Ciccia, A., & Seaton, C. (2003). Interactive be-
haviors in adolescent conversation dyads. Language, Speech, and
Hearing Services in Schools, 34, 117–127.
Willemsen-Swinkels, S. H. N., Buitelaar, J. K., Weijnen, F. G., &
van Engeland, H. (1998). Timing of social gaze behavior in
children with a pervasive developmental disorder. Journal of
Autism and Developmental Disorders, 28, 199–210.
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., & Sloetjes,
H. (2006, May). ELAN: A professional framework for multi-
modality research. Paper presented at the Fifth Interna-
tional Conference on Language Resources and Evaluation,
Genoa, Italy.
Woodhouse, L., Hickson, L., & Dodd, B. (2009). Review of visual
speech perception by hearing and hearing-impaired people:
Clinical implications. International Journal of Language &
Communication Disorders, 44, 253–270.
Yoshinaga-Itano, C., & Sedey, A. L. (1998). Language of early-
and later-identified children with hearing loss. Pediatrics,
102, 1161–1171.

Reproduced with permission of the copyright owner. Further reproduction prohibited without
permission.

Coordination of Gaze and Speech in Communication Between Children With Hearing Impairment and Normal-Hearing Peers

Recommended

Recommended

More Related Content

Similar to Coordination of Gaze and Speech in Communication Between Children With Hearing Impairment and Normal-Hearing Peers

Similar to Coordination of Gaze and Speech in Communication Between Children With Hearing Impairment and Normal-Hearing Peers (20)

More from haniiszah

More from haniiszah (6)

Recently uploaded

Recently uploaded (20)

Coordination of Gaze and Speech in Communication Between Children With Hearing Impairment and Normal-Hearing Peers