Quantitative Journal 1

ReCALL 26(1): 44–61. 2013 r European Association for Computer Assisted Language Learning 44
doi:10.1017/S0958344013000219
First published online 21 November 2013
Captions and reduced forms instruction: The
impact on EFL students’ listening
comprehension
JIE CHI YANG
Graduate Institute of Network Learning Technology,
National Central University, Taiwan
(email: yang@cl.ncu.edu.tw)
PEICHIN CHANG
Department of English, National Taiwan Normal University, Taiwan
(email: peichinchang@ntnu.edu.tw)
Abstract
For many EFL learners, listening poses a grave challenge. The difficulty in segmenting a
stream of speech and limited capacity in short-term memory are common weaknesses for
language learners. Specifically, reduced forms, which frequently appear in authentic informal
conversations, compound the challenges in listening comprehension. Numerous interventions
have been implemented to assist EFL language learners, and of these, the application of
captions has been found highly effective in promoting learning. Few studies have examined
how different modes of captions may enhance listening comprehension. This study proposes
three modes of captions: full, keyword-only, and annotated keyword captions and investigates
their contribution to the learning of reduced forms and overall listening comprehension.
Forty-four EFL university students participated in the study and were randomly assigned to
one of the three groups. The results revealed that all three groups exhibited improvement on the
pre-test while the annotated keyword caption group exhibited the best performance with the
highest mean score. Comparing performances between groups, the annotated keyword caption
group also emulated both the full caption and the keyword-only caption groups, particularly in
the ability to recognize reduced forms. The study sheds light on the potential of annotated
keyword captions in enhancing reduced forms learning and overall listening comprehension.
Keywords: annotation, captions, EFL, listening comprehension, reduced forms
1 Introduction
Listening poses grave challenges to English as foreign language (EFL) learners. In
order to comprehend, the learner must immediately process and segment an ongoing
stream of sounds. Written text, by contrast, allows readers to rely on word boundaries
to segment a stream of speech. What prevents EFL learners from comprehen-
ding speech is their weak ability to segment speech and recognize words (Chen, 2002;

Chen & Chang, 2011; Goh, 2000; Sun, 2002). Generally, second language (L2) or
EFL students possess a limited capacity in their short-term memory and tend
to quickly forget what was heard, as they feel rushed to comprehend the new input
(Goh, 2000).
Some researchers have referred to reduced forms, commonly found in spoken
English, as the hurdle to comprehension (Bowen, 1975; Brown & Hilferty, 1986; Ito,
2001; Rosa, 2002; Weinstein, 2001). Natural English is found to be replete with
reduced forms, perhaps more so in informal than in formal language use (Rosa,
2002). Therefore, reduced forms instruction has been considered critical to listening
comprehension. Reduced forms are defined as the ‘‘phonological simplifications
or variations’’ (Ito, 2001) and depict ‘‘the spontaneous pronunciation changes in
adjacent words or sounds spoken at a natural speed’’ (Bowen, 1975). Collectively,
reduced forms designate ‘‘the processes of contraction, elision, assimilation, and
reduction’’ (Brown & Hilferty, 1986).
Listening instructions, however, do not often prepare the students well for
authentic listening where reduced forms are frequent. Chaudron (1988) argued
that in-class modifications (in four aspects: phonological, lexical, syntactical, and
discursive) of natural speech widen the gap between in-class talk and informal and
natural talk. Fan (1993) also indicated that students in Asia were more predisposed,
by the prevalent teaching practice, to reading than to listening. To prevent widening
the gap in performance between reading and listening, Guillory (1998) indicated that
L2 listening instruction should refrain from over-facilitation of full captioning, and
keyword captioning might be a better alternative to promote listening comprehen-
sion where the learners can ‘‘listen more and read less’’ (op. cit.: 104). Despite the
importance of teaching reduced forms, more needs to be done in terms of research
and pedagogical material development (Rosa, 2002). The current study is a response
to a need for the facilitation of authentic listening instruction, aided by different
modes of caption interventions, that is, full caption, keyword-only (i.e., reduced-
form keywords) caption, and annotated keyword caption. The study investigated to
what extent the different modes of caption use centered on reduced forms benefit
students’ listening comprehension.
2 Literature review
Past research has investigated how different means of aid, mostly textual, impacted
listening comprehension. Five major strands in the literature were identified:
(1) captions and subtitles as the main intervention in listening instruction, (2) caption
use and proficiency level in listening instruction, (3) different forms of textual aids in
listening instruction, (4) language used in subtitles and (5) bimodal or multimodal
scaffold in listening comprehension.
The studies reviewed here show mixed results regarding the use of textual aids.
While a number of studies elucidated the positive impact of applying textual
aids, e.g., subtitles and transcripts, other studies did not find equally compelling
evidence. Similarly, while some studies encouraged multi-modal presentation of
listening lessons or tasks, some cautioned against it for concerns of possible cognitive
overload.
Captions and reduced forms instruction 45

2.1 Captions/subtitles as the main intervention in listening instruction
During the 1990s, a few researchers confirmed the benefits of captions and subtitles
in listening performance. The presence of subtitles, in Markham’s study (1999), was
found to significantly improve students’ ability to recognize words in the video and
in a later listening-only task. Guillory (1998) included three different caption modes,
i.e., full-text, keyword, and no caption, in listening instruction, and indicated that the
full-text caption group outperformed the other two even though the gap in perfor-
mance between full and keyword groups was insignificant. Guillory proposed that
the keyword version may have assisted the learners in much the same way as full
caption did but additionally, keyword captions could focus learners’ processing
capacity on the auditory channel and enhance listening comprehension.
More recently, Winke, Gass and Sydorenko (2010) found captions contributed
better to learning than no captions with regard to novel vocabulary recognition and
overall comprehension. When captions were present for the learners’ first viewing, a
better performance was observed. Caption use seemed also to benefit learners across
a range of proficiency levels on condition that the content and complexity were
attuned to the learners. Chen (2011) developed synchronized subtitles for EFL
college learners and found the intervention assisted the learners to summarize the key
points in the video better than those who did not have subtitles. Huang and Eskey
(2000) investigated the effect of captions on ESL students’ listening comprehension
and found that the application helped the students in terms of listening, general
comprehension, and vocabulary acquisition.
These studies have all established how captions or subtitles enhanced listening
comprehension, summarizing key points in listening tasks or word recognition.
While all the studies focused on captions or subtitles as the main medium of assis-
tance, Guillory (1998) further investigated different modes of captions and their
impact on listening and found that full and keyword captions had better potential to
enhance learning. In particular, keyword captions may assist in a way that focuses
listeners’ processing capacities on the auditory channel.
2.2 Caption use and proficiency level in listening instruction
The relationship between caption use and students’ proficiency levels has been
explored in some studies. Taylor (2005) found that captions benefited third-year
college students more than first-year students to whom the co-existence of sound,
image, and caption was distracting. Grgurovic´ and Hegelheimer (2007) found that
the higher proficiency group used more subtitles for a longer period of time and the
lower proficiency group used subtitles the least. They recommended the provision of
subtitles, so that students can be given more control to access the kind of help they
need to pace their learning. By contrast, Pujola (2002) observed that lower profi-
ciency students accessed more textual help both in terms of transcripts and scripts
whereas higher proficiency students use more rewind/replay functions.
Lwo and Lin (2012) compared four different types of captions (no captions,
Chinese captions, English captions, and Chinese plus English captions) and their
effects on students of different proficiency levels in a multimedia learning context
46 J. C. Yang and P. Chang

and demonstrated that the effect of different captions varied depending on students’
proficiency levels. Lower proficiency students performed better with the presence of
L2 (English) captions or L1 plus L2 captions than with no captions. Quite similarly,
Rooney (201 also observed better performance with lower proficiency students when
captions were implemented. In the study, three types of keyword captions (10%,
30% and 50% of the total scripts) were shown to 90 EFL learners of three profi-
ciency levels: elementary, intermediate and advanced. Significant improvement was
observed only in the elementary group, who viewed partial captions accounting for
50% of the total scripts. In terms of perception, the participants generally preferred
30% captions presented to them.
These studies revealed the importance of taking learners’ proficiency levels into
account because learners responded and performed differently when provided with
different types of captions. To cater to the needs of learners of different proficiency
levels, different designs and modes of delivery should therefore be carefully considered.
2.3 Different forms of textual aids in listening instruction
A number of studies combined more than one textual aid to investigate the different
effects. Studies conducted earlier have demonstrated the importance of applying
textual aids, including captions and transcripts. Chung (1999) introduced two
interventions, captions and advance organizers1
(AOs), and found that captions
played a more critical role in enhancing students’ comprehension, while the two
combined stimulated even better performance. Jafari and Hashim (2012) also utilized
AOs in the form of two types of written aids, key sentences (sentences which show
the gist of the chosen passages) and key vocabulary (difficult vocabulary and idio-
matic expressions critical to understanding the passages) to assist EFL learners in
listening comprehension. The two experimental groups which received each textual
aid respectively were found to improve more than those in the control group which
received no such intervention; however, no difference was observed in learners
of different proficiency levels. Hsu (1994) studied textual repetition in the form of
transcript, among other helps provided (audio repetition and a dictionary), and
found that transcripts were the most frequently accessed aids. A positive correlation
between the access of transcripts and listening comprehension scores was observed.
Students also conveyed their preference in using this aid. Grgurovic´ and Hegelheimer
(2007) offered not only subtitles but also transcripts as help options, revealing that
subtitles were more frequently used for longer periods of time than transcripts while
performance was affected by students’ proficiency levels.
Jones and Plass (2002) applied both textual and pictorial annotations to aid with
listening tasks and found that, in terms of both comprehension and vocabulary
acquisition, the two aids combined achieved better results. However, in delayed test,
students performed better with pictorial annotations available, independent of tex-
tual aids. Similarly, Maleki and Rad (2011) investigated whether ‘‘multidimensional
representation of input’’ (op. cit.: 28), on one hand, condensed and paraphrased
1
Advance organizer can be defined as the application of visuals or texts which offer a
structure in a way that can connect the learners’ prior knowledge to the new materials.

scripts, and on the other, static image (both complemented by listening texts and
verbal stimuli), may enhance listening comprehension in students of different pro-
ficiencies. They revealed that static images contributed to better performance than
scripts, and that visual aids were particularly beneficial to lower proficiency students
while textual aids were more advantageous to higher proficiency students.
Overall, these studies shed light on the benefits of introducing different textual aids
to stimulate listening comprehension. Of these, captions and subtitles enhanced
learning more than transcripts. And combining both textual and pictorial aids was
more effective than single textual format. Pictorial aids were particularly helpful in
two aspects: to help retain knowledge and to cater to low proficiency learners.
2.4 Language used in subtitles
Subtitles, usually denoting ‘‘on-screen text in the native language of the viewers’’
(Markham & Peter, 2003: 332), were introduced in both L1 and L2 to investigate the
effects on listening comprehension. In the 1990s, Garza (1991) introduced Russian
subtitles, the L2, for advanced ESL learners and found a positive correlation
between the application and students’ performance. Borras and Lafayette (1994) also
integrated L2 subtitles for ESL college students and found that subtitles positively
impacted students’ performance both in terms of comprehension and production.
Most recent studies seem to support the findings in the 1990s. Hayati and Mohmedi
(2011) found subtitles in the target language (L2) better assisted EFL learners, com-
pared to subtitles in L1 and no subtitles. When introducing multi-modal help (sound,
image, and subtitles) in listening tasks, Guichon and McLornan (2008) found that
subtitles in L2 were more helpful than those in L1 in terms of reducing lexical inter-
ference when similar modalities were held constant. However, Markham and Peter’s
(2003) study revealed opposite results. They argued that subtitles in L1 better promoted
comprehension, while subtitles in L2 ranked second and no subtitles ranked third.
In all, these studies pointed out that providing and rendering subtitles in L2
contributes to listening comprehension more than no subtitles and subtitles in L1 do.
2.5 Bimodal or multimodal scaffold in listening comprehension
Some studies scrutinized the potential advantages of bimodal or multimodal appli-
cations in listening instruction and how different modes of input afforded specific
listening tasks. Two studies conducted in the 1990s revealed conflicting findings
about bimodal scaffolding. Baltova (1999) reported that bimodal use of sound and
subtitles in L2 benefited L2 learners, even the inexperienced ones. However, Pujola
(2002) argued that the simultaneous presence of audiovisual and textual aids was
negatively correlated with performance.
Recently, Hayati and Mohmedi (2011) indicated that listening comprehension
instruction may benefit from the bimodal input, listening while reading, which would
enhance learners’ recall and recognition of words in later tests. Guichon and
McLornan (2008) introduced four modalities to scaffold listening and found that
students exposed to multi-modalities, i.e., sound, image, and subtitles, tended to
perform better. Sydorenko (2010) examined the effect of different input modalities

on students’ vocabulary gains, listening and written recognition of word forms and
concluded that captioned video helped students perform better on written recognition of
word forms and gains of word meanings while non-captioned video assisted students to
score better on aural recognition. Chang, Lei, and Tseng (2011) confirmed that double
modes, text plus sound, were more effective than single mode listening instruction.
However, while textual aids resulted in enhanced listening comprehension along with
reducing cognitive load, the effect lasted only temporarily. As soon as the text aids faded
out, learners had a difficult time adjusting to a single mode environment.
While these studies, save one (Pujola, 2002), converge on the general benefits of
multimodal input on listening comprehension, they also suggested that different
modalities contributed in various ways. To optimize learning, the application of
different modalities is recommended but thoughtful scaffolding should also be in
place to help learners to adjust to different modalities.
The literature reviewed above largely converges on the benefits of applying textual
aids in promoting listening comprehension. A few variables have been tested,
including modality, language used, learners’ proficiency levels, types of captions, all
of which exert great influence on performance. In particular, captions and subtitles
were found to play a critical role in learners’ listening comprehension. More may be
done to investigate how different modes of captioning can stimulate performance.
Built on and informed by the literature, this study probes the use of full, keyword,
and annotated keyword captions (a format similar to pictorial presentations) to
distinguish the different effects each mode may bring. Instead of assuming general
listening competence, this study singled out reduced forms instruction which is
considered critical to listening comprehension (Brown & Hilferty, 1986; Ito, 2001;
Rosa, 2002) to implement the different modes of captions.
Given that different modes of textual aids contributed differently to performance
(e.g., Guillory, 1998; Jones & Plass, 2002; Maleki & Rad, 2011), this study sought to
investigate how three modes of captions, full, keyword-only, and annotated keyword
captions, may contribute to reduced forms learning and improve listening comprehen-
sion. This study is driven by two research questions. The first question explored whether
an overall improvement could be seen after the captioned intervention. The second
question probed more deeply into whether the improvement is variant, regarding the
four types of listening comprehension questions participants were tested on.
1. How are the different modes of captioned aids (full, keyword-only, and
annotated keyword captions) helpful in improving students’ overall listening
comprehension?
2. What types of listening comprehension question (dictation cloze, short
dialogue comprehension, reduced-form recognition, and reduced-form mark-
ing) do the different modes of captions help more in improving students’
performance?
3 Methods
Past research has shed light on how various modes and modalities may contribute to
listening comprehension differently. This study built on this by introducing three

modes of captions, in the form of full, keyword, and annotated keyword captions,
to ascertain the benefits of each mode in enhancing listening for reduced forms.
To answer the research questions, an empirical study was conducted based on an
experimental design using a quantitative approach.
3.1 Participants
The study recruited 44 EFL students (15 male and 29 female), aged 23 on average in
their undergraduate or graduate studies at a university in northern Taiwan. These
participants represented a rather homogeneous group in terms of the university they
came from and their English proficiency level (per self-report). The participants were
randomly assigned to one of the three groups: 15 in the full caption group, 15 in the
keyword-only caption group, and 14 in the annotated keyword caption group.
3.2 Instruments
3.2.1 Different modes of captions. The study developed a mechanism to auto-
matically render full captions in videos for learning. The captions come from four
sources, animation, cartoons, movies, and TV series, and were rendered into
full, reduced-form keyword, and annotated keyword captions, by way of Support
Vector Machine (SVM) techniques (Wu, Lee & Yang, 2008). As a result, a total of
51 video clips were stored in a database. The total duration of the 51 clips is 65 minutes
with each clip ranging approximately from 30 to 120 seconds, which is considered
an appropriate time span for learning (Thompson & Rubin, 1996). Overall, the
mechanism makes handling large amounts of aural data possible to cater to the needs
of listening instruction and, more specifically, reduced forms instruction.
Reduced-form keyword captions were categorized into four types: assimilation,
liaison, reduction, and elision. As an attempt to better assist learning the four reduced
forms, pictorial symbols were added to make the four types salient. The system first
searched words and phrases with reduced forms from the captions and then added
annotation symbols to them – a blue dot to mark a sound being assimilated, a yellow
curve to link sounds and indicate liaison, a grey letter to show reduced sound, and a
red slash to signify omitted sound and indicate elision. Table 1 shows the four reduced-
form symbols for annotated keyword captions.
The three groups of participants all watched the same clips and only the modes of
captions were different. Unlike full captions, keyword-only captions display keywords
when reduced forms are present while annotated keyword captions observe the same
principle in addition to attaching the four symbols to explicitly mark the four types of
reduced forms. Figure 1 shows the three modes of captions. From the top, the first
illustration is an example of full captions, the second shows keyword-only captions
and the third typifies annotated keyword captions, which adds symbols to the keyword
captions to make reduced forms more salient.
3.2.2 Listening comprehension test. A pre-test and a post-test were administered,
comprising the same listening comprehension content. The test was developed based
on a range of listening textbooks (Chao, 2005; Hagen & Grogan, 1992; Stone, 2008),
and was checked by a native speaking English teacher. Questions were recorded by

native speakers, who spoke in a natural way, presenting an authentic spoken lan-
guage context.
The listening comprehension test consists of 41 questions for a total of 104 points.
There are four types of question: dictation cloze, short dialogue comprehension,
reduced-form recognition, and reduced-form marking. Each question type includes
a mixture of the four types of reduced forms (assimilation, liaison, reduction, and
elision). The first type is dictation cloze, with fifteen questions, each worth two
points, accounting for thirty points in total. Participants were required to listen
and fill in the blank of each sentence. The answer may be more than one word.
The second type is short dialogue comprehension. There are twelve short dialogues,
each followed by a single multiple choice question worth two points, for a total of
24 points. Participants were required to listen to the dialogue and answer the
question and choose the best answer. The third type is reduced-form recognition,
which contains ten questions, each followed by a single multiple choice question, two
points per question and twenty points in total. Participants were required to choose
the correct reduced-form annotation based on what they had heard. The fourth type
includes four sentences to be marked by the participants. Each correct marking earns
them three points for a total score of thirty points. Unlike the third type, here
participants were required to listen, and mark each sentence with appropriate
reduced-form symbols without multiple choices presented. The listening compre-
hension test is a paper-and-pencil test and participants were required to answer each
question within the designated time (for a sample of the questions, please refer to
Appendix A).
Table 1 Symbols for annotated keyword captions
Types of reduced forms Examples Explanations
Assimilation blue dot: mark a sound being
assimilated
Liaison yellow curve: link sounds and
indicate liaison
Reduction grey letter: show reduced sound
Elision red slash: signify omitted sound
and indicate elision
Fig. 1. Examples of the three modes of captions (full, keyword, annotated keyword captions)

3.3 Procedure
This study consisted of four steps, illustrated in Figure 2. The entire procedure took
approximately 115 minutes. The steps were as follows.
Step 1: Listening comprehension test (pre-test): Participants completed a listening
comprehension test of approximately 15 minutes to assess their baseline
knowledge regarding reduced forms.
Step 2: Orientation and demo: Before the intervention, participants were required to
access a reduced-form instruction online, which provided definition and
examples of the four types of reduced forms. In addition, they were also given
a demonstration of how to use the tool properly. During the demonstration
process, the participants were encouraged to pose questions. As it turned out,
all the students familiarized themselves with the system and completed the
process in 20 minutes.
Step 3: Intervention: Participants were divided into three groups, who watched the
video clips with full, keyword-only, and annotated keyword captions
respectively. Except for different modes of captions, the sequence and the
number of the clips were all the same to ensure participants received equal
input. Once the video clip started playing, it could not be paused, and it
played through until the end. The intervention lasted about 65 minutes.
Step 4: Listening comprehension test (post-test): After watching the video clips, partici-
pants took the same listening comprehension test again for about 15 minutes.
3.4 Data analysis
To answer the first research question, a paired sample t-test, a one-way analysis of
covariance (ANCOVA), and a Fisher’s Least Significant Difference (LSD) post hoc test
were performed. The paired sample t-test was employed to compare the pre-test and post-
test scores of the three groups. To examine whether significant difference existed between
groups, a one-way analysis of covariance (ANCOVA) was employed. The independent
variable is the different groups, including full caption, keyword-only caption, and
annotated keyword caption groups. The dependent variable is the post-test score of the
listening comprehension test and the covariate is the participants’ score on the pre-test.
To further evaluate differences among the three groups, the Fisher’s LSD post hoc test
was carried out under the condition that the result of ANCOVA is significant.
To answer the second research question, a one-way ANCOVA and a Fisher’s LSD
post hoc test were again employed to examine how the three types of intervention
Listening comprehension test (pre-test)
Orientation and demo
Intervention (the instruction)
Listening comprehension test (post-test)
Fig. 2. The procedure of this study

impacted the participants’ post-test performance in the four question types (dictation
cloze, short dialogue comprehension, reduced-form recognition, and reduced-form
marking). The independent variable is the different groups. The dependent variable
is the post-test score of each question type and the covariate is the participants’ score
on the pre-test of each question type. Similarly, the Fisher’s LSD post hoc test was
carried out to further evaluate differences among the three groups.
4 Results and discussion
To compare the pre-test and post-test scores of the three groups, the paired-sample
t-test was applied. Table 2 shows the results in terms of mean scores (M), number of
participants per group (N), standard deviation (SD), and t value. The results indicate
that all the participants in the three groups exhibited significant improvement
(p , 0.001) from the pre-test. In particular, the annotated keyword caption group
demonstrated most significant improvement in the mean score by 16.93 while the full
caption group shows least increase by 10.54.
To examine whether a significant difference existed between groups, the one-way
ANCOVA was employed. The independent variable is the different groups. The
dependent variable is the post-test score and the covariate is the pre-test score. A
preliminary analysis evaluating the homogeneity-of-regression assumption indicates
that the relationship between the covariate and the dependent variable does not differ
significantly as a function of the independent variable (F(2, 38) 5 .820, p5 .448), and
Levene’s test does not differ significantly (F(2, 41) 5 .323, p 5 .726). The ANCOVA is
significant (F(2, 38) 5 5.348, p , .01) (see Table 3). The Fisher’s LSD post hoc test was
carried out to further evaluate differences among the three groups. The results indicate
a significant difference in performance between the annotated keyword caption group
and the full caption group (p , .05). This confirms previous findings, which suggest that
full captions or scripts may not be all beneficial. For example, Chang et al. (2011)
argued that while textual aids stimulated better performance the effect was temporary.
Once such scaffold faded out, students had difficulty adjusting to authentic listening
tasks. Preparing students with keyword captions might be a more profitable alternative
not only in facilitating learning but also to ease the transitional phase to authentic
listening tasks. Rooney (2011) demonstrated that implementing partial captions was
effective but while learners performed better with 50% than 10% of the scripts, they
Table 2 Pre-test and post-test scores for the three groups
Groups Tests M N SD t
Full caption Pre-test 45.53 15 22.24 25.907***
Post-test 56.07 15 22.15
Keyword-only caption Pre-test 40.87 15 17.39 26.843***
Post-test 53.87 15 20.90
Annotated keyword caption Pre-test 45.64 14 15.24 28.279***
Post-test 62.57 14 17.48
***
p , .001

recommended even fewer captions at the proportion of 30% of total scripts. Our
finding also resonated with Jones and Plass (2002) and Maleki and Rad (2011) in terms
of the efficacies of introducing pictorial aids. Our study implemented annotated key-
word captions that were found to facilitate both reduced forms learning and overall
listening comprehension. Guillory (1998) also suggested that keyword captions have
better potential to focus listeners on listening rather than on reading off the screen.
To examine how the three types of intervention impacted the participants’ post-
test performance in the four question types, the one-way ANCOVA was again
employed to each question type. The independent variable is the different groups.
The dependent variable is the post-test score and the covariate is the pre-test score
of each question type. For dictation cloze, a preliminary analysis evaluating
the homogeneity-of-regression assumption indicates that the relationship between
the covariate and the dependent variable does not differ significantly as a function
of the independent variable (F(2, 38) 5 .053, p 5 .949), and Levene’s test does not
differ significantly (F(2, 41) 5 2.879, p 5 .068). The ANCOVA is not significant
(F(2, 38) 5 1.005, p 5 .376) (see Table 4), indicating no significant difference among
the three groups in the question type of dictation cloze.
In terms of short dialogue comprehension, again, the relationship between the
covariate and the dependent variable does not differ significantly as a function of
the independent variable (F(2, 38) 5 .166, p 5 .848), and Levene’s test does not
Table 3 Analysis of covariance for test score by different groups
Source
Sums of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Post hoc
(LSD)
Group 588.337 2 294.168 5.348 .009 .220 Annotated
. Full
Pre-test(covariate) 14779.628 1 14779.628 268.692 .000 .876
group*pre-test 90.247 2 45.123 .820 .448 .041
Error 2090.221 38 55.006
Total 162449.000 44
R Squared 5 .881 (Adjusted R Squared 5 .865)
Annotated 5 Annotated keyword caption group, Full 5 Full caption group
Table 4 Analysis of covariance for dictation cloze by different groups
Source
Sums of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
group 17.224 2 8.612 1.005 .376 .050
group*pre-test .900 2 .450 .053 .949 .003
Error 325.668 38 8.570
Total 12479.000 44

differ significantly (F(2, 41) 5 .885, p 5 .420). The ANCOVA is not significant (F(2,
38) 5 .173, p 5 .841) (see Table 5). Similar to dictation cloze questions, no significant
difference is found when comparing the performances of the three groups in the
question type of short dialogue comprehension.
For reduced-form recognition, similarly, the relationship between the covariate and
the dependent variable does not differ significantly as a function of the independent
variable (F(2, 38) 5 1.579, p 5 .219), and Levene’s test does not differ significantly
(F(2, 41) 5 2.795, p 5 .073). The ANCOVA is significant (F(2, 38) 5 4.193, p, .05) (see
Table 6). To further examine differences among the three groups, the Fisher’s LSD
post hoc test was performed. The results show that a significant difference exists
between the annotated keyword caption group and the keyword-only caption group
(p , .01). In other words, students performed better in the annotated keyword caption
group than in the keyword-only caption group.
Lastly, in the reduced-form marking question type, again, the relationship between
the covariate and the dependent variable does not differ significantly as a function of
the independent variable (F(2, 38) 5 2.040, p 5 .144), and Levene’s test does not differ
significantly (F(2, 41) 5 .074, p 5 .928). The ANCOVA is significant (F(2, 38) 5 5.426,
p , .01) (see Table 7). To further evaluate differences among the three groups, the
Fisher’s LSD post hoc test was employed. The results show a significant difference
between the annotated keyword caption group and the full caption group (p , .05).
Table 5 Analysis of covariance for short dialogue comprehension by different groups
Source
Sums of
Squares df
Mean
Square F Sig.
Partial
Eta Squared
Group 3.887 2 1.943 .173 .841 .009
group*pre-test 3.716 2 1.858 .166 .848 .009
Error 425.703 38 11.203
Total 9956.000 44
Table 6 Analysis of covariance for reduced-form recognition by different groups
Source
Sums of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Post hoc
(LSD)
Group 72.476 2 36.238 4.193 .023 .181 Annotated
.Keyword
group*pre-test 27.301 2 13.650 1.579 .219 .077
Error 328.412 38 8.642
Total 9184.000 44
Annotated 5 Annotated keyword caption group, Keyword 5 Keyword-only caption group

In other words, students performed better in the annotated keyword caption group
than in the full caption group.
Overall, the results indicate that all the three groups improved from pre-test after
the intervention. When comparing the mean score of the three groups, the annotated
keyword caption group held the highest mean score over the other two groups. When
comparing the performance between groups, the annotated keyword caption group
significantly outperformed those in the full caption group. Furthermore, among the
four question types, the annotated keyword caption group excelled in reduced-form
recognition when compared to the keyword-only caption group and in the reduced-
form marking versus the full caption group.
Generally, the improvement made on reduced-form recognition both by way of
annotated keyword captions and keyword captions was found to positively impact
listening comprehension. Specifically, annotated keyword captions were found to
enhance learners’ ability to recognize reduced forms better than both keyword captions
and full captions, potentially resulting from the addition of the pictorial symbols which
contributed to their knowledge retention. While past research has investigated the
effect of different kinds of textual aids on listening comprehension, few have combined
pictorial symbols and captions to ascertain the maximum effect of captions in listening
comprehension given the prevalence of captions in language teaching. Many studies
probed the benefits of different modalities, for example, pictorial symbols versus text,
but not exploring different modes of the same modality which this study investigated.
Overall, our study has found that more textual scaffolds, caption and symbols
combined, contributed to better listening comprehension while implementing full
captions is the least helpful of the three modes of captions. In addition to decreasing
the number of captions, which has been tested and recommended by scholars (e.g.,
Guillory, 1998; Rooney, 2011), our study further affirms the importance of inserting
pictorial modes to enhance reduced forms performance which has direct bearing on
general listening comprehension ability.
5 Conclusion and implications
The current study explores how different types of captions, full, keyword-only,
and annotated keyword captions, impact EFL students’ listening comprehension.
Table 7 Analysis of covariance for reduced-form marking by different groups
Source
Sums of
Squares df
Mean
Square F Sig.
Partial Eta
Squared
Post hoc
(LSD)
Group 262.146 2 131.073 5.426 .008 .222 Annotated
.Full
group* pre-test 98.538 2 49.269 2.040 .144 .097
Error 917.873 38 24.155
Total 11410.000 44
Annotated 5 Annotated keyword caption group, Full 5 Full caption group

Informed by past studies, which proposed that reduced forms instruction is critical to
improving listening comprehension, annotated keyword captions were developed
specifically to alert learners to the different types of reduced forms by way of
pictorial symbols. Previous studies have demonstrated a positive outcome when
introducing captions or subtitles to enhance listening comprehension (Chung, 1999;
Grgurovic´ & Hegelheimer, 2007; Huang & Eskey, 2000; Jones & Plass, 2002;
Markham, 1999; Winke et al., 2010). However, few investigated whether enhanced
visual effect to keyword captions, i.e., annotated keyword captions, may enhance
learning outcome further. This study revealed that annotated keyword captions not
only helped the participants to score higher in listening comprehension test but this
mode of caption also assisted learners to perform better in both recognizing
and marking reduced forms, which is positively related to their overall listening
comprehension performance.
The results suggest that reduced forms instruction has the potential to help EFL
learners to improve overall listening comprehension. In reduced forms instruction,
keyword captions may be more helpful than full captions and the captions are best
executed with reduced-form relevant pictorial symbols. Doing this may enhance
learners’ visual memory and retain their knowledge, resulting in better listening
comprehension performance.
There are a few limitations to the present study. Given the short-term training and
instruction, salient difference may not be readily observable. First, the participants
may resort to their short-term memory in answering questions. To address this,
a delayed test may be implemented to examine whether better performance resul-
ting from watching annotated keyword captions remains constant. Second, given
that this study recruited a relatively small sample of participants, generalization is
unlikely regarding how the three types of captions may be distinguished in effect. For
future research, larger sample and a long-term treatment may provide additional
evidence and expand understanding in listening comprehension instruction. Finally,
while participants in the study shared homogeneous academic background in terms
of the university they came from and their English proficiency level (per self-report),
for future research, participants could be grouped by alternative variables such as
gender, proficiency levels (subjected to pre-test before participation), and so forth
before the intervention.
The study has found that annotated keyword captions may be beneficial in terms
of assisting with the learning of reduced forms. Improvement in reduced forms
performance has also been positively linked to overall listening comprehension. For
future pedagogical design, less caption is recommended. Specifically, annotated
keyword captions, enhanced with pictorial symbols, may better focus learners’
attention and improve their listening comprehension.
Acknowledgements
The authors would like to thank Ms. Jia Fen Liu for assisting in the system develop
ment. The authors would also like to thank all the subjects who participated in
the study. This study was partially supported by grant (NSC 100-2628-S-008-002-
MY3) from the National Science Council of Taiwan.

References
Baltova, I. (1999) Multisensory language teaching in a multidimensional curriculum: The use
of authentic bimodal video in core French. Canadian Modern Language Review, 56(1):
32–48.
Borras, I. and Lafayette, R. C. (1994) Effect of multimedia courseware subtitling on the
speaking performance of college students of French. The Modern Language Journal, 78:
61–75.
Bowen, J. D. (1975) Patterns of English Pronunciation. Rowley, Mass: Newbury House.
Brown, J. D. and Hilferty, A. G. (1986) The effectiveness of teaching reduced forms for
listening comprehension. RELC Journal, 17(2): 59–70.
Chang, C. C., Lei, H. and Tseng, J. S. (2011) Media presentation mode, English listening
comprehension and cognitive load in ubiquitous learning environments: Modality effect or
redundancy effect? Australasian Journal of Educational Technology, 27(4): 633–654.
Chao, Y. C. (2005) Say It Right. Taipei: Di-Teng Publishing.
Chaudron, C. (1988) Second Language Classrooms: Research on Teaching and Learning.
Cambridge: Cambridge University Press.
Chen, H. J. (2011) Developing and evaluating SynctoLearn, a fully automatic video and
transcript synchronization tool for EFL learners. Computer Assisted Language Learning,
24(2): 117–130.
Chen, S. W. (2002) Problems in listening comprehension for learners of EFL. Studies in
English Language and Literature, 10: 57–70.
Chen, I. J. and Chang, C. C. (2011) Content presentation modes in mobile language listening
tasks: English proficiency as a moderator. Computer Assisted Language Learning, 24(5):
451–470.
Chung, J. M. (1999) The effects of using video texts supported with advance organizers and
captions on Chinese college students’ listening comprehension: An empirical study. Foreign
Language Annals, 32(3): 296–308.
Fan, Y. (1993) Listening: Problems and solutions. English Teaching Forum, 31(1): 16–19.
Garza, T. J. (1991) Evaluating the use of captioned video materials in advanced foreign
language learning. Foreign Language Annals, 24(3): 239–258.
Goh, C. C. M. (2000) A cognitive perspective on language learners’ listening comprehension
problems. System, 28: 55–75.
Grgurovic´ , M. and Hegelheimer, V. (2007) Help options and multimedia listening: Students’
use of subtitles and the transcript. Language Learning & Technology, 11(1): 45–66.
Guichon, N. and McLornan, S. (2008) The effects of multimodality on L2 learners: Implications
for CALL resource design. System, 36(1): 85–93.
Guillory, H. G. (1998) The effects of key word captions to authentic French video in foreign
language instruction. CALICO Journal, 15(1-3): 89–108.
Hagen, S. A. and Grogan, P. E. (1992) Sound Advantage: A Pronunciation Book (1st ed.).
Englewood Cliffs, NJ: Prentice Hall Regents.
Hayati, A. and Mohmedi, F. (2011) The effect of films with and without subtitles
on listening comprehension of EFL Learners. British Journal of Education Technology,
42(1): 181–192.
Hsu, J. (1994) Computer assisted language learning (CALL): The effect of ESL students’ use
of interactional modifications on listening comprehension, Unpublished doctoral disserta-
tion, Iowa State University, Ames, IA.
Huang, H. and Eskey, D. (2000) The effects of closed-captioned television on the listening
comprehension of intermediate English as a second language students. Educational Tech-
nology Systems, 28: 75–96.

Ito, Y. (2001) Effect of reduced forms on ESL learners’ input-intake process. Second Language
Studies, 20(1): 99–124.
Jafari, K. and Hashim, F. (2012) The effects of using advance organizers on improving EFL
learners’ listening comprehension: A mixed method study. System, 40(2): 270–281.
Jones, L. C. and Plass, J. L. (2002) Supporting listening comprehension and vocabulary
acquisition in French with multimedia annotations. The Modern Language Journal, 86(4):
546–561.
Lwo, L. and Lin, M. C. T. (2012) The effects of captions in teenagers’ multimedia L2 learning.
ReCALL, 24(2): 188–208.
Maleki, A. and Rad, M. S. (2011) The Effect of Visual and Textual Accompaniments
to Verbal Stimuli on the Listening Comprehension Test Performance of Iranian
High and Low Proficient EFL Learners. Theory and Practice in Language Studies, 1(1):
28–36.
Markham, P. L. (1999) Captioned videotapes and second-language listening word recognition.
Foreign Language Annals, 32(3): 321–328.
Markham, P. L. and Peter, L. (2003) The influence of English language and Spanish language
captions on foreign language listening/reading comprehension. Journal of Educational
Technology Systems, 31(3): 331–341.
Pujola, J. T. (2002) CALLing for help: Researching language learning strategies using help
facilities in a web-based multimedia program. ReCALL, 14(2): 235–262.
Rooney, K (2011) Impact of keyword caption ratio, language proficiency, and attitude on foreign
language listening comprehension, Unpublished doctoral dissertation, Capella University,
Minneapolis.
Rosa, M. (2002) Don’cha know? A survey of ESL teachers’ perspectives on reduced forms
instruction. Second Language Studies, 21(1): 49–78.
Stone, T. (2008) Listen and Speak Out. Taipei: Di-Teng Publishing.
Sun, K. C. (2002), Investigation of English listening difficulties of Taiwan students. Pro-
ceedings of the 11th
Conference on English Teaching and Learning in the Republic of China,
518–525, Taipei, Crane.
Sydorenko, T. (2010) Modality of input and vocabulary acquisition. Language Learning &
Technology, 14(2): 50–73.
Taylor, G. (2005) Perceived processing strategies of students watching captioned video. Foreign
Language Annals, 38(3): 422–427.
Thompson, I. and Rubin, J. (1996) Can strategy instruction improve listening comprehension?
Foreign Language Annals, 29: 331–342.
Weinstein, N. J. (2001) Whaddaya say?: Guided practice in relaxed speech (2nd ed.). White
Plains, NY: Longman.
Winke, P., Gass, S. and Sydorenko, T. (2010) The effects of captioning videos used for foreign
language listening activities. Language Learning & Technology, 14(1): 65–86.
Wu, Y. C., Lee, Y. S. and Yang, J. C. (2008) Robust and efficient multiclass SVM models for
phrase pattern recognition. Pattern Recognition, 41(9): 2874–2889.

Appendix A
Type I Dictation cloze (in total 15 questions)
1. Let’s leave in an hour.
2. Do you want the water hot or cold
3. We’d
?
better dress up for the party tonight.
Type II Short dialogue comprehension (in total 12 questions)
1. W: Look at Sue’s baby.
M: Before you know it, she’ll be a year old.
Q: What does the man mean?
a) The baby is almost a year old.
b) The baby was a year old before.
c) You’ll know when she’s a year old.
d) The baby’s older than a year.
The correct answer is (a).
2. W: I ran for 45 minutes without stopping.
M: No wonder you’re so worn out.
Q: What does the man mean?
a) He’s not surprised she’s tired.
b) He’s wondering what she wore.
c) He’s surprised she was out.
d) He’s wondering why she’s exhausted.
3. M: Are you going to the jazz festival?
W: I wouldn’t miss it for the world. It’s supposed to be wonderful.
Q: What does the woman mean?
a) She’s sorry she’s going to miss it.
b) She plans on attending.
c) She’s going to something else wonderful.
d) She’s heard it’s the best in the world.
The correct answer is (b).
:
:

Type III Reduced-form recognition (in total 10 questions)
1. I have a sweet tooth.
2. I’m not skilled at sports.
3. I took an international flight to France!
The correct answer is (c).
Type IV Reduced-form marking (in total 4 questions)
1. He works on weekends.
2. A piece of cake.
Answers:

Reproduced with permission of the copyright owner. Further reproduction prohibited without
permission.

Quantitative Journal 1

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to Quantitative Journal 1

Similar to Quantitative Journal 1 (20)

More from amirahjuned

More from amirahjuned (6)

Quantitative Journal 1