This document summarizes a meta-analysis on the effects of multimedia on early literacy development in children at risk. It found that multimedia had a small to moderate positive effect on literacy outcomes like letter learning, phonological awareness, and comprehension. Effects were larger for children from low-SES backgrounds, second language learners, and those at risk for reading failure. Computer applications and e-books had relatively large effects, while effects were greatest for letter learning, comprehension, and phonological awareness outcomes.

1. The Effects of Multimedia
on Early Literacy
Development of Children
at Risk: A Meta-Analysis
Victor van Daal and
Jenny Miglis Sandvik

2. The Reading Centre
www.lesesenteret.no
Synthesis of Multimedia Effects on Literacy
Achievements in Children at Risk
Victor H.P. van Daal*
&
Jenny Miglis Sandvik
Reading Centre, University of Stavanger, Norway
Presented at EETC 2012
*In 2010-2011 Fellow-in-Residence at
Netherlands Institute of Advanced Study
victor.v.daal@uis.no & jenny@sandviks.com
1

3. The Reading Centre
www.lesesenteret.no
Introduction:
Redefinition of literacy
• In the old days: ability to read and write.
• Now we have multimedia: integration of text, images and sound,
presented electronically/digitally.
• Examples: TV, DVDs, computer software, electronic books, talking
books, internet, video games, smart phone apps, interactive toys,
and more.
• Literacy is now: ability to communicate through multimedia.
• Multimedia more efficient than verbal/written delivery of
instruction:
– NAEYC (National Association for the Education of Young People), 1996:
• ‘used appropriately, technology can enhance children’s cognitive and social
abilities’
• ‘computers should be integrated into early childhood practice physically,
functionally, and philosophically’
2

4. The Reading Centre
www.lesesenteret.no
Introduction:
Theory on multimedia
• Pro
– Dual coding (visual and auditory) results in enhanced
comprehension (Sadoski & Paivio, 2007).
– Supports ‘children of the digital age’ (Marsh, 2005).
• Con
– Use of technology is developmentally inappropriate, cognitive
overload (Kirschner, 2002).
– Teacher resistance to incorporating technology into lessons
(Turbill, 2001).
– It costs a lot, but produces little (Yelland, 2005).
– Use of technology undermines the very nature of childhood, ‘death
of childhood’ (Buckingham, 2000).
• Overviews: Buckingham, 2000; Lankshear & Knobel, 2003;
Stephen & Plowman, 2003.
3

5. The Reading Centre
www.lesesenteret.no
Introduction:
The practice
• Rideout & Hamel, 2006:
– 1,000 American households with children under age 6
surveyed.
– in a typical day, 83% use some type of screen media.
– 27% reports that their children use a computer several
times a week or more.
– 69% felt computers helped their children’s learning.
• Therefore, in-depth research on the topic is long
overdue.
4

6. The Reading Centre
www.lesesenteret.no
Review of research (I)
• Kamil, Intractor & Kim (2000) reviewed 350 articles
on the effects of multimedia on literacy:
– only few related to early literacy.
– multimedia facilitate comprehension through ‘mental
model building’.
– children who come from language and cultural minority
backgrounds can benefit from multimedia.
• Lankshear & Knobel (2003) found only 22 articles
focusing on young children:
– majority of these 22 studies dealt with decoding.
– effects of technology on early literacy development were
‘radically under-researched’.
5

7. The Reading Centre
www.lesesenteret.no
Review of research (II)
• Burnett (2009) reviewed 22 quantitative and 16
qualitative studies on technology and literacy:
– technology was used in the same way as traditional print
teaching methods.
– strengths of multimedia were not exploited at all.
– therefore, effects difficult to ascertain.
• Zucker, Moody & McKenna (2009) looked at effects of E-
books in 7 randomised trial studies and 20 narrative:
– small to medium effect sizes for comprehension.
– effect on decoding could not be assessed (only 2 studies).
– mixed results in narrative studies: overall positive, but
sometimes more time was spent on games than
educational content (De Jong & Bus, 2002).
6

8. The Reading Centre
www.lesesenteret.no
Review of research (III - CAI)
• Mainframe computers were too expensive and too slow
(Fletcher & Atkinson, 1972; Krendl & Williams, 1990; Slavin,
1991).
• Meta-analytic studies found effect sizes of 0.25 (SE = 0.07),
Kulik & Kulik (1991) and 0.16 (SE = 0.08), Ouyang (1993).
• Qualitative studies:
– Torgesen & Horen (1992): computer should be integrated with
teacher-driven curriculum.
– Van der Leij (1994): concentrating on a specific subskill is more
effective.
– Wise & Olson (1998): talking computers should be combined with
PA training.
– National Reading Panel (2000): talking computers promising (20
studies reviewed).
7

9. The Reading Centre
www.lesesenteret.no
Review of research (IV - CAI continued)
• Blok, Oostdam, Otter & Overmaat (2002):
– 45 studies with 75 experimental conditions.
– overall effect size: 0.254 (SE = 0.056).
– variance of effect size could be explained by:
• effect size of pretest: 34%.
• language of instruction: 27%, English-medium studies
0.319 SD more effective than non-English (Dutch and
Danish).
– overall disappointing, especially as in all studies children
at risk of literacy underachievement took part.
• Has CAI become more effective over the last decade?
8

10. The Reading Centre
www.lesesenteret.no
Questions of the current study
• 1. Can multimedia facilitate the early literacy
development of young children (0-8 years old) at risk of
literacy underachievement (low progress in reading, low-
SES background, or Second Language learners)?
• 2. Which literacy-related learning outcomes are most
influenced by the use of multimedia?
• 3. Which multimedia are more effective?
• 4. Are there any multimedia X literacy outcome
interactions?
• 5. What works in multimedia? How do parameters of
multimedia affect effect sizes?
9

11. The Reading Centre
www.lesesenteret.no
Method: We looked for...
• Quantitative research published in peer-reviewed
journals between 2000 and 2010, with children 0-8
years as participants.
• Children at risk of literacy failure.
• Also mainstream children.
• Studies with at least one of the following outcomes:
Alphabetic Knowledge, PA, RAN, Writing, Phonological
Memory, Reading Readiness, Oral Language, Visual
Processing and Concepts of Print (NELP, National
Institute for Literacy, 2008).
• Published in English.
10

12. The Reading Centre
www.lesesenteret.no
Method: How we looked for studies...
• Multimedia and early literacy search terms devised
by:
– Cross-checking reference lists found in most recent
meta studies and in results of pilot searches.
– Consulting reference books:
• Handbook of Early Literacy Research
• Handbook of Research on New Literacies
• International Handbook of Literacy and Technology, Vol. II
• Data bases searched:
– PsychINFO
– ERIC
11

13. The Reading Centre
www.lesesenteret.no
Results:
Searches (I)
• References for several hundred (!) potential studies were located.
• Abstracts were examined and subsequently 92 studies were downloaded for
further inspection.
• Again abstracts reviewed and, if needed, full texts were evaluated: 51 studies
complied with the search criteria.
• 15 had to be excluded, because at least one relevant statistic was missing.
• Of the 36 remaining, 24 reported on children at risk:
– 10 studies on second language learners
– 5 studies on low-SES children
– 9 studies with underachieving readers
• Applications dealt with:
– VIDEO (8 outcomes, 2 studies)
– TV (16 outcomes, 1 study)
– E-books (48 outcomes, 13 studies)
– Computer Assisted Instruction (127 outcomes, 24 studies)
12

14. The Reading Centre
www.lesesenteret.no
Results:
Searches (II)
• Altogether we found 220 literacy outcomes/experimental
conditions (most popular journals: Computers & Education,
Journal of Educational Psychology, Journal of Research in
Reading), 50% of which was published 2008 and 2010
• 195 out of 220 with kindergarten and first grade
participants
• Final categorisation of literacy outcomes:
Letter learning, Phonological Awareness, Concepts of print,
Vocabulary, Comprehension, Reading, Spelling, Syntax
• Majority of studies in English-speaking countries:
– USA (14), UK (5), Canada (2)
– Israel (6, Hebrew), Netherlands (8, Dutch), France 1
13

15. The Reading Centre
www.lesesenteret.no
Results:
Analysis method
• Designs:
– posttest only control group
– pretest-posttest, no control group
– pretest-posttest control group
– 1 or 2 studies with a control task
• Meta-analysis:
– Cohen’s d: difference between experimental and control
group (at pretest and at posttest), divided by pooled
variance.
– Small samples: corrections by means of Hedges’ g.
– For grouped outcomes: 95% confidence interval for mean
effect.
– See: Borenstein, Hedges, Higgins & Rothstein (2009).
14

16. The Reading Centre
www.lesesenteret.no
Results:
Effect sizes (I)
• Posttest control group design
– ES = 0.229 (95% confidence interval: 0.094 - 0.367)
– 46 outcomes, 11 studies, all CAI
• Pretest posttest control group
– ES = 0.311 (95% confidence interval: 0.243 - 0.395)
– 60 outcomes, 10 studies, CAI
• Pretest posttest control group
– ES = 0.360 (95% confidence interval: 0.224 - 0.514)
– 40 outcomes, 10 studies, E-books
15

17. The Reading Centre
www.lesesenteret.no
Results:
Effect sizes for risk groups (II)
• No risk
– ES = 0.228 (0.075 - 0.380)
• Reading failure
– ES = 0.512 (0.099 - 0.926)
• Low SES
– ES = 0.663 (0.415 - 0.910)
• SL
– ES = 0.749 (0.594 - 0.904)
16

18. The Reading Centre
www.lesesenteret.no
Results:
Effect sizes for media (III)
• Computer
– ES = 0.510 (0.395 - 0.625)
• E-book
– ES = 0.430 (0.001 - 0.859)
• TV*
– ES = 0.835 (0.190 - 1.479)
• VIDEO**
– ES = 0.245 (0.105 - 0.383)
17

19. The Reading Centre
www.lesesenteret.no
Results:
Effect sizes for outcomes (IV)
• Letter learning: ES = 0.654 (0.272 - 1.036)
• Comprehension: ES = 0.619 (0.150 - 1.080)
• PA: ES = 0.565 (0.407 - 0.718)
• Vocabulary: ES = 0.565 (0.339 - 0.790)
• Spelling: ES = 0.561 (0.209 - 1.334)
• Reading: ES = 0.379 (0.009 - 0.748)
• Concepts of print: ES = 0.351 (-0.048 - 0.750)
• Compare with baseline:
– RAN: ES = 0.195 (-0.148 - 0.537)
18

20. The Reading Centre
www.lesesenteret.no
Results:
Some (very low) correlations of ES and...
• Year of publication: .06
• Number of sessions: .06
• Duration (in weeks): .08
19

21. The Reading Centre
www.lesesenteret.no
Summary
• First study to indicate that literacy-related multimedia
applications have a small to moderate effect on
literacy learning outcomes of children at risk.
• CAI slightly more effective than E-books.
• Second language learners profited most, then SES,
then children at risk of reading failure.
• Multimedia applications do have effects on Letter
learning, Comprehension, Vocabulary, Spelling and
PA. Less so on Reading and Concepts of print.
20

22. The Reading Centre
www.lesesenteret.no
Discussion
• Results depend on sort of control group
– In all studies a ‘proper’ control group was used
– ‘Added value’
– Baseline: RAN
– Net added value effect: .40 - .15
• What worries me (a bit):
– no effect of duration/sessions on ES
• The way forward:
– Given a further increase of Apps:
It’s time for a thorough systematic review of how
efficient they are based on randomised controlled trials
21