This meta-analysis reviewed research on online learning and compared its effectiveness to face-to-face instruction. It found that online learning was slightly more effective than face-to-face (average effect size of +0.14). Blended approaches with both online and face-to-face instruction were more effective than either alone (average effect size of +0.35). Certain practices like more time on task and learner reflection were associated with more effective online learning. The findings were limited by not being able to control for variables across studies.

1. Evaluation of Evidence-Based Practices in Online Learning A Meta-Analysis and Review of Online Learning Studies U.S. Department of Education http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf

2. Overview A meta-analysis of on-line learning. Looking specifically at How does the effectiveness of online learning (OL) compare with that of face-to-face (F2F) instruction? Mean effect = +0.14, p < 0.05 Does supplementing F2F instruction with OL instruction enhance learning? Mean effect = +0.35, p < 0.0001 What practices are associated with more effective OL learning? Mean effect = various, p-values = various What conditions influence the effectiveness of OL learning? None of them!

3. Observations The really want to find results for K-12, but alas-- (See caveats, page xii) Greatest effect for undergraduate and older students, but not K-12, secondary or graduate students. Weird. Hard to measure since time spent, pedagogy and curriculum are by definition different. “ [T]he studies . . . do not demonstrate that online learning is superior as a medium.” Page xvii “ Studies in which learners in the online condition spent more time on task than to students in the F2F condition found a greater benefit for online.” Duh. I’m seeing a creepy amount of p-values of 0.05. Trying too hard? Buried deeply is the definition of their effect sizes. Most as “smallish”.

4. Differentiation of Studies OL as replacement or enhancement of F2F Pedagogy: Expository, Interactive, active Synchronous or asynchronous Can’t control conditions: Year, demographics, teacher credentials, state systems, etc.

5. Questions Quasi-Experimental Scaling for progress? Yes, sort of . . .

6. Findings Best is F2F + OL (Page xv) Contradicts! Purely OL = F2F + OL (Blended) Page 38 Seven of eight studies found no significant differences among media combinations. Page 40 Best if students given a choice of resources. Page 41 Online Quizzes don’t work! Hoorah! (page xvi) More media is not always better. (page xvi) Learner Reflection is good . (page 44) Instructors as on-line moderators are unnecessary. (page 46) Don’t script online interaction (page 46)

7. Surprises One study found that synchronous OL had a negative effect, whereas asynchronous OL was significantly positive. Suggested that the negative synchronous result was due to old-fashioned techniques. Page 7 Contradicts Year published was not statistically significantly! Page 30

8. Best Quote Although earlier meta-analysis of distance education found it equivalent to classroom instruction several reviewers have suggested that this pattern may chance. They argue that online learning as practiced in the 21 st century can be expected to outperform earlier forms of distance education in terms of effects on learning.

9. Truth in Advertising “Finally, readers should be cautioned that the literature on alternative online learning practices has been conducted for the most part by professors and others instructors who are conducting research using their own courses. Moreover, the combinations of technology, content and activities used in different experimental conditions have often been ad hoc rather than theory based.

10. Authors Title Effect Size 95% Con Int Null (2-tail) Retention Rate (%) #Units g SE Lower Limit Upper Limit Z -Value Online Face-to-Face LaRose(1998) Audiographic telecourses for the Web: An experiment +0.070 0.281 -0.481 0.621 0.25 Unknown Unknown 49 students Lowry (2007) Effects of online versus face-to-face professional development with a team-based learning community approach on teachers’ application of a new instructional practice -0.281 0.335 -0.937 0.370 -0.84 80 93.55 53 students Mentzer & Teclehaimanot (2007) A comparison of face-to-face and Web-based classrooms -0.796 0.339 -1.460 -0.131 -2.35* Unknown Unknown 36 students Nguyen (2008) Randomized controlled trial of an Internet-based versus face-to-face dyspnea self-management program for patients with chronic obstructive pulmonary disease: Pilot study +0.292 0.316 -0.327 0.910 0.93 Unknown Unknown 39 participa Ocker and Yaverbaum (1999) Asynchronous computer-mediated communication versus face-to-face collaboration: Results on student learning, quality and satisfaction -0.030 0.214 -0.449 0.389 -0.14 Unknown Unknown 43 students Padalino and Peres (2007) E-learning: A comparative study for knowledge apprehension among nurses 0.115 0.281 -0.437 0.666 0.41 Unknown Unknown 49 participa Peterson and Bond (2004) Online compared to face-to-face teacher preparation for learning standards-based planning skills +0.100 0.214 -0.320 0.520 0.47 Unknown Unknown 4 sections Schmeeckle (2003) Online training: An evaluation of the effectiveness and efficiency of training law enforcement personnel over the Internet -0.106 0.198 -0.494 0.282 -0.53 Unknown Unknown 101 students Schoenfeld-Tacher, McConnell and Graham (2001) Do no harm: A comparison of the effects of online vs. traditional delivery media on a science course +0.800 0.459 -0.100 1.700 1.74 100 99.94 Unknown Exhibit 4a. Purely Online Versus Face-to-Face (Category 1) Studies Included in the Meta-Analysis (continued) Notice that negative effect sizes were from studies with broad titles, whereas positive effect sizes were from studies of OL learning of specific tasks.