These are the slides from my research-focused job talk at Georgia Regents University in February 2013.

1. What do we know? Where do we
go?
Examining the Effective Design,
Delivery and Support of K-12 Online
Learning
Michael K. Barbour
Assistant Professor
Wayne State University

2. Newfoundland and Labrador
• area of the island is 43,359 square
miles, while Labrador covers 112,826
square miles
• population of 514,536 in 2011 (down
from 551,795 in 1996)
• 67,933 students in 2011 (down from
118,273 in 1996)
• 268 schools in 2006 (down from 432
in 1996)
• average school size 220 pupils

3. • Graduated from a large, urban high school
• Became a teacher in a regional, rural school
• Started a web-based Advanced Placement social studies
distance education program
• Became interested in:
– how interaction through discussion forums affected student
understanding and performance
– achievement and retention differences between web-based and
classroom students

4. Centre for Distance Learning and Innovation
• created in 2000
• offered first courses in
2001-02
• by 2005-06
– 35 courses
– ~1,500 student enrolments
– ~1000 students
– ~100 different schools
Images from http://www.cdli.ca/index.php?PID=AnnounceFull&NewsID=6655&PHPSESSID=e510a9218ee89f32fb5283032a839364 and
http://www.waet.uga.edu/canada/canada.htm

5. History of K-12 Online Learning
in the North America

6. Year Report Status
1997 Clark 3 States
2001 Clark 8 States / 40,000-50,000 students
Vail 30 States
2004 Watson et al. 11 of 22 States
Huerta & Gonzales 15 States
2005 Watson et al. 21 States
Setzer & Lewis 328,000 students
2006 Watson et al. 24 States
Gray & Tucker 139,000 students
2007 Watson et al. 42 States
Picciano & Seaman 700,000 students
2008 Watson et al. 44 States
2009 Watson et al. 45 States / 320,000 supplement & 175,000 full-time
Picciano & Seaman 1,000,000 students
2010 Watson et al. 48 States / 1,500,000 students
Wicks 2,000,000 students
2011 Watson et al. 50 States
4,000,000 students
2012 Ambient Insights 6,000,000 students

9. 9

10. Digital Learning Now
1. All students are digital learners.
2. All students have access to high quality digital content and online
courses.
3. All students can customize their education using digital content
through an approved provider.
4. Students progress based on demonstrated competency.
5. Digital content, instructional materials, and online and blended
learning courses are high quality.
6. Digital instruction and teachers are high quality.
7. All students have access to high quality providers.
8. Student learning is the metric for evaluating the quality of content
and instruction.
9. Funding creates incentives for performance, options and
innovation.
10. Infrastructure supports digital learning.

11. Digital Learning Now
1. All students are digital learners.
2. All students have access to high quality digital content and online
courses.
3. All students can customize their education using digital content
through an approved provider.
4. Students progress based on demonstrated competency.
5. Digital content, instructional materials, and online and blended
learning courses are high quality.
6. Digital instruction and teachers are high quality.
7. All students have access to high quality providers.
8. Student learning is the metric for evaluating the quality of content
and instruction.
9. Funding creates incentives for performance, options and
innovation.
10. Infrastructure supports digital learning.

12. So let’s look at the research…

13. Literature Reviews
1. Rice (2006)
– Journal of Research on Technology in
Education
1. Barbour & Reeves (2009)
– Computers and Education
1. Cavanaugh, Barbour, & Clark (2009)
– International Review of Research in Open

14. What does the literature say?
• “based upon the personal experiences of
those involved in the practice of virtual
schooling” (Cavanaugh et al., 2009)
• described the literature as generally falling
into one of two general categories: the
potential benefits of and challenges facing K-
12 online learning (Barbour & Reeves, 2009)

15. What about research?
• “a paucity of research exists when
examining high school students enrolled
in virtual schools, and the research base
is smaller still when the population of
students is further narrowed to the
elementary grades”
(Rice, 2006)

16. Is this a problem?
“indicative of the foundational descriptive work
that often precedes experimentation in any
scientific field. In other words, it is important to
know how students in virtual school engage in
their learning in this environment prior to
conducting any rigorous examination of virtual
schooling.”
(Cavanaugh et al., 2009)

17. Analysis of Primary & Secondary Focused Articles
in the Main Distance Education Journals (2005-
10)
Australia Canada New Zealand United States
American Journal of Distance
Education (United States) 8
Distance Education
(Australia) 2 4
Journal of Distance
Education (Canada) 1 4
Journal of Distance Learning
(New Zealand) .5* 1 .5*
Total
3 4.5* 1 12.5*
* One article had a focus on both Canada and the United States

18. What does the research say?
1. Comparisons of student performance based upon
delivery model (i.e., classroom vs. online)
2. Studies examining the qualities and characteristics
of the teaching/learning experience
– characteristics of
– supports provided to
– issues related to isolation of online learners (Rice, 2006)
1 Effectiveness of virtual schooling
2 Student readiness and retention issues (Cavanaugh
et al., 2009)

19. So, what does the student
performance research say?

20. Student Performance
• performance of virtual and
classroom students in Alberta
were similar in English and
Social Studies courses, but that
classroom students performed
better overall in all other
subject areas (Ballas & Belyk,
2000)
• over half of the students who
completed FLVS courses scored
an A in their course and only
7% received a failing grade
(Bigbie & McCarroll, 2000)

21. Student Performance
• students in the six virtual schools
in three different provinces
performed no worse than the
students from the three
conventional schools (Barker &
Wendel, 2001)
• FLVS students performed better
on a non-mandatory assessment
tool than students from the
traditional classroom
(Cavanaugh et al., 2005)

22. Student Performance
• FLVS students performed
better on an assessment of
algebraic understanding than
their classroom counterpart
(McLeod et al., 2005)
• CDLI students performed as
well as classroom-based
students on final course scores
& exam marks (Barbour &
Mulcahy, 2007; 2008)

23. Let’s look a little closer...

24. Students and Student Performance
Ballas & performance of virtual and participation rate in the
Belyk, 2000 classroom students similar assessment among virtual
in English & Social Studies students ranged from 65% to
courses, but classroom 75% compared to 90% to
students performed better 96% for the classroom-based
in all other subject areas students
Bigbie & over half of the students between 25% and 50% of
McCarroll, who completed FLVS students had dropped out
2000 courses scored an A in of their FLVS courses over
their course and only 7% the previous two-year
received a failing grade period

25. Students and Student Performance
Cavanaugh et FLVS students performed speculated that the virtual
al., 2005 better on a non- school students who did
mandatory assessment take the assessment may
tool than students from have been more
the traditional classroom academically motivated and
naturally higher achieving
students
McLeod et FLVS students performed results of the student
al., 2005 better on an assessment performance were due to
of algebraic understanding the high dropout rate in
than their classroom virtual school courses
counterparts

26. Student Performance and Students
So are we really
comparing apples to
apples?

27. The Students
• the vast majority of VHS
Global Consortium students
in their courses were
planning to attend a
four-year college (Kozma,
Zucker & Espinoza, 1998)
• “VHS courses are
predominantly designated as
‘honors,’ and students
enrolled are mostly college
bound” (Espinoza et al., 1999)

28. The Students
The preferred characteristics
include the highly motivated,
self-directed, self-disciplined,
independent learner who
could read and write well,
and who also had a strong
interest in or ability with
technology (Haughey &
Muirhead, 1999)

29. The Students
• “only students with a high
need to control and structure
their own learning may
choose distance formats
freely” (Roblyer & Elbaum,
2000)
• IVHS students were “highly
motivated, high achieving,
self-directed and/or who liked
to work independently” (Clark
et al., 2002)

30. The Students
• the typical online student
was an A or B student
(Mills, 2003)
• 45% of the students who
participated in e-learning
opportunities in Michigan
were “either advanced
placement or
academically advanced”
students (Watkins, 2005)

31. But does this represent all of
our online students?

32. Student Reality???
• two courses with the highest enrollment of online
students in the US are Algebra I & Algebra II
(Patrick, 2007)
• largest proportion of growth in K–12 online
learning enrollment is with full-time cyber schools
(Watson et al., 2008)
• many cyber schools have a higher percentage of
students classified as “at-risk” (Klein, 2006)
• at-risk students are as those who might otherwise
drop out of traditional schools (Rapp, Eckes &
Plurker, 2006)

33. Literature
indicates K-12
online learning
students are...

34. Reality of most or
a large segment
K-12 online
learning
students?

35. Including Wider Range of Students
State of Colorado – 2006 Online Education
Performance Audit
– “Online student scores in math, reading, and writing
have been lower than scores for students statewide
over the last three years.”
– “The difference in performance between online
students and all students statewide is larger in higher
grades.”
– “Our analysis of Colorado Student Assessment Program
results and repeater, attrition, and dropout rates
indicate that online schools may not be providing
sufficiently for the needs of their students.”

36. Including Wider Range of Students
State of Wisconsin – Legislative Audit of Virtual
Charter Schools (2010)
– “Virtual charter school pupils’ median scores on the
mathematics section of the Wisconsin Knowledge and
Concepts Examination were almost always lower than
statewide medians during the 2005-06 and 2006-07
school years.”
– “Because of the relative newness of virtual charter
schools and their substantial growth since inception,
readily available information on the performance of
virtual charter school pupils would be of value to
parents, school districts, legislators, and other
policymakers.”

37. Including Wider Range of Students
State of Colorado – iNews Network Investigation
(2011)
– “Half of the online students wind up leaving within a
year. When they do, they’re often further behind
academically then when they started.”
– “Online schools produce three times as many dropouts
as they do graduates. One of every eight online
students drops out of school permanently – a rate four
times the state average.”
– “Online student scores on statewide achievement tests
are consistently 14 to 26 percentage points below
state averages for reading, writing and math over the
past four years.”

38. Including Wider Range of Students
State of Minnesota – 2011 K-12 Online Learning
Legislative Audit
– “Full-time online students dropped out much more
frequently.”
– “Compared with all students statewide, full-time online
students had significantly lower proficiency rates on
the math MCA-II but similar proficiency rates in
reading.”
– “During both years [i.e., 2008-09 and 2009-10], full-
time online students enrolled in grades 4 through 8
made about half as much progress in math, on
average, as other students in the same grade.”

39. Including Wider Range of Students
Miron, G. & Urschel, J. (2012). Understanding and
improving full-time virtual schools. Denver, CO:
National Education Policy Center.
– “…students at K12 Inc., the nation’s largest virtual school
company, are falling further behind in reading and math scores
than students in brick-and-mortar schools.”
– “These virtual schools students are also less likely to remain at
their schools for the full year, and the schools have low
graduation rates.”
– “Children who enroll in a K12 Inc. cyberschool, who receive full-
time instruction in front of a computer instead of in a classroom
with a live teacher and other students, are more likely to fall
behind in reading and math. These children are also more likely
to move between schools or leave school altogether – and the
cyberschool is less likely to meet federal education standards.”

40. The Other Side of the Story…
University of Arkansas Internal Evaluation of the Arkansas
Virtual Academy School (ARVA)
When comparing student performance in mathematics, the researchers found:
• students in the face-to-face group increased their performance by 1% more
than the online group from grades 3 to 5 (not statistically significant)
• students in the online group increased their performance by 5% more than
the face-to-face group from grades 4 to 6 (not statistically significant)
• students in the online group increased their performance by 2% more than
the face-to-face group from grades 5 to 7 (not statistically significant)
• students in the online group increased their performance by 16% more than
the face-to-face group from grades 6 to 8 (statistically significant at the
p=0.10 level)

41. The Other Side of the Story…
University of Arkansas Internal Evaluation of the Arkansas
Virtual Academy School (ARVA)
When comparing student performance in literacy, the researchers found:
• students in the face-to-face group increased their performance by 3% more
than the online group from grades 3 to 5 (not statistically significant)
• students in the online group increased their performance by 11% more than
the face-to-face group from grades 4 to 6 (statistically significant at the
p=0.10 level)
• students in the online group increased their performance by 2% more than
the face-to-face group from grades 5 to 7 (not statistically significant)
• students in the online group increased their performance by 7% more than
the face-to-face group from grades 6 to 8 (not statistically significant)

42. The Other Side of the Story…
University of Arkansas Internal Evaluation of the Arkansas
Virtual Academy School (ARVA)
There were methodological limitations in the sample (all of
which favored the online students):
• the online sample had several of its lowest performing
students removed before they had repeated a grade or had
dropped out over the two-year period.
• the online sample was a more affluent group.
• the online sample had significant fewer minority students.

43. What’s Really Driving this Growth??

44. What About the Other Research?

45. Methodologically Limited Findings
Online 7 principles of Interviews with teachers and course
Course effective online developers at a single virtual school,
Design course content with no verification of whether the
for adolescent interviewees’ perceptions were actually
Barbour learners effective or any student input at all for
(2005; 2007) that matter.
Online 37 best Interviews with teachers at a single
Teaching practices in virtual school selected by the virtual
asynchronous school itself. Their teachers’ beliefs
DiPietro et online teaching were not validated through observation
al. (2008) of the teaching or student performance.

46. The Challenge
Whether online
learning can
be suitable for
all K-12 students?
(Mulcahy, 2002)

47. The Challenge
How do we create
environments
where all K-12
students can be
successful when
they learn online?

48. Teaching in K-12 Online Learning
Virtual School Designer: Course
Development
• design instructional materials
• works in team with teachers and a virtual
school to construct the online course, etc.
Virtual School Teacher: Pedagogy and
Class Management
• presents activities, manages pacing, rigor, etc.
• interacts with students and their facilitators
• undertakes assessment, grading, etc.
Virtual School Site Facilitator: Mentoring and Advocating
• local mentor and advocate for student(s)
• proctors & records grades, etc.
Davis (2007)

49. Looking for better ways…

50. My Own Research Agenda
• Examining the
preparation of teachers
to design, deliver and
support K-12 online
learning
• Exploring ways to better
prepare students to be
successful in K-12 online
learning environments

51. My Own Research Agenda
• Examining the policy and
regulation of K-12
distance education
in Canada & elsewhere
• Working with individual
K-12 online learning
programs to help them
to effectively design,
deliver and support K-12
online learning

52. My Own Research Agenda
• Countering the
dominant narrative
presented by the
neo-liberal
supporters of K-12
online learning in
the United States
(and elsewhere)

53. Your
Questions
and
Comments

54. Assistant Professor
Wayne State University, USA
mkbarbour@gmail.com
http://www.michaelbarbour.com