Virtual schools use different metrics to calculate course completion rates, making comparisons difficult. A survey of 86 US virtual schools found: 1. Trial periods for withdrawing from courses without penalty varied widely, from 1 to 185 days. Most common were 2-4 weeks. 2. Course completion definitions ranged from passing regardless of grade, to passing with a certain percentage. Most considered passing the course as completion. 3. Variability in these policies hinders comparing student retention and attrition across virtual schools. Standardizing metrics could facilitate more meaningful comparisons.

1. How are virtual schools calculating course completion rates?
Abigail Hawkins Dr. Michael K. Barbour
Brigham Young University Wayne State University
Introduction Results Trial period length by days (n = 57) Conclusions
There has been a rapid proliferation of virtual schools Of the 159 sampled, 86 schools responded to the survey, a There was a significant variation in trial period length and
throughout the United States. One challenge of such rapid 54.1% response rate varying by school type and region. 1
course completion definitions. We found that trial periods were a
growth has been adopting common metrics across virtual 3
fairly common practice among virtual schools and varied in
schools to measure quality. Currently, no common metrics exist 7 length. We also found that the majority of schools defined a
to calculate course completion rates Percentage of schools sampled & respondents 8 successful course completion as passing the course, though
(Pape, Revenaugh, Watson, and Wicks 2006; 10 there was wide variation in what percentage constituted a
Smith, Clark, and Blomeyer 2005, Watson and Ryan 2006). % of % of 14 “passing grade.” This documented variability strengthens the
v
Unfortunately completion rates are a key indicator commonly 15 call to standardize course completion metrics (Pape et al., 2006;
used to measure the quality of virtual schools (Blomeyer and
Characteristics Sample Respondents Length
Smith et al., 2005; Watson et al., 2007).
20
School Type in days
Dawson 2005). The absence of a common metric makes it 21
difficult to draw comparisons on student retention and attrition Cyber Charter 21.1 18.6 28 The implications of this study are a need greater transparency
v
among virtual schools. For Profit 5.6 1.2 30 and consistency in policies to facilitate meaningful comparisons
35 of retention rates. Though schools can have their internal
Multi-district / 6.8 8.1 60 policies for calculating course completions, for statistical and
Policies that may affect completion rates Consortium >179 reporting purposes, all virtual schools should be required to start
Trial period policy: Period when students can withdraw from a Private 13.0 14.0 counting students at the same time and in the same manner.
0 2 4 6 8 10 12 14
course with no penalty and not be counted as officially being Single-district 30.4 30.2 Ideally, when and how students are counted ought to be similar
enrolled (Watson et al., 2008). Number of Schools Figure 3. to how brick and mortar schools calculate course
State-led 14.9 18.6
completions, thus allowing for meaningful comparisons of
Completion definition policy: Determination of students who University-led 6.8 9.3 retention rates among virtual schools and brick and mortar
are officially counted as successfully completing a course. Total Percentage 100% 100% Course completion definitions schools.
Some schools define a student as completing a course if they
All but two of the 86 schools provided qualitative definitions of a
master the material, others if they pass the course with a 60% Region completion. There was wide variation ranging from from “remaining Results from this study also provide a scope of polices currently
or better, and others if they simply remain in the course. Central States 25.5 26.7 in the course regardless of the grade” to passing the course with an in practice that can aid policymakers’ efforts to create common
Northeastern States 11.2 9.3 A- or better. Responses fell into three main categories: (1) metrics among virtual schools.
Research Questions completion of the course within the allotted time frame regardless of
Southeastern States 20.5 22.1
1. What are the trial period policies for US virtual the grade received, (2) completion of the course within an allotted
Western States 41.6 41.9 time frame with a passing grade, and (3) deferring judgment to the
schools?
Total Percentage 100% 100% brick and mortar school the student physically attends. Chi Square
a. How much variability exists across different types of
virtual schools and regions of the country?
Prevalence of trial periods
Test for Independence (p=.05) indicated no significant relationships Literature cited
between completion definitions and school type nor geographical Blomeyer, R. L. and M. Dawson. 2005. Policy context of virtual schools. In
2. How do US virtual schools define course completions? location. Virtual Schools: Planning for Success,eds. Z. L. Berge and T. Clark, 61-
 Majority of schools Defer Judgment
a. How much variability exists across different types of 76. New York: Teachers College Press.
sampled had trial periods 4%
50 virtual schools and regions of the country? Pape, L., M. Revenaugh, J. Watson, and M. Wicks. 2006. Measuring
(59 out of 86). No
outcomes in K-12 online education programs: The need for common
40 Trial
 All but 1 state-led school Perio
metrics. Distance Learning 3 (3): 51-59.
30 Figure 1. States with had trial period. d 31% Smith, R., T. Clark, and R. L. Blomeyer. 2005. A synthesis of new research on
Significant K-12 Online Grade
Trial K-12 online learning. Naperville, IL: Learning Point Associates.
20 Programs  Cyber charter and private Perio Irrelevant
Figure 4. Course completion Watson, J. F., and J. Ryan, 2006. Keeping pace with k-12 online learning: A
10 schools were evenly split. d 69% 33%
definitions where grade is or review of state-level policy and practice. Vienna, VA: North American
Grade Relevant is not a determining factor in Council for Online Learning. Available online at
0 Michael, I pulled these figures from
 Northeastern states least 63% counting students as http://www.kpk12.com/downloads/KeepingPace_2006.pdf
1994 1997 2005 2006 2007 2008 several sources – CELT, Clark, & All
of Watson’s keeping pace docs. How
successful completers.
likely to have trial periods
do I cite this? Watson, J. F., and J. Ryan. 2007. Keeping pace with k-12 online learning: A
(only 28.6% compared to review of state-level policy and practice. Vienna, VA: North American
70% Western and 83.3% Figure 2. Percent of schools Council for Online Learning. Available online at
Southeastern). with and without a trial period. http://www.kpk12.com/downloads/KeepingPace_2007.pdf
Examples of variation where grade is relevant
Length of trial periods
Methods  Trail period lengths ranged from 1 day to 185 days. Definition n For further information
 Surveyed 159 schools taken from NACOL’s 2007 Online Passing course 37 Please contact Abby Hawkins at abbyhawkins7@gmail.com or Dr. Michael Barbour
 Mean length was 19.61, when two extreme outliers were excluded. Passing course and final 2 at mkbarbour@gmail.com.
Learning
Clearinghouse List and state-led schools listed in Watson’s  2 and 4 week trial periods were the most common, accounting for over Completing four quarters with 60% or better 2 The entirety and more detailed discussion of the literature, methods, findings, and
(2007) 50% of trial period lengths. See Figure 3. Passing with D/64% or better 1 implications will be printed in the AECT 2009 Conference Proceedings.
Keeping Pace with K12 Online Learning.
6
 One instance where an event, the submission of the first assignment, Passing with C-/70% or better
 Emailed three-question survey directly to
principal, director, or marked the end of a trial period. Passing with B-/80% or better 4
registrar when possible. Passing with A-/90% or better 1
 No significant difference between trial period length by school type or
 Made 4 contact attempts (2 by email, fax, telephone) to geographic location according to one-way ANOVA test. Total 53
increase
response rates.
 Sampled only K-12 virtual schools thatdo I need to mention the
Michael,
were state approved Michael, check out the side bar on
regionally accredited. goegraphy and virtual school type the far left off tof the ppt slide to see
taxonomy and where we got them an alternative view (table) of course
from? completion definition information.