This document summarizes a report on the evaluation and approval of K-12 online and blended learning programs. It examines policies and practices in different states for initial and ongoing evaluation of online providers and courses. Key dimensions of evaluation include whether it is done at the provider or course level, timeframe (front-end or ongoing), if approval is required, the geographic reach, modes of instruction (fully online or blended), and if it is for full-time or supplemental instruction. The report also provides recommendations for rigorous evaluation processes and periodic external audits to ensure program quality. An international model from British Columbia is presented as an example approach.

1. Cases of Quality: Case Studies of
the Approval and Evaluation of K-
12 Online and Blended Providers
Michael Barbour & Tom Clark

2. Barbour, M. K., Clark, T., DeBruler, K., & Bruno, J. A.
(2014). Evaluation and approval constructs for
online and blended courses and providers. Lansing,
MI: Michigan Virtual Learning Research Institute at
MVU. Retrieved from
http://media.mivu.org/institute/pdf/eval_construc
ts.pdf
MVLRI Fellows

3. • Growth in K-12 online & blended learning programs
& enrollments, in MI & U.S.
• MI Legislature lifts ban on cyber charters (PA 227,
2010)
• Removes restrictions, creates pro-growth policies
(PA 129, 2012)
• Growth is outpacing research on quality in K-12 OLL
Overview

4. • MVU tasked to develop MVLRI (PA 201, 2012)
• Provide leadership for MI online & blended learning
• Key MVLRI task: research, develop, and recommend
annually to the department criteria by which cyber
schools and online course providers should be
monitored and evaluated to ensure a quality
education for their pupils (p.44).
Overview

5. • Purpose: To examine existing policies and practices
related to the evaluation and approval of K-12
online learning in the U.S.
– RQ1: How do states evaluate the quality of online
learning courses?
– RQ2: How do states initially evaluate the quality of
online learning programs?
– RQ3: How do states ensure the quality of online
learning programs on an on-going basis?
Methodology

6. Six Dimensions of Consideration
Evaluation & Approval:
Level
Provider / Course
Evaluation & Approval:
Timeframe
Front-End/Ongoing
Approval Requirement Optional / Required
Geographic Reach Multi-Dist / Single Dist
Modes of Instruction Fully Online / Blended
Instruction Full-time / Supplemental

7. • Georgia
o Provider Level - All virtual instruction programs must
be approved by DOE
o Course Level - Established a clearinghouse of courses
that the DOE reviews the course content prior to
including it
• Maryland
o Course Level - Reviewed either by a team of reviewers
at the MSDE or local district, MSDE also recognizes
courses reviewed and certified by Quality Matters
Level of Evaluation and Approval

8. • California
o CLRN Review Process - Provide online courses
evaluations with regard to their alignment with
Common Core or state content standards and iNACOL
quality standards
Approval Requirements

9. • Washington
o originally developed approval and evaluation criteria
in response to multi-district providers and has
adapted the multi-district process to single-district
providers with limited success
Geographic Reach

10. • Refers to differential approval and evaluation procedures
based on how course content and instruction are delivered
o A necessary pre-condition for this dimension is a clear
definition of online course/learning & blended
course/learning, which specifies the delivery,
communication and contact expectations and sets a
threshold for the distinction between online & blended
• Maryland online course review required for all courses in which
80% of the content & instruction is delivered online
• Minnesota requires provider approval in cases where more than
50% of instruction is delivered online
Mode of Instruction

11. • Colorado
o Only front-end approval after removing many of its
on-going monitoring and reporting requirements
o Authorizers must submit an application detailing
evidence of adequate resources and capacity to
oversee the online program
Evaluation & Approval Procedures

12. • Education of students participating in full-time
online programs, as opposed to taking 1-2 courses
online, should be of special concern to states
• Effective processes for evaluating the quality of
online programs are needed
• Periodic external program audits by dedicated
teams of experts can play a valuable role in ensuring
program quality, and can provide a mechanism for
starting program shutdown when absolutely needed
Recommendations

13. • Communicate expected quality levels in instructional
services, leadership practices, and content
o Standards for K-12 Distributed Learning in BC
o Standards for Digital Learning Content in BC
• Each instructional practice or leadership practice
standard is a statement about a high-level
expectation accompanied by several observable
supporting evidence statements that provide
guidance without being specific
An International Model - BC

14. • Process begins with a DL school’s internal review
o Documentation supporting the internal review is based
on the standards, research and growing descriptions of
emerging practice
• Each year, several DL schools are selected for an external
review
o Using primarily qualitative methodologies, a small team
visits the school to validate the internal review, observe
instructional and leadership practices
o The review team leader prepares a report with specific
input from the DL school principal
An International Model - BC

15. • The report serves as a template for specific actions
within the school, but also identifies promising or
exemplary practices that can be shared with other DL
schools
• After several months, the Ministry of Education asks
the school to provide a status report on the external
team’s recommendations
http://tinyurl.com/bc-dl-review
An International Model - BC

16. An International Model - BC

17. • Some states have sought to limit access to full-time
online learning programs (and the research evidence
suggests that there is some merit to this approach)
• Rigorous monitoring and performance requirements
should allow states, over time, to ensure that full-
time programs are of high quality.
• Rigorous state requirements may also provide an
incentive for full-time program providers to move to
blended learning models, where it is easier to achieve
quality learning results.
Recommendations

18. Thank you!
Michael Barbour Tom Clark
MVLRI Fellow MVLRI Fellow
mkbarbour@gmail.com tom@taconsulting.net