Barbour, M. K., & Clark, T. (2016, October). Cases of quality: Case studies of the approval constructs for K-12 online and blended courses and providers. A paper presentation at the annual convention of the Association for Educational Communications and Technology, Las Vegas, NV.

1. Cases of Quality: Case Studies of
the Approval and Evalua8on of
K-12 Online and Blended Providers
Michael Barbour & Tom Clark

2. Barbour, M. K., Clark, T., DeBruler, K., & Bruno, J. A.
(2014). Evalua&on and approval constructs for
online and blended courses and providers. Lansing,
MI: Michigan Virtual Learning Research InsGtute at
MVU. Retrieved from hMp://media.mivu.org/
insGtute/pdf/eval_constructs.pdf
MVLRI Fellows

3. • Growth in K-12 online & blended learning programs
& enrollments, in MI & U.S.
• MI Legislature liUs ban on cyber charters (PA 227,
2010)
• Removes restricGons, creates pro-growth policies
(PA 129, 2012)
• Growth is outpacing research on quality in K-12 OLL
Overview

4. • MVU tasked to develop MVLRI (PA 201, 2012)
• Provide leadership for MI online & blended learning
• Key MVLRI task: research, develop, and recommend
annually to the department criteria by which cyber
schools and online course providers should be
monitored and evaluated to ensure a quality
educa&on for their pupils (p.44).
Overview

5. • Purpose: To examine exis8ng policies and prac8ces
related to the evalua8on and approval of K-12
online learning in the U.S.
– RQ1: How do states evaluate the quality of online
learning courses?
– RQ2: How do states iniGally evaluate the quality of
online learning programs?
– RQ3: How do states ensure the quality of online
learning programs on an on-going basis?
Methodology

6. Six Dimensions of Considera8on
Evalua&on & Approval:
Level
Provider / Course
Evalua&on & Approval:
Timeframe
Front-End/Ongoing
Approval Requirement OpGonal / Required
Geographic Reach MulG-Dist / Single Dist
Modes of Instruc&on Fully Online / Blended
Instruc&on Full-Gme / Supplemental

7. • Georgia
o Provider Level - All virtual instrucGon programs must
be approved by DOE
o Course Level - Established a clearinghouse of courses
that the DOE reviews the course content prior to
including it
• Maryland
o Course Level - Reviewed either by a team of reviewers
at the MSDE or local district, MSDE also recognizes
courses reviewed and cerGfied by Quality MaMers
Level of Evalua8on and Approval

8. • California
o CLRN Review Process - Provide online courses
evaluaGons with regard to their alignment with
Common Core or state content standards and iNACOL
quality standards
Approval Requirements

9. • Washington
o originally developed approval and evaluaGon criteria
in response to mulG-district providers and has
adapted the mulG-district process to single-district
providers with limited success
Geographic Reach

10. • Refers to differenGal approval and evaluaGon procedures
based on how course content and instrucGon are delivered
o A necessary pre-condiGon for this dimension is a clear
definiGon of online course/learning & blended course/
learning, which specifies the delivery, communicaGon
and contact expectaGons and sets a threshold for the
disGncGon between online & blended
• Maryland online course review required for all courses in which
80% of the content & instruc&on is delivered online
• Minnesota requires provider approval in cases where more than
50% of instruc&on is delivered online
Mode of Instruc8on

11. • Colorado
o Only front-end approval aUer removing many of its
on-going monitoring and reporGng requirements
o Authorizers must submit an applicaGon detailing
evidence of adequate resources and capacity to
oversee the online program
Evalua8on & Approval Procedures

12. • EducaGon of students parGcipaGng in full-Gme
online programs, as opposed to taking 1-2 courses
online, should be of special concern to states
• EffecGve processes for evaluaGng the quality of
online programs are needed
• Periodic external program audits by dedicated
teams of experts can play a valuable role in ensuring
program quality, and can provide a mechanism for
starGng program shutdown when absolutely needed
Recommenda8ons

13. • Communicate expected quality levels in instrucGonal
services, leadership pracGces, and content
o Standards for K-12 Distributed Learning in BC
o Standards for Digital Learning Content in BC
• Each instrucGonal pracGce or leadership pracGce
standard is a statement about a high-level
expectaGon accompanied by several observable
supporGng evidence statements that provide
guidance without being specific
An Interna8onal Model - BC

14. • Process begins with a DL school’s internal review
o DocumentaGon supporGng the internal review is based
on the standards, research and growing descripGons of
emerging pracGce
• Each year, several DL schools are selected for an external
review
o Using primarily qualitaGve methodologies, a small team
visits the school to validate the internal review, observe
instrucGonal and leadership pracGces
o The review team leader prepares a report with specific
input from the DL school principal
An Interna8onal Model - BC

15. • The report serves as a template for specific acGons
within the school, but also idenGfies promising or
exemplary pracGces that can be shared with other DL
schools
• AUer several months, the Ministry of EducaGon asks
the school to provide a status report on the external
team’s recommendaGons
h3p://7nyurl.com/bc-dl-review
An Interna8onal Model - BC

16. An Interna8onal Model - BC

17. • Some states have sought to limit access to full-Gme
online learning programs (and the research evidence
suggests that there is some merit to this approach)
• Rigorous monitoring and performance requirements
should allow states, over Gme, to ensure that full-
Gme programs are of high quality.
• Rigorous state requirements may also provide an
incenGve for full-Gme program providers to move to
blended learning models, where it is easier to achieve
quality learning results.
Recommenda8ons

18. Thank you!
Michael Barbour Tom Clark
MVLRI Fellow MVLRI Fellow
mkbarbour@gmail.com tom@taconsulGng.net
Kristen DeBruler & MVRLI colleagues
kdebruler@mivu.org