Diving Head First into a Empty             PoolExamining the Research Guiding    K-12 Online Learning        Michael K. Ba...
History of K-12 Online Learning     in the North America
Year   Report              Status1997   Clark               3 States2001   Clark               8 States / 40,000-50,000 st...
6
Digital Learning Now1.   All students are digital learners.2.   All students have access to high quality digital content a...
Digital Learning Now1.   All students are digital learners.2.   All students have access to high quality digital content a...
What Does The Literature Say?• “based upon the personal experiences of  those involved in the practice of virtual  schooli...
What About Research?• “a paucity of research exists when  examining high school students enrolled  in virtual schools, and...
Analysis of Primary & Secondary Focused Articles  in the Main Distance Education Journals (2005-                        10...
Is This A Problem?“indicative of the foundational descriptive workthat often precedes experimentation in anyscientific fie...
What Does The Research Say?1. Comparisons of student performance based upon   delivery model (i.e., classroom vs. online)2...
So Let’s Look At The Research…
So, What Does That StudentPerformance Research Say?
Student PerformanceStudy                   FindingBallas & Belyk (2000)   performance of virtual and classroom students in...
Let’s look a little closer...
Students and Student PerformanceStudy              SampleBallas & Belyk     participation rate in the assessment among(200...
K-12 Distance Education Meta-Analysis• Cavanaugh (2001) - 16 studies  – +0.147 in favor of K-12 distance education• Cavana...
Analyzing Meta-Analyses
Analyzing Meta-Analyses                Teacher                Effects      Zone of                             Desired Eff...
K-12 Distance Education Meta-Analysis• Cavanaugh (2001) - 16 studies  – +0.147 in favor of K-12 distance education• Cavana...
Student Performance and StudentsSo are we reallycomparing apples toapples?
The StudentsStudy                   SampleKozma et al. (1998)     vast majority of VHS students in their courses were     ...
But Is This Representative Of All K-12 Online Students?
Student Reality???• two courses with the  highest enrollment of  online students in the US  are Algebra I & Algebra II  (P...
Student Reality???• many cyber schools have  a higher percentage of  students classified as  “at-risk” (Klein, 2006)• at-r...
K-12 onlinelearners in themajority of theresearchliterature are...
…however, thismay berepresentative ofthe majority of K-12 online
So, What Does The StudentPerformance Research Say About Full-Time K-12 Online Learning?
Including Wider Range of StudentsState of Colorado – 2006 Online Education  Performance Audit  – “Online student scores in...
Including Wider Range of StudentsState of Wisconsin – Legislative Audit of Virtual  Charter Schools (2010)  – “Virtual cha...
Including Wider Range of StudentsState of Colorado – iNews Network Investigation  (2011)  – “Half of the online students w...
Including Wider Range of StudentsState of Minnesota – 2011 K-12 Online Learning  Legislative Audit  – “Full-time online st...
Including Wider Range of StudentsMiron, G. & Urschel, J. (2012). Understanding and improving full-time virtual schools. De...
The Other Side of the Story…University of Arkansas Internal Evaluation of the Arkansas  Virtual Academy School (ARVA)When ...
The Other Side of the Story…University of Arkansas Internal Evaluation of the Arkansas  Virtual Academy School (ARVA)When ...
The Other Side of the Story…University of Arkansas Internal Evaluation of the Arkansas  Virtual Academy School (ARVA)Onlin...
Are More Students Really At-RiskMiron, G. & Urschel, J. (2012). Understanding and improving full-time  virtual schools. De...
What’s Really Driving this Growth??
The ChallengeWhether onlinelearning canbe suitable forall K-12 students?(Mulcahy, 2002)
The ChallengeHow do we createenvironmentswhere all K-12students can besuccessful whenthey learn online?
What About the Other Research?
Methodologically Limited FindingsOnline         7 principles of    Interviews with teachers and courseCourse         effec...
What Do We Know?1. Support at the local level can   determine student success2. Teacher interaction in the   asynchronous ...
What Do We Know?4. Data has the potential to improve   program delivery and individual   student performance (but generall...
Potential Useful Models1. Requirement to target at-risk or dropped out   students (Michigan)2. Tying funding to completion...
My Own Research Agenda• Examining the  preparation of teachers  to design, deliver and  support K-12 online  learning• Exp...
My Own Research Agenda• Examining the policy and  regulation of K-12  distance education  in Canada & elsewhere• Working w...
My Own Research Agenda• Countering the  dominant narrative  presented by the  neo-liberal  supporters of K-12  online lear...
Looking for better ways…
YourQuestions  andComments
Assistant Professor  Wayne State University, USA     mkbarbour@gmail.comhttp://www.michaelbarbour.com
Job Talk: Research (2013) - University of Colorado (Denver)
Job Talk: Research (2013) - University of Colorado (Denver)
Upcoming SlideShare
Loading in …5
×

Job Talk: Research (2013) - University of Colorado (Denver)

562 views

Published on

These are the slides from my research-focused job talk at University of Colorado (Denver) in March 2013.

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
562
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Cavanaugh (2001) - developmental effects Cavanaugh et al. (2004) - reverse effects Means et al. (2009) - online = teacher effects & blended = developmental effects + teacher effects
  • Another problem is what we measure... 1. Correlation does not equal causality 2. Single studies measure if there is a difference between two groups beyond chance Need for meta-analysis...
  • Cavanaugh (2001) - developmental effects Cavanaugh et al. (2004) - reverse effects Means et al. (2009) - online = teacher effects & blended = developmental effects + teacher effects
  • Job Talk: Research (2013) - University of Colorado (Denver)

    1. 1. Diving Head First into a Empty PoolExamining the Research Guiding K-12 Online Learning Michael K. Barbour Assistant Professor Wayne State University
    2. 2. History of K-12 Online Learning in the North America
    3. 3. Year Report Status1997 Clark 3 States2001 Clark 8 States / 40,000-50,000 students Vail 30 States2004 Watson et al. 11 of 22 States Huerta & Gonzales 15 States2005 Watson et al. 21 States Setzer & Lewis 328,000 students2006 Watson et al. 24 States Gray & Tucker 139,000 students2007 Watson et al. 42 States Picciano & Seaman 700,000 students2008 Watson et al. 44 States2009 Watson et al. 45 States / 320,000 supplement & 175,000 full-time Picciano & Seaman 1,000,000 students2010 Watson et al. 48 States / 1,500,000 students Wicks 2,000,000 students2011 Watson et al. 50 States 4,000,000 students2012 Ambient Insights 6,000,000 students
    4. 4. 6
    5. 5. Digital Learning Now1. All students are digital learners.2. All students have access to high quality digital content and online courses.3. All students can customize their education using digital content through an approved provider.4. Students progress based on demonstrated competency.5. Digital content, instructional materials, and online and blended learning courses are high quality.6. Digital instruction and teachers are high quality.7. All students have access to high quality providers.8. Student learning is the metric for evaluating the quality of content and instruction.9. Funding creates incentives for performance, options and innovation.10. Infrastructure supports digital learning.
    6. 6. Digital Learning Now1. All students are digital learners.2. All students have access to high quality digital content and online courses.3. All students can customize their education using digital content through an approved provider.4. Students progress based on demonstrated competency.5. Digital content, instructional materials, and online and blended learning courses are high quality.6. Digital instruction and teachers are high quality.7. All students have access to high quality providers.8. Student learning is the metric for evaluating the quality of content and instruction.9. Funding creates incentives for performance, options and innovation.10. Infrastructure supports digital learning.
    7. 7. What Does The Literature Say?• “based upon the personal experiences of those involved in the practice of virtual schooling” (Cavanaugh et al., 2009)• described the literature as generally falling into one of two general categories: the potential benefits of and challenges facing K- 12 online learning (Barbour & Reeves, 2009)
    8. 8. What About Research?• “a paucity of research exists when examining high school students enrolled in virtual schools, and the research base is smaller still when the population of students is further narrowed to the elementary grades” (Rice, 2006)
    9. 9. Analysis of Primary & Secondary Focused Articles in the Main Distance Education Journals (2005- 10) Australia Canada New Zealand United StatesAmerican Journal of DistanceEducation (United States) 8Distance Education(Australia) 2 4Journal of DistanceEducation (Canada) 1 4Journal of Distance Learning(New Zealand) .5* 1 .5*Total 3 4.5* 1 12.5* * One article had a focus on both Canada and the United States
    10. 10. Is This A Problem?“indicative of the foundational descriptive workthat often precedes experimentation in anyscientific field. In other words, it is important toknow how students in virtual school engage intheir learning in this environment prior toconducting any rigorous examination of virtualschooling.” (Cavanaugh et al., 2009)
    11. 11. What Does The Research Say?1. Comparisons of student performance based upon delivery model (i.e., classroom vs. online)2. Studies examining the qualities and characteristics of the teaching/learning experience – characteristics of – supports provided to – issues related to isolation of online learners (Rice, 2006)1 Effectiveness of virtual schooling2 Student readiness and retention issues (Cavanaugh et al., 2009)
    12. 12. So Let’s Look At The Research…
    13. 13. So, What Does That StudentPerformance Research Say?
    14. 14. Student PerformanceStudy FindingBallas & Belyk (2000) performance of virtual and classroom students in Alberta were similar in English and Social Studies courses, but that classroom students performed better overall in all other subject areasBigbie & McCarroll over half of the students who completed FLVS(2000) courses scored an A in their course and only 7% received a failing gradeBarker & Wendel students in the six virtual schools in three different(2001) provinces performed no worse than the students from the three conventional schoolsCavanaugh et al. FLVS students performed better on a non-(2005) mandatory assessment tool than students from the traditional classroomMcLeod et al. (2005) FLVS students performed better on an assessment of algebraic understanding than their classroom counterpartsBarbour & Mulcahy little difference in the overall performance of(2008) students based upon delivery modelBarbour & Mulcahy no difference in student performance based upon(2009a) method of course delivery
    15. 15. Let’s look a little closer...
    16. 16. Students and Student PerformanceStudy SampleBallas & Belyk participation rate in the assessment among(2000) virtual students ranged from 65% to 75% compared to 90% to 96% for the classroom- based studentsBigbie & McCarroll between 25% and 50% of students had(2000) dropped out of their FLVS courses over the previous two-year periodCavanaugh et al. speculated that the virtual school students(2005) who did take the assessment may have been more academically motivated and naturally higher achieving studentsMcLeod et al. results of the student performance were due(2005) to the high dropout rate in virtual school courses
    17. 17. K-12 Distance Education Meta-Analysis• Cavanaugh (2001) - 16 studies – +0.147 in favor of K-12 distance education• Cavanaugh et al. (2004) - 14 studies – -0.028 for K-12 distance education• Means et al. (2009) - 46 studies (5 on K-12) – +0.24 favoring online over face-to-face – +0.35 favoring blended over face-to-face*
    18. 18. Analyzing Meta-Analyses
    19. 19. Analyzing Meta-Analyses Teacher Effects Zone of Desired EffectsDevelopmentalEffectsReverseEffects
    20. 20. K-12 Distance Education Meta-Analysis• Cavanaugh (2001) - 16 studies – +0.147 in favor of K-12 distance education• Cavanaugh et al. (2004) - 14 studies – -0.028 for K-12 distance education• Means et al. (2009) - 46 studies (5 on K-12) – +0.24 favoring online over face-to-face – +0.35 favoring blended over face-to-face*
    21. 21. Student Performance and StudentsSo are we reallycomparing apples toapples?
    22. 22. The StudentsStudy SampleKozma et al. (1998) vast majority of VHS students in their courses were planning to attend a four-year collegeEspinoza et al., 1999 VHS courses are predominantly designated as ‘honors,’ and students enrolled are mostly college boundHaughey & Muirhead preferred characteristics include the highly(1999) motivated, self-directed, self-disciplined, independent learner who could read and write well, and who also had a strong interest in or ability with technologyRoblyer & Elbaum only students with a high need to control and(2000) structure their own learning may choose distance formats freelyClark et al. (2002) IVHS students were highly motivated, high achieving, self-directed and/or who liked to work independentlyMills (2003) typical online student was an A or B studentWatkins (2005) 45% of the students who participated in e-learning opportunities in Michigan were either advanced placement or academically advanced students
    23. 23. But Is This Representative Of All K-12 Online Students?
    24. 24. Student Reality???• two courses with the highest enrollment of online students in the US are Algebra I & Algebra II (Patrick, 2007)• largest proportion of growth in K–12 online learning enrollment is with full-time cyber schools (Watson et al., 2008)
    25. 25. Student Reality???• many cyber schools have a higher percentage of students classified as “at-risk” (Klein, 2006)• at-risk students are as those who might otherwise drop out of traditional schools (Rapp, Eckes & Plurker, 2006)
    26. 26. K-12 onlinelearners in themajority of theresearchliterature are...
    27. 27. …however, thismay berepresentative ofthe majority of K-12 online
    28. 28. So, What Does The StudentPerformance Research Say About Full-Time K-12 Online Learning?
    29. 29. Including Wider Range of StudentsState of Colorado – 2006 Online Education Performance Audit – “Online student scores in math, reading, and writing have been lower than scores for students statewide over the last three years.” – “The difference in performance between online students and all students statewide is larger in higher grades.” – “Our analysis of Colorado Student Assessment Program results and repeater, attrition, and dropout rates indicate that online schools may not be providing sufficiently for the needs of their students.”
    30. 30. Including Wider Range of StudentsState of Wisconsin – Legislative Audit of Virtual Charter Schools (2010) – “Virtual charter school pupils’ median scores on the mathematics section of the Wisconsin Knowledge and Concepts Examination were almost always lower than statewide medians during the 2005-06 and 2006-07 school years.” – “Because of the relative newness of virtual charter schools and their substantial growth since inception, readily available information on the performance of virtual charter school pupils would be of value to parents, school districts, legislators, and other policymakers.”
    31. 31. Including Wider Range of StudentsState of Colorado – iNews Network Investigation (2011) – “Half of the online students wind up leaving within a year. When they do, they’re often further behind academically then when they started.” – “Online schools produce three times as many dropouts as they do graduates. One of every eight online students drops out of school permanently – a rate four times the state average.” – “Online student scores on statewide achievement tests are consistently 14 to 26 percentage points below state averages for reading, writing and math over the past four years.”
    32. 32. Including Wider Range of StudentsState of Minnesota – 2011 K-12 Online Learning Legislative Audit – “Full-time online students dropped out much more frequently.” – “Compared with all students statewide, full-time online students had significantly lower proficiency rates on the math MCA-II but similar proficiency rates in reading.” – “During both years [i.e., 2008-09 and 2009-10], full- time online students enrolled in grades 4 through 8 made about half as much progress in math, on average, as other students in the same grade.”
    33. 33. Including Wider Range of StudentsMiron, G. & Urschel, J. (2012). Understanding and improving full-time virtual schools. Denver, CO: National Education Policy Center. – “…students at K12 Inc., the nation’s largest virtual school company, are falling further behind in reading and math scores than students in brick-and-mortar schools.” – “These virtual schools students are also less likely to remain at their schools for the full year, and the schools have low graduation rates.” – “Children who enroll in a K12 Inc. cyberschool, who receive full- time instruction in front of a computer instead of in a classroom with a live teacher and other students, are more likely to fall behind in reading and math. These children are also more likely to move between schools or leave school altogether – and the cyberschool is less likely to meet federal education standards.”
    34. 34. The Other Side of the Story…University of Arkansas Internal Evaluation of the Arkansas Virtual Academy School (ARVA)When comparing student performance in mathematics, the researchers found:• students in the F2F group increased their performance by 1% more than the online group from grades 3 to 5 (not statistically significant)• students in the online group increased their performance by 5% more than the F2F group from grades 4 to 6 (not statistically significant)• students in the online group increased their performance by 2% more than the F2F group from grades 5 to 7 (not statistically significant)• students in the online group increased their performance by 16% more than the F2F group from grades 6 to 8 (statistically significant at the p=0.10 level)
    35. 35. The Other Side of the Story…University of Arkansas Internal Evaluation of the Arkansas Virtual Academy School (ARVA)When comparing student performance in literacy, the researchers found:• students in the F2F group increased their performance by 3% more than the online group from grades 3 to 5 (not statistically significant)• students in the online group increased their performance by 11% more than the F2F group from grades 4 to 6 (statistically significant at the p=0.10 level)• students in the online group increased their performance by 2% more than the F2F group from grades 5 to 7 (not statistically significant)• students in the online group increased their performance by 7% more than the F2F group from grades 6 to 8 (not statistically significant)
    36. 36. The Other Side of the Story…University of Arkansas Internal Evaluation of the Arkansas Virtual Academy School (ARVA)Online cohorts performed statistically significantly better than F2F cohorts in 2 of 8 measures!There were methodological limitations in the sample (all of which favored the online students):• the online sample had several of its lowest performing students removed before they had repeated a grade or had dropped out over the two-year period.• the online sample was a more affluent group.• the online sample had significant fewer minority students.
    37. 37. Are More Students Really At-RiskMiron, G. & Urschel, J. (2012). Understanding and improving full-time virtual schools. Denver, CO: National Education Policy Center. – “K12 Inc. virtual schools enroll approximately the same percentages of black students but substantially more white students and fewer Hispanic students relative to public schools in the states in which the company operates” – “39.9% of K12 students qualify for free or reduced lunch, compared with 47.2% for the same-state comparison group.” – “K12 virtual schools enroll a slightly smaller proportion of students with disabilities than schools in their states and in the nation as a whole (9.4% for K12 schools, 11.5% for same-state comparisons, and 13.1% in the nation).” – “Students classified as English language learners are significantly under-represented in K12 schools; on average the K12 schools enroll 0.3% ELL students compared with 13.8% in the same-state comparison group and 9.6% in the nation.”
    38. 38. What’s Really Driving this Growth??
    39. 39. The ChallengeWhether onlinelearning canbe suitable forall K-12 students?(Mulcahy, 2002)
    40. 40. The ChallengeHow do we createenvironmentswhere all K-12students can besuccessful whenthey learn online?
    41. 41. What About the Other Research?
    42. 42. Methodologically Limited FindingsOnline 7 principles of Interviews with teachers and courseCourse effective online developers at a single virtual school,Design course content with no verification of whether the for adolescent interviewees’ perceptions were actuallyBarbour learners effective or any student input at all for(2005; 2007) that matter.Online 37 best Interviews with teachers at a singleTeaching practices in virtual school selected by the virtual asynchronous school itself. Their teachers’ beliefsDiPietro et online teaching were not validated through observational. (2008) of the teaching or student performance.
    43. 43. What Do We Know?1. Support at the local level can determine student success2. Teacher interaction in the asynchronous environment is important a. Possibly through the lens of procedural, instructional, and social3. Preparing the student to learn online
    44. 44. What Do We Know?4. Data has the potential to improve program delivery and individual student performance (but generally goes unused)5. Smaller, targeted programs have shown best results6. Managed growth has prevented academic missteps
    45. 45. Potential Useful Models1. Requirement to target at-risk or dropped out students (Michigan)2. Tying funding to completion and performance (Arizona-defeated proposal)3. Focus on quality assurance (British Columbia)4. Limiting growth (Multiple states)5. Funding full-time K-12 online learning at lower rates (Multiple states)
    46. 46. My Own Research Agenda• Examining the preparation of teachers to design, deliver and support K-12 online learning• Exploring ways to better prepare students to be successful in K-12 online learning environments
    47. 47. My Own Research Agenda• Examining the policy and regulation of K-12 distance education in Canada & elsewhere• Working with individual K-12 online learning programs to help them to effectively design, deliver and support K-12 online learning
    48. 48. My Own Research Agenda• Countering the dominant narrative presented by the neo-liberal supporters of K-12 online learning in the United States (and elsewhere)
    49. 49. Looking for better ways…
    50. 50. YourQuestions andComments
    51. 51. Assistant Professor Wayne State University, USA mkbarbour@gmail.comhttp://www.michaelbarbour.com

    ×