• Save
MidWest REL - Rural K–12 Students’ Learning in Virtual School Environments
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

MidWest REL - Rural K–12 Students’ Learning in Virtual School Environments

  • 348 views
Uploaded on

Barbour, M. K. (2010, August). Rural K–12 students’ learning in virtual school environments. A keynote presentation at the Regional Educational Laboratory – Midwest’s Online Learning Opportunities......

Barbour, M. K. (2010, August). Rural K–12 students’ learning in virtual school environments. A keynote presentation at the Regional Educational Laboratory – Midwest’s Online Learning Opportunities for Rural Schools Conference, Madison, WI.

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
348
On Slideshare
348
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Benefits = Expanding educational access; Providing high-quality learning opportunities; and Allowing for educational choice Challenges = Student readiness issues and retention issues
  • American Journal of Distance Education (United States) - 8 US Journal of Distance Education (Canada) - 4 Cdn / 1 Aus Distance Education (Australia) - 2 Aus / 4 US Journal of Distance Learning (New Zealand) - 1 NZ / 1 Cdn / 1 US-Cdn Last five years - 24 articles out of a total of 262 related to K-12 distance education
  • This is actually quite normal
  • Common link between both assessments is the pre-occupation of researchers with comparing student performance in an effort to show the effectiveness of online
  • As research into comparing student performance between face-to-face and online environments is both the common theme and, by far, the dominant theme... Let’s take a closer look at this body of research.
  • Canadian province of Alberta - online students do as well as classroom students in Social Studies and English, classroom students better in other areas
  • In their two-year evaluation, Bigbie and McCarrol found that more than 50% of Florida Virtual students get As in their courses and very few students failed I n examining 6 virtual schools in 3 Canadian provinces, Barker and Wendel found that online and classroom students performed the same
  • In a NCREL funded study, Cavanaugh and her colleagues found online students in Florida performed better than classroom students Similarly in another NCREL funded study, McLeod and his colleagues found online students in Florida performed better in algebra
  • Even the meta-analysis, or combination of individual studies, found that online student did as well or better.
  • But does this tell really tell the full story???
  • Ballas & Belyk had dramatically differing participation rates - how would the 20%-30% missing from the online group have scored? Bigbie & McCarroll had a significant drop-out rate in the online courses - how would the results have differed had those students stayed enrolled?
  • Cavanaugh and her colleagues speculated that the online students were simply better students McLeod and his colleagues speculated their results were due to the fact that weaker students had dropped out of the online course
  • Let’s examine who the literature says is enrolled in K-12 online learning...
  • First year evaluation of VHS - majority are planning to attend a four year college Second year evaluation - most are honors students and college bound
  • Highly motivated, self-directed, self-disciplined, independent learners who could read and write well, and had a strong interest in or ability with technology
  • need to control and structure their learning highly motivated, high achieving, self-directed, independent workers
  • A or B students half are academically advanced or AP students
  • These have all been studies that use sample data. Maybe the data was simply skewed, by chance, towards the higher ability students.
  • Using population data from a Canadian province where distance education is primarily used in rural jurisdictions. Distance education students out-performed classroom based students each year, and distance students in rural schools were among the highest achievers in most years Note the small differences in performance (~3% or less)
  • Even when we break this out into subject areas, the distance students still do better than the classroom students and the distance students attending rural schools do as well or better than everyone else But even though this is population data, are we actually comparing apples to apples?
  • Newfoundland and Labrador still have streaming in English language arts and mathematics. Students can enroll in an academic stream of courses and be eligible for college or university, or they can enroll in a basic stream and be able to graduate from high school and attend trade school. The online program in the province does not offer basic courses, so all basic courses are taught in the classroom.
  • Note that the basic stream enrollment for English language arts is higher in rural schools where students would have to take an online course to enroll in the academic stream, as compared to the province average and the average of all of the schools using distance education.
  • Also note that the basic stream enrollment for mathematics is higher in rural schools where students would have to take an online course to enroll in the academic stream, compared to the province average. Even when using population data, there is compelling evidence that the population of students included in the research is simply a higher calibre of students, so we would expect them to do better when we compare performance.
  • However, is that really the description of all K-12 online learning students?
  • Supplemental - algebra Full-time - higher proportion of at-risk students
  • The research is based upon the best and the brightest.
  • However, we know from practice that this does not reflect all or even the majority of K-12 online learners. So the population of students the research focuses on is one of the main limitations of the usefulness (and even the believability) of much of that research.
  • Another problem is what we measure... 1. Correlation does not equal causality 2. Single studies measure if there is a difference between two groups beyond chance Need for meta-analysis...
  • A K-12 online learning example from earlier Cathy took 16 individual studies and combined the results to determine an overall effect size.
  • Things that hurt student learning
  • 0.15 - The amount a student would increase simply from being a year older and a year wiser / maturity
  • 0.25 - The amount student learning increases based upon an average teacher
  • 0.40 - The magic number... If it doesn’t reach beyond 0.4, it likely isn’t worth it. Some scholars have argued as high as 0.6 or 0.8. Recall earlier I mentioned three different meta-analysis related to K-12 online learning.
  • Cavanaugh (2001) - developmental effects Cavanaugh et al. (2004) - reverse effects Means et al. (2009) - online = teacher effects & blended = developmental effects + teacher effects
  • In fact, if you look at many of the factors that proponents of K-12 online learning trumpet, most have little impact on student learning beyond what an average teacher and the normal process of aging would have. So, what do Hattie’s findings tell us?
  • Good teachers and the act of teaching well can have significant impacts Some design and delivery lessons applicable to K-12 online learning: direct instruction, mastery learning, worked examples, concept mapping, setting goals But this is just the research on student performance, what about the other research?
  • Most of the remainder research is also problematic - primarily due to methodological limitations and overreaching Barbour - principles of effective online course design based upon interviews with teachers at a single virtual school DiPietro et al. - best practice in online teaching based upon interviews with teachers at a single virtual school
  • Which naturally leads to the question of how should we be doing educational research when it comes to K-12 online learning?
  • Begins with the involvement of the local stakeholders in identifying the challenges to be addressed and the interventions to be used. In addition to trying to solve the local problem, there is a focus on the development of theory to explain what occurs in the local context. Finally, there are multiple cycles of analysis and revisions to ensure that changes become part of the routine of those involved in the system. Probably the only example or closest example of DBR in the K-12 online learning field is the Virtual High School Global Consortium.
  • The VHS Global Consortium was established through a five year, $7.4 million federal grant, as such there was an expectation for evaluations and research. This research was conducted with the VHS staff as a full participant (i.e., being involved in identifying the issues that needed to be examined, assisting in the design and completion of the research, implementing the recommendations, and then repeating the process to ensure the recommendations had the desired outcomes). As a result of these cycles of inquiry that examined a variety of problems in this specific context, along with the close relationship between VHS staff and the SRI International evaluation team in the design of both the virtual school and the evaluations, much of what is still known about virtual schools comes from this refined approach. It is also worth noting that the VHS has not only continued to operate since the end of the initial federal funding, but has thrived. In fact, the VHS is one of the largest and most successful virtual schools, and one of the few virtual schools that doesn’t rely upon direct government funding.
  • It all depends. If all we have is a single method that we use to design, delivery and support online learning; how is that really different than the teacher that lectures every single day, expecting their students - who are frantically taking notes - to keep up. The way we design, delivery and support online learning targeted to at-risk students must be different than online learning targeted to AP students. One of the best ways to figure out how to do that and do it well, is to have our research focus on local challenges through methodologies like design-based research.

Transcript

  • 1. Rural K–12 Students’Learning in Virtual School Environments Michael K. Barbour Assistant Professor Wayne State University
  • 2. Agenda1. What does the literature and research say?2. What does this mean for K-12 online learning?3. What should we do next?
  • 3. Literature Reviews1. Rice (2006) – Journal of Research on Technology in Education1. Barbour & Reeves (2009) – Computers and Education1. Cavanaugh, Barbour, & Clark (2009) – International Review of Research in Open
  • 4. What does the literature say?• “based upon the personal experiences of those involved in the practice of virtual schooling” (Cavanaugh et al., 2009)• described the literature as generally falling into one of two general categories: the potential benefits of and challenges facing K- 12 online learning (Barbour & Reeves, 2009) 4
  • 5. What about research?• “a paucity of research exists when examining high school students enrolled in virtual schools, and the research base is smaller still when the population of students is further narrowed to the elementary grades” (Rice, 2006) 5
  • 6. Is this a problem?“indicative of the foundational descriptive workthat often precedes experimentation in anyscientific field. In other words, it is important toknow how students in virtual school engage intheir learning in this environment prior toconducting any rigorous examination of virtualschooling.” (Cavanaugh et al., 2009) 6
  • 7. What does the research say?1. Comparisons of student performance based upon delivery model (i.e., classroom vs. online)2. Studies examining the qualities and characteristics of the teaching/learning experience – characteristics of – supports provided to – issues related to isolation of online learners (Rice, 2006)1. Effectiveness of virtual schooling2. Student readiness and retention issues (Cavanaugh et al., 2009) 7
  • 8. So, what does the studentperformance research say? 8
  • 9. Student Performance• performance of virtual and classroom students in Alberta were similar in English and Social Studies courses, but that classroom students performed better overall in all other subject areas (Ballas & Belyk, 2000)
  • 10. Student Performance• over half of the students who completed FLVS courses scored an A in their course and only 7% received a failing grade (Bigbie & McCarroll, 2000)• students in the six virtual schools in three different provinces performed no worse than the students from the three conventional schools (Barker & Wendel, 2001)
  • 11. Student Performance• FLVS students performed better on a non-mandatory assessment tool than students from the traditional classroom (Cavanaugh et al., 2005)• FLVS students performed better on an assessment of algebraic understanding than their classroom counterparts (McLeod et al., 2005)
  • 12. Meta-Analysis• Cavanaugh (2001) – +0.147 in favor of K-12 distance education• Cavanaugh et al. (2004) – -0.028 for K-12 distance education• Means et al. (2009) – +0.24 favoring online over face-to-face – +0.35 favoring blended over face-to-face 12 12
  • 13. Let’s look a little closer... 13
  • 14. Students and Student PerformanceBallas & performance of virtual and participation rate in theBelyk, 2000 classroom students similar assessment among virtual in English & Social Studies students ranged from 65% to courses, but classroom 75% compared to 90% to students performed better 96% for the classroom-based in all other subject areas studentsBigbie & over half of the students between 25% and 50% ofMcCarroll, who completed FLVS students had dropped out2000 courses scored an A in of their FLVS courses over their course and only 7% the previous two-year received a failing grade period 14
  • 15. Students and Student PerformanceCavanaugh et FLVS students performed speculated that the virtualal., 2005 better on a non- school students who did mandatory assessment take the assessment may tool than students from have been more the traditional classroom academically motivated and naturally higher achieving studentsMcLeod et FLVS students performed results of the studental., 2005 better on an assessment performance were due to of algebraic understanding the high dropout rate in than their classroom virtual school courses counterparts 15
  • 16. Student Performance and StudentsSo are we reallycomparing apples toapples?
  • 17. The Students• the vast majority of VHS Global Consortium students in their courses were planning to attend a four-year college (Kozma, Zucker & Espinoza, 1998)• “VHS courses are predominantly designated as ‘honors,’ and students enrolled are mostly college bound” (Espinoza et al., 1999)
  • 18. The StudentsThe preferred characteristicsinclude the highly motivated,self-directed, self-disciplined,independent learner whocould read and write well,and who also had a stronginterest in or ability withtechnology (Haughey &Muirhead, 1999)
  • 19. The Students• “only students with a high need to control and structure their own learning may choose distance formats freely” (Roblyer & Elbaum, 2000)• IVHS students were “highly motivated, high achieving, self-directed and/or who liked to work independently” (Clark et al., 2002)
  • 20. The Students• the typical online student was an A or B student (Mills, 2003)• 45% of the students who participated in e-learning opportunities in Michigan were “either advanced placement or academically advanced” students (Watkins, 2005)
  • 21. From sample to population... 21
  • 22. Barbour & Mulcahy - Student Performance 22
  • 23. Barbour & Mulcahy - Student Performance 23
  • 24. Streaming...• English language arts• mathematics• academic stream - graduation, college, university, etc.• basic stream - graduation, trade school• K-12 online learning program only offers academic streamed courses
  • 25. Enrollment - English Language Arts 25
  • 26. Enrollment - Mathematics 26
  • 27. But does this represent all of our online students? 27
  • 28. Student Reality???• two courses with the highest enrollment of online students in the US are Algebra I & Algebra II (Patrick, 2007)• the largest growth in K–12 online learning enrollment is with full-time cyber schools (Watson et al., 2008)• many cyber schools have a higher percentage of students classified as “at-risk” (Klein, 2006)• at-risk students are as those who might otherwise drop out of traditional schools (Rapp, Eckes & Plurker, 2006) 28
  • 29. Literatureindicates K-12online learningstudents are... 29
  • 30. Reality of most K-12 online learningstudents? 30
  • 31. Problem With Student Performance Studies 31
  • 32. Cavanaugh (2001)• Allen & Thompson (1995)• Blanton et al. (1997)• Burkman (1994)• Center for Applied Special Technology (1996)• Erickson (1992)• Gray (1996)• Hinnant (1994)• Libler (1991)• Martin & Rainey (1993) 32 32
  • 33. Problem of Effect SizesReverseEffects 33
  • 34. Problem of Effect SizesDevelopmentalEffects 34
  • 35. Problem of Effect Sizes Teacher Effects 35
  • 36. Problem of Effect Sizes Zone of Desired Effects 36
  • 37. Synthesis of Meta-Analysis• Cavanaugh (2001) - 16 studies – +0.147 in favor of K-12 distance education• Cavanaugh et al. (2004) - 14 studies – -0.028 for K-12 distance education• Means et al. (2009) - 46 studies (5 on K-12) – +0.24 favoring online over face-to-face* – +0.35 favoring blended over face-to-face* 37 37
  • 38. Results of Interest• Second and third chance programs (d=0.50)• Matching style of learning (d=0.40)• Computer assisted instruction (d=0.37)• Decreasing disruptive behavior (d=0.34)• Programmed instruction (d=0.24)• Individualized instruction (d=0.23)• Class size (d=0.21)• Charter schools (d=0.20)• Web-based learning (d=0.18)• Home-school programs (d=0.16)• Teacher training (d=0.11)• Teacher subject matter knowledge (d=0.09)• Distance education (d=0.09)• Student control over learning (d=0.04) 38 38
  • 39. Results to Consider• Providing formative evaluation (d=0.90)• Micro teaching (d=0.88)• Teacher clarity (d=0.75)• Providing feedback (d=0.73)• Teacher-student relationships (d=0.72)• Teaching strategies (d=0.60)• Cooperative vs. individualistic learning (d=0.59)• Study skills (d=0.59)• Direct instruction (d=0.59)• Mastery learning (d=0.58)• Worked examples (d=0.57)• Concept mapping (d=0.57)• Goals (d=0.56)• Peer tutoring (d=0.55)• Cooperative vs. competitive learning (d=0.54) 39 39
  • 40. What about the other research? 40
  • 41. Problematic ResearchOnline 7 principles of Interviews with teachers and courseCourse effective online developers at a single virtual school,Design course content with no verification of whether the for adolescent interviewees’ perceptions were actuallyBarbour learners effective (or any student input for that(2005; 2007) matter)Online 37 best Interviews with teachers at a singleTeaching practices in virtual school selected by the virtual asynchronous school itself. Their teachers’ beliefsDiPietro et online teaching were not validated through observational. (2008) of the teaching or student performance. 41
  • 42. Is there a better way? 42
  • 43. Design-Based Research Reeves (2006) 43
  • 44. Virtual High School Global Consortium • annual evaluations – e.g., Espinoza, Dove, Zucker & Kozma, 1999; Kozma, Zucker & Espinoza, 1998; Kozma, Zucker, Espinoza, McGhee, Yarnall & Zalles, 2000 • content-specific investigations – e.g., Elbaum, McIntyre & Smith, 2002; Yamashiro & Zucker, 1999 • final evaluation – e.g., Zucker & Kozma, 2003 44 44
  • 45. The ChallengeWhether online learning can be suitable for all K-12 students? (Mulcahy, 2002)
  • 46. YourQuestions andComments
  • 47. Assistant Professor Wayne State University, USA mkbarbour@gmail.com http://www.michaelbarbour.comhttp://virtualschooling.wordpress.com