Pedagogical and Assessment Design for Online Learning

1,610 views

Published on

6 March 2010 (Saturday) | 09:00 - 10:00 | http://citers2010.cite.hku.hk/abstract/32 | Dr. Barbara MEANS, Center for Technology in Learning, SRI International

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,610
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • The Open Learning Initiative is a project started at Carnegie Mellon University with funding from the William and Flora Hewlett Foundation.
  • Pedagogical and Assessment Design for Online Learning

    1. 1. Pedagogical and Assessment Designs for Online Learning Barbara Means Center for Technology in Learning SRI International Keynote presentation for the CITE Research Symposium E-Learning Designs & Designs for Learning March 6, 2010 Hong Kong © 2007 SRI International
    2. 2. Outline of Talk  Emergence of online learning at secondary level in U.S.  Variations in online learning designs  Research synthesis on the effectiveness of online learning  Role of formative assessment in enhancing learning outcomes for blended instruction © 2007 SRI International 2
    3. 3. Growing Importance of Online Learning  Long a part of corporate and military training  Online learning has become a big part of higher education and of self-directed adult learning  Now it is becoming part of K-12 education (mostly at the secondary level) © 2007 SRI International
    4. 4. K-12 adoption of online learning rising in U.S. Enrollments up from 45,000 in 2000 to 1,000,000 in 2007 Online Learning Enrollment 0.08 0.07 0.06 0.05 0.04 Online Learning 0.03 Enrollment 0.02 0.01 0 1998 2000 2002 2004 2006 2008 Source: Estimates of secondary school online enrollments from Picciano & Seaman, 2009 Copyright Clayton M. Christensen
    5. 5. K-12 adoption of online learning rising in U.S. Enrollments up from 45,000 in 2000 to 1,000,000 in 2007 0.10 0.09 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 Online Learning Enrollment 0 1998 2000 2002 2004 2006 2008 2010 2012 2014 Source: Estimates of secondary school online enrollments from Picciano & Seaman, 2009 Copyright Clayton M. Christensen
    6. 6. K-12 adoption of online learning rising in U.S. 0.10 0.09 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 Online Learning Enrollment 0 1998 2000 2002 2004 2006 2008 2010 2012 2014 Clayton Christensen predicts that by 2019 half of all high school courses will be taken online. Copyright Clayton M. Christensen
    7. 7. U.S. districts and states are starting to encourage online learning  Most school districts (75% in 2007) provide online learning options for at least some of their students  Several state-funded programs offer online high school courses (Florida, Illinois, Michigan, Kentucky)  24 of 50 states now have statewide full-time online schools (“cyberschools”)  Two states (Michigan and Alabama) require students to take an online course in order to earn a high school diploma  One state (Florida) requires all of its districts to make online learning options available to their students © 2007 SRI International 7
    8. 8. Typical Reasons for Offering Online Course Options  Provide a course not otherwise available  Provide more college-level (Advanced Placement) course options  Give students a second chance through “credit recovery” programs © 2007 SRI International 8
    9. 9. Different Forms of Online Learning  Fully online v. hybrid or “blended”  Module within a course, supplemental formal course, degree program  Scheduled course (“class-linked”) v. self-paced  Instructor-developed v. professionally developed © 2007 SRI International
    10. 10. Instructional Design Elements in Online Learning  Communication type - Synchronous - Asynchronous  Learner control  Instructional approach - Expository - Active - Interactive © 2007 SRI International
    11. 11. Expository Learning Environments Present content to the learner © 2007 SRI International 11
    12. 12. Active Learning Designs Support the practice of skills and elicit student responses Carnegie Learning’s Algebra I Tutor © 2007 SRI International 12
    13. 13. Interactive Learning Designs Involve collaboration with other learners or an instructor with the focus of engagement emerging over time Political Science 105 from University of North Carolina Greensboro © 2007 SRI International 13
    14. 14. SRI’s Meta-Analysis of Online Learning Research  Systematic search for studies from 1996 through July 2008 on the effectiveness of online learning  Limited to studies that - compared an e-learning or blended condition to face-to-face instruction -measured learning objectively with the same measure for treatment and control groups -used an experimental or quasi- experimental design with control for any pre-existing differences between groups -provided the statistical information needed to compute an effect size  Out of 1,132 articles reviewed, 51 study effects met the requirements for meta-analysis © 2007 SRI International
    15. 15. Meta-Analysis Findings  Average effect size of +0.24 favoring the online condition  Advantage over face-to-face instruction was larger for studies using blended approaches (E.S. = +0.35) than for studies using pure e-learning (E.S. = +0.14)  Advantage over face-to-face instruction was larger for studies where - online students spent more time learning than did those in the face-to- face class - the online and face-to-face conditions varied in terms of content and instructional approach Taken as a whole, the findings suggest that the observed advantage of online learning is a product of redesigning the learning experience, not of the medium per se. © 2007 SRI International
    16. 16. SRI’s Review of Studies Comparing Different Online Learning Designs  Online learning usually more effective when - it stimulates more active engagement with the content - it includes prompts for learner reflection -learners have an element of control over their interactions with the software  Some common practices usually had no effect -adding additional media not related to the content to be learned - adding multiple-choice quizzes Results were inconclusive with respect to different instructional approaches www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf © 2007 SRI International
    17. 17. Conjectures on the Blended Advantage  More learning time  More interactivity  Selective re-teaching of difficult concepts and skills © 2007 SRI International
    18. 18. The Concept of Formative Assessment  Assessment for learning rather than assessment of learning  Occurs during the course of learning while there is still time to improve outcomes  A quality of the way in which assessment information is used, not inherent in the test itself  One of the most powerful levers for improving learning outcomes (Black & Wiliam, 1998) © 2007 SRI International
    19. 19. e-Learning Offers New Options for Formative Assessment  We can do much more than just interspersing sets of multiple-choice questions into instructional materials  Technology enables automated capture of complex learner behaviors  Assessment can be embedded in the learning in ways that feel natural rather than forced © 2007 SRI International 19
    20. 20. Learning in a Multi-User Virtual Environment Chris Dede’s River City MUVE (Science, 2009) Enlargement of Microscope © 2007 SRI International 20
    21. 21. River City  Teaches concepts from biology, ecology and epidemiology  Students work in teams of three, moving through the city to run tests in response to the mayors’ challenge  Teams keep online journals, analyze data, form hypotheses, and write up their research in a report for the mayor  Researchers developed measures of science concept knowledge, science inquiry skills, and sense of efficacy as a scientist based on student actions within the River City environment © 2007 SRI International 21
    22. 22. Such online learning environments can be designed to capture . . .  Where a student went  With whom the student communicated and what was said  The artifacts the student activated  Databases the student used  Data that the student gathered using virtual scientific instruments  Screenshots and notations the student entered in a virtual notebook  Hints that the learner requested All of this information can be used for assessment. © 2007 SRI International 22
    23. 23. Examples from Carnegie-Mellon’s Online Learning Initiative (OLI) 12 web-based colleges courses that can be taken entirely online
    24. 24. Examples from Carnegie-Mellon’s Online Learning Initiative (OLI)  Some of the systems contain virtual laboratories © 2007 SRI International 24
    25. 25. Examples from Carnegie-Mellon’s Online Learning Initiative (OLI)  Some of the systems contain virtual laboratories © 2007 SRI International 25
    26. 26. Examples from Carnegie-Mellon’s Online Learning Initiative (OLI)  Others incorporate intelligent tutoring and carefully designed exercises © 2007 SRI International 26
    27. 27. Examples from Carnegie-Mellon’s Online Learning Initiative (OLI)  Others incorporate intelligent tutoring and carefully designed exercises © 2007 SRI International 27
    28. 28. The Boxplot The boxplot graphically represents the distribution of a quantitative variable by visually displaying The five number summary and any observation that was classified as a suspected outlier using the 15(IQR) criterion. Here is how a boxplot is constructed (this is for the “Best Actress” dataset – to see the dataset click here.) To see a static version of this movie, click here. Use the pull-down menu to label the various points on the boxplot. X That is not quite right. I think you might be Source: Carnegie-Mellon University, OLI confusing Q1 and Q3. Recall that Q1, the first quartile of the distribution, Course in Statistics is represented By the bottom edge of the box. Q3, the third quartile of the 28 © 2007 SRI International
    29. 29. Multiple Feedback Loops Source: Thille, 2009 © 2007 SRI International 29
    30. 30. Formative Assessment Is Woven into the Online Learning  Interactive simulations that students can manipulate are woven into the courseware. Students interact with the simulation and then respond to probes that get at their understanding of how the simulation works.  “Did I Get This?” quizzes follow the presentation of new material so that students can check their understanding without fear of hurting their course grade.  Short essay questions that call on students to make connections between different concepts are embedded throughout the online course material.  “Muddiest Point” requests ask students what they thought was confusing. © 2007 SRI International 30
    31. 31. In Blended OLI Courses Instructors Take Advantage of eLearning Assessments Each student works with the online course materials outside of class. © 2007 SRI International 31
    32. 32. In Blended OLI Courses Instructors Take Advantage of eLearning Assessments Source: Heffernan & Thille, Carnegie Mellon Online Learning Initiative web presentation The e-learning system aggregates information from all the students who used it. © 2007 SRI International 32
    33. 33. In Blended OLI Courses Instructors Take Advantage of eLearning Assessments The instructor sees what concepts students had trouble with. © 2007 SRI International 33
    34. 34. In Blended OLI Courses Instructors Take Advantage of eLearning Assessments In class, the instructor focuses on concepts that students struggled with when they worked on line. © 2007 SRI International 34
    35. 35. The OLI Statistics Course Study  Students were assigned at random to take college statistics on line or in a conventional class.  Students in the online version of the class finished the material in half a semester or half the time taken by students in the conventional class.  The two classes took the same final examination, and the online students’ scores were significantly better than those of the students in the conventional class.  No difference between the two groups on a long-term retention test.  Study has been replicated in a community college and a large public university. Source: Lovett, Meyer, & Thille, 2008 © 2007 SRI International 35
    36. 36. Principles for e-Learning Design  Incorporate a range of online activities that engage learners with content.  Give learners options for different ways to learn content and with options for review and support (scaffolding).  Embed assessment within the learning activities and use multiple assessment approaches.  Make assessment feedback available to the student as well as to the instructor.  Use information from students’ online activity to shape the content of face-to-face instruction. © 2007 SRI International 36
    37. 37. Thank you! NIH Media and Children May 15,

    ×