Effects of Advance Organizers on Learning and Retention from a Fully Web-based Class

3,226 views

Published on

The purpose of this study is to investigate the short-term and long-term effects of two kinds of advance organizers (AOs), a visual concept map and a text outline. The AOs were administered in a fully Web-based course in health care ethics. The outcome measures are students’ knowledge acquisition and application in two posttests.
This study was conducted through a post-test only control group design with a random assignment. The population of the study involved 166 college students who participated in this online class in their junior or senior year. The voluntary research participants were randomly assigned into the two treatment groups and one control group.
The treatment of AO was administered as an integral part of a one-week-long online module on the topic of patient-physician relationships. Students of the two treatment groups were presented with one of the two AOs, while the control group was instructed to proceed to textbook reading without an AO. Then, students were tested on the subject matter with two parallel posttests. Both posttests were composed of a multiple-choice question quiz and a set of scenario-based essay questions. The students took posttest I at the end of the instructional week, and posttest II four weeks after. A survey and interviews were also conducted to supplement the quantitative results with contextual information.
The findings do not demonstrate a statistically significant AO effect among the treatment groups and the control group. However, in agreement with the previous research, this study shows a positive but inconclusive benefit of using AOs for students’ short-term knowledge acquisition. The students using a concept map consistently obtained higher learning achievements than individuals using a text outline. More importantly, this study reiterated the proposition that students of lower-learning abilities benefit more from using an AO for online learning than those of higher-learning abilities.
The current study extends our knowledge on the use of AOs in fully Web-based educational environments. The results indicated that although AOs more often than not have small facilitative effects for learners, they are not equally effective for all learners in all learning situations. The incorporation of the instructional strategies, such as AOs, in Web-based courses and programs might benefit online learners, especially those students of lower verbal and analytical abilities, or of lower prior knowledge of the material-to-be-learned.

Published in: Education, Technology
  • Be the first to comment

Effects of Advance Organizers on Learning and Retention from a Fully Web-based Class

  1. 1. Effects of Advance Organizers on Learning and Retention from a Fully Web-based Class Baiyun Chen, Ph.D. Candidate/M.A. College of Education University of Central Florida March 5, 2007
  2. 2. INTRODUCTION
  3. 3. Introduction <ul><li>Online learning has entered the mainstream of higher education in the United States (Allen & Seaman, 2004, 2005, 2006) </li></ul>(+18%) (+24%) (+35%)
  4. 4. Introduction <ul><li>Online courses have become one of the major learning modalities at UCF (Center for Distributed Learning, 2006). </li></ul>
  5. 5. Problem Statement <ul><li>Online learning challenges (Dias & Sousa, 1997) </li></ul><ul><ul><li>Cognitive overload </li></ul></ul><ul><ul><li>Learner disorientation </li></ul></ul><ul><li>An increasing need to exploit research-based pedagogical strategies in online learning (Bonk & Dennen, 2003) </li></ul>
  6. 6. Problem Statement <ul><li>Use of advance organizers (AOs) as an effective learning strategy in traditional and computer-assisted face-to-face classroom/lab </li></ul><ul><li>Dearth of studies on AOs in fully web-based classes. </li></ul>
  7. 7. Purpose of the Study <ul><li>To investigate both short-term and long-term effects of AOs on students’ academic achievements in a fully web-based class </li></ul>
  8. 8. Hypotheses <ul><li>Null hypothesis I: There is no difference in the short-term knowledge-based and performance-based learning achievements among students in the graphic AO (concept map), text AO (outline) and control groups. </li></ul><ul><li>Null hypothesis II: There is no difference in the long-term knowledge-based and performance-based learning achievements among students in the graphic AO (concept map), text AO (outline) and control groups. </li></ul>
  9. 9. Literature Review: Definition <ul><li>Advance organizers (AOs)– relatively brief introductions </li></ul><ul><ul><li>Abstract, inclusive, and general </li></ul></ul><ul><ul><li>Relatable to existing relevant ideas </li></ul></ul><ul><ul><ul><ul><ul><li>— D.P. Ausubel, 1968, 1978, & 2000 </li></ul></ul></ul></ul></ul>
  10. 10. Literature Review: AO Examples <ul><li>Step-By-Step Chart </li></ul><ul><li>Series of Events Chain </li></ul><ul><li>Sequence Organizer </li></ul><ul><li>Cause and Effect Chain </li></ul><ul><li>Minute Paper </li></ul><ul><li>Timeline </li></ul><ul><ul><ul><ul><ul><li>— Calandra, 2002; Minchin Jr., 2004 </li></ul></ul></ul></ul></ul>
  11. 11. Literature Review: AO Framework <ul><li>Students with AOs perform better than students without AOs </li></ul><ul><li>AO effect is at least as great in longer studies as in shorter ones </li></ul><ul><li>Graphic AOs are at least as effective as text AOs </li></ul><ul><li>Students having low learning-ability are helped more by AOs than other students </li></ul><ul><ul><li>— Ausubel, 1968, 2000; Kenny, 1993, Mayer, 1976b; Stone, 1983 </li></ul></ul>
  12. 12. Literature Review: Research Evidence <ul><li>Advance organizers (AOs) </li></ul><ul><ul><li>Positive effects in traditional f2f classroom </li></ul></ul><ul><ul><ul><li>— Ausubel, 1960, 1961, 1962, 1963; Luiten et al., 1980; Mayer, 1979b; Stone, 1983 </li></ul></ul></ul><ul><ul><li>Positive effects in computer-assisted f2f classroom/lab </li></ul></ul><ul><ul><ul><li>— Calandra, 2002; Kenny, 1993; Lang & Barron, 2002; Hale, 2003; Minchin Jr., 2004; Tseng et al., 2002; Yeh & Lehman, 2001 </li></ul></ul></ul><ul><ul><li>Positive effects of concept mapping in Web-based class </li></ul></ul><ul><ul><ul><li>— Zittle, 2001 </li></ul></ul></ul>
  13. 13. METHODS
  14. 14. Methods: Population <ul><li>Participants </li></ul><ul><ul><li>166 undergraduate students enrolled in a fully Web-based class in Fall 2006 at UCF </li></ul></ul><ul><li>Sample </li></ul><ul><ul><li>Participants who complete all instruments of the study (112) </li></ul></ul>
  15. 15. Methodology: Research Design <ul><li>R E 1 X 1 ( Graphic Advance Organizer ) O 1 O 2 </li></ul><ul><li>R E 2 X 2 ( Text Advance Organizer ) O 3 O 4 </li></ul><ul><li>R C (No Advance Organizer) O 5 O 6 </li></ul>Note : R – Random Assign; E 1 – Experimental Group 1 E 2 – Experimental Group 2; C – Control Group Posttest I: O 1 , O 3 , O 5 ; Posttest II: O 2 , O 4 , O 6
  16. 16. Methodology: Intervention– AO Concept Map
  17. 17. Methodology: Intervention– AO Outline
  18. 18. Methodology: Instruments <ul><li>Posttest I </li></ul><ul><ul><li>Quiz 1 (9 questions) </li></ul></ul><ul><ul><li>Scenario 1 (3 questions) </li></ul></ul><ul><li>Posttest 2 </li></ul><ul><ul><li>Quiz 2 (9 questions) </li></ul></ul><ul><ul><li>Scenario 2 (3 questions) </li></ul></ul><ul><li>Additional analysis: </li></ul><ul><ul><li>Student survey (19 questions) </li></ul></ul>
  19. 19. Methodology: Procedures <ul><li>Week 1 </li></ul>Week 2 Week 6
  20. 20. ANALYSES AND FINDINGS
  21. 21. Data Analyses <ul><li>Descriptive </li></ul><ul><ul><li>Means, Standard Deviations, Effect Sizes </li></ul></ul><ul><li>ANOVA </li></ul><ul><ul><li>Post test I (All learners; differentiated learners) </li></ul></ul><ul><ul><li>Post test II (All learners; differentiated learners) </li></ul></ul><ul><li>Repeated-Measure Regression (RMR) </li></ul><ul><ul><li>Prior knowledge </li></ul></ul><ul><ul><li>Previous online learning experience </li></ul></ul><ul><ul><li>Computer abilities </li></ul></ul><ul><ul><li>Study time </li></ul></ul><ul><ul><li>GPA </li></ul></ul>
  22. 22. Descriptive Data <ul><li>Population </li></ul><ul><ul><li>166 student participants </li></ul></ul><ul><ul><li>112 completed all instruments </li></ul></ul><ul><li>Instruments </li></ul><ul><ul><li>Posttest I </li></ul></ul><ul><ul><ul><li>Quiz 1 (144 responses) </li></ul></ul></ul><ul><ul><ul><li>Scenario 1 (131 responses) </li></ul></ul></ul><ul><ul><li>Posttest II </li></ul></ul><ul><ul><ul><li>Quiz 2 (128 responses) </li></ul></ul></ul><ul><ul><ul><li>Scenario 2 (128 responses) </li></ul></ul></ul><ul><ul><li>Student Survey (144 responses) </li></ul></ul>
  23. 23. Descriptive Data <ul><li>Demographic Information (144 responses) </li></ul><ul><ul><li>Age (126 students older than 21-year-old) </li></ul></ul><ul><ul><li>Class standing (103 senior students) </li></ul></ul><ul><ul><li>Major (83 Health Service Administration) </li></ul></ul><ul><ul><li>Gender (111 female; 33 male) </li></ul></ul><ul><ul><li>Ethnicity (83 white/Caucasian) </li></ul></ul>
  24. 24. Results: Descriptive Statistics (Post I) Mean & Std Deviation of Tests Note : Group 1—Experimental group with concept map Group 2—Comparison group with outline Group 3—Control group 25 22.76 22.46 22.60 22.89 Mean Scenario 1 (Posttest I-Part B) 0.19 0.25 ES 2.08 17.75 59.38 Total 90 Note: Full Score 0.06 0.19 ES 2.14 2.24 2.28 Std Deviation 18.67 16.33 18.19 Std Deviation 56.59 60.00 61.11 Mean Quiz 1 (Posttest I-Part A) 3 2 1 Group
  25. 25. Results : Hypothesis I Tests of Between-Subject Effects Dependent Variable: Quiz 1 & Senario 1 0.66 0.422 2.087 2 4.175 SENARIO1 0.44 0.826 260.89 2 521.78 QUIZ1 p Value F Mean Square df Sum of Squares Source
  26. 26. Conclusion I <ul><li>There is no difference in the short-term knowledge-based and performance-based learning achievements among students in the concept map, outline and control groups. </li></ul>
  27. 27. Results: Descriptive Statistics (Post II) Means & Std Deviations of Tests Note : Group 1—Experimental group with concept map Group 2—Comparison group with outline Group 3—Control group 25 21.56 21.69 21.36 21.67 Mean Scenario 2 (Posttest II-Part B) -0.22 -0.09 ES 3.05 16.55 58.91 Total 90 Note: Full Score -0.11 -0.01 ES 2.74 3.45 2.91 Std Deviation 16.17 17.82 14.06 Std Deviation 61.11 57.39 59.78 Mean Quiz 2 (Posttest II-Part A) 3 2 1 Group
  28. 28. Results: Hypothesis II Tests of Between-Subject Effects Dependent Variable: Quiz 2 & Scenario 2 0.85 0.16 1.496 2 2.99 Scenario 2 0.57 0.57 148.19 2 296.38 Quiz 2 p Value F Mean Square df Sum of Squares Source
  29. 29. Conclusion II <ul><li>There is no difference in the long-term knowledge-based and performance-based learning achievements among students in the concept map, outline and control groups. </li></ul>
  30. 30. Results: AO Time Effects on Quizzes (High-Scorers) Descriptive Analysis of Quiz Scores (High-Scorers) ANOVA Summary Table: Quiz 1 & Quiz 2 (High-Scorers) 90 62.99 67.14 60.71 62.14 Mean Quiz 2 (Posttest II-Part A) 71.30 Total 90 Note: Full Score 71.43 70.36 72.14 Mean Quiz 1 (Posttest I-Part A) 3 2 1 Group 0.000* p Value 0.18 15.88 2404.82 1 2404.82 Time PES F Mean Square df Sum of Squares Source
  31. 31. Results: AO Time Effects on Scenarios (High-Scorers) Descriptive Analysis of Scenario Scores (High-Scorers) ANOVA Summary Table: Scenario 1 & Scenario 2 (High-Scorers) 25 22.51 22.43 22.54 22.55 Mean Scenario 2 (Posttest II-Part B) 23.88 Total 25 Note: Full Score 23.86 24.04 23.74 Mean Scenario 1 (Posttest I-Part B) 3 2 1 Group 0.000* p Value 0.19 18.16 73.44 1 73.44 Time PES F Mean Square df Sum of Squares Source
  32. 32. Results: AO Time Effects on Quizzes (Low-Scorers) Descriptive Analysis of Quiz Scores (Low-Scorers) ANOVA Summary Table: Quiz 1 & Quiz 2 (Low-Scorers) 90 53.88 52.67 53.53 55.29 Mean Quiz 2 (Posttest II-Part A) 41.84 Total 90 Note: Full Score 39.33 42.94 42.94 Mean Quiz 1 (Posttest I-Part A) 3 2 1 Group 0.0008* p Value 0.40 31.18 3569.56 1 3569.57 Time PES F Mean Square df Sum of Squares Source
  33. 33. Results: AO Time Effects on Scenarios (Low-Scorers) Descriptive Analysis of Scenario Scores (Low-Scorers) ANOVA Summary Table: Scenario 1 & Scenario 2 (Low-Scorers) 25 20.36 21.09 20.21 19.75 Mean Scenario 2 (Posttest II-Part B) 19.93 Total 25 Note: Full Score 19.91 20.18 19.60 Mean Scenario 1 (Posttest I-Part B) 3 2 1 Group 0.347 p Value 0.03 0.91 3.564 1 3.564 Time PES F Mean Square df Sum of Squares Source
  34. 34. Results: Survey Survey: Experience with AO Note : Students’ responses to their uses of AO. 48 52 Total 40 83.3% Agree 35 67.3% Agree Usefulness 20 41.7% Before textbook 26 50.0% Before textbook When read 25 41.9% Once 23 44.2% Once How many times read AO 23 47.9% 1-5 min 24 46.2% 6-10 min Time spent on AO N % Mean N % Mean Outline Concept Map
  35. 35. DISCUSSIONS
  36. 36. Discussions: Short-term Learning Achievements <ul><li>No statistically significant difference in scores </li></ul><ul><ul><li>Short duration of the AO intervention </li></ul></ul><ul><ul><li>Little differentiation in scenario-question scores </li></ul></ul><ul><ul><li>Problem w/ Web-based experiment implementation </li></ul></ul><ul><li>Graphic AO might have a positively measurable effect on immediate learning </li></ul>
  37. 37. Discussions: Long-term Learning Achievements <ul><li>No statistically significant difference in scores </li></ul><ul><ul><li>Students of high-learning abilities </li></ul></ul><ul><ul><li>Utilization of teacher-constructed organizers </li></ul></ul>
  38. 38. Discussions: High-Scorers <ul><li>AOs did not assist high-scorers for knowledge acquisition or retention. </li></ul><ul><li>Teacher-constructed AOs might have restrained students’ long-term knowledge retention. </li></ul>
  39. 39. Discussions: Low-Scorers <ul><li>Considerable AO effect on low-scorers for knowledge acquisition and retention. </li></ul><ul><li>Small to medium effect sizes </li></ul><ul><ul><li>Quiz 1: d concept map =0.33; d text outline =0.31 </li></ul></ul><ul><ul><li>Quiz 2: d concept map =0.22; d text outline =0.05 </li></ul></ul>
  40. 40. Implications for AO Theory <ul><li>Not Supported </li></ul><ul><ul><li>Students with AOs perform better than students without AOs; </li></ul></ul><ul><ul><li>AO effect is as great in longer studies than in shorter ones; </li></ul></ul><ul><li>Supported </li></ul><ul><ul><li>Graphic AOs more effective than text AOs </li></ul></ul><ul><ul><li>AOs help students with low learning abilities </li></ul></ul>
  41. 41. Implications for Research Methods <ul><li>Use of posttest-only control group design </li></ul><ul><li>Use of ANOVA analysis </li></ul>
  42. 42. Implications for Practice <ul><li>Expand the AO framework to a fully Web-based environment </li></ul><ul><li>Integrate AOs for online remedial programs </li></ul><ul><li>Use AOs for new online learners </li></ul>
  43. 43. Recommendations <ul><li>Student population in other disciplines and geographical locations </li></ul><ul><li>Use of Solomon four-group design </li></ul><ul><li>Extend intervention duration </li></ul><ul><li>Improve validity and reliability of instruments </li></ul><ul><li>Study the participatory (student-constructed) organizers </li></ul>
  44. 44. THANK YOU!

×