Effects of Advance Organizers on Learning and Retention from a Fully Web-based Class Baiyun Chen, Ph.D. Candidate/M.A. College of Education University of Central Florida March 5, 2007

INTRODUCTION

Introduction Online learning has entered the mainstream of higher education in the United States (Allen & Seaman, 2004, 2005, 2006) (+18%) (+24%) (+35%)

Online learning has entered the mainstream of higher education in the United States (Allen & Seaman, 2004, 2005, 2006)

Introduction Online courses have become one of the major learning modalities at UCF (Center for Distributed Learning, 2006).

Online courses have become one of the major learning modalities at UCF (Center for Distributed Learning, 2006).

Problem Statement Online learning challenges (Dias & Sousa, 1997) Cognitive overload Learner disorientation An increasing need to exploit research-based pedagogical strategies in online learning (Bonk & Dennen, 2003)

Online learning challenges (Dias & Sousa, 1997)

Cognitive overload

Learner disorientation

An increasing need to exploit research-based pedagogical strategies in online learning (Bonk & Dennen, 2003)

Problem Statement Use of advance organizers (AOs) as an effective learning strategy in traditional and computer-assisted face-to-face classroom/lab Dearth of studies on AOs in fully web-based classes.

Use of advance organizers (AOs) as an effective learning strategy in traditional and computer-assisted face-to-face classroom/lab

Dearth of studies on AOs in fully web-based classes.

Purpose of the Study To investigate both short-term and long-term effects of AOs on students’ academic achievements in a fully web-based class

To investigate both short-term and long-term effects of AOs on students’ academic achievements in a fully web-based class

Hypotheses Null hypothesis I: There is no difference in the short-term knowledge-based and performance-based learning achievements among students in the graphic AO (concept map), text AO (outline) and control groups. Null hypothesis II: There is no difference in the long-term knowledge-based and performance-based learning achievements among students in the graphic AO (concept map), text AO (outline) and control groups.

Null hypothesis I: There is no difference in the short-term knowledge-based and performance-based learning achievements among students in the graphic AO (concept map), text AO (outline) and control groups.

Null hypothesis II: There is no difference in the long-term knowledge-based and performance-based learning achievements among students in the graphic AO (concept map), text AO (outline) and control groups.

Literature Review: Definition Advance organizers (AOs)– relatively brief introductions Abstract, inclusive, and general Relatable to existing relevant ideas — D.P. Ausubel, 1968, 1978, & 2000

Advance organizers (AOs)– relatively brief introductions

Abstract, inclusive, and general

Relatable to existing relevant ideas

— D.P. Ausubel, 1968, 1978, & 2000

Literature Review: AO Examples Step-By-Step Chart Series of Events Chain Sequence Organizer Cause and Effect Chain Minute Paper Timeline — Calandra, 2002; Minchin Jr., 2004

Step-By-Step Chart

Series of Events Chain

Sequence Organizer

Cause and Effect Chain

Minute Paper

Timeline

— Calandra, 2002; Minchin Jr., 2004

Literature Review: AO Framework Students with AOs perform better than students without AOs AO effect is at least as great in longer studies as in shorter ones Graphic AOs are at least as effective as text AOs Students having low learning-ability are helped more by AOs than other students — Ausubel, 1968, 2000; Kenny, 1993, Mayer, 1976b; Stone, 1983

Students with AOs perform better than students without AOs

AO effect is at least as great in longer studies as in shorter ones

Graphic AOs are at least as effective as text AOs

Students having low learning-ability are helped more by AOs than other students

— Ausubel, 1968, 2000; Kenny, 1993, Mayer, 1976b; Stone, 1983

Literature Review: Research Evidence Advance organizers (AOs) Positive effects in traditional f2f classroom — Ausubel, 1960, 1961, 1962, 1963; Luiten et al., 1980; Mayer, 1979b; Stone, 1983 Positive effects in computer-assisted f2f classroom/lab — Calandra, 2002; Kenny, 1993; Lang & Barron, 2002; Hale, 2003; Minchin Jr., 2004; Tseng et al., 2002; Yeh & Lehman, 2001 Positive effects of concept mapping in Web-based class — Zittle, 2001

Advance organizers (AOs)

Positive effects in traditional f2f classroom

— Ausubel, 1960, 1961, 1962, 1963; Luiten et al., 1980; Mayer, 1979b; Stone, 1983

Positive effects in computer-assisted f2f classroom/lab

— Calandra, 2002; Kenny, 1993; Lang & Barron, 2002; Hale, 2003; Minchin Jr., 2004; Tseng et al., 2002; Yeh & Lehman, 2001

Positive effects of concept mapping in Web-based class

— Zittle, 2001

METHODS

Methods: Population Participants 166 undergraduate students enrolled in a fully Web-based class in Fall 2006 at UCF Sample Participants who complete all instruments of the study (112)

Participants

166 undergraduate students enrolled in a fully Web-based class in Fall 2006 at UCF

Sample

Participants who complete all instruments of the study (112)

Methodology: Research Design R E 1 X 1 ( Graphic Advance Organizer ) O 1 O 2 R E 2 X 2 ( Text Advance Organizer ) O 3 O 4 R C (No Advance Organizer) O 5 O 6 Note : R – Random Assign; E 1 – Experimental Group 1 E 2 – Experimental Group 2; C – Control Group Posttest I: O 1 , O 3 , O 5 ; Posttest II: O 2 , O 4 , O 6

R E 1 X 1 ( Graphic Advance Organizer ) O 1 O 2

R E 2 X 2 ( Text Advance Organizer ) O 3 O 4

R C (No Advance Organizer) O 5 O 6

Methodology: Intervention– AO Concept Map

Methodology: Intervention– AO Outline

Methodology: Instruments Posttest I Quiz 1 (9 questions) Scenario 1 (3 questions) Posttest 2 Quiz 2 (9 questions) Scenario 2 (3 questions) Additional analysis: Student survey (19 questions)

Posttest I

Quiz 1 (9 questions)

Scenario 1 (3 questions)

Posttest 2

Quiz 2 (9 questions)

Scenario 2 (3 questions)

Additional analysis:

Student survey (19 questions)

Methodology: Procedures Week 1 Week 2 Week 6

Week 1

ANALYSES AND FINDINGS

Data Analyses Descriptive Means, Standard Deviations, Effect Sizes ANOVA Post test I (All learners; differentiated learners) Post test II (All learners; differentiated learners) Repeated-Measure Regression (RMR) Prior knowledge Previous online learning experience Computer abilities Study time GPA

Descriptive

Means, Standard Deviations, Effect Sizes

ANOVA

Post test I (All learners; differentiated learners)

Post test II (All learners; differentiated learners)

Repeated-Measure Regression (RMR)

Prior knowledge

Previous online learning experience

Computer abilities

Study time

GPA

Descriptive Data Population 166 student participants 112 completed all instruments Instruments Posttest I Quiz 1 (144 responses) Scenario 1 (131 responses) Posttest II Quiz 2 (128 responses) Scenario 2 (128 responses) Student Survey (144 responses)

Population

166 student participants

112 completed all instruments

Instruments

Posttest I

Quiz 1 (144 responses)

Scenario 1 (131 responses)

Posttest II

Quiz 2 (128 responses)

Scenario 2 (128 responses)

Student Survey (144 responses)

Descriptive Data Demographic Information (144 responses) Age (126 students older than 21-year-old) Class standing (103 senior students) Major (83 Health Service Administration) Gender (111 female; 33 male) Ethnicity (83 white/Caucasian)

Demographic Information (144 responses)

Age (126 students older than 21-year-old)

Class standing (103 senior students)

Major (83 Health Service Administration)

Gender (111 female; 33 male)

Ethnicity (83 white/Caucasian)

Results: Descriptive Statistics (Post I) Mean & Std Deviation of Tests Note : Group 1—Experimental group with concept map Group 2—Comparison group with outline Group 3—Control group 25 22.76 22.46 22.60 22.89 Mean Scenario 1 (Posttest I-Part B) 0.19 0.25 ES 2.08 17.75 59.38 Total 90 Note: Full Score 0.06 0.19 ES 2.14 2.24 2.28 Std Deviation 18.67 16.33 18.19 Std Deviation 56.59 60.00 61.11 Mean Quiz 1 (Posttest I-Part A) 3 2 1 Group

Results : Hypothesis I Tests of Between-Subject Effects Dependent Variable: Quiz 1 & Senario 1 0.66 0.422 2.087 2 4.175 SENARIO1 0.44 0.826 260.89 2 521.78 QUIZ1 p Value F Mean Square df Sum of Squares Source

Conclusion I There is no difference in the short-term knowledge-based and performance-based learning achievements among students in the concept map, outline and control groups.

There is no difference in the short-term knowledge-based and performance-based learning achievements among students in the concept map, outline and control groups.

Results: Descriptive Statistics (Post II) Means & Std Deviations of Tests Note : Group 1—Experimental group with concept map Group 2—Comparison group with outline Group 3—Control group 25 21.56 21.69 21.36 21.67 Mean Scenario 2 (Posttest II-Part B) -0.22 -0.09 ES 3.05 16.55 58.91 Total 90 Note: Full Score -0.11 -0.01 ES 2.74 3.45 2.91 Std Deviation 16.17 17.82 14.06 Std Deviation 61.11 57.39 59.78 Mean Quiz 2 (Posttest II-Part A) 3 2 1 Group

Results: Hypothesis II Tests of Between-Subject Effects Dependent Variable: Quiz 2 & Scenario 2 0.85 0.16 1.496 2 2.99 Scenario 2 0.57 0.57 148.19 2 296.38 Quiz 2 p Value F Mean Square df Sum of Squares Source

Conclusion II There is no difference in the long-term knowledge-based and performance-based learning achievements among students in the concept map, outline and control groups.

There is no difference in the long-term knowledge-based and performance-based learning achievements among students in the concept map, outline and control groups.

Results: AO Time Effects on Quizzes (High-Scorers) Descriptive Analysis of Quiz Scores (High-Scorers) ANOVA Summary Table: Quiz 1 & Quiz 2 (High-Scorers) 90 62.99 67.14 60.71 62.14 Mean Quiz 2 (Posttest II-Part A) 71.30 Total 90 Note: Full Score 71.43 70.36 72.14 Mean Quiz 1 (Posttest I-Part A) 3 2 1 Group 0.000* p Value 0.18 15.88 2404.82 1 2404.82 Time PES F Mean Square df Sum of Squares Source

Results: AO Time Effects on Scenarios (High-Scorers) Descriptive Analysis of Scenario Scores (High-Scorers) ANOVA Summary Table: Scenario 1 & Scenario 2 (High-Scorers) 25 22.51 22.43 22.54 22.55 Mean Scenario 2 (Posttest II-Part B) 23.88 Total 25 Note: Full Score 23.86 24.04 23.74 Mean Scenario 1 (Posttest I-Part B) 3 2 1 Group 0.000* p Value 0.19 18.16 73.44 1 73.44 Time PES F Mean Square df Sum of Squares Source

Results: AO Time Effects on Quizzes (Low-Scorers) Descriptive Analysis of Quiz Scores (Low-Scorers) ANOVA Summary Table: Quiz 1 & Quiz 2 (Low-Scorers) 90 53.88 52.67 53.53 55.29 Mean Quiz 2 (Posttest II-Part A) 41.84 Total 90 Note: Full Score 39.33 42.94 42.94 Mean Quiz 1 (Posttest I-Part A) 3 2 1 Group 0.0008* p Value 0.40 31.18 3569.56 1 3569.57 Time PES F Mean Square df Sum of Squares Source

Results: AO Time Effects on Scenarios (Low-Scorers) Descriptive Analysis of Scenario Scores (Low-Scorers) ANOVA Summary Table: Scenario 1 & Scenario 2 (Low-Scorers) 25 20.36 21.09 20.21 19.75 Mean Scenario 2 (Posttest II-Part B) 19.93 Total 25 Note: Full Score 19.91 20.18 19.60 Mean Scenario 1 (Posttest I-Part B) 3 2 1 Group 0.347 p Value 0.03 0.91 3.564 1 3.564 Time PES F Mean Square df Sum of Squares Source

Results: Survey Survey: Experience with AO Note : Students’ responses to their uses of AO. 48 52 Total 40 83.3% Agree 35 67.3% Agree Usefulness 20 41.7% Before textbook 26 50.0% Before textbook When read 25 41.9% Once 23 44.2% Once How many times read AO 23 47.9% 1-5 min 24 46.2% 6-10 min Time spent on AO N % Mean N % Mean Outline Concept Map

DISCUSSIONS

Discussions: Short-term Learning Achievements No statistically significant difference in scores Short duration of the AO intervention Little differentiation in scenario-question scores Problem w/ Web-based experiment implementation Graphic AO might have a positively measurable effect on immediate learning

No statistically significant difference in scores

Short duration of the AO intervention

Little differentiation in scenario-question scores

Problem w/ Web-based experiment implementation

Graphic AO might have a positively measurable effect on immediate learning

Discussions: Long-term Learning Achievements No statistically significant difference in scores Students of high-learning abilities Utilization of teacher-constructed organizers

No statistically significant difference in scores

Students of high-learning abilities

Utilization of teacher-constructed organizers

Discussions: High-Scorers AOs did not assist high-scorers for knowledge acquisition or retention. Teacher-constructed AOs might have restrained students’ long-term knowledge retention.

AOs did not assist high-scorers for knowledge acquisition or retention.

Teacher-constructed AOs might have restrained students’ long-term knowledge retention.

Discussions: Low-Scorers Considerable AO effect on low-scorers for knowledge acquisition and retention. Small to medium effect sizes Quiz 1: d concept map =0.33; d text outline =0.31 Quiz 2: d concept map =0.22; d text outline =0.05

Considerable AO effect on low-scorers for knowledge acquisition and retention.

Small to medium effect sizes

Quiz 1: d concept map =0.33; d text outline =0.31

Quiz 2: d concept map =0.22; d text outline =0.05

Implications for AO Theory Not Supported Students with AOs perform better than students without AOs; AO effect is as great in longer studies than in shorter ones; Supported Graphic AOs more effective than text AOs AOs help students with low learning abilities

Not Supported

Students with AOs perform better than students without AOs;

AO effect is as great in longer studies than in shorter ones;

Supported

Graphic AOs more effective than text AOs

AOs help students with low learning abilities

Implications for Research Methods Use of posttest-only control group design Use of ANOVA analysis

Use of posttest-only control group design

Use of ANOVA analysis

Implications for Practice Expand the AO framework to a fully Web-based environment Integrate AOs for online remedial programs Use AOs for new online learners

Expand the AO framework to a fully Web-based environment

Integrate AOs for online remedial programs

Use AOs for new online learners

Recommendations Student population in other disciplines and geographical locations Use of Solomon four-group design Extend intervention duration Improve validity and reliability of instruments Study the participatory (student-constructed) organizers

Student population in other disciplines and geographical locations

Use of Solomon four-group design

Extend intervention duration

Improve validity and reliability of instruments

Study the participatory (student-constructed) organizers

THANK YOU!