Analysis of Self-Directed Mastery Learning of Honors Physics Capstone Defense University of Florida June 1, 2011 by Wendy AthensGood morning. My capstone project on self-directed mastery learning in a physics classroomhas been a great learning experience. It was my ﬁrst formal study of my teaching practicesand it was enlightening. I couldn’t have reached this point without the insight and support ofmy committee members, and I thank you very much.
Benjamin Bloom Why mastery learning? LFM •Every student can master concepts given sufficient time and support •Formative testing with feedback & correctives •~ 1 sigma learning gains • Attitudinal gains Enrichment Activities Unit 2 Formative Unit 1 Formative Assessment A Correctives Assessment BThe origin began two summers ago while reading Benjamin Bloom’s paper called “Learningfor Mastery”, published in 1968. This journal article resonated with me and gave me hopethat I could improve learning outcomes in my physics classroom by ﬂexing time andproviding more 1-to-1 tutoring. Bloom’s fundamental belief was that time, not aptitude, wasthe variant of learning given proper support. The basic structure is a formative testing -feedback -correctives loop with enrichment activities for advanced students.
Why self-directed? SDL • Important life skill • Social constructivist • Engaging - appeals to interests & learning styles • Frees instructor for one-to-one tutoringProviding 1-to-1 time with students drove me to self-directed learning, because I had keep the other studentsproductively occupied while I met with one. This led me to the SDL literature, and I was grateful because in aknowledge-driven society, SDL is key to survival and contribution, and much recent attention has been focused onbuilding these skills earlier in life. The term “self-directed” might mislead - SDL is hardly independent learningbut social constructivist, requiring feedback from instructor and peers to progress. I came to ﬁnd LFM and SDL tobe mutually reinforcing / symbiotic.
Research Design Three types of support: expert/peer/computer tutor (masteringphysics®) VOLOs Google Apps infrastructure Time TicketsLast summer, I constructed the technology infrastructure to support the self-directed mastery learningenvironment. I used Google Apps-based infrastructure that supported more 2-way communication betweenteacher-student and student-student. I embedded self-paced learning contracts called VOLOs, which providedstudents with a limited choice of learning activities. This image shows three key supports available to students inblue: the instructor, peers, and the computer tutor (primarily masteringphysics). The course was 25% whole-class/traditional and 75% self-directed. I used Google Forms to create Time Tickets to track students’ use of timeon a daily basis.
VOLO Power of choice Learning contract Animations & simulations Clear rubric !The self-direction was accomplished mainly through the VOLOs. You can see the VOLO for the capstone learningunit in this slide. It’s this light blue tic-tac-toe box providing links to all activities and worksheets. Students wererequired to complete three learning activities, and could choose two more. The simple rubric for VOLOs was 5boxes completed => 100% for the VOLO grade. The students would highlight the 5 they chose at the outset of theunit, then mark “DONE” when completed. I gave students the ability to edit the class website and maintain apersonal web page like this one.
Context • Honors Physics - 2 sections • Private high school • N = 24 • 50%/50% male/female • 20% minority • Pilot study • Year-long effortThe context for this study were two sections of honors physics at a private high school in Fort Myers for a total of24 students, split evenly male/female. I kicked off the year using the self-directed mastery learning frameworkand during the pilot worked out many of the glitches. The capstone study began in Semester 2 and ran for 4weeks. The focus of the learning unit was conservation of energy.
RQ #1: RQ #2: RQ #3: What SDL activities How many students Did successful and did honors physics achieved mastery unsuccessful students choose in and how did they students differ in their self-directed spend their time? their perceptions of mastery learning the self-directed environment? mastery learning environment? Three research questionsThe three research questions pursued in this study are shown here. It was interesting to see what learningactivities students chose and how they spent their time. It was even more interesting to compare students whowere successful vs. unsuccessful. Success was deﬁned as clearing the unit test in four tries. Mastery learning isbased on formative testing and these students had a pre-test and four similar tests (not the same). They receivedindividual feedback from me after each test. I was surprised that all of my students did not clear the test quickly,but I’m getting ahead of myself...
RQ #1: Data What SDL activities • VOLO activities did honors physics • Time Tickets students choose in their self-directed Data Analysis mastery learning • # SDL activities completed environment? • Minutes per SDL activity • Likert rating of educational & enjoyment value of each SDL activityTo answer the ﬁrst research question I simply had to look at the personal page of each student and see whichactivities they chose. The Time Ticket data detailed their time usage. In the post survey, students were alsoasked to rate the educational value and enjoyment value of these activities.
Data RQ #2: • Unit test scores How many students achieved mastery • VOLO activities and how did they • Time Tickets spend their time? Data Analysis • # students who cleared unit test given four tries • SDL activities completed • Minutes per SDL activityThe second research question split my students into two groups: either they were or were not successful inclearing the unit test within the timeframe of the learning unit. From these two groups comparisons were madewith respect to time allocation, activity choices, and perceptions.
Data RQ #3: • Survey results Did successful and unsuccessful Data Analysis students differ in •Compare Likert scores of their perceptions of successful/unsuccessful students the self-directed mastery learning •Mann-Whitney U Test for environment? individual questions and categories • Comparison of favorite resources 3 categories: SDL readiness Problem solving & physics thinking SDL-LFM environmentStudents were administered a post survey containing 45 questions coming from three validated sources inaddition to custom questions about the self-directed mastery learning environment. I used Guglielmino’s Self-Directed Learning Readiness Scale to extract questions about SDL readiness, Halloun’s Views About SciencesSurvey (VASS) and the Colorado Learning Attitudes About Sciences Survey (CLASS) to extract questions aboutlearning physics and gaining a sense of concept mastery. By comparing responses of successful vs. unsuccessfulstudents, I gained a sense of where student readiness and viewpoints differed.
ResultsThe results of this capstone study surprised me. I really thought when I wrote the unit tests that most studentswould clear it by the second try and that everyone would love choosing their learning activities and controllingtheir time.
Success ! 7 out of 24 successful Total time: Successful 633 min, unsuccessful 500 min Successful spent 25% more time in learning activities Successful spent triple the time in retesting !What I found instead was that students struggled to clear the unit test and only 7 of 24 students were successfulwithin the timeframe of the study. This happened even though I was spending more 1-to-1 time than usualtutoring them and providing individualized feedback (in fact it turned out that I spent an average of 16 min perstudent per class period versus the 4 min that is normally possible). This data points to a limitation of the study,which was the accuracy of the Time Tickets, however, it was my sense that rather than lecturing I was providingmore targeted 1-to-1 instruction for students.When students were split successful/unsuccessful, a very interesting pattern of time emerged. Successfulstudents consistently invested more time in learning activities. In fact the only learning activity that they did notspend more time in was the “most enjoyable” one, Roller Coaster Tycoon. Overall, successful students invested25% more time in all learning activities, triple the time in retesting and 44% more time with the instructor.
Problem solving ability and SDL readiness: alignment with Perceptions expert thinking about the U = 23, about physics: self-directed p = 0.0366 mastery learning U = 25, Source: SDLRS p = 0.0121 environment: Source: VASS, U = 25, p = 0.0121 CLASS Survey results Perceptions of successful vs. unsuccessful students showed statistically significant differences using Mann-Whitney U Test (alpha = 0.05)Post survey results were reviewed on a per question basis and also in three categories. The three categories wereperceptions about SDL, problem-solving stance and alignment with expert thinking about physics, and the self-directed mastery learning environment. I used the Mann Whitney U Test to determine statistical differences in thedistribution of the Likert ratings of the successful vs. unsuccessful students. I chose the Mann Whitney U Testbecause it was deemed more appropriate for small sample sizes that are non-normally distributed and for datathat is categorical (Huck, 2008).
Individual survey prompts that were statistically different #14 Understanding what I read is a problem for me. U = 23, p = 0.0366; average Likert: 3.4 | 4.2 #30 I am satisfied with my grade for this unit. U = 16, p = 0.0076; average Likert: 4.0 | 2.2 #27 I prefer traditional lecture with weekly lab compared to self-paced VOLO work. U = 90, p = 0.0232; average Likert: 1.7 | 3.0 #8 I cannot learn physics if the teacher does not explain things well in class. U = 91, p = 0.0209; average Likert: 2.0 |2.9All individual survey responses are available for your reference at the end of the slides. Differences in Likertratings are apparent throughout. This slide shows the four standalone survey prompts that were highlighted tohave statistically different responses between successful and unsuccessful students. The ﬁrst is such a surprisefor a junior honors physics student at a private high school. Are we dealing with some literacy issues? Thesecond is understandable since all of these students care about grades -- some more so than learning, as we willsee. #27 was expected because based on learning styles and feedback from the pilot, I knew some of mystudents preferred a more traditional classroom, and in fact I had increased the traditional content to 25% fromsomething lower based on their feedback. The ﬁnal prompt about needing the teacher gets to the heart of SDL,and students taking ownership for their own learning. Of course all students want their teacher to explain thingswell, but to say “I CANNOT learn physics...” points to self-efficacy differences between students.
Epilogue ! Feedback on unit performance followed by 5 more days to improve scores... 51% gain in test scores! Class average jumped from 57% to 79%!Before we draw conclusions, it is important to reveal an interesting epilogue to the capstone story because thisevent lends credence to the conclusions. As already stated, only 7 of 24 students passed the unit test within 4tries and within 4 weeks. The average unit test score was a very poor 57% and the overall grades were as shownhere. This data was revealed to students, then they were given 5 more days to improve their scores and retakethe test. We also reviewed the version 4 test. Suddenly everything changed! Now 8 more students cleared theunit test and the class average moved to 79%. Again, this was a similar, not the same, test, and I didn’t view it asbeing any more/less difficult than the previous.
Conclusions Bottom line: Only a minority of these honors physics students were ready for SDL.The bottom line of this study is that only a minority (29%) of these students ﬂourished in the self-directed masterylearning environment and achieved concept mastery.
Time on task Successful students invested 25% more time in learning activities and triple the time in retesting. Consistent with Abar & Loken, 2010. Time, not aptitude, is the variant of learning. - Bloom, 1968 Conclusion: Time on task contributed to success; SDL instructor must raise student “Time plus energy equals learning. awareness to manage their own There is no substitute for time on task.” -Chickering & Gamson, 1991, p. 66 learning (Idros, 2010; Taylor, 1995)One conclusion from this study is that time invested in learning yields concept mastery. Time on task ISimportant, and this quote from Chickering and Gamson expresses the idea so clearly. Since time on taskcontributes to success, the SDL instructor must raise students awareness of the importance of their timemanagement and personal learning goal-setting skills.
SDL Readiness Differences SDL is dependent on the learner, not on the instructional strategy (Bolhuis, 2003) Tolerance for uncertainty, motivation, knowledge base, goal-orientation, and self- management are factors Modeling of SDL strategies and expert thinking is recommended (Pape, 2003; Verschaffel, 1999) Conclusion: Gradual transition to SDL needed with modeling (Abdullah, 2001; Bolhuis, 2003; Breslow, 2003; Gibbons, 2002)Another conclusion of this study is that capstone students needed SDL coaching, or “modeling”, as recommendedby Pape and Verschaffel. Modeling with discourse is promoted in the literature as a great starting point to buildSDL skills. Instructors model how they categorize problems, how they eliminate certain approaches and adoptothers, and how they solve the problem. Students should practice what the instructor demonstrates to really lockit in. SDL readiness differences are attributed to differences in tolerance of uncertainty, motivation, knowledge-base, goal-orientation, and self-management. When viewed from the lens that these are the skills necessary forsuccess in college, the workplace, and life in general, educators must invest effort in building SDL skills instudents in parallel to content instruction. Instructors need to incorporate emotional support through positivefeedback and encouragement as well as providing Learning to Learn strategies suggested by Pape and Verschaffel.
Perceptions about Physics Successful students showed greater confidence in problem solving and closer alignment with expert thinking. Successful students showed intention in the selection of SDL activities and fit new information into cognitive framework. Conclusion: Metacognition is an important aspect of SDL and relates to concept mastery (Bransford, 2000; Gagne, 1998; Halloun & Hestenes, 1998)The third conclusion of the study was that successful students were much more intentional inwhy they picked certain learning activities and they were aware of what they gleaned fromeach activity and how it ﬁt into their cognitive framework. Considering this was iteration #2following the pilot, these students had already adopted “best practices” that made themsuccessful including particular resources. The pattern recognition of experts is what I amtrying to represent with this chess board: experts see patterns and can quickly categorizeproblems, which frees their cognitive resources for thinking. Novices remain mired in details.According to post survey prompts, successful students had already developed a conceptualframework around which they constructed new knowledge. The recommendation toinstructors therefore becomes, how can I organize information to make it more intelligibleand connected for the student? Especially in SDL settings, concept maps, whole-class “bigideas” lectures, and peer critique help tie together concepts and create “sense-making”.
Learning Environment Perceptions The majority of students were positive about the SDL-LFM learning environment (86% | 76%). Peer interaction was the most time-consuming SDL activity. Formative testing and masteringphysics® were praised. Conclusions: Positive responses were an outcome of the power of choice (Knowles, 1975; Tomlinson, 2003); frequent and individualized feedback (Bloom, 1984; Chickering & Gamson, 1991); adequate support (Kirschner, 2006); lack of competition (Bloom, 1968); peer interactions must be more purposeful (Crouch & Mazur, 2001; Merrill & Gilbert 2008). Technology infrastructure was effective in the background and supported the learning process.The third conclusion of the study was that students were positive about the self-directedmastery learning environment, more so for successful students. The literature supports thisoutcome for several reasons. Giving students a choice of learning activity is motivating initself, besides the fact that they can follow their interests. Regarding feedback, feedback isthe fuel of learning and students ﬂourish with frequent and individualized feedback.Regarding support, SDL students must have adequate support to be successful and capstonestudents used all 3 forms of support and expressed they were adequately supported.Formative testing was praised because it gave students a chance to learn from their mistakesin a targeted way and also removed competitive pressure. Finally, peer interaction wasimportant and was the most signiﬁcant use of time, however it could be more reﬁned. Thetechnology infrastructure ran appropriately and effectively in the background.
Profile of a Self-Directed Honors Physics StudentA proﬁle of a self-directed honors physics student emerged from this study.
SDL readiness Actively constructed knowledge into cognitive framework Prior knowledge through discourse and feedback Prioritized time Enjoyed the learning process Metacognitive intention Persisted Used many resources Motivated Profile of a Self-Directed Honors Physics StudentThe attributes that seemed important for students to be successful in the capstone settingare shown here. Self-direction contributed to concept mastery. For the four weektimeframe they were given, successful students prioritized their time well and used manyresources, including human resources of the instructor and peers. They persisted informative testing and doing homework. They were motivated to work and they wereintentional about how they used their time. They ﬁlled in gaps of knowledge throughdiscourse and self-study.
Recommendation Content- and Process-Oriented Instruction (Bolhuis, 2003)Because self-directed learning skills facilitate concept mastery and are deemed important lifeskills in our society, an instructor needs to balance teaching content with teaching self-direction. Bolhuis (2003) expresses this balancing act as process-oriented instruction, whichfocuses on four principles.
Constructive friction Recommendation Content- and Process-Oriented Instruction (Bolhuis, 2003)The ﬁrst is to maintain healthy “constructive friction” and move gradually to SDL becauselearners differ in readiness.
Constructive friction Content Recommendation Content- and Process-Oriented Instruction (Bolhuis, 2003)The second is to focus on building content knowledge, especially by making the learningmore meaningful through real-world, problem-based learning.
Constructive friction Content Emotional support Recommendation Content- and Process-Oriented Instruction (Bolhuis, 2003)The third is to pay attention to the emotional aspects of learning through positive feedback,helping students embrace uncertainty, fostering motivation, and modeling the self-gratiﬁcation of solving a hard problem.
Social context Constructive friction Content Emotional support Recommendation Content- and Process-Oriented Instruction (Bolhuis, 2003)The fourth principle is to treat learning as a social phenomenon and facilitate social skills andeffective cooperative learning so learners can more effectively construct their knowledgethrough discourse with the instructor and peers.