<g> tells us the % of what they could have learned that they did learn.
Incorrect or incomplete responses are often particularly useful for classroom discussion.
Just in Time Teaching - Jeff Loats @ LMU
Prepare for a talk about teaching and learning … … by a physicist.Liberal-arts majors may be annoying sometimes, but there’s nothing more obnoxious than a physicists first encountering a new subject.
Just in Time Teaching A 21st Century Teaching Technique Jeff Loats Metropolitan State University of Denver Department of Physics“Learning technologies should be designed to increase, and not to reduce, the amount of personal contact between students and faculty on intellectual issues.” Study Group on the Conditions of Excellence in American Higher Education, 1984 LMU – Sept. 2012
The Evidence Standard 3Teachers can feel a bit bombarded… hybrid courses,brain-based learning, technology in the classroom,learner-centered teaching, etc.For me, striving to be a scholarly teacher meansfollowing the evidence, regardless of the source.Some common themes, supported by evidence: • Focus and attention • Using emotions appropriately • Repetition and practice • Feedback
Narrowing Our Focus 4• Focus and attention –No such thing as multitasking, etc.• Using emotions appropriately –A little anxiety is good, a bit more is bad, etc.• Repetition and practice• Feedback Just in Time Teaching has something unique to offer here
5In (roughly) what area do you teach?A) HumanitiesB) Natural sciences & mathematicsC) Professions & applied sciencesD) Social sciencesE) Teacher education (there is no surer way for me to offend a good chunk of you)
6In your teaching do you have a method for holdingstudents accountable for preparing for class?A) I don’t, but I ask/threaten really well.B) I use a paper method (quiz, journal, others?)C) I use a digital method (clickers, others?)D) I use Just in Time Teaching.E) I have some other method.
Overview 7Talk:1. Motivation for change2. Basics of Just in Time Teaching3. Mock example4. Evidence for effectiveness5. SummariesWorkshop:• Try JiTT from both sides (students/instructors)• Feedback from students• Roadblocks and bottlenecks
Feedback That Works 8“Improvement of performance is actually a function oftwo perceptual processes. The individual’s perceptionof the standards of performance, and her/hisperception of his/her own performance.” The Feedback Fallacy – Steve Falkenberg (via Linda Nilson)
9 The Physics Education Revolution• About ~20 years ago, physics teachers began treating education as a research topic!• Their findings were grim: From Mazur, New Faculty Workshop presentation, 2004• Eric Mazur (at Harvard!) found this:"But the students dofine on my exams!"
How Do Others Do? 10Conclusion: Traditional physics lectures are all similarly (in)effective for improving conceptual understanding.
11 Enter Physics Education Research (PER): An effort to find empirically tested ways to improve the situation. Many top university physics departments now have a PER research group. University of Illinois at Urbana-Champaign University of ColoradoUniversity of Washington
12 Technique & Technology Technique: Just in Time Teaching (JiTT) Technology:Web based question & response tools
Just in Time Teaching 13“Just-in-Time Teaching (JiTT for short) is a teachingand learning strategy based on the interaction betweenweb-based study assignments and an active learnerclassroom.” ~ Gregor Novak, Co-Author of the JiTT book• Online assignments (“WarmUps”), completed before class to promote preparation and thought.• Responses are read “just in time”.• Instructor modifies that day’s plan accordingly.• Aggregate and individual (anonymous) responses are displayed in class.
What JiTT is Not 14JiTT techniques rely heavily on web-based tools. JiTT is not about … online courses or distance learning. … computer-graded homework. … delivering content via the web.The goals of JiTT:• Student preparation.• Obvious communication loop.• Improve student ownership and buy-in of class.• Establish a community effort towards learning.
15For an average day in your class, what fraction of yourstudents do their preparatory work before your class.A) 0% - 20%B) 20% - 40%C) 40% - 60%D) 60% - 80%E) 80% - 100%
Questions – Practice & Repetition 16• WarmUp questions about (new) material: – Every-day language. – Some simple comprehension questions . – Mostly higher level questions (a la Bloom). – Perhaps any question is better than none.• Connections to evidence: – Pre-class work reduces the working memory load during class, a factor experts easily neglect. – Multimodal practice (not learning styles): Reading, writing and discussion are modes of practice that JiTT brings to the classroom daily.
Questions – Deliberate Practice 17Two questions end every WarmUp:“What aspect of the material did you find the mostdifficult or interesting. Give this a bit of thought andbe specific in your answer.”“How much time did you spend on the pre-class workfor tomorrow?” [Results: A pretty steady average of ~40 minutes across many courses/levels/cohorts.]• Connections to evidence: – Forced practice at metacognition: Students regularly evaluate their own interaction with the material.
Closing The Loop 18• Student responses are: – Graded on thoughtful effort that demonstrates they did the reading. – Sampled and categorized by the instructor in some way to create a class response profile. – Quoted anonymously in class to demonstrate a variety of aspects: Misconceptions, good efforts, superior explanations, metacognition, etc.• Closing the loop: – Instructor responds to some students digitally. – Class time emphasizes interactive engagement.
Effective Feedback 19Recall Faulkenberg’s criteria for feedback: – Feedback doesn’t work if students don’t correctly perceive the performance standards. – Feedback doesn’t work if students cannot correctly evaluate their own performance.• JiTT offers chances to clarify standards in low- stakes situations. Allows us to show model responses that are not teacher-generated.• Closing the loop allows students to judge whether they have correctly evaluated their own performance.
JiTT Web Tools 20More in workshop• JiTT can be done using many different online tools: – Course management systems (Blackboard). (Easy to set up and tools are good, but not perfect) – Free service from JiTTDL.org. (Designed just for JiTT but extra login, and the site has not been improved in ~4 years) – Email (No setup, very direct, but tends to be overwhelming and discouraging) – Blogging tools (WordPress). (Very possible for a tech-savvy user, but you would need to have a “hackerish” mentality).
21Here is an example WarmUp question of mine.Answer with your clicker for this simulation.(A full demonstration is in the workshop following).A bucket of water can be whirled in a vertical circlewithout the water falling out, even at the top of thecircle when the bucket is upside down. Explain why.A) Nothing holds it inB) An outward force holds the water in the bucketC) The water’s acceleration and velocity are in different directionsD) An inward force holds the water in the bucket
Example From My Class 22A bucket of water can be whirled in a vertical circlewithout the water falling out, even at the top of thecircle when the bucket is upside down. Explain… ~15% → An outward force holds it in ~30% → An inward force holds it in ~20% → Talked (correctly!) about acceleration & velocity… but didnt really answer. ~10% → Nailed it! (or close enough)
Class Response: Water Bucket 23“The water doesnt come out because you twirling thebucket is applying the force of spinning, and the waterjust kind of counteracts that motion.”“Because the water naturally wants to keep travelingin the same direction its being whirled around in thewater attempts to continue going up in a straight linebut the bottom of the bucket forces it to stay in thebucket, like when you are pushed by the door of a carwhile making a turn.”
Does It Work? 24Mazur: After 1 year using active engagement methods:• Big jump in conceptual understanding.• Is this just new energy into an old course?
Outside Harvard 25• Mazur’s methods were applied all over the country, with impressive results:• A conceptually focused, learner-centered class can create dramatically improved conceptual gains.• The evidence indicates that problem solving skills are improved when time is devoted to concepts.
Effectiveness of JiTT 26• Back in 2004 JiTT was used by hundreds of faculty, in more than 25 disciplines at over 100 institutions.• Dozens of publications in different fields indicate concrete differences in JiTT driven classes: – Improved student preparation for class – Improved use of out-of-class time and… – Increased attendance & engagement in class – Improvement in affective measures – Increase in content knowledge
Does It Work In Biology? 27• Marrs, K., 2005, Assessment of JiTT on Student Learning In several biology courses the author documents: Improved study habits & preparation before class From “References for Just in Time Teaching”: http://serc.carleton.edu/sp/pkal/justintime/references.html#pedagogy
Does It Work In Biology? 28• Marrs, K., 2005, Assessment of JiTT on Student Learning ‘Crammed’ in ‘Crammed’ in Biology N100 other courses A students 16% 44% B students 34% 63% C students 41% 65% D students 64% 71% F students 68% 69% From “References for Just in Time Teaching”: http://serc.carleton.edu/sp/pkal/justintime/references.html#pedagogy
Does It Work In Biology? 29• Marrs, K., 2005, Assessment of JiTT on Student Learning In several biology courses the author documents: Improved study habits & preparation before class Increased retention rates (DFW from 29% to 21%) Increased cognitive gains on conceptual knowledge… AND increased content knowledge! From “References for Just in Time Teaching”: http://serc.carleton.edu/sp/pkal/justintime/references.html#pedagogy
Does It Work In Art History? 30• Cookman, 2009 Using JiTT to Foster Active Learning in a Humanities Course This chapter (available online) describes Thinking About Reading questions (TARs). His methods, student reactions and lessons learned are all included. The course is History of 20th Century Photography. Part of “Just in Time Teaching Across the Disciplines and Across the Academy” Book Google this title.
Does It Work In Art History? 31• Cookman, 2009 Using JiTT to Foster Active Learning in a Humanities Course Sample: “Summarize in your own words Knightley’s argument that Capa staged this photograph. Summarize in your own words Whelan’s argument that the photograph is authentic. Whose argument do you find more convincing? Why?” Part of “Just in Time Teaching Across the Disciplines and Across the Academy” Book Google this title.
What Might Stop You 32• In terms of the technique:• Time, coverage, not doing your part, student/colleague pushback.• In terms of the technology:• Learning curve, technology failures, perfectionism• In generally reforming your teaching:• Inventing the wheel, not finding support, doing too much at once.
My Summary 33• In terms of content and changes to the classroom, Just in Time Teaching may be among the easiest research-based instructional strategies that you can consistently integrate into your teaching.• From an evidence-based perspective, JiTT addresses areas that are often neglected by traditional techniques.• Student report that WarmUps make them better prepared for class and help them learn the material.• As with all reforms, we should be prepared to find that students know less than we might hope.
Your Summary! 34For yourself or to share with me (if you’re willing):What part of JiTT concept/process is the fuzziest foryou after this talk?What is the biggest reason you might not give JiTT atry in one course next term?
35 Works Cited• Falkenberg, S. (1996). “The Fedback Fallacy”. Retrieved March, 2012 from http://people.eku.edu/falkenbergs/feedback.htm• Nilson, L. (20011) ”The Mind Has a Mind of Its Own”, teleseminar recorded by Emphasis on Excellence, Jun 2011• Mazur, E. 2004 ”Introduction to Peer Instruction” talk presented at New Physics & Astronomy Faculty Workshop, 2004, UMD.• Hake, R.R. 1998a. “Interactive-engagement vs traditional methods: A six thousand-student survey of mechanics test data for introductory physics courses,” Am. J. Phys. 66(1): 64-74; online at <http://www.physics.indiana.edu/~sdi/ajpv3i.pdf> (84 kB). Jeff Loats Jeff.Loats@gmail.com 303-900-2175 Take a card and visit slideshare.net/jeffloats