The document discusses the concept of Just-In-Time Teaching (JITT), which involves online pre-class assignments to enhance student engagement and preparedness for class. It presents the structure of JITT, its benefits, and evidence of its effectiveness across various disciplines, noting improvements in student performance and engagement. Additionally, it shares strategies for implementing JITT, including maintaining consistent communication with students and designing effective warm-up questions.

1. Name
School
Department
THE CHANGING LANDSCAPE OF TEACHING
@SPS 4500 #2
OCTOBER 9TH, 2015
DR. JEFF LOATS
DEPARTMENT OF PHYSICS

2. WHERE WE LEFT OFF…
2
Teaching techniques matter
We should let evidence guide our attention
Eric Mazur’s story from ~1990…

3. ELSEWHERE?
3

4. In your teaching do you have a method for holding
students accountable for preparing for class?
Previous anonymous poll results (compiled):
~17% → I don’t, but I ask/threaten really well
~50% → I use a paper method (quiz, journal…)
~11% → I use a digital method (clickers, etc.)
~17% → I have some other method
(N ~ 211)
A)
B)
C)
E)


5. OVERVIEW
1. Motivation for change
2. Basics of Just-in-Time Teaching
3. Mock example
4. Choose our own adventure?
5. Summaries

6. TECHNIQUE & TECHNOLOGY
Technique:
Just-in-Time Teaching
Technology:
Online question & response tools
Learne
r
Teacher

7. JUST-IN-TIME TEACHING
Online pre-class assignments
called WarmUps
First half - Students
• Conceptual questions, answered in sentences
• Graded on thoughtful effort
Second half - Instructor
• Responses are read “just in time”
• Instructor modifies that day’s plan accordingly.
• Aggregate and individual (anonymous) responses
are displayed in class.
Learne
r
Teacher

8. EXAMPLE: WHIRLING BUCKET
A bucket of water can be whirled in a
vertical circle without the water falling
out, even at the top of the circle when the
bucket is upside down. Explain…
~15% → An outward force holds it in
~30% → An inward force holds it in
~20% → Talked (correctly!) about
acceleration & velocity… but
didn't really answer.
~10% → Nailed it! (or close enough)

9. EXAMPLE: WHIRLING BUCKET
“The water doesn't come out because you
twirling the bucket is applying the force of
spinning, and the water just kind of counteracts
that motion.”
“Because the water naturally wants to keep
traveling in the same direction its being
whirled around in the water attempts to
continue going up in a straight line but the
bottom of the bucket forces it to stay in the
bucket, like when you are pushed by the door
of a car while making a turn.”

10. For your “typical” college class, estimate the
fraction of students who do their preparatory work
before class?
A) 0% - 20%
B) 20% - 40%
C) 40% - 60%
D) 60% - 80%
E) 80% - 100%
10
29%
32%
20%
14%
5%
(𝑁~238)

11. JITT STRUCTURE & RESPONSE
RATES
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
%Responsed
Class #
Response Rate by Day
College Physics I, N = 78
Worth 10% of final grade
Due 10 PM the night before class
Assignments available for prior 2-3 days
College Physics I

12. WARMUP QUESTIONS
• Every-day language
• Occasional simple comprehension question
• Mostly “higher level” questions
• Any question is better than none (don’t be precious)
Connections to evidence:
–Pre-class work reduces working memory load
during class.
–Multimodal practice (not learning styles):
JiTT brings reading, writing and discussion as
modes of practice.

13. METACOGNITION
Two questions in every WarmUp:
First:“What aspect of the material did you find
the most difficult or interesting.”
Last: “How much time did you spend on the pre-
class work for tomorrow?”
Connections to evidence:
–Forced practice at metacognition:
Students regularly evaluate their own
interaction with the material.

14. THE JITT FEEDBACK LOOP
Student responses:
• Graded on thoughtful effort
• Sampled and categorized for display
• Quoted anonymously
Closing the loop:
• Respond to some students digitally
• Class time shifts to active engagement.

15. JUST-IN-TIME TEACHING
A different student role:
• Actively prepare for class
(not just reading/watching)
• Actively engage in class
• Compare your progress & plan accordingly
A different instructor role:
• Actively prepare for class with you
(not just going over last year’s notes )
• Modify class accordingly
• Create interactive engagement opportunities
Learne
r
Teacher

16. Which topic would you like to spend our
remaining time on?
A) Evidence for effectiveness
B) Best tools for JiTT
C) Getting student “buy-in”
D) Writing good questions
16

17. Evidence For Effectiveness

18. STUDIED EFFECTIVENESS
Used at hundreds of institutions
Dozens of studies/articles, in many disciplines:
Bio, Art Hist., Econ., Math, Psych., Chem., etc.
–Increase in content knowledge
–Improved student preparation for class
–Improved use of out-of-class time
–Increased attendance & engagement in class
–Improvement in affective measures

19. JITT VS. FINAL GRADE
CORRELATIONS
College Physics I, Fall 2013
0
20
40
60
80
100
0 10 20 30 40 50 60 70 80 90 100
CumulativeScore(withoutwarm-ups)
WarmUp Score
WarmUps vs. Cumulative Score
Correlation r = 0.71

20. PROGRESSIVE EXAMS
CORRELATIONS
College Physics I:
Important disclosure: This was not a hypothesis we were
testing, it appeared as we analyzed the data. Could be
0.18
0.33
0.43
0.54
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
Mini Exam
(week 4)
Exam 1
(week 7)
Exam 2
(week 11)
Final Exam
(week 16)
NoneWeakStrongModerate
Correlations between Total WarmUp Score
and Sequence of Exams

21. Mean on 1-5 scale
Preparation for class 4.06
Engagement during
class 3.93
STUDENT SURVEY RESULTS
9% 10%
81%
10%
18%
73%
10%
22%
68%
0%
20%
40%
60%
80%
100%
Harmful Neutral Helpful
How did WarmUps affect your...
Preparation Engagement Learning
N = 781

22. STUDENT SURVEY QUOTES
Physics:
“Initially, it was hard for me to get used to the
warm-ups. It seemed like along with the
homework assignments there was a lot of things
to do. Eventually I got used to it and ultimately
the warmups really helped me to learn the
material and stay caught up with the class.”
“If it weren't for warm ups, the amount of time I
spent reading the book would have dropped by
75%”

23. Best Tools for JiTT

24. WHAT TOOLS TO USE?
The crucial part:
Daily reading, grading & using responses
• Automatic full credit for any response
• View all responses to a question together
• Grade responses on the same page with
minimal clicks
Wishlist:
Easy (quick!) individual feedback

25. SMALL ASIDE: TEXT EXPANDER
25
Every professor should have this!
You define a snippet like “ttyl” which instantly
gets replaced by “Talk to you later!”
Windows:
– Texter, PhraseExpres
(FREE, some advanced features, some flaws)
– Breevey ($40, worth it if you hit problems)
– AutoHotKey (free advanced automation tool)
Mac:
– TypeIt4Me, TextExpander, Typinator
(All cost $20-$30. Generally worth it!)

26. WHAT TOOLS TO USE?
• CMS/LMS (Blackboard, D2L, Moodle, etc.)
Ready to use, tools… imperfect  awful
• Free service from JiTTDL.org.
Designed just for JiTT. Additional website, not
very “shiny” by 2015 standards.
• Students email responses
Easy… also overwhelming and awful
• Blogging tools (WordPress)?
• New tools (TopHat? Learning Catalytics?)

27. Getting Student Buy-In
(a.k.a. “the sales pitch”)

28. THE SALES PITCH
The way we talk to our students impacts
• How they approach the assignments
• How they feel about the work they do

29. OVERARCHING MESSAGE
Communicating with your students (humans)
• Message (explicit statements)
• Attitude (subtext, body language, etc.)
Consistent subtext:
"I am here to help you learn, and I have thought
about your learning trajectory carefully."
Consistent attitude:
I am comfortable and relaxed about my part of
this partnership.

30. DAY 1 – GENERAL
Describe components of the course
• How each one is graded, and why.
• How each one is important for learning and/or
assessment.
Keep justifications short and succinct
Be honest:
"This is my first time using this method, and
there is a lot of data on how and why this is
effective and what the best practices are."

31. DAY 1 – JITT
When discussing JiTT:
“Today is going to feel pretty ‘normal.’You’ll get
to see how this works starting next time, after
you’ve done your first warm-up.”
It isn’t “more assignments = more work,” but
rather “working in smaller chunks is more
effective and more efficient.”

32. ALLOW TIME
They (probably) won’t “buy it all” on Day 1
Emphasize that you will be consistent and they
will get to see its value over time
“This class is different, and I will say that to you,
but it really is something you will get to
see/experience every day.”

33. DAY 2 – JITT
Discuss their first experience with warm-ups
Share how many did them
Remind them of structure:
Release/due times, course value, grading
Remind them of the purpose of warm-ups:
–Student preparation
–Instructor preparation
(“Which I’ll show you now!”)

34. DAY 3 – JITT BITS
A different role for you:
• Actively prepare for class by engaging and
being reflective.
(not just reading/watching)
• Be ready to actively engage with the material
in class.
• Take regular “readings” on your experience
with the material compared to classmates.
Make plans accordingly.

35. DAY 3 – JITT BITS
A different role for me:
• I will actively prepare for class by engaging
and focusing on you.
(not just going over last year’s notes )
• I will modify the class plan based on what I
see in your preparatory work.
• I will consciously create chances for you to
grapple with the material in an active way.

36. STUDENTS: BUSY-WORK
DETECTORS
K-12 represents more than 13,000 hours of class
Students are experts at detecting what really
matters to an instructor:
• What does the instructor do with class time?
• What does the instructor talk about?
• Does the instructor push against the usual
“invisible contract” of the classroom?

37. DEMONSTRATING VALUE IN JITT
Ideas for demonstrating that you value JiTT
• Thank those who do them for giving you
insight into their learning.
• Bring at least one “difficult/interesting” item
from WarmUp to class each day.
• Give non-verbal cues that you value
discussing WarmUps as much (more) than
other course components.
• Be consistent!

38. CONSISTENCY
Be consistent with:
• Assignment releases
• Assignment due dates/times
• Follow-up in class
• Summative assessments (e.g., exam questions)
that build on WarmUp questions.

39. Writing Good Questions

40. FEATURES OF A GOOD QUESTION
40
What would a “good” response look like?
– A paragraph? (too long)
– One word? (too short)
Make sure the reading is needed to respond (but a
sentence straight out of the book shouldn’t work).
Make sure a beginner can take a crack at the question
Be concrete:
– “Explain in 2-3 sentences.”
– “Give two brief examples.”
– “Explain how you got your estimate.”
“Game out” their responses a bit.

41. WRITE A QUESTION AND SHARE...
41
Consider an intro. course in your discipline.
Consider a topic you discuss early in that course.
Write one question… shoot for “higher level.”
Good starting words: apply, analyze, evaluate,
sketch, use, compare, estimate, etc.
Take 3 minutes… then trade questions with your
neighbor and do your best to answer theirs.

42. Summary Slides

43. MY SUMMARY
JiTT may be among the easiest research-based
instructional strategies that can be consistently
integrated into any kind of teaching.
From an evidence-based perspective, JiTT
addresses often-neglected areas.

44. YOUR SUMMARY
To ponder: Why are most college courses taught
the way they are?
Email: jeff.loats@gmail.com
Twitter: @JeffLoats
Slides: www.slideshare.net/JeffLoats

45. JITT REFERENCES & RESOURCES
Simkins, Scott and Maier, Mark (Eds.) (2010) Just inTimeTeaching: Across the Disciplines, Across the
Academy, Stylus Publishing.
Gregor M. Novak, Andrew Gavrini, Wolfgang Christian, Evelyn Patterson (1999) Just-in-Time
Teaching: Blending Active Learning with WebTechnology. Prentice Hall. Upper Saddle River NJ.
K. A. Marrs, and G. Novak. (2004). Just-in-Time Teaching in Biology: Creating an Active Learner
Classroom Using the Internet. Cell Biology Education, v. 3, p. 49-61.
Jay R. Howard (2004). Just-in-Time Teaching in Sociology or How I Convinced My Students to
Actually Read the Assignment. Teaching Sociology,Vol. 32 (No. 4 ). pp. 385-390. Published by:
American Sociological Association
Stable URL: http://www.jstor.org/stable/3649666
S. Linneman, T. Plake (2006). Searching for the Difference: A ControlledTest of Just-in-Time
Teaching for Large-Enrollment Introductory Geology Courses. Journal of Geoscience Education, Vol.
54 (No. 1)
Stable URL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18
ON-DEMAND SLIDES

48. JITT STRUCTURE & RESPONSE RATES
0
10
20
30
40
50
60
70
80
90
100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
%Responsed
Class #
Response Rate by Day
College Physics I, N =
0
10
20
30
40
50
60
70
80
90
100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
%Responsed
Class #
Response Rate by Day
Intro. Sociology, N =
Worth 10% of final grade
Due 10 PM the night before class
Assignments available for prior 2-3 days
College Physics I Intro. to Sociology
Worth 5%, due @10 PM, available for
2-3 days… but got inconsistent in 2nd
half of the term. Posted late, no follow-
up, etc.