This document summarizes a presentation on implementing Just-in-Time Teaching (JiTT), a blended learning strategy. The presentation provides an overview of JiTT, shares data from courses that have used JiTT showing increased student preparation and performance, and offers recommendations for getting started with JiTT. Sample JiTT questions are also presented along with student responses to demonstrate how the strategy works.

1. Just in Time!
Applying a Blended Learning
Strategy across the Academy
Randi Smith, Jeff Loats,
Courtney Rocheleau, & Arlene Sgoutas
Teaching Professor Technology Conference
October 2015

2. Presentation Overview
• What is it?
Introduction to Just-in-Time Teaching
(JiTT)
• Does it work?
Data from our courses
• How do I get started?
Recommendations for implementing JiTT

3. In your teaching do you have a method for
holding students accountable for preparing for
class?
A) I don’t, but I ask/threaten really well.
B) I use a paper method (quiz, journal,
others?)
C) I use a digital method (clickers, others?)
D) I use Just-in-Time Teaching.
E) I have some other method.
3
17%
51%
11%
5%
17%
(~230 others)

4. Consider a typical day in your class. What
fraction of students did their preparatory work
before coming to class?
A) 0% - 20%
B) 20% - 40%
C) 40% - 60%
D) 60% - 80%
E) 80% - 100%
4
28%
34%
20%
14%
5%
(~260 others)

5. Student Preparation
Quotes from Sappington, Kinsey, & Munsayac (2002)
"72% of Connor-Greene’s (2000) sample
reported that they rarely or never read their
assignments by the due date.”
"Burchfield and Sappington (2000): On
any given day, less than a third of students
in this population had adequately
prepared for class."
Recent USPIRG survey: 70% of students
admit that they sometimes don’t even
obtain required textbooks.

6. JUST-IN-TIME TEACHING
Online pre-class assignments
called WarmUps
First half - Students
• Conceptual questions, answered in sentences
• Graded on thoughtful effort
Second half - Instructor
• Responses are read “just in time”
• Instructor modifies that day’s plan accordingly.
• Aggregate and individual (anonymous) responses
are displayed in class.
Learne
r
Teacher

7. JUST-IN-TIME TEACHING
A different student role:
• Actively prepare for class
(not just reading/watching)
• Actively engage in class
• Compare your progress & plan accordingly
A different instructor role:
• Actively prepare for class with these humans
(not just going over last year’s notes )
• Modify class accordingly
• Create interactive engagement opportunities
Learne
r
Teacher

8. BENEFITS OF JITT
• Student preparation
• Supports development of metacognitive skills
• Brings student voices into the classroom
• Promotes engagement with “higher-level”
questions (cf. Bloom’s taxonomy)
• Consistent with other research-based
instructional strategies
(learner-centered, universal design, etc.)

9. Example Questions &
Responses

10. Suppose you are
interested in
researching why and
how some women fake
orgasms. How would
you design a study to
answer those
questions?
Perhaps I’d tell a random group of women
to fake an orgasm. Additionally, a control
group that is not told to fake an orgasm
should be included. After both groups’
ensuing sexual encounters, ask very
specific and pointed questions about
sexual arousal, climax, or the lack thereof,

11. Give an example of a time you conformed due
to normative influence. Give an example of a
time you conformed due to informational
influence.
I gave into normative social influence when I
bought a pair of Uggs because I felt obligated
to follow the norm. I have given into
informational social influence when I moved to
a new school and had to ask students around
here how to act in certain situations because it
was not like my other school.

12. You can probably stand flatfooted on the
floor and then rise up and balance on your
tiptoes. Why are you unable to do this if you
stand so your toes are touching a wall?
It is impossible to do this because when you
tiptoe, your center of gravity moves forward
to be in equilibrium and when there is a door
in front of you, you can’t move your center of
gravity forward.

13. What is your first memory of gender defining
or impacting your life?
When I learned to pee. I felt like in a house full
of woman at 4 I know I different then everyone
else in this house at that age.
I recall getting mad when they would just give
me the girl toy at McDonalds without asking
me which one I wanted. A hot wheels car is
way more useful then some plastic airhead
anyways.

14. Sounds good in theory, but does it
actually work?
Convince me it’s worth my time…

15. Our Studies
• Have examined data from 18 courses
across 5 disciplines
– Upper- and lower-division courses
– Larger and smaller courses
– General studies, required major courses,
elective courses
• Vary in implementation of JiTT
– Weekly vs. biweekly
– 5-20% of final course grade

16. Data from Our Courses
Overall Response Rates:
46% - 74% average response rate across
the entire 15-week semester.
(mean of averages across 18 classes =
59%)
• Recall: ~33% in Burchfield & Sappington
(2000)

17. 0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14
PSY2850: Smith, N = 33
PSY2410: Rocheleau, N = 36
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10 11 12 13
WMS 1001: Sgoutas, N = 25
0
20
40
60
80
100
1 2 3 4 5 7 8 9 10 11 14 15 16 17 18 19 22 23 24 25 26 27
PHY2010: Loats, N = 78
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13
PSY1001: Smith, N = 22
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
PHY2020: Loats, N = 73

18. Course Performance
Effect sizes for most pedagogical techniques
are small to medium (see, e.g., Hattie, 2009)
Correlations with total grade (w/warm-ups
partialed out) across 18 classes range from
r = 0.38 – 0.84; Mr = .60 (r2 = .36)
• Per Cohen (1988),
r = .30 is medium effect,
r = .50 is large effect

19. Limitations
• Could be alternative
explanations/confounds that are
uncontrolled in these analyses; e.g.,
– Academic motivation
– Quality of answer
– Conscientiousness
– Interest in course
• But pattern repeated across wide variety
of courses

20. Wow—you guys are so convincing.
I’m totally sold on this JiTT stuff.
Now what?

21. Which topic would you like to spend our
remaining time on?
A) Getting student “buy-in”
B) What tool should you use for JiTT?
C) Writing good questions
21
Pause for questions…

22. Getting Student Buy-In
(a.k.a. “the sales pitch”)

23. THE SALES PITCH
13,000 hours in invisible contract indoctrination
Mindfulness in what we say and what we do.
Day 1 – Keep justifications short. Emphasize
purpose over mechanics.
Day 2 – Discuss their first experience & response
rates. Remind them about structure & purpose,
but mostly show them.
Day 3 – Return to “different roles” for both.
Demonstrate value, be consistent

24. Best Tools for JiTT

25. WHAT TOOLS TO USE?
The crucial part:
Daily reading, grading & using responses
• Automatic full credit for any response
• View all responses to a question together
• Grade responses on the same page with
minimal clicks
Wishlist:
Easy (quick!) individual feedback

26. SMALL ASIDE: TEXT EXPANDER
26
Every professor should have this!
You define a snippet like “ttyl” which instantly
gets replaced by “Talk to you later!”
Windows:
– Texter, PhraseExpres
(FREE, some advanced features, some flaws)
– Breevey ($40, worth it if you hit problems)
– AutoHotKey (free advanced automation tool)
Mac:
– TypeIt4Me, TextExpander, Typinator
(All cost $20-$30. Generally worth it!)

27. WHAT TOOLS TO USE?
• CMS/LMS (Blackboard, D2L, Moodle, etc.)
Ready to use, tools… imperfect  awful
• Free service from JiTTDL.org.
Designed just for JiTT. Additional website, not
very “shiny” by 2015 standards.
• Students email responses
Easy… also overwhelming and awful
• Blogging tools (WordPress)?
• New tools (TopHat? Learning Catalytics?)

28. Writing Good Questions

29. EXAMPLE: WHIRLING BUCKET
A bucket of water can be whirled in a
vertical circle without the water falling
out, even at the top of the circle when the
bucket is upside down. Explain…
~15% → An outward force holds it in
~30% → An inward force holds it in
~20% → Talked (correctly!) about
acceleration & velocity… but
didn't really answer.
~10% → Nailed it! (or close enough)

30. FEATURES OF A GOOD QUESTION
30
What would a “good” response look like?
– A paragraph? (too long)
– One word? (too short)
Make sure the reading is needed to respond (but a
sentence straight out of the book shouldn’t work).
Make sure a beginner can take a crack at the question
Be concrete:
– “Explain in 2-3 sentences.”
– “Give two brief examples.”
– “Explain how you got your estimate.”
“Game out” their responses a bit.

31. Summary Slides

32. A POSSIBLE PLAN
Choose one course you will teach next term.
A. Write two questions for each class meeting:
1. One lower-level (maybe multi-choice?).
One higher-level (sentences).
2. Give yourself 10 minutes to write each one
B. Write a standard (1st) metacognitive question
(What was most interesting or confusing …?)
C. Discuss one question at the top of class, and
one in the middle. Use the metacognitive
responses as break points or highlights.

33. OUR SUMMARY
JiTT may be among the easiest research-based
instructional strategies that you can consistently
integrate into your teaching.
From an evidence-based perspective, JiTT
addresses often-neglected areas.

34. YOUR SUMMARY
If you want to implement JiTT, what is your next
concrete action?
Randi Smith: rsmit216@msudenver.edu
Jeff Loats: jeff.loats@gmail.com, @JeffLoats
Courtney Rocheleau:
crochel1@msudenver.edu
Arlene Sgoutas: sgoutasg@msudenver.edu
Slides: www.slideshare.net/JeffLoats
Thanks for your attention!

35. JITT REFERENCES & RESOURCES
Simkins, Scott and Maier, Mark (Eds.) (2010) Just inTimeTeaching: Across the Disciplines, Across the
Academy, Stylus Publishing.
Gregor M. Novak, Andrew Gavrini, Wolfgang Christian, Evelyn Patterson (1999) Just-in-Time
Teaching: Blending Active Learning with WebTechnology. Prentice Hall. Upper Saddle River NJ.
K. A. Marrs, and G. Novak. (2004). Just-in-Time Teaching in Biology: Creating an Active Learner
Classroom Using the Internet. Cell Biology Education, v. 3, p. 49-61.
Jay R. Howard (2004). Just-in-Time Teaching in Sociology or How I Convinced My Students to
Actually Read the Assignment. Teaching Sociology,Vol. 32 (No. 4 ). pp. 385-390. Published by:
American Sociological Association
Stable URL: http://www.jstor.org/stable/3649666
S. Linneman, T. Plake (2006). Searching for the Difference: A ControlledTest of Just-in-Time
Teaching for Large-Enrollment Introductory Geology Courses. Journal of Geoscience Education, Vol.
54 (No. 1)
Stable URL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18
ON-DEMAND SLIDES

36. JITT VS. FINAL GRADE
CORRELATIONS
College Physics I, Fall 2013
0
20
40
60
80
100
0 10 20 30 40 50 60 70 80 90 100
CumulativeScore(withoutwarm-ups)
WarmUp Score
WarmUps vs. Cumulative Score
Correlation r = 0.71

37. Mean on 1-5 scale
Preparation for class 4.06
Engagement during
class 3.93
STUDENT SURVEY RESULTS
9% 10%
81%
10%
18%
73%
10%
22%
68%
0%
20%
40%
60%
80%
100%
Harmful Neutral Helpful
How did WarmUps affect your...
Preparation Engagement Learning
N = 781

38. STUDENT SURVEY QUOTES
Physics:
“Initially, it was hard for me to get used to the
warm-ups. It seemed like along with the
homework assignments there was a lot of things
to do. Eventually I got used to it and ultimately
the warmups really helped me to learn the
material and stay caught up with the class.”
“If it weren't for warm ups, the amount of time I
spent reading the book would have dropped by
75%”