Part 2 of a two-part presentation about both scholarly-teaching and Just-in-Time Teaching. This was the lunchtime keynote.

1. Name
School
Department
JUST IN TIME TEACHING
MAKING IT SHINE
(PART I1)
@ AIMS FALL FACULTY FORUM
DR. JEFF LOATS
DEPARTMENT OF PHYSICS

2. CHANGING THE CLASSROOM
2
Questions to think on:
• Are you best lecturer in the world on the
topics you teach?
• In the 21st-century, how should students spend
their 15 hours per credit with you?

3. WARM-UP: BIGGEST “TAKE AWAY”
Deslauriers, et al., 2011: “Improved Learning in a
Large-Enrollment Physics Class” (3 pages)
Controlled comparison of learning achieved
using two instructional approaches.
3 hours of traditional lecture given by an
experienced highly rated instructor.
VS.
3 hours of instruction given by a trained but
inexperienced instructor using research-based
best-practices.

4. WARM-UP: BIGGEST “TAKE AWAY”
What was the biggest “take away” idea that you
got from the article?
~50% → Active engagement shows impressive
learning outcomes compared to
lecture.
~30% → Changes were well-received by
students
~10% → Wondered why we still use poor
methods?

5. WARM-UP: BIGGEST “TAKE AWAY”
“It appears that the use of pre-class reading
assignments and online quizzes with clicker
questions and student discussion and small group
learning tasks (which I remember as the physics
lab) facilitated learning at least as well as did the
lecture method.”

6. WARM-UP: BIGGEST “TAKE AWAY”
“The most interesting findings were how much
better the experimental students performed on the
exam (approx 74% compared to 41%), despite being
nearly even with the control group on midterm
exams occurring before that.”
“The information related to how well received the
new instruction was for the students. I often hear
other faculty talk about how students sometimes
balk at anything new, I have to wonder how much of
that is actually the way students feel, or the
instructor.”

7. WARM-UP: BIGGEST “TAKE AWAY”
“The question that remains is, why are we still
teaching and doing the same things we have
always done?”
“The article seemed to lack a critical view on its
own experiment, not pointing out the added
expense, any potential challenges arising relying
on this technology, and suggesting further
explorations as how the investments and training
required.”

8. COMBINED IMPACT
8
Deslauriers, et al. (2011):
“We found increased student attendance, higher
engagement, and more than twice the learning in the
section taught using research-based instruction.”
With standard deviations around 13%, this meant an
effect size of 2.5!
“[…] other science and engineering classroom
studies report effect sizes less than 1.0. An effect
size of 2, obtained with trained personal tutors, is
claimed to be the largest observed for any
educational intervention.”

9. Consider:
“If I stopped using any “innovative” teaching
techniques, and just tried to do the best I could
with straight-up lecture, there would be no
concrete consequences to me.”
A) Strongly disagree
B) Disagree
C) Neither agree nor disagree
D) Agree
E) Strongly agree
9

10. Pause for questions…

11. Which topic would you like to spend our
remaining time on?
A) Evidence for effectiveness
B) Best tools for JiTT
C) Getting student “buy-in”
D) Writing good questions
11

12. Evidence For Effectiveness

13. STUDIED EFFECTIVENESS
Used at hundreds of institutions
Dozens of studies/articles, in many disciplines:
Bio, Art Hist., Econ., Math, Psych., Chem., etc.
–Increase in content knowledge
–Improved student preparation for class
–Improved use of out-of-class time
–Increased attendance & engagement in class
–Improvement in affective measures

14. JITT VS. FINAL GRADE
CORRELATIONS
College Physics I, Fall 2013
0
20
40
60
80
100
0 10 20 30 40 50 60 70 80 90 100
CumulativeScore(withoutwarm-ups)
WarmUp Score
WarmUps vs. Cumulative Score
Correlation r = 0.71

15. PROGRESSIVE EXAMS
CORRELATIONS
College Physics I:
Important disclosure: This was not a hypothesis we were
testing, it appeared as we analyzed the data. Could be
0.18
0.33
0.43
0.54
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
Mini Exam
(week 4)
Exam 1
(week 7)
Exam 2
(week 11)
Final Exam
(week 16)
NoneWeakStrongModerate
Correlations between Total WarmUp Score
and Sequence of Exams

16. Mean on 1-5 scale
Preparation for class 4.06
Engagement during
class 3.93
STUDENT SURVEY RESULTS
9% 10%
81%
10%
18%
73%
10%
22%
68%
0%
20%
40%
60%
80%
100%
Harmful Neutral Helpful
How did WarmUps affect your...
Preparation Engagement Learning
N = 781

17. STUDENT SURVEY QUOTES
Physics:
“Initially, it was hard for me to get used to the
warm-ups. It seemed like along with the
homework assignments there was a lot of things
to do. Eventually I got used to it and ultimately
the warmups really helped me to learn the
material and stay caught up with the class.”
“If it weren't for warm ups, the amount of time I
spent reading the book would have dropped by
75%”

18. Best Tools for JiTT

19. WHAT TOOLS TO USE?
The crucial part:
Daily reading, grading & using responses
• Automatic full credit for any response
• View all responses to a question together
• Grade responses on the same page with
minimal clicks
Wishlist:
Easy (quick!) individual feedback

20. SMALL ASIDE: TEXT EXPANDER
20
Every professor should have this!
You define a snippet like “ttyl” which instantly
gets replaced by “Talk to you later!”
Windows:
– Texter, PhraseExpres
(FREE, some advanced features, some flaws)
– Breevey ($40, worth it if you hit problems)
– AutoHotKey (free advanced automation tool)
Mac:
– TypeIt4Me, TextExpander, Typinator
(All cost $20-$30. Generally worth it!)

21. WHAT TOOLS TO USE?
• CMS/LMS (Blackboard, D2L, Moodle, etc.)
Ready to use, tools… imperfect  awful
• Free service from JiTTDL.org.
Designed just for JiTT. Additional website, not
very “shiny” by 2015 standards.
• Students email responses
Easy… also overwhelming and awful
• Blogging tools (WordPress)?
• New tools (TopHat? Learning Catalytics?)

22. Getting Student Buy-In
(a.k.a. “the sales pitch”)

23. THE SALES PITCH
13,000 hours in invisible contract indoctrination
Mindfulness in what we say and what we do.
Day 1 – Keep justifications short. Emphasize
purpose over mechanics.
Day 2 – Discuss their first experience & response
rates. Remind them about structure & purpose,
but mostly show them.
Day 3 – Return to “different roles” for both.
Demonstrate value, be consistent

24. OVERARCHING MESSAGE
Communicating with your students (humans)
• Message (explicit statements)
• Attitude (subtext, body language, etc.)
Consistent subtext:
"I am here to help you learn, and I have thought
about your learning trajectory carefully."
Consistent attitude:
I am comfortable and relaxed about my part of
this partnership.

25. DAY 1 – JITT
When discussing JiTT:
“Today is going to feel pretty ‘normal.’You’ll get
to see how this works starting next time, after
you’ve done your first warm-up.”
It isn’t “more assignments = more work,” but
rather “working in smaller chunks is more
effective and more efficient.”

26. STUDENTS: BUSY-WORK
DETECTORS
K-12 represents more than 13,000 hours of class
Students are experts at detecting what really
matters to an instructor:
• What does the instructor do with class time?
• What does the instructor talk about?
• Does the instructor push against the usual
“invisible contract” of the classroom?

27. DEMONSTRATING VALUE IN JITT
Ideas for demonstrating that you value JiTT
• Thank those who do them for giving you
insight into their learning.
• Bring at least one “difficult/interesting” item
from WarmUp to class each day.
• Give non-verbal cues that you value
discussing WarmUps as much (more) than
other course components.
• Be consistent!

28. CONSISTENCY
Be consistent with:
• Assignment releases
• Assignment due dates/times
• Follow-up in class
• Summative assessments (e.g., exam questions)
that build on WarmUp questions.

29. Writing Good Questions

30. EXAMPLE: WHIRLING BUCKET
A bucket of water can be whirled in a
vertical circle without the water falling
out, even at the top of the circle when the
bucket is upside down. Explain…
~15% → An outward force holds it in
~30% → An inward force holds it in
~20% → Talked (correctly!) about
acceleration & velocity… but
didn't really answer.
~10% → Nailed it! (or close enough)

31. EXAMPLE: WHIRLING BUCKET
“The water doesn't come out because you
twirling the bucket is applying the force of
spinning, and the water just kind of counteracts
that motion.”
“Because the water naturally wants to keep
traveling in the same direction its being
whirled around in the water attempts to
continue going up in a straight line but the
bottom of the bucket forces it to stay in the
bucket, like when you are pushed by the door
of a car while making a turn.”

32. FEATURES OF A GOOD QUESTION
32
What would a “good” response look like?
– A paragraph? (too long)
– One word? (too short)
Make sure the reading is needed to respond (but a
sentence straight out of the book shouldn’t work).
Make sure a beginner can take a crack at the question
Be concrete:
– “Explain in 2-3 sentences.”
– “Give two brief examples.”
– “Explain how you got your estimate.”
“Game out” their responses a bit.

33. WRITE A QUESTION AND SHARE...
33
Consider an intro. course in your discipline.
Consider a topic you discuss early in that course.
Write one question… shoot for “higher level.”
Good starting words: apply, analyze, evaluate,
sketch, use, compare, estimate, etc.
Take 3 minutes… then trade questions with your
neighbor and do your best to answer theirs.

34. Summary Slides

35. A POSSIBLE PLAN
Choose one course you will teach next term.
A. Write two questions for each class meeting:
1. One lower-level (maybe multi-choice?).
One higher-level (sentences).
2. Give yourself 10 minutes to write each one
B. Write a standard (1st) metacognitive question
C. Discuss one question at the top of class, and
one in the middle. Use the metacognitive
responses as break points or highlights.

36. YOUR SUMMARY
If you want to implement JiTT, what is your next
concrete action?
At Lunch: Scholarly Teaching & Making JiTT Shine
We will discuss an article… want to play along:
Google it: “Deslauriers Improved Learning”
Email: jeff.loats@gmail.com
Twitter: @JeffLoats
Slides: www.slideshare.net/JeffLoats

37. JITT REFERENCES & RESOURCES
Simkins, Scott and Maier, Mark (Eds.) (2010) Just in Time Teaching: Across the Disciplines, Across
the Academy, Stylus Publishing.
Gregor M. Novak, Andrew Gavrini, Wolfgang Christian, Evelyn Patterson (1999) Just-in-Time
Teaching: Blending Active Learning with Web Technology. Prentice Hall. Upper Saddle River NJ.
K. A. Marrs, and G. Novak. (2004). Just-in-Time Teaching in Biology: Creating an Active Learner
Classroom Using the Internet. Cell Biology Education, v. 3, p. 49-61.
Jay R. Howard (2004). Just-in-Time Teaching in Sociology or How I Convinced My Students to
Actually Read the Assignment. Teaching Sociology, Vol. 32 (No. 4 ). pp. 385-390. Published by:
American Sociological Association
Stable URL: http://www.jstor.org/stable/3649666
S. Linneman, T. Plake (2006). Searching for the Difference: A Controlled Test of Just-in-Time
Teaching for Large-Enrollment Introductory Geology Courses. Journal of Geoscience Education, Vol.
54 (No. 1)
Stable URL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18
Louis Deslauriers, Ellen Schelew and Carl Wieman (2011). Improved Learning in a Large-Enrollment
Physics Class. Science, Vol. 332 no. 6031 pp. 862-864 DOI: 10.1126/science.1201783
Sappington, J., Kinsey, K., & Munsayac, K. (2002). Two Studies of Reading Compliance Among
College Students. Teaching of Psychology , 29 (4), 272-274.
ON-DEMAND SLIDES