1. Name
School
Department
A TALE OF TWO CLASSES:
JUST IN TIME TEACHING (JITT)
IN SOCIOLOGY AND PHYSICS
@ SLOAN-C, JULY 2014
DR. TING JIANG
DEPARTMENT OF SOCIOLOGY
AND ANTHROPOLOGY
DR. JEFF LOATS
DEPARTMENT OF PHYSICS

2. OUTLINE
1. The Blended Learning Initiative
2. Basics of Just-in-Time Teaching
3. Structure & examples from each class
4. Analysis methods & results
5. Student attitudes & perspectives
6. Summaries

3. BLENDED LEARNING INITIATIVE
(2013)
Common:
Target intro courses with large enrollments
(instructors teaching First Year Success courses)
Offer small incentive/acknowledgement (laptop)
Perhaps uncommon:
Invite potential participants (vs. them applying)
Offer a small “menu” of teaching techniques
Sustained support: 6-8 “meetings” including
1-on-1, small-group meetings, workshops.

4. PARTICIPANT PERSPECTIVE
Spring 2013:
Exposure to new pedagogical approaches
Hands on guidance to convert a regular class to a
JiTT incorporated class
Fall 2013:
Continuous support every step of the way
Useful follow up sessions during the trial
semester

5. In your teaching do you have a method for holding
students accountable for preparing for class?
A)I don’t, but I ask/threaten really well.
B)I use a paper method (quiz, journal, others?)
C)I use a digital method (clickers, others?)
D)I use Just in Time Teaching.
E) I have some other method.
17%
50%
10%
6%
16%
From
~170
others

6. JUST IN TIME TEACHING
Online pre-class assignments
called WarmUps
First half - Students
• Conceptual questions, answered in sentences
• Graded on thoughtful effort
Second half - Instructor
• Responses are read “just in time”
• Instructor modifies that day’s plan accordingly.
• Aggregate and individual (anonymous) responses
are displayed in class.
Learner Teacher

7. Consider a typical day in your class. What fraction
of students did their preparatory work before
coming to class?
A) 0% - 20%
B) 20% - 40%
C) 40% - 60%
D) 60% - 80%
E) 80% - 100%
27%
33%
21%
14%
5%
From
~200
others

8. WARMUP QUESTIONS
• Every-day language
• Occasional simple comprehension question
• Mostly higher level questions (a la Bloom)
• Perhaps any question is better than none
Connections to evidence:
–Pre-class work reduces working memory load
during class.
–Multimodal practice (not learning styles):
JiTT brings reading, writing and discussion as
modes of practice.

9. METACOGNITION
First & last questions on every WarmUp:
“What aspect of the material did you find the
most difficult or interesting.”
“How much time did you spend on the pre-class
work for tomorrow?” [Multiple-choice survey]
Connections to evidence:
–Metacognition practice:
Students regularly evaluate their own
interaction with the material.

10. THE JITT FEEDBACK LOOP
Student responses:
• Graded on thoughtful effort
• Sampled and categorized for display
• Quoted anonymously
Closing the loop:
• Respond to some students digitally
• Shift class time toward active engagement

11. JUST IN TIME TEACHING
A different student role:
• Actively prepare for class
(not just reading/watching)
• Actively engage in class
• Compare your progress & plan accordingly
A different instructor role:
• Actively prepare for class with you
(not just going over last year’s notes )
• Modify class accordingly
• Create interactive engagement opportunities
Learner Teacher

12. JITT STRUCTURE & RESPONSE
RATES
0
10
20
30
40
50
60
70
80
90
100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
%Responsed
Class #
Response Rate by Day
College Physics I, N = 78
0
10
20
30
40
50
60
70
80
90
100
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
%Responsed
Class #
Response Rate by Day
Intro. Sociology, N = 23
Worth 10% of final grade
Due 10 PM the night before class
Assignments available for prior 2-3 days
College Physics I Intro. to Sociology
Worth 5% of final grade
Due 10 PM the night before class
Assignments available for prior 2-3 days

13. SOCIOLOGY EXAMPLE: LOOKING GLASS
SELF
Read p. 69 of the textbook, and explain what is
“looking-glass self.”
Responses:
“What we think of ourselves depends on how we
think others see us”
“The way we see ourselves as we imagine others
see us”
“Reflection of how we see ourselves from other's
perspectives.”

14. PHYSICS EXAMPLE: WHIRLING
BUCKET
A bucket of water can be whirled in a
vertical circle without the water falling
out, even at the top of the circle when the
bucket is upside down. Explain…
~15% → An outward force holds it in
~30% → An inward force holds it in
~20% → Talked (correctly!) about
acceleration & velocity… but
didn't really answer.
~10% → Nailed it! (or close enough)

15. PHYSICS EXAMPLE: WHIRLING
BUCKET
“The water doesn't come out because you
twirling the bucket is applying the force of
spinning, and the water just kind of
counteracts that motion.”
“Because the water naturally wants to
keep traveling in the same direction its
being whirled around in the water
attempts to continue going up in a straight
line but the bottom of the bucket forces it
to stay in the bucket, like when you are
pushed by the door of a car while making
a turn.”

16. JITT VS. FINAL GRADE
CORRELATIONS
College Physics I Intro. to Sociology
0
20
40
60
80
100
0 20 40 60 80 100
CumulativeScore(withoutwarm-ups)
WarmUp Score
WarmUps vs. Cumulative Score
Correlation
r = 0.71
0
20
40
60
80
100
0 20 40 60 80 100
CumulativeScore(withoutWarmUps)
Warmup Score
WarmUps vs. Cumulative Score
Correlation
r = 0.50

17. SOCIOLOGY – COMPARISON TO CONTROL
Comparison of the same course, same term, same
instructor, with and without JiTT.
Experimental group: 23 students, with JITT
Control group: 48 students, without JiTT
Same assessment tools: Exams and final papers
But… this difference is only marginally significant
with a p-value of 0.11.
Group Average on Final Exam
Control (without JiTT) 73.2
Experimental (with JiTT) 80.5

18. PHYSICS – PROGRESSIVE EXAMS
College Physics I:
Important disclosure: This was not a hypothesis we were testing, it
appeared as we analyzed the data. Could be spurious.
0.18
0.33
0.43
0.54
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
Mini Exam
(week 4)
Exam 1
(week 7)
Exam 2
(week 11)
Final Exam
(week 16)
NoneWeakStrongModerate
Correlations between Total WarmUp Score
and Sequence of Exams

19. STUDENT SURVEY RESULTS
Mean on 1-5 scale
Preparation for class 4.11
Engagement during class 4.17
Learning the material 3.9
College Physics I Intro. to Sociology
Mean on 1-5 scale
Preparation for class 4.43
Engagement during class 4.00
Learning the material 4.57
3%
10%
58%
5%
9%
57%
6%
15%
50%
0%
20%
40%
60%
80%
100%
Harmful Neutral Helpful
How did WarmUps affect your...
Preparation Engagement Learning
N = 71
0%
14%
86%
14% 14%
71%
0%
14%
86%
0%
20%
40%
60%
80%
100%
Harmful Neutral Helpful
How did WarmUps affect your...
Preparation Engagement Learning
N = 7

20. STUDENT SURVEY QUOTES
Sociology:
“JITT was help me retain the information from lecture
and helped my study for exams.”
“Very helpful. I wish other classes did it. Helped me stay
on track within the class”

21. STUDENT SURVEY QUOTES
Physics:
“Initially, it was hard for me to get used to the
warm-ups. It seemed like along with the
homework assignments there was a lot of things
to do. Eventually I got used to it and ultimately
the warmups really helped me to learn the
material and stay caught up with the class.”
“If it weren't for warm ups, the amount of time I
spent reading the book would have dropped by
75%”

22. OUR SUMMARY
Important similarities in results:
• Moderate to strong correlations between
course performance and WarmUp score
• Between-group and in-group comparisons help
indicate a robust result
• Positive student responses
Important confounding factors:
• Different experience levels of instructors
• Different disciplines
• Different student populations

23. YOUR SUMMARY
For yourself… or to share?
What one “nugget” do most want remember to
use yourself or to share with other faculty?
ContactTing: tjiang@msudenver.edu
Contact Jeff: Jeff.Loats@gmail.com
Twitter: @JeffLoats
Slides: bit.ly/jeffloats

24. ON-DEMAND SLIDES
JITT REFERENCES & RESOURCES
Simkins, Scott and Maier, Mark (Eds.) (2010) Just in Time Teaching: Across the Disciplines, Across the Academy, Stylus Publishing.
Gregor M. Novak, Andrew Gavrini, Wolfgang Christian, Evelyn Patterson (1999) Just-in-Time Teaching: Blending Active Learning with
Web Technology. Prentice Hall. Upper Saddle River NJ.
K. A. Marrs, and G. Novak. (2004). Just-in-Time Teaching in Biology: Creating an Active Learner Classroom Using the Internet. Cell
Biology Education, v. 3, p. 49-61.
Jay R. Howard (2004). Just-in-Time Teaching in Sociology or How I Convinced My Students to Actually Read the
Assignment. Teaching Sociology, Vol. 32 (No. 4 ). pp. 385-390. Published by: American Sociological Association
Stable URL: http://www.jstor.org/stable/3649666
S. Linneman, T. Plake (2006). Searching for the Difference: A Controlled Test of Just-in-Time Teaching for Large-Enrollment
Introductory Geology Courses. Journal of Geoscience Education, Vol. 54 (No. 1)
Stable URL:http://www.nagt.org/nagt/jge/abstracts/jan06.html#v54p18

25. T-TEST RESULTS
------------------------------------------------------------------------------
Group | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval]
---------+--------------------------------------------------------------------
JiTT | 23 80.45595 3.483984 17.06797 73.24878 87.66312
Regular | 48 73.19017 3.710472 25.7069 65.72567 80.65468
---------+--------------------------------------------------------------------
combined | 71 75.6121 2.748048 23.31796 70.13265 81.09155
---------+--------------------------------------------------------------------
diff | 7.265781 5.806398 -4.314713 18.84628
------------------------------------------------------------------------------
Degrees of freedom: 69
Ho: mean(JiTT) - mean(Regular) = diff = 0
Ha: diff < 0 Ha: diff != 0 Ha: diff > 0
t = 1.2513 t = 1.2513 t = 1.2513
P < t = 0.8925 P > |t| = 0.2150 P > t = 0.1075

26. WHAT TOOLS TO USE?
The crucial part:
Daily reading, grading & using responses
• Automatic full credit for any response
• View all responses to a question together
• Grade responses on the same page with
minimal clicks
Wishlist:
Easy (quick!) individual feedback

28. WHAT MIGHT STOP YOU?
In terms of the technique:
Time, coverage, not doing your part, pushback…
In terms of the technology:
Learning curve, tech. failures, perfectionism…
In any reform of your teaching:
Reinventing, no support, too much at once…