A discussion of Scholarly Teaching, with a focus on three areas: - Active engagement during class time - Effective preparation (students & instructors) - Feedback loops and iterative learning

1. Name
School
Department
Scholarly Teaching:
How we are all social
scientists
@ DIME
January 16th, 2019
Dr. Jeff Loats

2. WARM-UP: DISCIPLINE ABUSE
Can you think of a time when you were frustrated
by misconceptions or misunderstandings of your
field of expertise in the world at large?

3. DISCIPLINE VS. CAREER
3
Dunning-Kreuger effect:
• Expertise in a discipline probably makes us
underestimate our knowledge/skill.
• A lack of expertise in teaching probably makes us
overestimate our knowledge/skill.
Teaching is a fundamentally social activity,
regardless of topic or academic discipline.

4. SCHOLARLY TEACHING
4
Goal 1:
Apply the rigor and scholarship of our
academic disciplines to the discipline of
teaching.

5. STEALING FOR SUCCESS
5
A sheepish statement from a colleague:
“I’ve borrowed ideas and techniques from my
own teachers and from colleagues. Of course, I
always change them a bit to make them my
own.”
Have you “stolen” teaching ideas from
colleagues?

6. PLEASE STEAL THIS IDEA!
6
Are there fields in which “stealing” ideas is
acceptable? Encouraged? Required?
Practical skills: Electrician, “How To” videos
Safety concerns: Where do you store poisons?
Medicine: Ask your doctor, “Where do your
methods and ideas about treating my condition
come from?”
I want a scholarly doctor:
Aware of the best, most up-to-date research on
how to treat my condition .

7. SCHOLARLY TEACHING
Goal 1:
Apply the rigor and scholarship of our
academic disciplines to the discipline of
teaching.
Goal 2:
Choose teaching methods that are
strongly informed by the best empirical
evidence available.
7

8. CONSISTENT EVIDENCE-BASED
THEMES
8
• Active engagement during class time
• Effective preparation (students & instructors)
• Feedback loops and iterative learning

9. Consider a typical day in a typical (college) class.
What fraction of class time is spent on lecture-
based delivery of content?
Previous anonymous poll results (compiled):
N = 267
13%
18% 22% 29% 19%
0%
10%
20%
30%
40%
50%
60%
0-20% 20-40% 40-60% 60-80% 80-100%

10. CHANGING THE CLASSROOM
10
Are you best lecturer in the world on the topics
you teach?
Does the best lecturer in the world have a
YouTube channel?
In the 21st-century, how should students spend
their 15 hours per credit with you?

11. Consider a typical day in a typical (college) class.
What fraction of students did their preparatory
work before coming to class?
Previous anonymous poll results (compiled):
N = 398
33% 34%
18%
12%
4%0%
10%
20%
30%
40%
50%
60%
0-20% 20-40% 40-60% 60-80% 80-100%

12. CHANGING OUR PREPARATION
12
How do we “make room” for an active-
engagement classroom?
Shift appropriate parts of teaching & learning
outside of the classroom:
• Student preparation is a “low hanging fruit”
that enhances everything else.
• Instructors prepare by learning what our
students already think about the subject.

13. PRE-CLASS WORK
13
Evidence:
Sappington (1998):
Students who did well on a surprise reading
assessment “scored significantly better than the
Zero or Fail groups.” Effect size was 0.25.
Marrs (2003):
Students showed an average normalized gain of
∼52% on test questions reinforced by either
Warm Up questions or Cooperative Learning
(~60% if reinforced by both!).

14. HOW DO PEOPLE LIKE TO LEARN
14
Do we ever enjoy learning?
Possible candidates:

15. COMMON ELEMENTS?
15
Feedback is (nearly) instantaneous
Failure is expected (desired?)
The cost of failure is very low
Mastery requires iterative learning
Contrast this with a typical feedback loop in the
classroom…

16. “MANY CHANCES TO FAIL”
16
A line adopted from business:
“Fail early, fail often, fail well…”
Grounded in constructivist learning theory:
• Constructing new ideas often requires facing the
failure of previous ideas.
• Confusion and conflict make clear the need to
build functional ideas in place of those that failed.

17. ITERATIVE LEARNING LOOPS
17
On a given topic…
Before class: Engage with Just-in-Time Teaching
“warm-up” questions that enforce
reading & require thought
During class: Respond (digitally) to difficult
questions, peer discussions
After class: Online homework with immediate
feedback and low(ish) stakes.
Perhaps 10-20 chances to test their understanding
before they encounter a high-stakes exam.

18. Students have developed a robot dog
and a robot cat, both of which can
run at 8 mph and walk at 4 mph.
A the end of the term, there is a race!
The robot cat must run for half of its
racing time, then walk.
The robot dog must run for half the
race distance, then walk.
A) The cat wins B) The dog wins C) They tie
18

19. COMBINED IMPACT –
DESLAURIERS
19
Deslauriers, et al. (2011): “Improved learning in
a large-enrollment physics class.”
Novice teachers with evidence-based teaching
techniques more than doubled student learning,
compared to an experienced and highly-rated
traditional instructor.

20. COMBINED IMPACT –
DESLAURIERS
20
Deslauriers, et al. (2011): “Improved learning in
a large-enrollment physics class.”
“The standard deviation calculated for both
sections was about 13%, giving an effect size for
the difference between the two sections of 2.5
standard deviations. As reviewed in (4), other
science and engineering classroom studies report
effect sizes less than 1.0. An effect size of 2,
obtained with trained personal tutors, is claimed
to be the largest observed for any educational
intervention (16).”

21. COMBINED IMPACT – FREEMAN
Freeman, et al., 2014: “Active learning increases
student performance in science, engineering, and
mathematics”
Meta-analysis of studies on “active learning” vs
“traditional lecture.” Included 225 studies that
reported on exam scores an failure rates in STEM
courses.

22. COMBINED IMPACT – FREEMAN
Freeman, et al., 2014: “Active learning increases
student performance in science, engineering, and
mathematics”
One finding: The odds ratio for failing under
traditional lecture was 1.95 (translates to a 50%
higher chance of failing).
In medical trials, “a recent analysis of 143
randomized controlled medical trials that were
stopped for benefit found that they had a median
relative risk of 0.52, with a range of 0.22 to 0.66
(15).”

23. SUMMARY AND CHALLENGE
Challenge yourself to be a scholarly-teacher
Follow the evidence!
Be moderate… (perhaps follow the “10% rule”)
Engage with peers! Share, steal, and combine.

24. Your Summary
24
For yourself… or to share?
What nugget(s) from this talk do you want to
keep in mind in a month or a year?
Email: jeff.loats@gmail.com
Twitter: @JeffLoats
Slides: www.slideshare.net/JeffLoats

25. References
25
Dunning–Kruger effect:
http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
Louis Deslauriers, Ellen Schelew and Carl Wieman (2011). Improved Learning in
a Large-Enrollment Physics Class. Science, Vol. 332 no. 6031 pp. 862-864 DOI:
10.1126/science.1201783
Freeman, Scott; Eddy, Sarah L.; McDonough, Miles; Smith, Michelle K.;
Okoroafor, Nnadozie; Jordt, Hannah; Wenderoth, Mary Pat; Active learning
increases student performance in science, engineering, and mathematics (opens as
pdf), 2014, Proc. Natl. Acad. Sci.
Sappington, J., Kinsey, K., & Munsayac, K. (2002). Two Studies of Reading
Compliance Among College Students. Teaching of Psychology , 29 (4), 272-274.
Marrs, K.A. (2003). Just in Time Teaching enhances cognitive gains in biology. J.
Coll. Sci. Teach.

26. Aside: Learning Styles
“I think that many teachers teach in a way that
makes sense to them, according to their learning
style […]”
Best current evidence: Learning styles don’t exist
References:
• “The Myth of Learning Styles”
by Cedar Riener and Daniel Willingham
• YouTube: Learning Styles Don’t Exist
• Scholarly review: “Learning styles: Concepts
and evidence”, Pashler et al, 2008