Claire Yan, Senior Instructor, University of British Columbia Festival of Learning in Burnaby, B.C. - June 6-9, 2016

1. Using muddiest point forma1ve feedback
to encourage reflec1ve teaching and
improve student learning
Claire Yan
School of Engineering, UBC Okanagan
Stephen Krause
Ira A. Fulton School of Engineering, Arizona
State University
Fes?val of Learning, Burnaby BC, June 6-9, 2016

2. Agenda
• Forma?ve and summa?ve feedback
• Features of effec?ve feedback
• Muddiest point survey
– Background
– Implementa?on via Blackboard survey tool
– Impact on instructor’s teaching
– Impact on student learning
• Summary
2

3. 3
How can I quickly grab
students’ attention in class?
How do I quickly identifystudent problems?

4. Forma1ve Feedback Summa1ve Feedback
Time During a course, ongoing At the end of a course
Purpose • Help students iden?fy
strength and weakness
during the course of
learning
• Help the instructor
monitor student learning
progress
• Provide students final
evalua?on of their
learning in comparison
to a standard
4

5. Effec1ve Forma1ve Feedback
• Content
– Specific to the processes that students are engaging
in
– Sufficient to direct students’ subsequent learning
• Timing
– How soon? (the earlier the beZer)
– How oen? (the more frequent the beZer)
5
Ambrose, S.A., et al, How learning works: Seven Research-Based Principles for Smart Teaching, Jossey-
Bass, 2010

6. Muddiest Point (MP) Survey
• A means of forma?ve feedback developed by Dr.
Frederick Mosteller, 1989
• Focusing on student confusion on a topic
• Two ques?ons in a MP survey
– Muddiest point
“What topic remains the least clear to you?”
“Describe in a sentence what was confusing or needs more
explana?on.”
– Most interes1ng point
“In a sentence explain what you found the most interes?ng
about today's class.”
6

7. Blackboard survey tool for MP data collec1on
• MP survey for each class, opens from the end
of class un?l a few hours before the next
class
• Student responses are anonymous – honest
responses
7

8. Sample Student Responses to MP Survey
from a material science class
(experienced user)
• Muddiest Points
– “How do I determine which elements form Ionic or
Covalent bonds?”
– “I need clarifica?on on the characteris?cs associated with
certain types of bond”
• Most Interes1ng Points
– “How to make a garbage bag and a bulletproof vest out of
the same material.”
– “Covalent bond has several kinds of bonding such as 0-D,
1-D, 3-D.”
8
DON’T SUMMARIZE USE STUDENTS’ OWN LANGUAGE

9. MP Survey for Reflec1ve Teaching
• Address MP ques?ons at the beginning of each class
– Draw students’ aZen?on
– Resolve confusion and correct misconcep?ons
– Smooth transi?on from previous class to current class
• Design targeted class ac?vi?es and tutorial/
assignment problems
• Create enriched learning resources
– Youtube channel
hZps://www.youtube.com/user/MaterialsConcepts/feed
– Create ideas for new books
9

10. Successful
(experienced user)
Successful
(experienced user)
Successful
(first 1me user A )
Successful
(first 1me user)
Unsuccessful
(first 1me user B)
Class size ~ 80 N/A ~ 80 23 Two classes (27,
29)
Overall
response
rate
60-90%
80-90% (at the
end of
semester)
~ 70% ~ 52% ~ 58% (3x5 notecard)
~ 79% (assigned reading)
Very poor
Incen1ve Up to 5% grade
(extra credit)
Credit for
aZendance
Up to 2.5% grade
(extra credit)
• None for class
notecard;
• MP for assigned
reading is part of
homework
none
MP
feedback
Almost in every
class with
single slide of a
few MPs with
diagrams on
board
Almost in every
class
• 80% lectures
• 5-10 min at the
beginning of
class
• Hand back 3x5
notecard in the next
class with
instructor’s
comments
• Further discussion
on MP in class
N/A
10
Waters, C.K. et al, Revealing Student MisconcepDons and Instructor Blind Spots with Muddiest Point FormaDve
Feedback , 123rd ASEE annual conference and exposiDon, June 26-29, 2016

11. Change of Faculty Beliefs Measured by ATI
• Approaches to Teaching Inventory (ATI): an instrument (22 ques?ons)
developed by Trigwell and Prosser to understand how faculty approach
teaching: teacher focused strategies -> student focused strategies
11
Trigwell, K., and Prosser, M. (2004). Development and use of the approaches to teaching inventory. EducaDonal
Psychology Review, 16(4), 409-424
Waters, C.K. et al, Revealing Student MisconcepDons and Instructor Blind Spots with Muddiest Point FormaDve
Feedback , 123rd ASEE annual conference and exposiDon, June 26-29, 2016
37.5
21.7
-20
-10
0
10
20
30
40
Sucessful 1st ?me user
(A)
Unsucessful 1st ?me
user (B)
% of change in ATI scores between pre- and post- MP
ac1vi1es
Score in "Teacher
Focused" Column
Score in "Student
Focused" Column

12. Change of Faculty Beliefs Measured by RTOP
12
34
9
0
5
10
15
20
25
30
35
40
Sucessful 1st ?me user (A) Unsucessful 1st ?me user (B)
% of change in RTOP scores between pre- and
post- MP ac1vi1es
Waters, C.K. et al, Revealing Student MisconcepDons and Instructor Blind Spots with Muddiest Point FormaDve
Feedback , 123rd ASEE annual conference and exposiDon, June 26-29, 2016
• A high RTOP score
indicates a more
student-centered
teaching strategy.
• A low RTOP score
indicates a more
teacher-centered
strategy.

13. Impact on Student Learning
Data from an experienced user
• Student persistence (#students at final exam over #students
ini?ally registered in the course)
13
• MP promotes
reflec?ve learning
• Scaffolding with ?mely
feedback and enriched
learning resources
boosts students’
interest and
confidence in learning
Krause, S.J. et al, The Impact of Two-Way FormaDve Feedback and Web-Enabled Resources on Student Resource Use
and Performance in Materials Courses, 122nd ASEE annual conference and exposiDon, June 14-17, 2015
Before using MP
Aer using MP

14. Impact on Student Learning
Data from an experienced user
• Final exam scores
14 Krause, S.J. et al, The Impact of Two-Way FormaDve Feedback and Web-Enabled Resources on Student Resource Use
and Performance in Materials Courses, 122nd ASEE annual conference and exposiDon, June 14-17, 2015

15. Summary
• MP makes learning “visible” to both instructors and
students
• MP, if used effec?vely, may
– For faculty
• Promote reflec?ve teaching
• Help plan/create enriched learning resources and engagement
ac?vi?es
• Help foster student-centered teaching philosophy and prac?ce
– For students
• Encourage reflec?ve learning
• Promote responsibility for and confidence in learning
• To encourage student par?cipa?on
– Use incen?ves
– Faculty provides consistent and ?mely feedback to help
students see the value of MP
15

16. Acknowledgement
• Bethany Smith, MASc, Arizona State University
• Yong-Seok Park, PhD, Arizona State University
• Danielle Stevens, MASc, Arizona State University
Contact: Claire Yan ( yu.yan@ubc.ca )
16