Fair, fast, feasible, frugal feedback

1. Fair, Fast, Feasible and Frugal Feedback
Adrian Molyneux
Lecturer in learning technology
Software development lead
Doctoral student
@ajmolyneux
…”where’s the ‘f’ in feedback?”

2. Aims for today
Why should students receive feedback after
summative assessments?
A look at the the medical school OSCE marking
app and the Feedback Portal
OSCE = Objective Structured Clinical Examination

3. Aims for today
Why should students receive feedback after
summative assessments?
A look at the the medical school OSCE marking
app and the Feedback Portal

4. Why should we give feedback?
• The cynical reason
• The educational reason

5. Why should we give feedback?
• The cynical reason
• The educational reason

7. Feedback on my work has been prompt
0
10
20
30
40
50
60
70
80
90
100
2009 2010 2011 2012 2013 2014 2015 2016
National Student Survey Results
Keele Median for all UK Medical Schools
%
satisfied

8. I have received detailed comments on my work
0
10
20
30
40
50
60
70
80
90
100
2009 2010 2011 2012 2013 2014 2015 2016
National Student Survey Results
Keele Median for all UK Medical Schools
%
satisfied

9. Feedback has helped to clarify things I did not
understand
0
10
20
30
40
50
60
70
80
90
100
2009 2010 2011 2012 2013 2014 2015 2016
National Student Survey Results
Keele Median for all UK Medical Schools
%
satisfied

10. Why should we give feedback?
• The cynical reason
• The educational reason

11. Station 1 Station 2 Station 3 Station 4
OSCE
Up to 16 stations x 2 or 3 “lanes”
… Objective Structured Clinical Examination
~ 60% station
pass mark
75% OSCE
pass mark

12. OSCE for airline pilots

13. Station 1
Take off
Station 2
Talk to passengers
Station 3
Teamwork
Station 4
Landing

14. 7/10 PASS
8/10 PASS
9/10 PASS
3/10 FAIL

15. 7/10 PASS
3/10 FAIL
9/10 PASS
8/10 PASS

16. 6/10 PASS
6/10 PASS
6/10 PASS
6/10 PASS

17. OSCE for medical students

18. History-taking 7/10 PASS
Examination skills 8/10 PASS
Communication
skills 9/10 PASS
Diagnostic skills
3/10 FAIL

19. History-taking 6/10 PASS
Examination skills 6/10 PASS
Communication
skills 6/10 PASS
Diagnostic skills
6/10 PASS

20. The results of an important exam are never
black and white
PASS
FAIL

21. Students pass each exam hurdle then move
on

23. I’m fit to
practise
Just don’t ask
me to
diagnose

24. I’m pleased
I’ve passed…
But I’m going to
really learn how
to diagnose

25. Aims for today
Why should students receive feedback after
summative assessments?
A look at the the medical school OSCE marking
app and the Feedback Portal
- Please contact me via email for access to Feedback Portal screenshot slides

26. Marking domains
come from a
predefined list of
generic
consultation skills
(GeCoS)

27. Do students engage with the
feedback?

28. Evolution
PASS
FAIL
www.flickr.com
/photos/stacyly
nn/134533183
05

29. Feedback on my work has been prompt
0
10
20
30
40
50
60
70
80
90
100
2009 2010 2011 2012 2013 2014 2015 2016
National Student Survey Results
Keele Median for all UK Medical Schools
%
satisfied

30. I have received detailed comments on my work
0
10
20
30
40
50
60
70
80
90
100
2009 2010 2011 2012 2013 2014 2015 2016
National Student Survey Results
Keele Median for all UK Medical Schools
%
satisfied

31. Feedback has helped to clarify things I did not
understand
0
10
20
30
40
50
60
70
80
90
100
2009 2010 2011 2012 2013 2014 2015 2016
National Student Survey Results
Keele Median for all UK Medical Schools

32. Does performance affect use of
feedback?
0
20
40
60
80
100
120
140
160
Excellent
(passed all
stations)
Good
(passed 10 -11
stations)
Just pass
(passed 8-9
stations)
Fail
(passed 7 or
fewer stations)
Meanwebpagesviewed

33. www.flickr.com/photos/swedinbalchik/7231923840

34. Thank you
Adrian Molyneux
a.j.molyneux@keele.ac.uk
@ajmolyneux