Rethinking feedback practices: insights from research We are seeing a shift in understandings of feedback from information telling towards social and cultural perspectives aimed at helping students to self-regulate their own learning. Our conceptions of feedback inform how feedback is practiced, e.g., if feedback is thought of as information transmission to ‘tell’ the learner about their performance then our feedback practices will tend to be unidirectional from tutor to learner with a focus on content and delivery. However, simply providing feedback does not ensure that students have read, understood or engaged and learned from it. The aim of this keynote is to synthesise key research evidence into feedback, including the persistent so called ‘feedback gap’, and to draw recommendations for how feedback could be practiced and embedded into curricula in order to optimise learning of students and staff.

1. Rethinking
feedback
practices:
insights
from
research
Dr
Rola
Ajjawi
Centre
for
Medical
Educa6on
University
of
Dundee

2. Aims
• Compare
two
understandings
of
feedback
– Informa6on
transmission
– Cultural
and
rela6onal
feedback
• Review
research
about
these
• Consider
applica6on
of
feedback
research
• Educa6onal
principles

3. Importance of feedback
• Construc6ve
feedback
o<en
results
in
improved
performance
(Veloski
et
al
2006;
HaDe
and
Timperley
2007;
Shute
2008)
• Affec6ve
processes
of
increased
effort
and
mo6va6on
and/or
cogni6ve
processes
of
restructuring
knowledge
• Learners
require
reassurance
that
they
are
heading
in
the
right
direc6on
• Self-­‐assessment
is
flawed
(Kruger
and
Dunning
1999;
Eva
and
Regehr
2005)
>>>
1/3
of
feedback
results
in
a
nega6ve
effect
(Kluger
and
DeNisi
1996)

4. INFORMATION
TRANSMISSION

5. Most
common
approach
prevalent
and
cri6qued
in
medical
educa6on
is
informa6on
transmission
hZps://www.flickr.com/photos/theenmoy/14811974227

6. Defini6on
of
feedback
as
informa6on
transmission
“Specific
informa6on
about
the
comparison
between
a
trainee’s
observed
performance
and
a
standard,
given
with
the
intent
to
improve
the
trainee’s
performance”
(van
der
Ridder
et
al
2008
p193)
hZps://www.flickr.com/photos/61423903@N06/8084060025/

7. Feedback
in
medical
school
Students
conceptualise
feedback
as
a
one-­‐way
process,
as
informa6on
given
to
them
rather
than
something
that
occurred
with
them
and
including
them
(Murdoch-­‐Eaton
et
al
2012;
Urquhart
et
al
2014)

8. “…
which
was
just
like
amazing
–
it
was
like
gold
dust”
(Urquhart
et
al
2014)
hZps://www.flickr.com/photos/jenny-­‐pics/

9. The
feedback
gap
Educators
and
learners
at
all
levels
believe
feedback
is
valuable
for
learning
BUT
There
is
a
disparity
in
educators’
and
students’
percep6on
of
feedback
(Carless
2006)

10. Tutors think they provide more detailed feedback
than students perceive
Tutors view their feedback to be more useful
compared to what students think (Carless 2006)

11. Systema6c
Literature
Review
(Tai
2009)
Comparison of educators’ and trainees’ perceptions of feedback
Situation Educators Trainees
GP training, reporting no feedback (Baker &
16% 32%
Sprackling 1994)
Attending surgeons & residents reporting episodes of
feedback (Hutul 2006)
67 episodes
(from 16
attending
surgeons)
24 episodes
(from 33
surgical
residents)
Attending surgeons and residents reporting effective
feedback given by educator (Sender Liberman et al
2005)
90.9% 16.7%
Faculty members and medical students in clinical
years, asked to rate “provided sufficient feedback” on
a 7-point scale (Gil et al 1984)
5.17 3.76
First and second year medical students and their
preceptors asked to rate “provides feedback about
performance” on a 5-point scale (Qualters et al 1999)
3.88 3.23

12. hZp://www.psy.gla.ac.uk/~simon/DIALOGUE.html

13. A critique of information
transmission
• Lack of understanding of feedback
• Lack of learner engagement with
feedback
• Transmitted feedback creates
dependency on teacher
• Not utilising self/peer feedback
• High teacher effort — low efficiency
(Sadler 1989, 2010; Nicol 2010)

14. One-­‐way
feedback
culture
• Minimal
student
self-­‐evalua6on
• Minimal
student
prepara6on
for
the
sessions
• Almost
no
collabora6ve
development
of
strategies
for
improvement
• Tokenis6c
aZempt
at
dialogue
(Molloy
2009)

15. SHIFTING
CONCEPTIONS

16. Feedback as cultural and
relational
“Feedback should be conceptualised as a
dialogical and contingent two-way process
that involves coordinated teacher–student
and peer-to-peer interaction as well as
active learner engagement” (Carless 2006)

17. Purpose of feedback…
Feedback should develop the students’
capacity to make evaluative judgements
about their own and others work
Feedback should serve the function of
progressively enabling students to better
monitor, evaluate and regulate their own
learning, independently of the
teacher
(Boud et al 2010; Nicol & McFarlane-Dick 2006)

18. Feedback encapsulates:
1. Where am I going? (learner oriented goal)
2. How am I going? (current performance)
3. Where to next? (or how do I get there?)
(HaDe
&
Timperley,
2007)

19. What do learners want?
Timely,
specific,
observed,
construc6ve
and
ac6onable
from
a
credible
source
(Watling
et
al
2012)
Face-­‐saving
strategies,
embedded
within
ac6on
(Rizan
et
al
2014)
Nego6ated
goal-­‐oriented
(Farrell
et
al
in
progress)
Praise
(Boehler
et
al
2006)

20. The learner
Interplay between fear, confidence and reasoning
in receptivity to feedback (Eva et al 2011)
Tendency for ‘deflection’ when there is a discrepancy
between learners’ internal perceptions (self-evaluation)
and the external teacher’s perceptions (feedback)
(Boud & Molloy 2013)
Learner re-interprets the external feedback to make it
conform with their own hope, intention or interpretation
of their performance (Carless et al 2010)
Perceived credibility of feedback giver essential
(Watling & Lingard 2010; Watling et al 2012)

21. ‘Educational Alliance’
• Research
based
on
psychotherapeu6c
literature
‘therapeu6c
alliance’
• Learners
ac6vely
evaluate
feedback
in
the
context
of
a
series
of
credibility
judgments
regarding
the
feedback
source
and
strength
of
the
educa6onal
alliance.
– supervisor’s
inten6ons
– clinical
exper6se
– supervisor’s
perceived
professional
iden6ty
• Feedback
acceptance
and
future
behaviour
is
based
on
this
ac6ve
emo6onal
and
cogni6ve
processing
(Telio
et
al
2014)

22. Culture and feedback
• Very
liZle
research
on
culture
and
feedback
in
medical
educa6on
• Learning
cultures
influence
feedback
(how
its
judged,
done,
conceptualised…)
(Watling
et
al
2012)

23. APPLICATION

24. “Medical
educators
may
have
been
too
focused
on
a
narrow
view
of
feedback.
Building
an
approach
or
system
around
a
few
teacher-­‐
specific
behavioral
principles
(e.g.
6mely,
specific)
is
inadequate.
An
approach
to
improving
feedback
incorporates
teacher-­‐based
behaviors,
learner-­‐based
cogni6ve
principles,
and
a
focus
on
the
teacher-­‐learner
rela6onship”
(Bing-­‐you
&
Towbridge,
2009,
p.1331)

25. Teacher-­‐focussed
strategies
• Set-­‐up
feedback
expecta6ons
• Explicate
6mings
• Consider
turnaround
of
feedback
• Improve
quality
of
the
feedback
• Faculty
development
and
benchmarking
• Ac6vely
engage
learners
in
feedback
dialogue
• Ask
for
feedback

26. Learner-­‐focussed
strategies
• Educate
and
empower
students
to
take
an
ac6ve
role
in
feedback
• Engage
students
in
genera6ng
assessment
criteria
and
discussing
standards
• Create
opportuni6es
for
students
to
process
and
use
feedback
(e.g.
self-­‐
explaining)
• Engage
students
in
ways
to
develop
evalua6ve
judgements
(self-­‐
and
peer-­‐review)
• Generate
a
dialogue
about
feedback

27. Rela6onship
• Recognise
that
learners
are
ac6vely
exploring
the
supervisor's
credibility
and
authen6city
• Reflect
on
the
strength
of
the
educa6onal
alliance
and
seek
feedback
from
the
learner
• Reflect
on
the
opportuni6es
for
rela6onship
building
• Nego6ate
goals
explicitly
• Avoid
tokenis6c
use
of
feedback
models

28. Feedback processes
“Rather
than
viewing
feedback
as
a
single
occurrence
or
a
‘knee
jerk’
reac6on
to
a
piece
of
work,
it
should
be
seen
as
a
series
of
pedagogical
opportuni6es
op6mised
across
a
life6me
of
the
programme””
(Boud
&
Molloy
2013)

29. Curriculum assessment sequencing
(Russell
&
Bygate
2010)

31. Feedback
process
1. Orienta6on
to
standards
of
work
and
purpose
of
feedback
2. Ac6vity
1
3. Student
judges
work
4. Student
asks
for
specific
feedback
5. Others
judge
work
6. Student
compares
judgements
7. Dialogue
with
tutor
about
feedback
8. Plan
for
improved
work
9. Ac6vity
2
(Ajjawi
et
al
2013)

32. Goal-­‐oriented
feedback
during
brief
encounters
hZps://www.flickr.com/photos/nasa2explore/

33. SUMMARY

34. New feedback model (Orsmond
et
al
2013)
Rela6onal
feedback
approach
Informa6on
transmission
Encourages
dialogue
Monologue
tutor
directed
one
way
Involves
peers
Does
not
involve
peers
Explicitly
encourages
self-­‐
evalua6on
Does
not
explicitly
encourage
self-­‐evalua6on
Feedback
on
assessment
process
Feedback
on
assessment
product
Students
encouraged
to
be
proac6ve
in
working
with
feedback
Students
encouraged
to
be
reac6ve
to
working
with
feedback
Promotes
self-­‐regula6on
of
learning
Correc6ve
feedback
focused
on
single
episode

35. Educational principles
1. Feedback
should
be
dialogic
in
nature
2. Assessment
design
should
afford
opportuni6es
for
feedback
to
be
used
in
future
assignments
3. Feedback
should
develop
students’
evalua6ve
judgements
and
monitoring
of
their
own
work
4. Students
should
be
empowered
to
seek
and
query
feedback
from
different
sources

36. LEARNER
TEACHER
PROCESS
CULTURE/
CONTEXT

37. Rola Ajjawi
Centre for Medical Education
University of Dundee
Email: r.ajjawi@dundee.ac.uk
hZps://www.flickr.com/photos/jenosaur/

38. Thanks
to
The
team:
Dr
Rola
Ajjawi;
Dr
Susie
Schofield;
Ms
Karen
Barton;
Mr
Grant
Murray;
Dr
David
Walker;
Dr
Sean
McAleer;
Ms
Natalie
Lafferty;
Dr
Lorraine
Walsh
Our
reference
group
JISC
for
funding
our
project
HEA
for
funding
our
interim
workshop
And
of
course
our
students
and
staff
both
academic
and
administra6ve
for
all
the
input
and
pa6ence

39. References
(1/3)
• Ajjawi,
R.,
Schofield,
S.,
McAleer,
S.,
&
Walker,
D.
(2013).
Assessment
and
feedback
dialogue
in
online
distance
learning.
Medical
Educa6on,
47(5),
527-­‐528.
• Ajjawi,
R.
(2012).
Going
beyond
‘received
and
understood’
as
a
way
of
conceptualising
feedback.
Medical
Educa6on,
46(10),
1018-­‐1019.
• Baker,
M.,
&
Sprackling,
P.
D.
(1994).
The
educa6onal
component
of
senior
house
officer
posts:
differences
in
the
percep6ons
of
consultants
and
junior
doctors.
Postgraduate
Medical
Journal,
70(821),
198-­‐202.
• Bing-­‐You,
R.
G.,
&
Trowbridge,
R.
L.
(2009).
Why
medical
educators
may
be
failing
at
feedback.
JAMA,
302(12),
1330-­‐1331.
• Boehler,
M.
L.,
Rogers,
D.
A.,
Schwind,
C.
J.,
Mayforth,
R.,
Quin,
J.,
Williams,
R.
G.,
et
al.
(2006).
An
inves6ga6on
of
medical
student
reac6ons
to
feedback:
a
randomised
controlled
trial.
Medical
Educa6on,
40(8),
746-­‐749.
• Boud,
D.,
&
Associates.
(2010).
Assessment
2010:
Seven
proposi6ons
for
assessment
reform
in
higher
educa6on.
Sydney:
Australian
Learning
and
Teaching
Council
• Boud,
D.,
&
Molloy,
E.
(2013).
Rethinking
models
of
feedback
for
learning:
the
challenge
of
design.
Assessment
&
Evalua6on
in
Higher
Educa6on,
38(6),
698-­‐712.
• Carless,
D.
(2006).
Differing
percep6ons
in
the
feedback
process.
Studies
in
Higher
Educa6on,
31(2),
219-­‐233.
• Carless,
D.,
Salter,
D.,
Yang,
M.,
&
Lam,
J.
(2011).
Developing
sustainable
feedback
prac6ces.
Studies
in
Higher
Educa6on,
36(4),
395-­‐407.
• Cleland
JA,
Knight
LV,
Rees
CE,
Tracey
S
&
Bond
CM
(2008).
“Is
it
me
or
is
it
them?”:
Factors
that
influence
the
passing
of
underperforming
students.
Medical
Educa6on,
42:
800-­‐809
• Cho,
K.,
&
MacArthur,
C.
(2011).
Learning
by
reviewing.
Journal
of
Educa6onal
Psychology,
103(1),
73-­‐84.
doi:10.1037/a0021950
• Eva,
K.,
Armson,
H.,
Holmboe,
E.,
Lockyer,
J.,
Loney,
E.,
Mann,
K.,
et
al.
(2011).
Factors
influencing
responsiveness
to
feedback:
on
the
interplay
between
fear,
confidence,
and
reasoning
processes.
Advances
in
Health
Sciences
Educa6on,
1-­‐12.

40. References
(2/3)
• Eva
KW,
Regehr
G.
Self-­‐assessment
in
the
health
professions:
A
reformula6on
and
research
agenda.
Acad
Med.
2005;80
(10
suppl):S46
–S54
• Gil,
D.H.,
Heins,
M.D.
&
Jones,
P.B.
(1984)
Percep6ons
of
medical
school
faculty
members
and
students
on
clinical
clerkship
feedback,
Journal
of
Medical
Educa6on,
59,
pp.
856–863
• HaDe,
J.
&
Timperley,
H.
(2007)
The
Power
of
Feedback.
Review
of
Educa6onal
Research,
77,
81-­‐112.
• Hesketh,
E.
A.,
&
Laidlaw,
J.
M.
(2002).
Developing
the
teaching
ins6nct,
1:
Feedback.
Medical
Teacher,
24(3),
245-­‐248.
• Hutul,
O.
A.,
Carpenter,
R.
O.,
Tarpley,
J.
L.,
&
Lomis,
K.
D.
(2006).
Missed
Opportuni6es:
A
Descrip6ve
Assessment
of
Teaching
and
ADtudes
Regarding
Communica6on
Skills
in
a
Surgical
Residency.
Current
Surgery,
63(6),
401-­‐409.
• Kluger
AN,
DeNisi
A.
The
effects
of
feedback
interven6ons
on
performance:
a
historical
review,
a
meta-­‐
analysis,
and
a
preliminary
feedback
interven6on
theory.
Psychol
Bull
1996;119
(2):254–84.
• Murdoch-­‐Eaton
D,
Sargeant
J.(2012)
Matura6onal
differences
in
undergraduate
medical
students’
perspec6ves
about
feedback.
Med
Educ;46:711–21.
• Molloy,
E.
K.
(2009).
Time
to
Pause:
Feedback
in
Clinical
Educa6on.
In
C.
Delaney
&
E.
K.
Molloy
(Eds.),
Clinical
Educa6on
in
the
Health
Professions.
Sydney:
Elsevier.
• Nicol,
D.
(2010).
From
monologue
to
dialogue:
improving
wriZen
feedback
processes
in
mass
higher
educa6on.
Assessment
&
Evalua6on
in
Higher
Educa6on,
35(5),
501
-­‐
517.
• Nicol,
D.
J.,
&
Macfarlane-­‐Dick,
D.
(2006).
Forma6ve
assessment
and
self-­‐regulated
learning:
a
model
and
seven
principles
of
good
feedback
prac6ce.
Studies
in
Higher
Educa6on,
31(2),
199-­‐218.
• Orsmond
et
al
(2013)
Moving
feedback
forward:
from
theory
to
prac6ce.
Assessment
&
Evalua6on
in
Higher
Educa6on
38
(2):10.1080/02602938.2011.625472
• Qualters,
D.
M.,
Regan,
M.
B.,
O’Brien,
M.
C.,
&
Stone,
S.
L.
(1999).
Comparing
Ambulatory
Preceptors’
and
Students’
Percep6ons
of
Educa6onal
Planning.
Journal
of
General
Internal
Medicine,
14(3),
196-­‐199.
• Rizan
C,
Elsey
C,
Lemon
T,
Grant
A
&
Monrouxe
L
(2014)
Feedback
in
ac6on
within
bedside
teaching
encounters:
a
video
ethnographic
study.
Medical
Educa6on
48:
902-­‐920.

41. References
(3/3)
• Russell,
M.,
&
Bygate,
D.
(2010).
Assessment
for
Learning:
An
introduc6on
to
the
ESCAPE
project.
Blended
Learning
in
Prac.ce,
March,
38-­‐48.
• Sadler,
D.
R.
(1989).
Forma6ve
assessment
and
the
design
of
instruc6onal
systems.
Instruc6onal
Science,
18(2),
119-­‐144.
• Sadler,
D.
R.
(2010)
Beyond
feedback:
developing
student
capability
in
complex
appraisal.
Assessment
&
Evalua6on
in
Higher
Educa6on,
35:5,
535-­‐550
• Sender
Liberman,
A.,
Liberman,
M.,
Steinert,
Y.,
McLeod,
P.,
&
Meterissian,
S.
(2005).
Surgery
residents
and
aZending
surgeons
have
different
percep6ons
of
feedback.
Medical
Teacher,
27(5),
470–472.
• Shute,
V.
J.
(2008).
Focus
on
Forma6ve
Feedback.
Review
of
Educa6onal
Research,
78(1),
153-­‐189.
• Telio
S,
Ajjawi
R,
Regehr
G
(in
press).
The
‘educa6onal
alliance’
as
a
framework
for
reconceptualizing
feedback.
Academic
Medicine
• Tai
J.
H.
(2009)
Improving
medical
students‘
experiences
and
percep6ons
of
feedback
in
clinical
medical
educa6on:
an
inquiry
and
focussed
educa6onal
interven6on.
Unpublished
thesis.
CMHSE,
FMNHS,
Clayton,
Monash
University
• Urquhart,
L.
M.,
Rees,
C.
E.,
&
Ker,
J.
S.
(2014).
Making
sense
of
feedback
experiences:
a
mul6-­‐school
study
of
medical
students'
narra6ves.
Medical
Educa6on,
48(2),
189-­‐203.
• Van
De
Ridder,
J.
M.
M.,
Stokking,
K.
M.,
McGaghie,
W.
C.,
&
Ten
Cate,
O.
T.
J.
(2008).
What
is
feedback
in
clinical
educa6on?
Medical
Educa6on,
42(2),
189-­‐197
• Veloski,
J.,
Boex,
J.
R.,
Grasberger,
M.
J.,
Evans,
A.,
&
Wolfson,
D.
B.
(2006).
Systema6c
review
of
the
literature
on
assessment,
feedback
and
physicians’
clinical
performance:
BEME
Guide
No.
7.
Medical
Teacher,
28(2),
117-­‐128.
• Watling,
C.,
&
Lingard,
L.
(2012).
Toward
meaningful
evalua6on
of
medical
trainees:
the
influence
of
par6cipants’
percep6ons
of
the
process.
Advances
in
Health
Sciences
Educa6on,
17(2),
183-­‐194.
• Watling,
C.,
Driessen,
E.,
van
der
Vleuten,
C.
P.
M.,
&
Lingard,
L.
(2012).
Learning
from
clinical
work:
the
roles
of
learning
cues
and
credibility
judgements.
Medical
Educa6on,
46(2),
192-­‐200.
• Watling,
C.,
Driessen,
E.,
van
der
Vleuten,
C.
P.
M.,
Vanstone,
M.,
&
Lingard,
L.
(2012).
Understanding
responses
to
feedback:
the
poten6al
and
limita6ons
of
regulatory
focus
theory.
Medical
Educa6on,
46(6),
593-­‐603.

42. Content
(understanding
of
theory
/
principles
and
applica6on
to
own
context)
-­‐
Understanding
of
learning
theory
-­‐
Evidence
of
cri6cal
reflec6on
on
learning
theory
and
key
learning
and
teaching
principles
-­‐
Evidence
of
applica6on
of
learning
theory
to
own
prac6ce
Self-­‐evalua6on:
Tutor
feedback:
Style,
format
and
language
(e.g.
structure,
coherence,
flow,
formaDng,
use
of
language)
Self-­‐evalua6on:
Tutor
feedback:
Sources
and
references
(e.g.
range
of
references
cited,
relevance,
consistency,
accuracy
and
completeness
of
referencing)
Self-­‐evalua6on:
Tutor
feedback:
Which
aspect(s)
of
your
assignment
would
you
specifically
like
feedback
on?
Student
comment:
Tutor
feedback:
How
did
previous
feedback
inform
this
assignment?
Student
comment:
Name
of
Tutor:
Date: