Shared feedback literacy: Designing for productive feedback experiences

1. Shared feedback literacy:
Designing for productive feedback
experiences
David Carless, University of
Hong Kong,
@CarlessDavid
Keynote for Queen’s University
Belfast, January 20, 2022
The University of Hong Kong

2. Overview
1. Towards positive feedback experiences
2. Peer feedback and digital possibilities
3. Designing for feedback
4. Shared feedback literacy
5. Implications
The University of Hong Kong

3. FROM PRACTICE TO THEORY
The University of Hong Kong

4. The University of Hong Kong

5. Peer review + response
Research proposal assessment task (Year 2)
Anonymous peer review
Revise and respond (rebut) addressing peer
reviews
The University of Hong Kong

6. Cumulative peer review
Peer review within research-based curriculum
Development of critical evaluation skills
Multiple & sustained experiences
(Harland et al., 2017).
The University of Hong Kong

7. Knowledge creation
Subject: Knowledge of disciplinary content
Feedback: Skills of reviewing
Self: Confidence to involve in peer review
Peers: respecting other’s abilities
(Reddy et al., 2020)
The University of Hong Kong

8. Student peer review rationale
Being exposed to a body of related work
Enabling comparison between own
production and that of peers
(Nicol, 2021)
The University of Hong Kong

9. Key steps
1. Students attempt task;
2. Students review multiple other attempts;
3. Peer feedback composed & received;
4. Opportunities to revise work.
The University of Hong Kong

10. Digital peer feedback
Peer feedback using Screencasting &
Google docs
Opportunities to clarify peer feedback and
co-construct opportunities to use it
(Wood, 2021).
The University of Hong Kong

11. The University of Hong Kong

12. Facilitators
Teacher capacities: designing multiple peer
feedback sequences; offering curriculum time,
experience and support
Student capacities: willingness to give and
receive peer feedback; engage in dialogue;
trust and respect; take action
The University of Hong Kong

13. TOWARDS DESIGNING SATISFYING
FEEDBACK EXPERIENCES
The University of Hong Kong

14. Satisfying feedback experiences
Making feedback satisfying for teachers is
just as important as making it worthwhile for
students (Boud & Molloy, 2013a, p. 5)
The University of Hong Kong

15. Feedback does double duty
Competing audiences & functions of teacher
comments:
- Justifying grade
- Offering advice
- Specific vs generic comments
- Quality assurance dimensions
(Winstone & Carless, 2021)
The University of Hong Kong

16. Feedback as pedagogy
Reclaiming the pedagogic value of feedback
Enhancing future student learning as core
purpose of feedback
The University of Hong Kong

17. What staff might want
For staff, what would represent positive
outcomes of feedback processes?
The University of Hong Kong

18. Staff desired outcomes
Students engaging
Students taking responsibility
Students taking action
High quality learning outcomes
The University of Hong Kong

19. What students might want
For students, what would represent positive
outcomes of feedback processes?
The University of Hong Kong

20. Student desired outcomes
Support to achieve high grades
Feedback they can use
Perceptions of care, respect
Perceptions of fairness
The University of Hong Kong

21. CHALLENGES THAT SHARED
FEEDBACK LITERACY MIGHT
ADDRESS
The University of Hong Kong

22. Key feedback challenges
1. Timing & sequencing of feedback
2. Emotional side of feedback reactions
3. Feedback as teacher transmission
The University of Hong Kong

23. 1. Feedback sequences
Task 1  feedback  interlinked task 2
Position students as feedback users
The University of Hong Kong

24. 2. NSS Crisis? What crisis?
Scores for assessment & feedback lower
than other aspects
BUT
Emotional reactions to assessment results &
feedback probably a factor
(Buckley, 2020)
The University of Hong Kong

25. 3. Feedback designs
Shift from provision of comments to design
of learning environments
(Boud & Molloy, 2013b)
The University of Hong Kong

26. Paradigm shift
From teachers delivering comments
To what learners do: self-generated
feedback; using inputs
The University of Hong Kong

27. Comments  uptake
The University of Hong Kong
Teachers produce comments
Focus on delivery
Students generate insights
Focus on uptake
(Carless, 2015; Winstone & Carless, 2019)

28. Thoughts so far …?
Questions, comments?
The University of Hong Kong

29. SHARED FEEDBACK LITERACY
The University of Hong Kong

30. Defining student feedback literacy
Understandings, capacities & dispositions
needed to use feedback for improvement
(Carless & Boud, 2018).
The University of Hong Kong

31. Student feedback literacy
The University of Hong Kong
Making
Judgments
Valuing
Feedback
Working
with
Emotions
Taking Action
(Carless & Boud, 2018)

32. Socio-cultural theories
Meaning-making of feedback is mediated via
activity within social and cultural contexts
(Esterhazy & Damşa, 2019)
Learning situated within activity, context &
culture
The University of Hong Kong

33. Defining teacher feedback literacy
“Knowledge, expertise & dispositions to
design feedback processes in ways which
enable student uptake of feedback”
(Carless & Winstone, 2020, p. 4)
The University of Hong Kong

34. (Carless & Winstone, 2020)

35. Towards virtuous cycles
Teachers designing potentially positive
feedback experiences for students
Students engaging and acting
The University of Hong Kong

36. IMPLICATIONS
The University of Hong Kong

37. Theory vs practice
The literature has moved forward in how it
understands feedback – but not clear those
involved in feedback have been brought
along with it (Dawson et al. 2019)
The University of Hong Kong

38. Implementing peer feedback
• Selling benefits
• Scaffolding, modelling & coaching
• Trios rather than pairs (multiple reviews)
• Opportunities for dialogue then revision
The University of Hong Kong

39. Leveraging comparisons
Productive learning occurs when learners
compare their own work with that of multiple
other attempts at a similar task (Nicol, 2021)
The University of Hong Kong

40. Teacher role
Design learning environments for students
to generate feedback inputs
The University of Hong Kong

41. Digital interaction
Prioritizing learner generation of insights
Designing for dialogue & uptake
The University of Hong Kong

42. References
Adcroft, A. (2011). The mythology of feedback. Higher Education Research and Development, 30(4),
405-419. doi:10.1080/07294360.2010.526096
Boud, D., & Molloy, E. (2013a). Decision-making for feedback. In D. Boud & E. Molloy (Eds.),
Feedback in Higher and Professional Education. London: Routledge.
Boud, D., & Molloy, E. (2013b). Rethinking models of feedback for learning: The challenge of design.
Assessment & Evaluation in Higher Education, 38(6), 698-712.
Buckley, A. (2020). Crisis? What crisis? Interpreting student feedback on assessment. Assessment
and Evaluation in Higher Education, 46(7), 1008-1019. doi:10.1080/02602938.2020.1846015
Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Education, 31(2),
219-233. doi:10.1080/03075070600572132
Carless, D. (2015). Excellence in University Assessment. London: Routledge.
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of
feedback. Assessment & Evaluation in Higher Education.
https://doi.org/10.1080/02602938.2018.1463354.
Carless, D., & Winstone, N. (2020). Teacher feedback literacy and its interplay with student feedback
literacy, Teaching in Higher Education. https://doi.org/10.1080/13562517.2020.1782372
Crook, C., Gross, H., & Dymott, R. (2006). Assessment relationships in higher education: The tension
of process and practice. British Educational Research Journal, 32(1), 95-114.
Dawson, P., et al. (2019). What makes for effective feedback: Staff and student
perspectives. Assessment & Evaluation in Higher Education, 44(1), 25-36.
The University of Hong Kong

43. References (continued)
Esterhazy, R., & Damşa, C. (2019). Unpacking the feedback process: An analysis of undergraduate
students’ interactional meaning-making of feedback comments. Studies in Higher Education,
44(2), 260-274. doi:10.1080/03075079.2017.1359249
Harland, T., Wald, N., & Randhawa, H. (2017). Student peer review: Enhancing formative feedback
with a rebuttal. Assessment & Evaluation in Higher Education ,42(5), 801-811.
Mahoney, P., Macfarlane, S., & Ajjawi, R. (2019). A qualitative synthesis of video feedback in higher
education. Teaching in Higher Education, 24(2), 157-179. doi:10.1080/13562517.2018.1471457
Nicol, D. (2021) The power of internal feedback: Exploiting natural comparison processes.
Assessment & Evaluation in Higher Education, 46(5), 756-778.
https://doi.org/10.1080/02602938.2020.1823314
Reddy, K., Harland, T., Wass, R., & Wald, N. (2020). Student peer review as a process of knowledge
creation through dialogue. Higher Education Research & Development, 40(4), 825-837.
doi:10.1080/07294360.2020.1781797
Winstone, N., & Boud, D. (2020)/. The need to disentangle assessment and feedback in higher
education. Studies in Higher Education. doi:10.1080/03075079.2020.1779687
Winstone, N., & Carless, D. (2019). Designing effective feedback processes in higher education: A
learning-focused approach. London: Routledge.
Winstone, N., & Carless, D. (2021). Who is feedback for? The influence of accountability and quality
assurance agendas on the enactment of feedback processes. Assessment in Education, 28(3),
261-278. doi:10.1080/0969594X.2021.1926221
Wood, J. (2021). Making peer feedback work: The contribution of technology-mediated dialogic peer
feedback to feedback uptake and literacy. Assessment & Evaluation in Higher Education.
doi:10.1080/02602938.2021.1914544
The University of Hong Kong

44. THANK YOU
The University of Hong Kong

45. Mismatched views
Teachers think their feedback is more useful
than their students do (Carless, 2006)
Teachers anticipate more active roles than
students generally willing to commit
(Adcroft, 2011)
The University of Hong Kong

46. Teacher workload
Generation of information to students about
their work is time-consuming. Not justified if
no explicit expectation that it will be used.
(Boud & Molloy, 2013a, p. 206).
The University of Hong Kong

47. Student frustrations
Feedback often seems like a perversely belated
revelation of things that should have been made
clear earlier (Crook, Gross & Dymott, 2006)
The University of Hong Kong

48. Disentangling assessment & feedback
Distinguishing ‘grading’ from ‘feedback’
End of module – mainly summative
vs
During module – potential for students to act
on guidance
(see Winstone & Boud, 2020)
The University of Hong Kong

49. Shifts in priorities
Carless (2015) p. 240
The University of Hong Kong
Increase Decrease
Guidance within the taught
curriculum
Unidirectional comments after
completion of module
Written feedback comments on
first assessment task of module
Written feedback comments on
final task of module
Feedback for first year students Feedback for final year students

52. The University of Hong Kong
DISCOUNT CODE:
FLR40

53. Video feedback synthesis
Lack of opportunities for dialogue or
response to teacher video feedback inputs
Need for active student roles
(Mahoney et al. 2019)
The University of Hong Kong

54. The University of Hong Kong