The document discusses assessment practices and values in the era of generative AI. It proposes reducing assessment overload by streamlining assessments, cultivating a culture of productive feedback, and partnering with students in using AI. Interactive oral assessments and co-designing assessments with students are presented as possibilities. Developing student competencies for engaging critically with generative AI is also highlighted. The overall aim is to enact assessment approaches aligned with core values of learning-orientation, inclusion, and well-being.

1. Assessment values & their
enactment in the generative
AI era
David Carless
University of Hong Kong
Griffith University
November 9, 2023
The University of Hong Kong

2. Competing assessment functions
The University of Hong Kong
Judging student
achievement
Satisfying
accountability
needs
Stimulating
productive
student
learning

3. Core assessment values
Learning-oriented assessment
Cultivating a productive feedback culture
Partnering with students
Virtues: trust, care, respect, honesty
The University of Hong Kong

4. Supporting values
Inclusivity (e.g. Nieminen, 2022)
Choice & flexibility (e.g. Rideout, 2018)
Authenticity (e.g. McArthur, 2023)
Well-being of students & staff
(Jones et al. 2021)
The University of Hong Kong

5. REDUCE ASSESSMENT
OVERLOAD
The University of Hong Kong

6. Enabling reform
Students (& staff, esp sessional staff)
struggle with assessment overload
The University of Hong Kong

7. The assessment arms-race
Teachers competing for student attention
with grades as control & reward (Harland et
al. 2014; Harland & Wald, 2021)
The University of Hong Kong

8. Less can be more
Streamlining assessment is not lowering
standards & aids inclusivity/retention
The greatest enemy of understanding is
content coverage (Howard Gardner)
The University of Hong Kong

9. ASSESSMENT RE-DESIGNS:
SOME POSSIBILITIES
The University of Hong Kong

10. Assessment re-designs
Process as well as product
Assessment co-design with students
Interactive oral assessment
The University of Hong Kong

11. Process & product
Learning as developmental process over time
with iterative cycles of drafting & re-drafting
Digital traces e.g. Google Drive ‘version history’
(Sayers, 2023)
The University of Hong Kong

12. Co-design of assessment
Students as co-creators in learning together
Celebrating human diversity (Nieminen, 2022)
Inclusive range of students in co-design
The University of Hong Kong

13. Critical use of GenAI
GenAI integrated within assessment design
Students critiquing & improving on GenAI
Reflections tracing what
was done/learnt
The University of Hong Kong

14. Programmatic assessment
Cumulative & coherent program-wide
assessment & feedback strategy
The University of Hong Kong

15. INTERACTIVE ORAL
ASSESSMENT (IOA)
The University of Hong Kong

16. Abstract for
Sotiriadou, P., Logan, D., Daly,
A. & Guest, R. (2020). The
role of authentic assessment
to preserve academic integrity
and promote skill development
and employability. Studies in
Higher Education 45(11),
2132-2148.
The University of Hong Kong

17. Key Proposition
Make IOA a more central part of
assessment designs
The University of Hong Kong

18. Pandemic as catalyst
IOA instead of F2F exams
Differentiates students’ capacities
Teachers can adjust to students’ levels
The University of Hong Kong

19. Rationales
- Oral communication for life & workplace
- More personal & motivating
- Higher-order thinking & increased student
accountability (Burrows et al. 2021)
The University of Hong Kong

20. Video presentations or podcasts
Students record a short (3 minute?) oral
presentation and upload for assessment
E.g. Talk about a course topic that has
personal meaning for you & explain why
The University of Hong Kong

21. Vlogs as assessment
Short personal responses
Developing ideas, constructing arguments
Offering choice
(e.g. van den Berg & de Villiers, 2021)
The University of Hong Kong

22. Tackling challenge of scale
Concise presentations
Efficient moderation
IOA at critical points in the program
The University of Hong Kong

23. TOWARDS CULTIVATING A
PRODUCTIVE GRIFFITH
FEEDBACK CULTURE
The University of Hong Kong

24. Designing for feedback uptake
Assessment and feedback designs are
firmly interlinked
(After Boud & Molloy, 2013)
The University of Hong Kong

25. Productive feedback cultures
Relational climates where the seeking,
giving, receiving and acting upon honest, but
respectful, feedback is part of everyday
practices
The University of Hong Kong

26. Enablers for change
Willpower, determination, leadership
Shared vision between co-participants
Strategies to implement the vision
The University of Hong Kong

27. Strategies for a Griffith feedback culture
1. Building habits of feedback seeking & use
2. Teachers modelling acceptance and action on
student feedback on teaching
3. Teachers and students talking about feedback
as a concept (cf. Matthews et al. 2023)
…….
The University of Hong Kong

28. Feedback literacy
Teacher and student shared capacities to
seek, generate and use feedback
(Carless & Boud, 2018; Carless & Winstone,
2023)
The University of Hong Kong

29. Activating students
What do you think?
The University of Hong Kong

30. Is this the best you can do?
The University of Hong Kong

31. GenAI as a potential
feedback source for
student use
The University of Hong Kong

32. GenAI feedback literacies
What are students’ competencies
(automated feedback literacies) for
productive use of generative AI?
The University of Hong Kong

33. GenAI competencies (draft)
Prompt engineering D-I-A-L-O-G-U-E
Verification, judgment
Technical expertise e.g. how LLMs work
Learning from others in use of genAI
The University of Hong Kong

34. Conclusions
The University of Hong Kong

35. Concluding summary
+ reduce assessment overload
+ cultivate feedback cultures
+ build trust
+ partner with students in using AI
The University of Hong Kong

36. Ongoing aspiration
Useful assessment adjustments that
are congruent with the proposed values
&
Cater for the realities of the GenAI era
The University of Hong Kong

37. References
Boud, D. & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment &
Evaluation in Higher Education, 38(6), 698-712.
Burrows, N.L., Ouellet, J., Joji, J., & Man, J. (2021). Alternative assessment to lab reports: A phenomenology study of
undergraduate biochemistry students’ perceptions of interview assessment. Journal of Chemical Education, 98(5),
1518-1528. https://doi.org/10.1021/acs.jchemed.1c00150
Carless, D. (2009) Trust, distrust and their impact on assessment reform, Assessment and Evaluation in Higher
Education. 34, 1, 79-89.
Carless, D. (2015). Exploring learning-oriented assessment processes. Higher Education, 69(6), 963-976.
Carless, D. and Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback.
Assessment and Evaluation in Higher Education, 43(8), 1315-1325.
https://doi.org/10.1080/02602938.2018.1463354.
Carless, D., & Winstone, N. (2023). Teacher feedback literacy and its interplay with student feedback literacy. Teaching
in Higher Education, 28(1), 150-163. https://doi.org/10.1080/13562517.2020.1782372
Harland, T. & Wald, N. (2021) The assessment arms race and the evolution of a university’s assessment
practices, Assessment & Evaluation in Higher Education, 46:1, 105-117, DOI: 10.1080/02602938.2020.1745753
HEPI (2023). How are HE leaders responding to generative AI? https://www.hepi.ac.uk/2023/05/09/how-are-he-leaders-
responding-to-generative-ai/
Jones, E. Priestley, M., Brewster, L., Wilbraham, S., Hughes, G. & Spanner, L. (2021) Student wellbeing and
assessment in higher education: the balancing act, Assessment & Evaluation in Higher Education, 46:3, 438-
450, DOI: 10.1080/02602938.2020.1782344
Kramm, N. & McKenna, S. (2023). AI amplifies the tough question: What is higher education really for? Teaching in
Higher Education
The University of Hong Kong

38. References (continued)
Lodge, J. (2023). Assessing learning processes instead of artefacts won’t be easy.
https://www.linkedin.com/pulse/assessing-learning-processes-instead-artefacts-wont-easy-lodge/
Lodge, J., Thompson, K. & Corrin, L. (2023) Mapping out a research agenda for generative AI in tertiary education,
AJET, 39(1), 1-8. https://ajet.org.au/index.php/AJET/article/view/8695
Matthews, K., Sherwood, C., Enright E., & A. Cook-Sather (2023) What do students and teachers talk about when they
talk together about feedback and assessment? Expanding notions of feedback literacy through pedagogical
partnership, Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2023.2170977
Matthews, K., Tai, J., Enright, E., Carless, D., Rafferty, C. & Winstone, N. (2021). Transgressing the boundaries of
‘students as partners’ and ‘feedback’ discourse communities to advance democratic education. Teaching in Higher
Education https://doi.org/10.1080/13562517.2021.1903854
McArthur, J. (2023). Rethinking authentic assessment: Work, well-being, and society. Higher Education. 85, 85–101
doi:10.1007/s10734-022-00822-y
Nieminen, J. (2022). Assessment for Inclusion: Rethinking inclusive assessment in higher education. Teaching in
Higher Education
Nikolic, S. et al. (2023). ChatGPT versus engineering education assessment. European Journal of Engineering
Education, https://doi.org/10.1080/03043797.2023.2213169
Rideout, C. (2018). Students’ choices and achievement in large undergraduate classes using a novel flexible
assessment approach, Assessment & Evaluation in Higher Education
Sayers, D. (2023). A simple hack to ChatGPT-proof assignments using Google Drive.
https://www.timeshighereducation.com/campus/simple-hack-chatgptproof-assignments-using-google-drive
Sotiriadou, P., Logan, D., Daly, A. & Guest, R. (2020). The role of authentic assessment to preserve academic integrity
and promote skill development and employability. Studies in Higher Education 45(11), 2132-2148.
van den Berg, L. & de Villiers, J. (2021). Development of a conceptual framework to enhance sport students’
communication skills and content learning through vlogs as an assessment tool. Cogent Education
The University of Hong Kong

39. QUESTIONS
&
COMMENTS
The University of Hong Kong

40. SUPPLEMENTARY THOUGHTS
(Appendices)
The University of Hong Kong

41. Integrity is important but …
“Focusing on catching cheating is misplaced
effort”
Sir Tim O’Shea
Assessment for credentialing more than
learning overemphasizes assessment
security (Kramm & McKenna, 2023)
The University of Hong Kong

42. What leadership is needed?
Agile learning for unknown futures
Thoughtful & reflective leadership
Humility & flexibility
The University of Hong Kong

43. Need to build trust
How is trust developed across an institution?
Partnerships of mutual respect
The University of Hong Kong

44. Criticality
“One of the strengths of ChatGPT is that you
don’t know if what it’s telling you is true. We
can use ChatGPT to enable students to
think critically.” Tim O’Shea, HEPI blog
Ethics (e.g. Fawns & Nieminen, 2023)
The University of Hong Kong

45. Linkages & personalization
Links to real-life experiences e.g. events in
your context or discipline
Linking assignments to course readings
Personalizing assignments e.g. relate to
discussions in class
The University of Hong Kong

46. Staged assessment designs
Scaffolded series of tasks focused on students’
thinking processes in developing artefacts
(Lodge et al. 2023)
The University of Hong Kong
Week 4 Week 7 Week 9
Task 1 Task 2 Task 3

47. Staged assessment example
Stage 1: abstract or elevator pitch
Stage 2: annotated bibliography
Stage 3: draft for peer review & AI review
Stage 4: revise & submit
The University of Hong Kong

48. Barriers to reducing assessment
Reluctance to relinquish power
Covering and assessing content
The University of Hong Kong

49. Critical use of GenAI
Pairs or trios working with GenAI
What was done & why: + oral or written
reflections
The University of Hong Kong

50. Oral assessment rationales
Links to professional communication skills suggest
authenticity in oral presentations (Doherty et al.
2011).
Students perceive oral assessment as more
personal & leading to better learning outcomes
(Joughin, 2007).
The University of Hong Kong

51. Oral assessment & GenAI
Oral assessment should play a larger role in
post GenAI assessment (Nikolic et al. 2023)
Derived from ChatGPT performance on
engineering tasks (Nikolic et al. 2023)
The University of Hong Kong

53. Alternative assessment & exams
Exams have a long history but are they still
fit for purpose?
“I don’t want to memorize for an exam: I’ve
spent 15 years doing that in school”.
(Business student, Carless, 2015, p. 125)
The University of Hong Kong

54. The University of Hong Kong

55. Defining automated feedback literacies
Capacities to engage in dialogue with
automated systems, critically evaluate
outputs, and utilize them appropriately to
enhance work, knowledge or thinking.
The University of Hong Kong

56. AI specific competencies
Understanding how LLMs work
Learning from others in use of genAI
Identifying how genAI works well for you
Prompt engineering & probing (cf. dialogue)
The University of Hong Kong

57. Generic HE competencies
Critical thinking & reflection
Evaluative judgment
Growth mindset
Ethical use
The University of Hong Kong

58. Student automated feedback literacies (draft)
1. Appropriate prompts & continuing dialogue …
2. Critical engagement with AI outputs …
3. Co-learning with others …
4. Reflection and self-assessment …
5. Principled follow-up actions …
The University of Hong Kong

59. The University of Hong Kong