Barangay Council for the Protection of Children (BCPC) Orientation.pptx
Untubing AI in Assessment: A Primer for Future’s Sake
1. The Mini-Summit: AI in Assessment
Professor Mark Brown
Institute of Education
Dublin City University
26th September 2023
2. Two Truths and a Lie
I’m no expert but I have a growing
research interest in AI in education
I have a strong track record of research
on digital forms of Assessment
I haven’t taught in the schooling
sector for over 25 years
6. A few health warnings
Learning the art of brushing both ways
Untubing AI in Assessment:
Developing positive habits for healthy minds
7. Key messages
1. Avoid the trap of AI-centric thinking.
2. Proactively own, shape, and influence the narrative regarding
the assessment challenges that AI is attempting to solve.
3. Recognise that all was not well with Assessment (and
Feedback) practices before the emergence of Generative
Artificial Intelligence.
Educators need to…
8. • Training memories
• Limited relevance
• Lack of challenge
• Promoting transfer
• Academic integrity
• Culture of banking
• Playing the game
A
c
q
u
i
s
i
t
i
o
n
P
a
r
t
i
c
i
p
a
t
i
o
n
Knowledge Creation
10. 1. A few health warnings
• 92% of OER content in English
• Only 10 of 7,000 languages have
been used by GenAI
• Latest GenAI tools are less prone to
fake knowledge
• AI generated text can’t always be
easily detected
• Adoption of authentic assessment is
not AI proof
• Bad for the environment with
ChatGPT-3 generating 85,000kg of
CO2 emissions
• GenAI is not freely accessible
• ChatGPT is not just a big database
• ChatGPT tends to pose itself falsely
as a real person
• Few of us understand the ‘black box’
and can keep up with developments
• There are serious biases,
discrimination, privacy and ethical
issues
• Risk of propagating inequity and
marginalising those with already
marginalised voices
11. 2. Learning the art of brushing both ways
https://unsplash.com/photos/IW1uAv88Z0o
12. 2. Learning the art of brushing both ways
‘Design In’ ‘Design Out’
• Students can use GenAI tools in their
assessments
• Clear guidelines must be provided to
students (and parents and
caregivers)
• Potential biases, discrimination and
ethical issues have been considered
as well as they can be
• Students should NOT use GenAI
tools in their assessments
• Some assessments may need to be
redesigned to minimize their
vulnerability
• Students should not be asked to
undertake things that can be easily
done with GenAI tools
15. • In the face of AI, should we try to defend our existing Assessment
practices?
Straw poll…
• Is it possible to avoid spillage across ‘design in’ and ‘design out’
approaches, especially given AI’s ubiquity?
• To what extent do we risk a potential loss of confidence
locally and nationally in existing assessment practices if
we fail to appropriately respond to AI?
17. • From onerous to feasible
• From discrete to continuous
• From uniform to adaptive
• From inauthentic to authentic
• From antiquated to modern
What are the assessment
problems we face?
https://doi.org/10.1016/j.caeai.2022.100075
18. Henderson (2023) asks,
“What labours are involved that can be offloaded or co-constructed with AI?”
https://www2.ed.gov/documents/ai-report/ai-report.pdf
19. “If our educators’ evaluative expertise through the application of
the assessment criteria and standards cannot distinguish
between raw AI outputs compared with what students bring to
the task (with or without those AI) – then we have a problem
with our assessment designs – not with AI.
(Professor Michael Henderson, 2023)
“WHY SHOULD WE ASSESS SOMETHING THAT
AI CAN DO JUST AS WELL OR BETTER?”
https://der.monash.edu/defending-assessment-from-ai
20. Designing assessment for an AI world…
• Re-visiting what we are trying to assess
• Emphasising the process - product is a poor proxy of learning
• Finding ways of assessing the uniqueness of human processing
• Empowering learners to own the assessment process
• Focusing more on creative assessment designs – that elicit
evidence of learning that matters for ‘a life worth living’
22. Augmenting
Rather than Automating
• Learning to ask the right questions
• Learning how to make good use of
a devil’s advocate
• Learning to make judgments of
what counts as quality
• Learning that effortlessness is not
always the goal to strive for
• Learning to find the light through
the gaps/cracks
https://unesdoc.unesco.org/ark:/48223/pf0000385146
23. 3. Developing positive habits for healthy minds
Photo by Jelleke Vanooteghem on Unsplash
24. 3. Developing positive habits for healthy minds
https://lx.uts.edu.au/collections/artificial-intelligence-in-learning-and-teaching/resources/five-principles-for-effective-ethical-use-generative-ai/
Five principles…
28. Actions…
• Shaping regulations and policies
• Critical review of Assessment practices
• Actively engaging in communities of practice
• Supporting teachers’ professional development
• Developing, implementing and researching pilot initiatives
How?
32. Two Truths and a Lie
I’m no expert in AI but I have a
growing research interest in the area
I have a strong track record of research
on digital forms of Assessment
I haven’t taught in the schooling
sector for over 25 years