Sankey, M. (2022) Strategies for trusting online assessment in the age of artificial intelligence (AI). University of Divinity professional development webinar. 1 July. The necessity for us to rethinking assessment in the light of current trends. We’ve been slowly shifting the goal posts for a number of years now. But why? Academic integrity, cheating vs authentic assessment. Preparing students for the world of work. This shift has partly occurred due to the advent of new technologies and AI. Contemporary technologies have allowed us to re-invigorate different assessment types more common to the past. We will look at some things that were old but are now new again.

1. CRICOS Provider No: 00300K (NT/VIC) 03286A (NSW) RTO Provider No: 0373 TEQSA Provider ID PRV12069
Strategies for trusting online assessment in
the age of artificial intelligence (AI)
Professor Michael Sankey, Education Strategy
Director, Learning Futures and Lead Education Architect
Education Strategy
President, Australasian Council on Open Distance and eLearning (ACODE)
michael_sankey

2. Charles Darwin University acknowledges all
First Nations people across the lands on
which we live and work, and we pay our
respects to Elders both past and present.
2

3. • The necessity for us to rethinking assessment in the light of current trends
• We’ve been slowly shifting the goal posts for a number of years now. But why?
• Academic integrity, cheating vs authentic assessment
• Preparing students for the world of work
• This shift has partly occurred due to the advent of new technologies and AI
• Contemporary technologies have allowed us to re-invigorate different
assessment types more common to the past
• We will look at some things that were old but are now new again
I’m going to cover
3

4. • I am a visual arts teacher, I went to arts college and did well, so well that
I initially became a sessional lecturer, then a lecturer.
• Typically, I teach the way I was taught and most of my students seem to be
doing OK. I typically use a mastery model (stand and deliver) for my
teaching approach in the studio.
• I’ve been asked to do more of my teaching online and I hear from others
that I should be considering ‘constructive alignment’ (whatever that
means).
• If I want to apply to be a Senior Lecturer at some point, I really need to be
conscious and explicit about my teaching/pedagogical approach (whatever
that means), particularly as we move to more blended modes of delivery
Scenario
4

5. • It is the interaction between teachers, students, and the learning
environment and the learning tasks in and around that environment.
• If we say, we’re using a pedagogical approach, we talk about
‘Constructivism’.
• In reality we use a much broader range of approaches, as the need arises.
• We have developed out of necessity a very eclectic approach to pedagogy.
• It’s like having my set of golf clubs. If the hole is 260m, I’ll be using my 2
wood, then I’ll use my 6 iron to get to the green. I then get to use my putter.
• The next hole is only 180m, so I’ll use a different combination of clubs.
• But I always get to use my putter (assessment).
• Similarly, do we teach our 1st year students in the same as our 3rd years?
What is Pedagogy
5

6. 6

7. 7

8. 8

9. So how do we
rate and what
do we have to
support
students and
staff?
9
The Web Content Accessibility Guidelines (WCAG)

10. Productivity tools and behaviours
10

11. https://www.weforum.org/agenda/2021/11/artificial-intelligence-writing-universities-plagiarism
Advances in AI in L&T
11

12. • Writing assistance
• For example, if a piece of writing was 49% written by AI, with the
remaining 51% written by a human, is this considered original work?
• Grammarly, etc. Other cloud-based writing tools with automatic text
generation, extraction, prediction, mining, form-filling, paraphrasing,
translation and transcription, etc.
• Knowledge banks: Help with revision identifying
gaps targeting pre-produced sets of materials
• Tailoring exam Q’s
• We need to be really really interested
in this space before it over runs us
Yes and No
12

13. 13
https://intelligence.weforum.org/topics/a1Gb0000000pTDREA2?tab=publications

14. • Once a learning packages is adopted, this provides an underlying database that informs
an AI, and it can start working in three main ways:
1. It can perform adaptive teaching; it can change its pre-determined route of delivery
according to the input of other data and then react accordingly.
2. It can personalise delivery of material, where the AI tweaks the delivery of the
educational content according to a student’s behaviour.
3. The system can provide instant feedback, where the AI responds immediately to the
student query in a relevant manner.
• Other major developments are in assessments, invigilation and supporting social and
peer interactions, e.g. AI applications such as Pearson’s Intelligent Essay Assessor (IEA)
claims to grade assessments as accurately as a human.
• Some traditional educational institutions (Singapore’s Ngee Ann Polytechnic) are
trialling the use of AI for assessment, invigilation and content creation.
• https://campusmorningmail.com.au/news/ways-ai-will-change-teaching-and-learning/
From learning package to educational intelligence
14

15. 15

16. • There are 1281 sites associated with academic fraud and
contract cheating blocked by CDU.
• 84% of these are provided by TEQSA.
• It’s one thing to say we need to change assessment to be
more authentic, but we need to be able to back that up in
a consistent way.
• That means time (proxy for $’s). Or we don’t rush it.
• Two steps: Teachers need a meta understanding around
how they are teaching, and
• Need assessment mentors to help them.
Cheating vs Authentic Assessment
16

17. 17
3507
1570
853
1173
1698
10
3742
8060
3023
2690
8608
9971
9999
9117
1312
1278
3212
7533
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
1
2
3
4
5
6
7
8
9
10
11
12
Month
2020
2021
2022

18. Unit 1
Effective teaching
Unit 2
Technology enhanced
learning
Unit 3
Curriculum design
Unit 4
SoTL
Set up Portfolio
10%
Blog post
10%
Blog post
10%
Blog post
10%
Essay
30%
Essay
30%
Essay
30%
Essay + Prez
30%
Report
30%
Essay
30%
Prez
30%
Essay
30%
Report
30%
Posts
30%
Essay
30%
Teaching Plan
30%
Essays and Quiz’s eazy peazy for AI
18

19. AI doesn’t always get it right
19

20. Instead, can we please
think about:
Human voice
Images
Collaboration
Peer-review
Poster/infographic
ePortfolio/reflection
Work Integrated Learning
20

21. https://ro.uow.edu.au/jutlp/vol19/iss2/02/
21

22. Create an online social media advertisement
on the topic you’re learning
22
• Most students today use some form of social media
platform and are familiar with seeing ads being put in in
front of them.
• By students creating an image speaking to a particular topic
they can post this into a safe institutional collaboration tool;
Microsoft Teams or Yammer, a closed Facebook site, or an
ePortfolio platform, where other students can view it, ‘like’
it and comment on it.
• It could be put in Voice Thread with the students giving a
verbal explanation of why they have chosen to do this in a
particular way and what they were trying to convey.

23. • This can easily be done in OneNote or Padlet or even on a shared document
on Google.
• Students can do this in smaller groups or individually, at the same time or
over a set period.
• It is like pasting sticky notes on the wall in the classroom, but online.
• The key here is that there will be a synthesis of the ideas at some point,
again either done individually or by the group.
• This is then presented as the outcome of the brainstorming activity and
students can reflect on this.
Online brainstorming using sticky notes a/synchronous
23

24. Padlet
24

25. • This could be done synchronously or asynchronously. If live you would use
Zoom, Teams or Collaborate to have students present their ideas.
• If a recording is required, they could do this on their phones and post the
recording either into the LMS, Teams, or Steaming Media Platform.
• Voice Thread is also a good tools for this.
• The trick here is to ask other students to ask questions as though they are
the novice to try and tease out un-explored concepts.
• It a bit of a role play which adds an element of fun to this activity.
Ask students to do a description of a process, as though
presenting to a novice
25

26. • Infographics are all the rage now and students are exposed to these in all
walks of life. The trick here is to get them to precis their ideas and to bring
them back to the core constructs.
• Again, this can be accompanied by a description, either in writing or as an
audio explanation. This could be simply created in PowerPoint or Word, or a
more sophisticated tool, but the tool is not the point, it’s about how they
represent their ideas.
• This can be posted onto a forum, put on Voice Thread, hosted in Teams, or
presented live in a Zoom or Teams meeting. They could prerecord the
explanation also and post this with the visual.
• This would also make for a good peer-review activity in something like
Feedback Fruits or PebblePad.
Create a chart, mind map, infographic, or diagram of a
concept
26

27. 27
https://instructionaldesign.com.au/infographics/

28. • Pen pals may not be the common now, but the point is, we are getting the
student to summarise their learning for the week as though they were explaining
this to an old friend.
• Alternatively, they could create a 5-10-minute audio explanation as though they
were explaining it on the phone. They could record it on their phone.
• Initially, until they get the idea, this could be set us as a scaffolded scenario where
they are given some guidelines as to how much they should cover, or provide an
example so they can see what is required.
• Really there are many tools that could be used for this. It could be a blog or
journal page in an ePortfolio tool, written in Word and posted as an assignment.
• But in this case, I would not make it a shared document with other students as
this could be seen as a bit threatening by some.
Write a letter or email to a friend about what you
learned this week
28

29. • Asking student to act out, through something like a play (written), where
actors could be used to play out a scenario around a given topic being
studied. Think Snap Chat, or an Instagram Story (short and to the point).
• The art of creating a dialog from a concept gets them to see a topic from
different angles, putting on different shoes, as it were.
• A rhyming or acrostic poem may also get them to process information a wee
bit differently to what they normally do. Again, this could be done in an
ePortfolio tool as a blog or journal.
• If it is designed as a play, a group of students could even play this out in
Zoom or Teams. It could also be recorded separately and placed online.
Write a poem, play, or dialogue about the topic of the
week
29

30. 30
https://www.weareteachers.com/poems-about-teaching/

31. • Role play, students pretend to be somebody they are aspiring to be. This
provides valuable meta cognitive insights into their aspirations.
• Many board meeting are now held online and board members have to
present their ideas to their colleagues.
• Papers, memos, etc., can be provided ahead of time for others to read, the
person presenting does not have to rehearse all the concepts in the paper.
• Students post their work into a Team channel or forum set up for this.
• Ideally students take on different roles on the board. One might be the CEO,
another the chief finance person, another the CIO.
• Each one need to see what is being presented through that lens.
Create a policy memo or an executive summary for a
Board Meeting
31

32. 32

33. 33

34. • This can be done by creating an open text field after a question.
• Some systems allow for an audio response.
• Typically done in the LMS, could be done in Voice Thread or media
streaming platform like Kaltura, Panopto or ECO360.
• This is where the use of AI can work for you, as long-form text answers can
look for common words or strings.
• But AI engines need to be trained, so this would not be used in the first
instance, but could be used in subsequent iterations.
Give an explanation for a multiple-choice question
34

35. 35

36. https://michaelsankey.com/2022/06/30/some-authentic-assessment-activities-for-technology-
enhanced-learning-part-1/
36

37. 37

38. UDL
38

39. 39
https://towardsdatascience.com/the-old-saying-two-heads-are-better-than-one-4c7d133e16a4
https://www.copperbottom.net/blog/2016/11/7/why-two-heads-are-better-than-one
https://www.teachwire.net/news/two-heads-are-better-than-one-why-teachers-should-talk-to-their-tas
• Two heads are better than one
• Education designers are a great source of inspiration
• Your peers also are a great resource
• Partnership pedagogy (students)
• Be open to new ideas

40. “We are now realising that what was conceived as being
good online learning is being challenged by some of the
newer more student-centred approaches to learning and
teaching. Not the least of these being due to the
technologies that have evolved to allow us to be way more
collaborative.” (p.22)
Sankey 2022
Final word
40
https://ro.uow.edu.au/jutlp/vol19/iss2/02/
michael_sankey