A CAUDIT Webinar investigating the findings of the ACODE sector scan on online proctoring tools being used in Australasia for online exams. It looks at the issues risks and affordances

1. CAUDIT Webinar | ACODE
Online Proctoring
Professor Michael Sankey
Director, Learning Transformations
President, Australasian Council on Open,
Distance and eLearning (ACODE)

2. Overview
• Fortunately, we all have pretty robust LMS’s that double as online classrooms
• The one where people put up PDFs and PowerPoints and call it online learning
• The last 10 years have seen quite an improvement in how we use these spaces
• Despite this we have 1000’s of staff in the
sector that engage very little in ‘teaching’
online, as distinct from supporting teaching
in an online space
• Last 3 w’ks we have had 100’s of staff trained
https://arthistory.umd.edu/eventinfo/collaboratory-presents-online-teaching-best-practices-and-how-tos
michael_sankey

3. ACODE sector scan Oct/Nov
• 27 Uni’s
• 14 tools examined in trial (most) or in production
• Common Issues:
• Infrastructure for scaling up to support BYOD, eg
sufficient power points in rooms
• Networked PC labs only – wireless use for BYOD
has caused problems not yet solved
• Struggling to find system that can support diagrams,
annotation, maths formulae, etc
• Students not having appropriate equipment
• Strong preference for more authentic assessment
michael_sankey
• Inspera
• Safe Exam Browser
• ProctorU
• ExamSoft
• RPNow
• Tao
• Cadmus
• Respondus lockdown
• Exam4
• QMoD
• Home grown tool
• GradeScope
• Examity
• Proctoria

4. michael_sankey

5. Summary
• Over recent weeks there has been quite the fracas on how we are scrambling to get
exams on-line quickly, with various proctoring solutions being hastily looked at.
• Some getting over 100 queries a day from potential new customers.
• Most solutions need a webcam (not a tablet),
a stable internet connection (increasingly
problematic) and to do this in isolation.
• No one size fits all solution, but one thing
we can’t allow is for the technology cart
to be put in front of the pedagogical horse.
michael_sankey

6. Summary continued
• We know proctoring solutions work; we’ve been trialling them for a while
• But not run-out at scale (except of UNE). And their students are well informed
that this is what they are signing-up for.
• Not so our on-campus student some of whom do not have a computer with a
webcam or a stable internet connection (just watch the news).
• They are already stressed out, now we are asking them to participate in a largely
foreign form of assessment (an on-line proctored exam).
• Sure, there is professional accreditations at stake, but only a minority
• So, we can look to provide alternative assessment opportunities for our students.
• Yes, it will take a bit more time but our students are the ones we need to be really
concerned about here.
michael_sankey

8. One ACODE Rep wrote
• Don't. Just don't. We are in the midst of a global pandemic the like of which we have
never seen. People are dying, people are seriously ill. As academics and employees of
universities we are extremely privileged. We will nearly all have a safe place to live, a
roof over our heads, a space to work, a continuing income and food to eat. The same is
not true for many students. Some are still in a flat with 6 others all trying to use the
same internet to watch hours of recorded lectures. Some are back home, maybe
looking after siblings while parents do essential work, maybe back in a small bedroom
trying to organise study. Some will end up dealing with family or friends falling ill and
possibly dying. At 18, 19, 20 years old. I could go on. I shouldn't need to.
michael_sankey
• A survey 12 days ago in a computing science school found that more than 5% of
students were unable to successfully participate in online activities with the access
they had at home.

9. From the Uni that does a lot of this
Hi Michael, Our experience at UNE over the last 2-3 years of our Online Exams project
can be summed up with just one thing - be very careful not to treat the
implementation of online exams just as a technology procurement project. It is
primarily a communications and change management exercise. This isn't just about
"buying the car", rather staff and particularly students must "learn to drive it". If they
aren't given opportunities to do that, the best remote proctoring service in the world
won't avoid the conflation of exam anxiety with 'new technology' anxiety, and you'll
have a group of very stressed and very angry students headed for a very bad
experience. I think it will help the teams implementing online exams hugely if
CTOs/CIOs understand that.
michael_sankey

10. • Consider the students who have a disability and AIP and require accessible
technologies. There may be students who rely on the university provisioning
this technology during exams.
Some other concerns
michael_sankey
• Are there any technologies already available within the digital ecosystem at
the institution that support students with hand-written worked
examples? Students may be required to submit workings, calculations,
proofs or justifications for their answers. If this is required, there will need to
be a process provided for the online submission of the drawing or
handwriting of any diagrams/symbols.

11. • Could you provide a clear list of any technologies likely to be required that will (or
will not) operate in China
• Are there any existing technologies in your stacks (other than online proctoring)
that might be useful as short term fixes for lightly invigilated remote virtual
exams. e.g. are their monitoring and/or screen/desktop observation tools that an
invigilator might use to spot check student activity during the exam conducted via
Zoom or another conferencing technology?
Are there other tools CAUDIT?
michael_sankey

12. • The use of proctoring tools is problematic at scale for both the service
itself and for the university. We recommend caution. As many
universities will be moving to this option over the next few weeks, it
would be unlikely that proctoring solutions will be able to meet demand
globally
Proctoring tools (from ACODE 81)
michael_sankey

13. Exams
• Instead of in person exams, or employing expensive proctoring
software solutions, you could use the quiz tool in the LMS in
conjunction with Zoom (or its equivalent) so tutors/teachers can at least
watch the face of the students undertaking these quizzes. Realistically,
this could be done with classes of up to 16, or if multiple staff were
watching, up to 30 at one time
michael_sankey

14. • Run alternate assessments meeting the same learning outcomes, of
smaller multiple stakes assessments, to meet the same ends. For
example, where they might be a 60% exam these could become three
assessments of 20% each run over successive weeks
Breaking exams up
michael_sankey
• Randomise and tightly time questions in the LMS quiz tool, limiting the
opportunity for students to refer to other students or resources. One
would need to be realistic in the timings and warn students beforehand

15. michael_sankey

16. WIL it work or WIL it not work
• Alternate assessments for work integrated learning (WIL) could consider
freely available simulation labs that can be downloaded and run from
their own sites or through the LMS. Asking students to reflect on the
activities in these simulations could provide an adequate approach in
the short term
michael_sankey

17. Use of more audio based assessments
• All students have mobile devices, where they may not all have laptops
with cameras in them. A possible solution to this is to use the audio
capability of their mobile devices, in relation to assessment. For example,
students could be asked to respond to long-form answers via audio or
video recording that could then be submit through the institutions LMS
michael_sankey

18. Using multiple scenarios
• With many professional bodies relaxing their strict requirements for
proctored exams, take home, or open book exams could be considered. If
this is the case it would be suggested that instead of providing just one
scenario, multiple scenarios could be deployed
michael_sankey

19. michael_sankey