We present the results of a small case study in which we developed and tested a set of spreadsheets as a 'do-it-yourself' e-examination delivery and marking environment. A trial was conducted in a first year university level class during 2017 at Monash University, Australia. The approach enabled automatic marking for selected response questions and semi-automatic marking for short text responses. The system did not require a network or servers to operate therefore minimising the reliance on complex infrastructure. We paid particular attention to the integrity of the assessment process by ensuring separation of the answer key from the response composition environment. Students undertook a practice session followed by an invigilated exam. Student's perceptions of the process were collected using pre-post surveys (n = 16) comprising qualitative comments and Likert items. The data revealed that students were satisfied with the process (4 or above on 5 point scales). Comments revealed that their experience was in part influenced by their level of computer literacy with respect to enabling skills in the subject domain. Overall the approach was found to be successful with all students successfully completing the e-exam and administrative efficiencies realised in terms of marking time saved.

1. Do-it-yourself
e-Exams
Mathew Hillier
Monash University
Scott Grant
Monash University
ASCILITE Conference 25-28 Nov 2018 – Geelong, Australia
TRANSFORMINGEXAMS.COM
A Scalable Examination Platform for BYOD Invigilated Assessment

2. A key motivation - The gap
Real world of work Exams
2
World Economic Forum – How will digital change your working world.
https://agenda.weforum.org/wp-content/uploads/rtr2m8vm1-628x330.jpg
Exams at Monash Caufield in 2015 (mathew.hilier[at]monash.edu)
70,000 student university.
We are faced with a growing disconnect between
the way high stakes testing is conducted using pen
on paper exams and students’ everyday experiences
of study, work and life, let alone the future!
4IR
21C

3. A key idea …
SAMR Theory - Puentedura, R. (2012). Thinking About Change in Learning and Technology. Presentation
given September 25, 2012 at the 1st Global Mobile Learning Conference, Al Ain, UAE.
http://www.hippasus.com/rrpweblog/archives/2012/04/10/iPad_Intro.pdf

4. Study context – broader e-Exam project
Start > > > > > > > > > > > > > > Future >
Get Ready Phase 1 Phase 2 Phase 3 Phase 4 Phase 5
Institutional
approvals,
research
ethics,
hardware and
infrastructure
Paper
equivalent
small scale.
Post-paper
small to
medium.
Medium to
large scale.
Whitelisted
and logged
Internet
Open but fully
logged
Internet
Basic doc
exams to
begin!
Expanding the
app and media
landscape.
Adding the
power of an
LMS.
Network
BYOD exam.
Network
mixed mode
BYOD exam.
Crawling Walking Running Jumping Flying!
4
http://ta.vu/e-exam-roadmap
Extension work:
An offline e-learning platform
see moleap.org
The project has
progressed!
Moodle resistant to
network outages!
This paper!

5. Pedagogical Affordances
PDF
Scratch SDK
Exam doc Video
LMS quiz
Specialist applications
Sims
Authentic assessments. It is a platform, not an app!

6. Research Questions for spreadsheet pilot (phase 2.5)
Could a spreadsheet be used as a small-scale, do-it-yourself exam delivery system?
In particular, guided by literature, could we:
• Utilise standard and common spreadsheet software (leverage existing tools with
minimal barriers to use by teachers and students)
• Enable automated or semi-automated marking of selected response style questions
and short text responses without manual data entry (to realise work efficiencies of
technology and minimise errors)
• Not rely on a live network during the exam (to minimise the chance of disruption)
• Be hack proof - the student portion of the system must not to have answers ‘hidden’
within it.
Would students accept this approach?
In particular, ease of use, perceive it to be reliable and secure.

7. Doing it…
Room set up:
 Paper 'e-Exam quick start guide' and post-exam survey.
 A power socket was provided for each typist.
 Hand-writers were given a paper copy of the exam questions and response booklets. These
were available to typists upon request.
Both typists and hand-writers sat in the same room.
1) Students enter the room and seated.
2) Typists were given an e-Exam USB stick containing the questions.
3) Students start their computer with a USB stick progressing to the e-Exam desktop. A desktop
background image provides a visual check that all have booted from the correct USB.
4) Students enter their student ID and name into the e-Exam Starter screen.
5) All students now wait.
6) Invigilator announces the start of the exam and students click the ‘Start Exam’ button.
7) Auto-save occurs every 2 minutes.
8) At the end of the exam, the student uses File>Save one last time and Exit the Office suite.
9) When done they shut down the computer.
10) Students return the USB sticks.
11) Students complete the post-exam survey before leaving the room.

8. e-Exam workflow using spreadsheets (offline mode)
Post-exam: assessment
8.
Responses
retrieved
from USBs.
2. Create master USB
(load exam, configure, test)
Pre-exam: prepare exam
4. USBs sent to
exam venue
OR
USBs can be
recycled next
exam.
Linux Live
USB.
Libre Office
Spreadsheet
5. Pre-session:
Student laptop
setup &
practice.
3. USBs duplicated
1. Teacher creates spreadsheets
(exam and matching gradebook)
7. Collect
USBs
10. Marking
undertaken.
Checked by teacher.
9.
Responses
collated with
RDBmerge
6. Exam venue:
a. Students enter room
b.Given USB
c. Start laptop from USB
d.Do exam
e. Finalise and shutdown
f. Return USB
g. Leave room
11.
Return
feedback
to student.
NOTE!!!
NEW!!!Our
robust online
Moodle version
halves USB
handselling

9. Start up
Select
LOTE
Enter student info.

10. Spread sheet
The system then opens the
question and response
document automatically.

11. Language tools Language tools available
according to LOTE selection
Respond in designated cells
(other cells are locked).

12. Participation
Intro to Chinese (first year).
22 students at pre-exam practice.
16 typed the exam.
7 females and 9 males.
Caveat: sample was small and not random -
descriptive of these groups only.

13. Pre and post response trends
Strongly Disagree 1 2 3 4 5 Strongly Agree
Caveat: Small, not random samples – descriptive of these groups only.
1 2 3 4 5
Written instructions were easy to follow
It was easy to start my computer using the e-Exam USB stick
I can use the e-Exam system just as well as my own laptop system
It was easy to use the office suite (word processor/spread sheet)
It was easy to answer multiple-choice questions in the e-Exam…
Overall, I feel the e-Exam System is easy to use
I feel the e-Exam System is reliable against technical failures
I feel the e-Exam System is secure against cheating
I now feel relaxed about using the e-Exam system for my exam
I would recommend the e-Exam System to others
I would like to use a computer for exams in the future
Pre Post
Sign Test
n/s
0.03
n/s
n/s
n/s
n/s
n/s
n/s
n/s
n/s
n/s

14. Study Process
1) Two weeks prior: practice session + pre-survey.
2) Exam day: In-class, graded, supervised assessment task + post survey.

15. Summary - Key Findings
a) The offline Spreadsheet based e-Exam was rated well by the typists:
4+ out of 5.
b) The spreadsheet worked - each student completed successfully.
Responses were auto saved to USB each two minutes, retrieved following exam and
processed using a second ‘gradebook’ spreadsheet.
c) Marking was more efficient due to some auto marking and readable responses.
Possible next steps for spreadsheet version…
i) Trials in units (subjects) that would best take advantage of a spreadsheet
For example: accounting, finance, maths, statistics, science.
ii) Further development to streamline the workflow with respect to setup and
retrieval and marking.
iii) However project focus has shifted to the now proven ‘Robust’ online Moodle
approach!

16. Thank you
For further information contact:
Mathew.Hillier@gmail.com
TransformingExams.com
Presentations at ASCILITE2018:
1. Do-it-yourself e-Exams [8] Mon 11:30am D2.212
2. Towards authentic e-exams at scale – Robust networked Moodle [51]
Following this session! Same Room! Mon 12noon D2.212
3. Integrated mixed reality spatial learning analytics into secure
electronic exams [183]. Tue 11:20am D2.212