We present the results of a small case study in which we developed and tested a set of spreadsheets as a 'do-it-yourself' e-examination delivery and marking environment. A trial was conducted in a first year university level class during 2017 at Monash University, Australia. The approach enabled automatic marking for selected response questions and semi-automatic marking for short text responses. The system did not require a network or servers to operate therefore minimising the reliance on complex infrastructure. We paid particular attention to the integrity of the assessment process by ensuring separation of the answer key from the response composition environment. Students undertook a practice session followed by an invigilated exam. Student's perceptions of the process were collected using pre-post surveys (n = 16) comprising qualitative comments and Likert items. The data revealed that students were satisfied with the process (4 or above on 5 point scales). Comments revealed that their experience was in part influenced by their level of computer literacy with respect to enabling skills in the subject domain. Overall the approach was found to be successful with all students successfully completing the e-exam and administrative efficiencies realised in terms of marking time saved.
Computer application in Secondary education (Mind Map)Dr. Sushma N Jogan
This is an innovative method of presenting the content in the form of different shapes and colours in an effective way. This is the task of assignment given to my M.Ed student of II semester and it has really presented in a creative manner. Hope this will be helpful in understanding the content in an easy way.
Semi-Detailed Lesson Plan - Input and Output Devices of ComputersAnjenette Columnas
A Semi-Detailed Lesson Plan about Input and Output Devices of Computers in the Computer Subject. I'm now a professional teacher and this lesson plan was used during my teaching demonstration in Sicayab National High School.
Computer application in Secondary education (Mind Map)Dr. Sushma N Jogan
This is an innovative method of presenting the content in the form of different shapes and colours in an effective way. This is the task of assignment given to my M.Ed student of II semester and it has really presented in a creative manner. Hope this will be helpful in understanding the content in an easy way.
Semi-Detailed Lesson Plan - Input and Output Devices of ComputersAnjenette Columnas
A Semi-Detailed Lesson Plan about Input and Output Devices of Computers in the Computer Subject. I'm now a professional teacher and this lesson plan was used during my teaching demonstration in Sicayab National High School.
ASCILITE 2018: Towards authentic e-Exams at scale: robust networked Moodlemathewhillier
We present the design and user evaluation of a resilient online e-Exam platform that is capable of working without a network for most of the exam session, including the conclusion of an exam, without loss of data. We draw upon the education and technology acceptance literature as a basis for evaluation. The technology approach takes advantage of the Moodle learning management system quiz module as a means to provide an electronic workflow for assessments and builds on a range of open source components to construct the robust solution. The approach also enables rich, constructed assessment tasks by providing authentic 'e-tools of the trade' software applications and a consistent operating system on each student's BYO laptop. The robust Moodle exam deployment was trialled in two undergraduate units (subjects) at an Australian university. Students undertook a sequence of practice, mid term and a final examinations using the platform. Additional software and audio files were utilised as part of the exams. Student feedback on their experience was collected using pre and post surveys covering a range of issues related to technology acceptance.
Occe2018: Student experiences with a bring your own laptop e-Exam system in p...mathewhillier
This study investigated student's perceptions of a bring-your-own (BYO) laptop based e-Exam system used in trials conducted at an Australian Pre-University college in 2016 and 2017. The trials were conducted in two different subjects, in Geography and Globalisation. Data was gathered using pre-post surveys (n = 128) that comprised qualitative comments and Likert items. Student's perceptions were gathered relating to the ease of use of the e-Exam system, technical reliability, suitability of the assessment task to computerisation and the logistical aspects of the exam process. Many of the typists were taking a computerised supervised test for the first time. A divergence of opinions between those that typed and those that hand-wrote regarding student's future use intentions became more prominent following the exam event.
ASCILITE 2018: Towards authentic e-Exams at scale: robust networked Moodlemathewhillier
We present the design and user evaluation of a resilient online e-Exam platform that is capable of working without a network for most of the exam session, including the conclusion of an exam, without loss of data. We draw upon the education and technology acceptance literature as a basis for evaluation. The technology approach takes advantage of the Moodle learning management system quiz module as a means to provide an electronic workflow for assessments and builds on a range of open source components to construct the robust solution. The approach also enables rich, constructed assessment tasks by providing authentic 'e-tools of the trade' software applications and a consistent operating system on each student's BYO laptop. The robust Moodle exam deployment was trialled in two undergraduate units (subjects) at an Australian university. Students undertook a sequence of practice, mid term and a final examinations using the platform. Additional software and audio files were utilised as part of the exams. Student feedback on their experience was collected using pre and post surveys covering a range of issues related to technology acceptance.
Occe2018: Student experiences with a bring your own laptop e-Exam system in p...mathewhillier
This study investigated student's perceptions of a bring-your-own (BYO) laptop based e-Exam system used in trials conducted at an Australian Pre-University college in 2016 and 2017. The trials were conducted in two different subjects, in Geography and Globalisation. Data was gathered using pre-post surveys (n = 128) that comprised qualitative comments and Likert items. Student's perceptions were gathered relating to the ease of use of the e-Exam system, technical reliability, suitability of the assessment task to computerisation and the logistical aspects of the exam process. Many of the typists were taking a computerised supervised test for the first time. A divergence of opinions between those that typed and those that hand-wrote regarding student's future use intentions became more prominent following the exam event.
21st Century School Presentation - Acorn High SchoolLisa Nielsen
This presentation provides an overview of how Acorn high school has made strides to become a school that prepares students for 21st century teaching and learning.
Online Intelligent Semantic Performance Based Solution: The Milestone towards...AM Publications
As we analyse the computer application undergraduate logical-based courses in an assorted
environment of online assignments and exams and offline lectures, and exhibit the impact on academic routine of
factors such as classroom attendance, web-based course complement, and homework. We present grades from both
ordinary front ends and where the latter method controls for unobserved variation among students. A system
tailored intelligent instructional evaluation will generate the students, teachers & administration concepts,
discussing the predisposition in estimation when the ordinary evaluation method is used, resulting from the fact
that it ignores unobserved assorted. It also reduces the administrator’s load and helps provide the flexibility to
teacher’s need for mass evaluation. The Online Intelligent Semantic Performance based Solution is web
applications that ascertain an association between the institutes and the students. Institutes enter on the site, the
concepts they want in the exam. The questions based on the relevant concept and the syllabus is displayed as a test
to the eligible students. The answers entered by the students are then evaluated and their score is calculated and
saved. This score then can be accessed by the institutes to determine the passes students or to evaluate their
performance. It has been successfully applied to the distance evaluation of basic operating skills of computer
science, such as the course of computer skills in Universities and the local examination for the under graduates in
faridabad, Haryana.
ASCILITE 2018: Integrating mixed reality spatial learning analytics into secu...mathewhillier
We present an approach to using mixed reality (MR) technologies in supervised summative electronic exams. The student learning experience is increasingly replete with a rich range of digital tools, but we rarely see these same e-tools deployed for higher stakes supervised assessment, despite the increasing maturity of technologies that afford authentic learning experiences. MR, including augmented and virtual reality, enables educators to provide rich, immersive learner centred experiences that have unique affordances for collecting a range of learning analytics on student performance. This is especially so in disciplines such as health, engineering, and physical education requiring a spatial dimension. Yet, in many institutions, paper-based exams still dominate, in some measure due to concerns over security, integrity and scalability. This is despite a key concern for educators and institutions in producing employment ready 21st century graduates being the authenticity of assessments used for high stakes judgements. We therefore present a proposal for how MR pedagogies can be deployed for use in supervised examination contexts in a manner that is secure, reliable, and scalable.
Occe2018: writing e-exams in pre-university collegemathewhillier
This study examined student's expressed strategies, habits and preferences with respect to responding to supervised text based assessments. Two trials of a computerised examination a system took place in an Australian Pre-University college in 2016 and 2017. Students in several classes studying Geography and Globalisation completed a sequence of practice and assessed work. Data was collected using pre-post surveys about their preferred writing styles, habits strategies in light of their choice to type or handwrite essay and short answer exams. Comparisons were made between those that elected to handwrite and those who chose to type the exam were conducted with several areas being significant. The performance (grades), production (word count) of the typists and hand-writers were also correlated and compared.
Moodle quiz: towards post-paper e-assessmentmathewhillier
Using Moodle quiz for assessments that begin to leverage the affordances of ICTs that go beyond the capabilities of paper equivalents - post paper assessment task design examples.
This is intended to be used frame by frame during a lecture while explaining the differences in each approximation to the volume of the solid depicted in the diagram. The thinner slabs will give a better approximation to the volume. Calculus is all about taking limits...
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
2. A key motivation - The gap
Real world of work Exams
2
World Economic Forum – How will digital change your working world.
https://agenda.weforum.org/wp-content/uploads/rtr2m8vm1-628x330.jpg
Exams at Monash Caufield in 2015 (mathew.hilier[at]monash.edu)
70,000 student university.
We are faced with a growing disconnect between
the way high stakes testing is conducted using pen
on paper exams and students’ everyday experiences
of study, work and life, let alone the future!
4IR
21C
3. A key idea …
SAMR Theory - Puentedura, R. (2012). Thinking About Change in Learning and Technology. Presentation
given September 25, 2012 at the 1st Global Mobile Learning Conference, Al Ain, UAE.
http://www.hippasus.com/rrpweblog/archives/2012/04/10/iPad_Intro.pdf
4. Study context – broader e-Exam project
Start > > > > > > > > > > > > > > Future >
Get Ready Phase 1 Phase 2 Phase 3 Phase 4 Phase 5
Institutional
approvals,
research
ethics,
hardware and
infrastructure
Paper
equivalent
small scale.
Post-paper
small to
medium.
Medium to
large scale.
Whitelisted
and logged
Internet
Open but fully
logged
Internet
Basic doc
exams to
begin!
Expanding the
app and media
landscape.
Adding the
power of an
LMS.
Network
BYOD exam.
Network
mixed mode
BYOD exam.
Crawling Walking Running Jumping Flying!
4
http://ta.vu/e-exam-roadmap
Extension work:
An offline e-learning platform
see moleap.org
The project has
progressed!
Moodle resistant to
network outages!
This paper!
6. Research Questions for spreadsheet pilot (phase 2.5)
Could a spreadsheet be used as a small-scale, do-it-yourself exam delivery system?
In particular, guided by literature, could we:
• Utilise standard and common spreadsheet software (leverage existing tools with
minimal barriers to use by teachers and students)
• Enable automated or semi-automated marking of selected response style questions
and short text responses without manual data entry (to realise work efficiencies of
technology and minimise errors)
• Not rely on a live network during the exam (to minimise the chance of disruption)
• Be hack proof - the student portion of the system must not to have answers ‘hidden’
within it.
Would students accept this approach?
In particular, ease of use, perceive it to be reliable and secure.
7. Doing it…
Room set up:
Paper 'e-Exam quick start guide' and post-exam survey.
A power socket was provided for each typist.
Hand-writers were given a paper copy of the exam questions and response booklets. These
were available to typists upon request.
Both typists and hand-writers sat in the same room.
1) Students enter the room and seated.
2) Typists were given an e-Exam USB stick containing the questions.
3) Students start their computer with a USB stick progressing to the e-Exam desktop. A desktop
background image provides a visual check that all have booted from the correct USB.
4) Students enter their student ID and name into the e-Exam Starter screen.
5) All students now wait.
6) Invigilator announces the start of the exam and students click the ‘Start Exam’ button.
7) Auto-save occurs every 2 minutes.
8) At the end of the exam, the student uses File>Save one last time and Exit the Office suite.
9) When done they shut down the computer.
10) Students return the USB sticks.
11) Students complete the post-exam survey before leaving the room.
8. e-Exam workflow using spreadsheets (offline mode)
Post-exam: assessment
8.
Responses
retrieved
from USBs.
2. Create master USB
(load exam, configure, test)
Pre-exam: prepare exam
4. USBs sent to
exam venue
OR
USBs can be
recycled next
exam.
Linux Live
USB.
Libre Office
Spreadsheet
5. Pre-session:
Student laptop
setup &
practice.
3. USBs duplicated
1. Teacher creates spreadsheets
(exam and matching gradebook)
7. Collect
USBs
10. Marking
undertaken.
Checked by teacher.
9.
Responses
collated with
RDBmerge
6. Exam venue:
a. Students enter room
b.Given USB
c. Start laptop from USB
d.Do exam
e. Finalise and shutdown
f. Return USB
g. Leave room
11.
Return
feedback
to student.
NOTE!!!
NEW!!!Our
robust online
Moodle version
halves USB
handselling
11. Language tools Language tools available
according to LOTE selection
Respond in designated cells
(other cells are locked).
12. Participation
Intro to Chinese (first year).
22 students at pre-exam practice.
16 typed the exam.
7 females and 9 males.
Caveat: sample was small and not random -
descriptive of these groups only.
13. Pre and post response trends
Strongly Disagree 1 2 3 4 5 Strongly Agree
Caveat: Small, not random samples – descriptive of these groups only.
1 2 3 4 5
Written instructions were easy to follow
It was easy to start my computer using the e-Exam USB stick
I can use the e-Exam system just as well as my own laptop system
It was easy to use the office suite (word processor/spread sheet)
It was easy to answer multiple-choice questions in the e-Exam…
Overall, I feel the e-Exam System is easy to use
I feel the e-Exam System is reliable against technical failures
I feel the e-Exam System is secure against cheating
I now feel relaxed about using the e-Exam system for my exam
I would recommend the e-Exam System to others
I would like to use a computer for exams in the future
Pre Post
Sign Test
n/s
0.03
n/s
n/s
n/s
n/s
n/s
n/s
n/s
n/s
n/s
14. Study Process
1) Two weeks prior: practice session + pre-survey.
2) Exam day: In-class, graded, supervised assessment task + post survey.
15. Summary - Key Findings
a) The offline Spreadsheet based e-Exam was rated well by the typists:
4+ out of 5.
b) The spreadsheet worked - each student completed successfully.
Responses were auto saved to USB each two minutes, retrieved following exam and
processed using a second ‘gradebook’ spreadsheet.
c) Marking was more efficient due to some auto marking and readable responses.
Possible next steps for spreadsheet version…
i) Trials in units (subjects) that would best take advantage of a spreadsheet
For example: accounting, finance, maths, statistics, science.
ii) Further development to streamline the workflow with respect to setup and
retrieval and marking.
iii) However project focus has shifted to the now proven ‘Robust’ online Moodle
approach!
16. Thank you
For further information contact:
Mathew.Hillier@gmail.com
TransformingExams.com
Presentations at ASCILITE2018:
1. Do-it-yourself e-Exams [8] Mon 11:30am D2.212
2. Towards authentic e-exams at scale – Robust networked Moodle [51]
Following this session! Same Room! Mon 12noon D2.212
3. Integrated mixed reality spatial learning analytics into secure
electronic exams [183]. Tue 11:20am D2.212
Editor's Notes
ABSTRACT
Electronic examination and assessment systems pose a challenge in both finding and/or creating a solution that is effective and practical from a real world and software engineering perspective. Any proposed solution must be flexible to allow for a range of assessments, taking into account the requirements and expectations of multiple stakeholders such as student, teachers, administrators and technical support. This includes being easy to use, open enough to prevent vendor lock-in, practical to implement, secure and efficient to support.
In this presentation, we explore the current incarnation of an e-Exam platform, in particular its approach of using purely open source components, the mechanics of the system and design decisions taken in developing a modular and automated foundation for a systematic electronic learning and assessment approach. This, together with a Bring Your Own Device strategy for hardware provision demands a high level of common hardware compatibility in an ever-changing technology landscape.
This talk describes general experiences with e-exams such as advantages, challenges, preparing a good e-exam and tips on the implementation
Puentedura, R. (2012). Thinking About Change in Learning and Technology. Presentation given
September 25, 2012 at the 1st Global Mobile Learning Conference, Al Ain, UAE. http://www.hippasus.com/rrpweblog/archives/2012/04/10/iPad_Intro.pdf
Spare slide
Example of a question that requires a very complex response, which could not be posed without a computer
How your system works, in a nutshell. A diagram or photo to illustrate the critical element of the technological implementation
Illustration of the technological implementation of the workflow for running an e-exam on ‘robust’ Moodle infrastructure.
Exam content resides on a server. USB sticks serve as a standardised operating and application suite. USBs are immediately reusable without manual processing*.
System is resistant to network outages and uses secure browser with keys for access to server. Responses are auto saved every minute to server or in case of network outage saved to USB in an encrypted file. Once the exam starts the quiz content is cached to the browser, from then onwards the network is not required for successful exam continuance and completion. USBs can be used in a generic manner but can also be customised with additional software tools or to pre-load large media files.
The above workflow was used successfully for e-Exam trials in two units in semester 1 2018. Each unit had a sequence of a practice session followed by two or three exams.
Our trials used WIFI dongles to expand the range of compatible laptops as development of the system continued to integrate more drivers. The last trials in the sequence did not need the drivers because we had expanded the compatibility or students borrowed compatible laptops.
e-Exam USBs are configurable for multiple use cases. This includes the use of local files in fully offline mode, the use of the robust online mode via SEB, or the use of online whitelisted internet based resources.
The e-Exam USB can be used to provide a unified examination environment across BYO laptops, institution owned laptops and computers including computer labs provided boot from USB is enabled.
* A full USB auto reset feature is still under development that will be applicable when response submission is entirely handled via Moodle where a live network connection exists at the end of the exam.