What are the benefits of E-Assessments aka Computer Based Testing, what you need to look out for when adopting it and what are the challenges ahead

1. BENEFITS OF E-ASSESSMENTS
AND HOW TO ENCOURAGE
ADOPTION WITHIN A SCHOOL
JEFFERY TAY
NATIONAL UNIVERSITY OF SINGAPORE

2. WHO AM I?
• Working at National University of Singapore, Centre for Instructional Technology
• Responsible for eLearning systems in NUS
• >15 years experience in IT
• >10 years experience in computer based testing

3. E-ASSESSMENTS AKA COMPUTER BASED TESTING
• PC Labs
• Massive infrastructure
• Direct paper replacement
• Prison mode lockdown
• Glorified typewriter

4. WHY PC LABS
• Equality
• Rote learning/Memorization
• Not every student had a laptop
• For the fortunate few, well..
Oct 1988

5. DIGITAL REVOLUTION
• Laptops got smaller and lighter
• Finally someone figured a laptop needs to be portable
• In 2011, Ultrabooks introduced and is now the de-facto
• Smart phones, tablets

6. DIGITAL REVOLUTION
• Connectivity
• 28.8K  2G 3G  4G
• Wireless B  A  G  N  C

7. INTRODUCING THE ALWAYS CONNECTED STUDENT

8. THE ALWAYS CONNECTED STUDENT
• What they do not do quite so much
• Write
• Think in black and white
• Who are they
• NEEDS their personal device and
connectivity
• Wiling to try new technology
• Acclimate to technology easily
• Almost everything they do has an
electronic footprint
• Typing
• Learning
• Socializing
• Planning/Scheduling
• …

9. WHAT ABOUT FACULTY AND SCHOOL?
• Environmentally friendly, Save the trees!
• Reduce costs e.g transport, logistics, manpower
• Reduce human errors e.g transcribe from pen and paper to excel
• Student Retention
• Student Success e.g via remediation
• Introducing technology to syllabus

10. ADOPTION OF CBT IN SCHOOLS
Freie Universität Berlin
Numberofstudents
0
10
20
30
40
50
60
70
80
90
2009 2010 2011 2012 2013 2014 2015
National University of Singapore
Total
Numberofhighstakese-assessments

11. VARIETY OF VENDORS

12. BENEFITS OF CBT
Store questions securely
so that they can be
reused in future
Question Bank Secure
Ensures device is secure for
high stakes exams
Item Analysis
Accurately measure
validity and difficulty of
question and answer
choices
Student Reporting
Help students
understand their
strength & weakness
Learning Outcome
Reporting
Useful for accreditation,
curriculum reform, and
to decrease fall out
rates.
Grading Exams
Automatically grade
exams and provide
feedback to students on
their performance in
real-time
Blueprint Exams
Ensures faculty are
testing students on
appropriate or proper
concepts
Longitudinal Data
Track learning objectives
and student progression
over of the course of a
semester, year, or any
timeframe
Rubrics
Ease of marking and
greater transparency
during marking
“Cloud”
Centralized platform
for ease of access

13. POTENTIAL PITFALLS
CBT is NOT the holy grail
for assessments

14. PHYSICAL LIMITATIONS
• Problems
• Do you want to go for PC labs or bring your
own laptops (BYOD)?
• Do you have the facility?
• What do you need to do to ensure there is
sufficient capacity?
• Ensuring good connectivity and power supply
• Possible solutions
• Work out the pros and cons for each
• Think long term, a PC lab is not cheap both in
terms of capital as well as operational cost
• Management support
• Lock in funds for retrofitting/renovation

15. PRODUCT AND ACCEPTABLE RISKS
• Problems
• There are many products out there, which
one is best for me?
• What are my worse case scenarios?
• How do I mitigate against potential
bottlenecks?
• Possible solutions
• Talk to vendors, request for demos
• Ask for customer references and speak to
them
• Run small pilots
• Carve out policies to cover e-assessment
scenarios

16. PEOPLE, PEOPLE AND PEOPLE
• Problems
• You do not know what you do not know
• Aversion to change
• Faculty performance
• Possible Solution
• Management support
• Start small (non high-stakes!) and slowly scale
up
• WIIFT (What’s in it for them?)
• Motivation
• Give control
• Clearly define the goals, be realistic!
• Treat them as your friend, not as a test
subject

17. CRITICAL MASS ACHIEVED
• Humans are creatures of habit
• After a while, e-Assessments become the norm.
• Whats next?

18. WHERE DO YOU GO FROM HERE?
• Technology doesn’t rest and neither should you!
Multiple form factors
Machine learning / Predictive
Remote proctoring

19. QUESTIONS?
• Jeffery Tay
• cittka@nus.edu.sg