Presentation of the Project Digital Exam II - SUNET Inkubator - for 1st European Conference on digital Exams and Assessment (ECDEA 2018), 8:th of June 2018 in Gothenborg, Sweden

1. Project Digital Exam II
SUNET Inkubator
Presentation ECDEA.org, June 2018
Mats Brenner, Project lead
URL: https://wiki.sunet.se/display/Inkubator/Digital+Tentamen+II

2. National university project – Digital Exam II
Project Digital exam conducted in
the year 2015. In autumn 2017 and
2018 the project continues as Digital
Exam II.
The report computerized exam & e-
exam (170 pages) – which gives an
overview how digitization of the
work processes can be carried out
from the ax to the loaf of the
traditional paper exam.
URL: http://bit.ly/2zmN4y6 or at
https://wiki.sunet.se/display/Inkubator/Digital+Tentamen+II

3. Critical design principles – Digitization
• Maintainability: Whether the solution can be adapted or reused to
meet more than one need.
• Compatibility: Whether the solution can share information with other
solutions such that users can benefit.
• Efficiency: Whether the solution drives cost and time savings.
REF: When.. ”decide that the fundamental design principles for supporting educational technology are agility and
interoperability – all technology must enable adaptation to changing environments.” URL:
https://encoura.org/helping-institution-realize-vision-five-edtech-companies-watch/ och
http://pubs.opengroup.org/architecture/togaf8-doc/arch/chap29.html

4. The entire work process is digitised
Preparation Exam Assessment
Archive &
Learning
Analytic

5. The benefits of e-exam
The advantages (Brenner & Sandström, 2015) with
computerized exam & e-exams are, among other things, that
assessment/correction can take place faster (30-70%) and
better with higher quality, as it becomes easier when the
students ' answers are digital – especially through:
• Using digital Rubrics
• That more teachers are in and review/make judgement with
the correction Protocol (Moderation)
• That the students ' answers are and the teachers '
assessment is anonymous
• That teachers can sit anywhere to make their
correction/assessment
• That teachers can comment on documents more easily and
more flexibly, mark on images etc. as feedback to students

6. Key factors in the introduction &
implementation as following
Leadership from the top
Institutional commitment and investment
Robust and reliable infrastructure
Effective and accessible support & development for academic staff
Ability to demonstrate the benefits of experience for students,
teachers and administrative staff evidence decision making and a
continuous improvement cycle

7. Project Digital Exam II – We are increasing the
pace!
Mission
The main goal of the project Digital Exam II is to create a continued
foundation for solutions around the digital exam and further develop
the collaboration and exchange of experience between universities.

8. Status of the universities?
Management mode: Sophiahemmet University, GU, KAU, ORU, LU, SU, HiG, etc
On the move: Several universities have pilot projects, feasibility studies, project
groups or procurement e.g. MDH, LU, UMU, UU, etc.
Several universities have test licenses with suppliers – have piloting
Universities that have not started: Linnaeus University, Linköping University,
Södertörn University, Mid Sweden University
See the project website > questions and answers

9. Several ready-made system solutions are

10. New forms of examination - faster analysis

11. Common and less common forms of examinations
Forms of Examation Individual / Group Learning Managent
System (LMS)
Other Learning Tools
Project work Group Virtutal goup room Project web tool gantter.com,
Google Docs, Wiki
Role Play Group Record video or podcast Web conference tools
Peer Review Individual Self- or peed assessment
tool
Screencast, Podcast, iPeer &
PeerWise
Seminar disussion Gruop Discusson Board Web conference tools
Green / gold paper in Wiki Individual (Group) Wiki Wikipedia
Portfolio, Journal Individual Blog, Wiki, E-portfolio E-portfolio, box.com,
myshowcase.me
Case study Individual (Group) Wiki Google Docs, box.com
Minute Paper Individual (Group) Discussion Board Webcast, Google Docs
Book review
Poster exhibition
Individual (Group) Blog, Wiki, record video or
podcast
Screencast, mindmap tool,
web survey, Google Docs
Multiple-choice Individual E-exam, mobile test E-exam, test or survey tool

12. Paper Exam – one of several forms of examination
Students at univ and pupils in schools in Sweden
are not so accustomed to summative (High
Stake) test situations.
Driving licens test, yes. Here is an experience of
digital testing/certification.
However, we do not have entrance exams (only
for artistic courses) and not for exams such as in
Finland.
Test and high stake written examinations feels
strange and unaccustomed, we have not
learned study techniques to focus on a greater
achievement for longer periods of time, with
time pressure. Higher education tests are not
critical for applying for studies.
Teacher training has few or no written exams
high stake exam – mostly used home exams and
group assignments.
REF: http://www.provainfact.se/publish/examination/9127-
Examinationsformer___vanliga_och_mindre_vanliga/index.html#/9291

13. The Last paper Bastion…
The University of Gävle has 47% distance
students and 17000 students-therefore only
about 8000 paper exams.
Örebro University 66000 paper exams, with
17000 students.
Compare with the University of Skövde,
about 8000 students with 22000 paper
exams.
Gothenburg University (GU) with Chalmers
circa 1250000 paper exams, about 10000
digital exams 2017 according to GU.
Linköping has at least 130000-140000
students who do paper exams (5000 exams)
Larger universities such as Uppsala and Lund
have over 100000 paper exams each. In total
we can probably count on over 500000
paper exams/year.
Some digitize!
The universities have too many paper exams
still...

14. Trad paper exam – Pros and cons
Benefits
The trad paper exam is quite secure, since it reduces the risks of cheating.
The form is familiar to most students.
With thoughtful strategies, tasks and assessment criteria, deep learning
can be stimulated.
Cons
The trad paper exam can at carlessness encourage non-deep learning.
Students may be pressured by time limitations and to be in right shape at
the moment of writing
Information that stimulates deep learning can provide a high correction
burden for the teacher.

15. Advantages: E-exam for Teacher/Examiner
• Overview and easy with more forms of representation (multimodalt)
• Time-saving, less working hours needed to receive manually to correct/assess
• Easier to assess, find those who have made the same mistakes and thus get a better and
more equal assessment
• Saves time in design and scoring
• May have several examiners
• Can have negative points, give extra points
• Using plausible answers and good distractors
• May have templates for exam (number of questions, type of questions, limit/standardize)
• Can use third-party applications (streaming computor software)
• Save and reuse questions in questions banks
• More easily evaluate questions and question types
• Plagiarism detection goes to implement
• Better – readable, reallocated text, can insert images/sketches
• Adaptation for students with functional variations improves

16. Learning Analytics (Object analysis) – For quality Assurance

17. Why have some not got started yet?
Leadership from the top
Institutional commitment and investment
Robust and reliable infrastructure
Effective and accessible support & development for academic staff
Ability to demonstrate the benefits of experience for students,
teachers and administrative staff evidence-based decision making and
a continuous improvement cycle

18. Digital exam is... (not optimize the old paper process), it
is Digital Transformation & Business Development
• Take position, Start – do pilot test
• Choose the right waves – do international analysis (monitor, act or ignore)
• Make the most of the waves (what still comes) – have commitment, use other people's
experiences and make investments
• Mobilization – Create piloting and create project group with IT Department, exam booking, ICT-
management organization and disseminate experiences from other universities
• Coordination – collaborate with all necessary stakeholders for routines and competence
development and procure robust system solution.
• Acceleration – Scale up, integrate and continual improvements with the support of the software

19. Recommendations
• Do not create your own system solutions – for costly
• Start with piloting with interested teachers/examiners – gain time
and experience
• Gather all stakeholders – anchor
• Management decisions and documents – good, but must not be
shelf heaters
• Use the Digital Exam II project's resources – visit other universities
• Do not optimize the old paper process. Perform digital
transformation

20. In planning
• Webinar – please send in suggestions
• Questions & Answers to the website are expanded
• Support visits to universities that have not started
• Reference Architecture & LIS Integration for digital
exam
• Joint e-service for digital exam

21. Questions and thanks!
Mats Brenner, Project management Project Digital Exam II –
SUNET Inkubator
Contact: mats.brenner@hig.se, Phone: 026-64 82 18, 070-350 02 33
Web page/Contact addresses: Http://bit.ly/2zmN4y6 or
Https://wiki.sunet.se/display/Inkubator/Digital+Tentamen+II