Norwegian national project for digitalization of assessments for universities and university colleges, first phase is school type exams (essays and other written exams in controlled environments)
1. Digital assessments
a shared solution
Ingrid Melve,
NORDUnet2014, Uppsala, 2014-09-24
2.
3.
4. Digital assessment and exam day
4
eCampus: national program for video, collaboration tools
and IT architecture for flexible higher education
UNINETT, the Norwegian research network
Building infrastructure, supporting inter-institutional ICT
DIGG: digital exam systems development project
8 universities, 15 university colleges and UNNETT
Specifications for digital exam: rooms, security, integration etc
Multiple vendors develop software and help in testing
Aiming for joint procurement in 2016
5.
6. 2014-06-20 6
High
level
expert
group
eCampus
project
Local
pilots
Digital assessment for all
students ASAP
7. Project goals
7
Technical solution for exams
Defining needs, standardization
Working with vendors to develop solutions
Large scale roll-out: live test
Change ”school” exams
23 universities and colleges
Students get text processing tools
Change room requirements
Change workflow
Kill paper for millions of exams per semester
National mandate: find solution
8. Assessment process: From manual to digital routines
Prepare Examination Grading
Feedback on
grade and
complaints
Locality
Candidate
Examination
/
questions
Medium
Answers
Status
Info Journal
Digital
Manual
Digital Digital Digital
Manual Manual Manual
FS
Examination
-
registry
System for register
examinations
Formalities
Overviewt,e.g.
language-variants
Shortage, reminders
Archiving: Need of
history archiving.
Critical: Access
FS
Answers
Systems for digital exam
Challenges:
•Control access
•Scaling
•Context
•Infrastructur
•Costs
•Competence
Critical: Stability
FS
System for digital
censorship
-Process support
-Transparent censorship
Support for varying
methodology
Critical: Access,
Security, Transfer
results.
In general:
•Must support digital og manualroutines
•Access to systems. People with loose connection to institution
•Digital signatures
•Clarify regulations
•Different forms of assessment
•Role of Learning Management Systems
•Hand over information to internal and national archives
Need for system
support.
Case management
system
Integration with archive
system
Archiv
e
Inform the
students
Ministry of Education
(DBH)
Norwegian State
Education Loan Fund
Figure by NTNU
9. First time we do
9
Digitalization and
dematerialization of exam
process
Policy alignment for
assessment, at practical level
BYOD for critical situations
Integration of multiple
components for assessments
12. Workflow
12
Digital process
Supported by FS (shared
student registry)
Principals
Teacher, reviewer, grading teacher
Study administration, library etc
Student
Green IT, remove paper
13.
14. How does it play together?
14
Architecture
What is the framework and
expectations?
Best Current Practice
Aligning practice from
multiple institutions
Testing integrations/APIs
15.
16. Integration
16
Shared extract
Who are the candidates?
Who are grading exams?
What are the grades?
Transparency and complaints
Workflow handover to other
systems: archive, student
registry etc
17.
18. Integration (gory details)
18
Establish integration point
(Mule)
Define APIs for exam info
Test with multiple vendors
Adjust APIs
Harden integration point
Test, test, test, test
Exam systems
Student registry
Calendar, room planning
Check plagiarism
Grading system
Case processing system
(complaints)
Archive
19. BYOD – student´s stuff
19
Software requirements
Hardware requirements
Practical stuff
Must have 5-15% PCs to lend to students
Power supply
Security aspects
Specifications and good
practice documentation
20. Examination rooms
20
Power
Wifi
Do you see the screen?
Security inspections
BCP document in draft
Publish in November
21.
22.
23.
24. Next steps
24
Large scale testing
15% digital for 2015
60% for 2016
Best Current Practice docs
National alignment of practices
Develop software (vendors)
Development contracts
Paperwork and procurement rules
Joint procurement Q1 2016
Editor's Notes
Prepare:
When the professor is starting his work on preparing an assessment paper, he should be able to work digitally. He should be able to co work with his college professor, and they should be able to share the document I “real time”
Maybe it exists a “national repository” of e-learning objects (for example a video) and there might be a repository of premade questions
When the professor(s) is done, normally the paper must be controlled and approved before it is distributed on.
The assessment paper must then be digitally distributed to the examination office, who are responsible for the execution of the actual examination.
Examination
It exists a multiple variation on how to execute an exam
But the main purpose is to ensure that the student give a digital alternative for digital assessment
A universal design and inclusive learning environments to ensure accessibility for students with disabilities.
Grading:
The professor and his co-censor should be able to work with the same answer, separately at the same time.
Feedback and complains:
Must be integrated with the archive system
Challenges due to; Security, Control access, Scaling, operational stability, Context, Infrastructure, Cost, Competence, Need for support
* The Common Student System (Felles studentsystem FS) is a study administration system developed for universities, scientific colleges and national university colleges. FS is developed by the University Center of Information Technology (USIT), University of Oslo.
FS is the sole study administration system for universities and national university colleges in Norway. It is utilized by 51 institutions, including all Norwegian universities, all national university colleges, the Police College, the Defense Colleges, and several private colleges as well as the Norwegian Universities and Colleges Admission Service (NUCAS) [Samordna opptak]. FS is the only study administration system that supports the NUCAS admissions process.
All Norwegian Universities still need more knowledge and understanding about needs and required functionality to handle digital assessment. So instead of doing a traditional procurement process, and signing up contract with some vendors for buying a defined system, we are looking into a so called Pre-commercial Procurement process