Presentation delivered by Bert Wylin, Televic Education, at the 2011 eAssessment Scotland conference

1. eAssessment Scotland
Dundee
Bert Wylin - b.wylin@televic.com

2. Overview

3. References – use cases
France Belgium France UK
Belgium France Belgium Belgium
Kenya Belgium UK Belgium

4. References – use cases
France Belgium France UK
Belgium France Belgium Belgium
Kenya Belgium UK Belgium

5. Case studies for today
Belgium France

6. Case studies for today
Belgium France

7. Case study - Selor
• Scientific screening, adaptation and extension of the
national language exam for federal employees
• Introduction of smart (technological) question types allowed for the use of
authentic language materials (with multimedia) in the exam.
• New online preparatory learning environment,
including thousands of exercises and simulation exams,
with tracking and logging of people’s progress and results,
allowing a useful training before the exam.
• High stakes: job selection, promotion, bonuses
The Belgian minister of Public Services
and Enterprises getting information
about the Salto language tests before
taking an exam herself.

8. Case study - Selor
Facts and figures
• Belgian federal government selection and
recruitment office.
• + 200.000 exams per year
• + 500 job selections per year
• 2 exam rooms with 600 pc’s for
synchronous e-testing
• Decentralized exam rooms in all provinces.
• Online subscription for candidates
• SSO login with the Selor Admin system
• Exam results automatically returned into Selor
Admin System
The Belgian minister of Public Services
and Enterprises getting information
about the Salto language tests before
taking an exam herself.

9. Case study - Airbus
Airbus is the world leader in airplane manufacturing.
The project
Airbus screens and permanently evaluates all new candidates for the new A350
composite airplane.
Airbus organizes specialized training, both external and in-company and screens all staff
members involved in this process.
Edumatic Exam is used for both evaluation and self-assessment.
Facts and figures
• AMA-T Airbus Manufacturing Academy – Toulouse
organizes and manages more than 35.000 trainings per year
• For the new A350 FAL (Final Assembly Line) 1.000 new employees
are needed.
• Edumatic Exam is self-operated by Airbus AMA-T and the external
training centers
Airbus is preparing for the manufacturing of
a new composite airplane, the A350.

10. Case study – K.U.Leuven
Edumatic as one of the systems in the university’s Toledo platform for
e-learning.
Edumatic was co-developed with research groups from various
faculties:
• Language faculty for the unique translation correction algorithm, the
voice recording questions and the questions including ASR and TTS
• The Law faculty for the unique Open question (with fully automated
correction and feedback)
• The Faculty of Business and Economics for the Accounting question
types.
Hundreds of teachers (professors and assistants) are working as authors to
create questions and as coaches to do student follow-up.

11. Case study – K.U.Leuven
Facts and figures
• Students: 36 923 students
• Staff: K.U.Leuven employs 9 560 people.
• Senior Academic staff: 1 463 people
• Junior academic staff: 5 136
• Administrative and Technical Staff : 3 098 people
• One undergraduate campus in Kortrijk
• E-learning with Edumatic:
• More than 10.000 students online every year
• 228 different courses online
• More than 92.000 active (used) items in the
database
• Peak uses in exam periods, more than 3.000
student online (really) simultaneously.
• Fully automatic Single Sign On (SSO) webservice
with Blackboard learning platform.
• E-testing with Edumatic Exam:
More than 47.760 online exams for the faculty of Students taking an exam at K.U.Leuven.
Business and Economics since 2008.

12. Trends in eAssessment
…triggered by
Customer’s needs
more assessment than ever
more tracking and logging than ever
more flexibility asked than ever
more technological means/devices than ever…
…Technology evolutions
allowing for previously impossible innovative approaches
wifi, mobile, tablets, cloud computing, bandwidth available, multimedia options (webcam, audio recording…),
trackpads, touchpads, data warehousing, webservices, multiplatform, standards, tagging, social networks,…

13. Trends in eAssessment
…solutions in eAssessment
1. SSO or webservices based user management
2. Webservices based results exchange
3. Tag based structuring and filtering
 create test scenarios based on metadata
4. High performance correction algorithms
 open questions
5. Full multimedia playing and recording
 authentic question types and materials
6. Mobility, wireless and new devices
7. Scalability

14. How do you solve…?
User management
• Manually
• By group import
with Excel sheets
• By SSO – Single Sign On
Webservices that connect existing systems
People only have to login once (SSO!)
Deeplinking as an interesting feature
• SSO with Selor ADMIN and website • SSO with GEAF
• Launched from Selor Monitoring tool • Login and results

15. How do you solve…?
Organizing/structuring your DB
• Tree structured
• Branches
• Metadata
• Media library
• Shows linked items
• Link to external (IPR protected) resources
• Metadata mgmt for legal issues (ml) • Metadata for items
• Content input from external partners • Metadata for screening external
training centers

16. How do you solve…?
“We want it like the paper version…”
• Navigation (linear, non-linear)
• Skip/Remember this one
• Calculator
• …
• Brainteaser option evaluated as • Brainteaser/Casse-tête option
extremely useful at the end of each series
• Calculator

17. How do you solve…?
Open questions
• Developed an “Open question” question type
• Based on keywords recognition
(including sophisticated scoring)
• Scoring suggestion by the system
and/or
human correction/editing
• Now also with ASR and TTS (FR, EN, NL)
• Fully testing the efficiency of • Not used
open question evaluation

18. How do you solve…?
Voice recording
(and face recording)
• Developed a “Recording” question type
• No installation (Flash based)
• Saved and scored (suggestion) by the system (with ASR)
and/or
human correction/editing
• Excellent for interpreter training
• Voice recording for speaking skills • Used for self-assessment and
• Voice and face recording for interpreters survey questions
(Selor for EPSO) “Give your comments on…”

19. How do you solve…?
New devices
Smartphones and tablets
• Omnipresent and cheap (smartphones)
• Connected
• Mobile web version
• iOS limitations (no Flash)
• “Touch” limitations
• Small screen limitations
• App version
• Android and iOS (free or paid)
• Connect, download, work
• In-app (purchase of) modules
• Upload/Synchronize results

20. How do you solve…?
Unequipped rooms/Large groups
• Voting application
• Edumatic download to standalone
• Televic AvidaNet Teacher Control Center
• Project teacher’s screen
• Students have voting devices (all iOS and Android devices)
• Many question types available
• Real time answer monitoring
• Full scoring, tracking and logging
• Results sync back to server
• Large groups (+ 1000!) • Unequipped rooms on
work floor

21. How do you solve…?
Test room security
• Televic AvidaNet classroom software
• Selor Monitoring Tool • AvidaNet

22. How do you solve…?
Crashes, electricity cuts,
toilet breaks…
• Innovative features
• “Resume” function
• Don’t come back function (or See only once)
• Pause function
• Add time option
• Selor Monitoring Tool
• Resume

23. How do you solve…?
Scalability
• Decentralized assessments or exam
• Huge groups (> 1000 people)
• Cloud computing
• Switching on/off servers (energy saving)
• For extra capacity and reliability
• Pay what you use/need
• Synchronize between the local and the cloud system
• Azure Cloud
• Decentral (all over Europe)
• Decentral
• Huge groups

24. How do you solve…?
Reporting Services
• Huge tracking and logging database
• Very difficult to produce “generic” reports that satisfy all customers
• Very specific combinations of data in many different views
• Reporting Web Services
• “Harvesting”: customer can harvest in our database
• Selor Admin is connected via Webservice
• Airbus GEAF is connected

25. Edumatic Roadmap
RELEASE 3.6 (Sept)
• COBALT
• Reporting
• Per person
• Per group
• Per metadata
• Dialog (ASR & TTS)
• Voting devices
RELEASE 3.7 (Autumn 2011)
• Survey questions and management
• Adaptive testing
• Custom paths
• IRT algorithm paths
• Math question type +
RELEASE 3.8 (Spring 2012)
• Offline and standalone version
• Including standalone authoring

26. Edumatic Mobile
All releases
• Mobile candidate/student version
• iOS and Android compatible
• Voting
Edumatic SMS
• Spring 2012

27. EDUMATIC
How to… (example)