Case study - Selor• Scientific screening, adaptation and extension of the national language exam for federal employees• Introduction of smart (technological) question types allowed for the use of authentic language materials (with multimedia) in the exam.• New online preparatory learning environment, including thousands of exercises and simulation exams, with tracking and logging of people’s progress and results, allowing a useful training before the exam.• High stakes: job selection, promotion, bonuses The Belgian minister of Public Services and Enterprises getting information about the Salto language tests before taking an exam herself.
Case study - SelorFacts and figures• Belgian federal government selection and recruitment office.• + 200.000 exams per year• + 500 job selections per year• 2 exam rooms with 600 pc’s for synchronous e-testing• Decentralized exam rooms in all provinces.• Online subscription for candidates• SSO login with the Selor Admin system• Exam results automatically returned into Selor Admin System The Belgian minister of Public Services and Enterprises getting information about the Salto language tests before taking an exam herself.
Case study - AirbusAirbus is the world leader in airplane manufacturing.The projectAirbus screens and permanently evaluates all new candidates for the new A350composite airplane.Airbus organizes specialized training, both external and in-company and screens all staffmembers involved in this process.Edumatic Exam is used for both evaluation and self-assessment.Facts and figures• AMA-T Airbus Manufacturing Academy – Toulouse organizes and manages more than 35.000 trainings per year• For the new A350 FAL (Final Assembly Line) 1.000 new employees are needed.• Edumatic Exam is self-operated by Airbus AMA-T and the external training centers Airbus is preparing for the manufacturing of a new composite airplane, the A350.
Case study – K.U.LeuvenEdumatic as one of the systems in the university’s Toledo platform fore-learning.Edumatic was co-developed with research groups from variousfaculties:• Language faculty for the unique translation correction algorithm, the voice recording questions and the questions including ASR and TTS• The Law faculty for the unique Open question (with fully automated correction and feedback)• The Faculty of Business and Economics for the Accounting question types.Hundreds of teachers (professors and assistants) are working as authors tocreate questions and as coaches to do student follow-up.
Case study – K.U.Leuven Facts and figures• Students: 36 923 students• Staff: K.U.Leuven employs 9 560 people. • Senior Academic staff: 1 463 people • Junior academic staff: 5 136 • Administrative and Technical Staff : 3 098 people• One undergraduate campus in Kortrijk• E-learning with Edumatic: • More than 10.000 students online every year • 228 different courses online • More than 92.000 active (used) items in the database • Peak uses in exam periods, more than 3.000 student online (really) simultaneously. • Fully automatic Single Sign On (SSO) webservice with Blackboard learning platform.• E-testing with Edumatic Exam: More than 47.760 online exams for the faculty of Students taking an exam at K.U.Leuven. Business and Economics since 2008.
Trends in eAssessment…triggered byCustomer’s needs more assessment than ever more tracking and logging than ever more flexibility asked than ever more technological means/devices than ever……Technology evolutions allowing for previously impossible innovative approacheswifi, mobile, tablets, cloud computing, bandwidth available, multimedia options (webcam, audio recording…),trackpads, touchpads, data warehousing, webservices, multiplatform, standards, tagging, social networks,…
Trends in eAssessment…solutions in eAssessment1. SSO or webservices based user management2. Webservices based results exchange3. Tag based structuring and filtering create test scenarios based on metadata4. High performance correction algorithms open questions5. Full multimedia playing and recording authentic question types and materials6. Mobility, wireless and new devices7. Scalability
How do you solve…?User management• Manually• By group import with Excel sheets• By SSO – Single Sign On Webservices that connect existing systems People only have to login once (SSO!) Deeplinking as an interesting feature• SSO with Selor ADMIN and website • SSO with GEAF• Launched from Selor Monitoring tool • Login and results
How do you solve…?Organizing/structuring your DB• Tree structured • Branches• Metadata• Media library • Shows linked items • Link to external (IPR protected) resources• Metadata mgmt for legal issues (ml) • Metadata for items• Content input from external partners • Metadata for screening external training centers
How do you solve…?“We want it like the paper version…”• Navigation (linear, non-linear)• Skip/Remember this one• Calculator• …• Brainteaser option evaluated as • Brainteaser/Casse-tête option extremely useful at the end of each series• Calculator
How do you solve…?Open questions• Developed an “Open question” question type • Based on keywords recognition (including sophisticated scoring) • Scoring suggestion by the system and/or human correction/editing • Now also with ASR and TTS (FR, EN, NL)• Fully testing the efficiency of • Not used open question evaluation
How do you solve…?Voice recording(and face recording)• Developed a “Recording” question type • No installation (Flash based) • Saved and scored (suggestion) by the system (with ASR) and/or human correction/editing • Excellent for interpreter training• Voice recording for speaking skills • Used for self-assessment and• Voice and face recording for interpreters survey questions (Selor for EPSO) “Give your comments on…”
How do you solve…?New devicesSmartphones and tablets• Omnipresent and cheap (smartphones)• Connected• Mobile web version • iOS limitations (no Flash) • “Touch” limitations • Small screen limitations• App version • Android and iOS (free or paid) • Connect, download, work • In-app (purchase of) modules • Upload/Synchronize results
How do you solve…?Unequipped rooms/Large groups• Voting application• Edumatic download to standalone• Televic AvidaNet Teacher Control Center • Project teacher’s screen • Students have voting devices (all iOS and Android devices) • Many question types available • Real time answer monitoring • Full scoring, tracking and logging • Results sync back to server• Large groups (+ 1000!) • Unequipped rooms on work floor
How do you solve…?Test room security• Televic AvidaNet classroom software• Selor Monitoring Tool • AvidaNet
How do you solve…?Crashes, electricity cuts,toilet breaks…• Innovative features • “Resume” function • Don’t come back function (or See only once) • Pause function • Add time option• Selor Monitoring Tool• Resume
How do you solve…?Scalability• Decentralized assessments or exam• Huge groups (> 1000 people)• Cloud computing • Switching on/off servers (energy saving) • For extra capacity and reliability • Pay what you use/need• Synchronize between the local and the cloud system• Azure Cloud • Decentral (all over Europe)• Decentral• Huge groups
How do you solve…?Reporting Services• Huge tracking and logging database• Very difficult to produce “generic” reports that satisfy all customers• Very specific combinations of data in many different views• Reporting Web Services• “Harvesting”: customer can harvest in our database• Selor Admin is connected via Webservice • Airbus GEAF is connected
Edumatic RoadmapRELEASE 3.6 (Sept)• COBALT• Reporting • Per person • Per group • Per metadata• Dialog (ASR & TTS)• Voting devicesRELEASE 3.7 (Autumn 2011)• Survey questions and management• Adaptive testing • Custom paths • IRT algorithm paths• Math question type +RELEASE 3.8 (Spring 2012)• Offline and standalone version • Including standalone authoring
Edumatic MobileAll releases• Mobile candidate/student version • iOS and Android compatible• VotingEdumatic SMS• Spring 2012