faculty of arts ict in education | 1IDEM DITO:Towards one system for digital testingat the University of GroningenSymposium: A Broader Experience, Bremen, 22 March 2012Sake Jager, University of Groningen
faculty of arts ict in education | 2About the project› IDEM DITO: DIgitaal TOetsen = Digital Testing Different faculties: one system› Period: April 2011-May 2012› University funding: ICT strategy fund› Participants: Faculty of Medical Sciences Faculty of Arts Faculty of Economics and Business Educational Support and Innovation› Aims to examine: Which / what kind of testing system works for our organisation? Who are going to use it? How do we manage it? What is the cost? (Which return?)
faculty of arts ict in education | 3 Institutional context› University testing policy › Internationalization More and better testing More RUG students Improve student turn outs abroad More visiting students› Professionalisation Exam Erasmus Mundus boards collaborations Better preparation for supervision and standards setting tasks › Central test facility 288 computers› Professional development Multifunctional use faculty Flexible testing Didactic qualifications configuration Personal portfolio Specific applications for testing?
faculty of arts ict in education | 4Situation at start project› Blackboard 9.1 learning environment: Management and support: Educational Support and Innovation Centre Decentralized support Faculties Link to student registration (Progress) and scheduling (Ocasys)› Tests in Blackboard (with Respondus) Diagnostic/formative: as part of learning Summative: exams and (language) tests› Didactic benefits: Multimedia: Arts, Culture and Media programme More frequent tests in relation to mandatory study advice› Logistic / organisational issues: Lack of test facilities: multiple takes of same test Limitations Blackboard: re-sits, item banks, version management, test statistics.
faculty of arts ict in education |5 | 12-1-2013Potential benefits of digital tests › Reduction of correction time, paper savings (tests, answer sheets), easier distribution and workflow › Compiling item banks and re-use items › Better legibility compared to handwritten work › Immediate access to distribution of scores, difficulty items, and other statistical data › Multiple question types; pictures, graphics, photos, sound, video, animations; concurrent use of additional program (e.g. SPSS) › Immediate results (scores) for students, feedback
faculty of arts ict in education | 6Example case› Law teacher› 200 international students› Student laptops and word for exams› “Handing in” of assignments on USB sticks› Legibility no longer an issue› “This is much more efficient than pen-and- paper”› But: Even here, further efficiency gains are possible!!
faculty of arts ict in education | 7Plan of action› Specification of evaluation criteria › Desk research (15) › Presentation testing applications (8) › Pilots with teachers, students, test specialists, technicians, four packages (QMP, LPlus, Compass / Teletoets, Quayn) › Advice on organization and management
faculty of arts applied linguistics |8 | 12-1-2013
faculty of arts applied linguistics | Pilots completedPilot Faculty Date ObjectiveEssay-exam FEB 2 December 2011 Open questions / essayTeacher pilot 1 FMS 20 December 2011 Teacher evaluation of useabilityHistory Arts 20 December 2011 Multiple question types (mc, ordering, open) FeedbackArts, Culture, Arts 10, 31 January; 6 BlackboardMedia February 2012Pilot Master FMS 17 January 2012 Student comparison of (two) packagesstudents MedicineTeacher pilot 2 FMS 25 January 2012 Teacher evaluation of useability
faculty of arts applied linguistics | Pilots in progressPilot Faculty ObjectiveLanguage Centre English Arts Multiple question types (mc, cloze, essay) Correction open questions AudioTest results and analysis ESI Comparison with legacy application for analysis Export of test dataLanguage test Russian Arts Multipe question types (mc, ordering, open) Non-Western fonts (teacher, student)Connectivity Blackboard ESI Examine possibilities (together with suppliers)and SAS packagesArts, Culture, Media Arts Test taking and correction open questionsessay test
faculty of arts ict in education | 11Some preliminary results› QMP: Pro: No failures, import, question bank, support Con: stand-alone authoring, text editor, expertise required› Quayn: Pro: Wysiwyg interface, easy to use, web-based Con: Flash required, under development› Compass / Teletoets: Pro: Interface, text editor, version management, web-based Con: Test matrix, language support› LPLus: Pro: Versatile, many options Con: Architecture (databases, not fully web-based)
faculty of arts ict in education | 12Emerging scenario of use› Test requirements covered by combination of Blackboard and dedicated testing program› Registration as now (Progress)› Separate Blackboard server, specific BB course for each test, temporarily available› Test in Blackboard or dedicated program (link in BB course, no separate registration)› Consolidation of grades in Progress› Tests prepared in Word, Blackboard or test program› Gradual increase in use of dedicated testing application
faculty of arts ict in education | 13 Additional actions› Teacher training Test design Test analysis Digital test construction› Exam scheduling Identify courses for digital testing Prioritise these› Management plan Authentication and identification Usage policy test facility PC configuration Maintenance and support
faculty of arts applied linguistics ict in education | 14Thank you for your attentionContact: S. Jager http://www.rug.nl/let/idemdito http://www.rug.nl/staff/s.jager firstname.lastname@example.org