Innovations in IT and how it can (and will…) impact assessment
•
2 likes•271 views
Powerpoint presented by Mark Molenaar and Jason Fagan at the E-ATP 2014 Budapest
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (14)
Similar to Innovations in IT and how it can (and will…) impact assessment
Similar to Innovations in IT and how it can (and will…) impact assessment (20)
More from Cito
More from Cito (7)
Recently uploaded
Recently uploaded (20)
Innovations in IT and how it can (and will…) impact assessment
- 1. Gaining advantage through assessment Growing Talent In Europe What’s Next? Innovations in IT and how it can (and will…) impact assessment
- 2. Gaining advantage through assessment Growing Talent In Europe Who Are We? ▪ Mark Molenaar Chief Information Officer Cito/Questify ▪ Jason Fagan Software Developer Trifork 9/25/2014 2
- 3. Gaining advantage through assessment Growing Talent In Europe What Do We Do? ▪ Projects together ▪ CvTE – Facet ▪ IBE – Switzerland ▪ CBR – Dutch Driver’s License (Theory) ▪ Using (open) standards ▪ QTI in particular 9/25/2014 3
- 4. Growing Talent In Europe Gaining advantage through assessment Facet (CvTE)
- 5. Growing Talent In Europe Gaining advantage through assessment IBE (Zwitserland)
- 6. Growing Talent In Europe Gaining advantage through assessment CBR
- 7. Gaining advantage through assessment Growing Talent In Europe Recent Developments ▪ Moving away from paper based assessments ▪ First to computer, now to tablets ▪ 5 years ago the iPad did not exist! ▪ Launched in 2010 ▪ Lot of apps developed, also in the area of assessment ▪ Large target audience 9/25/2014 7
- 8. ▪ A natural evolution to BYOD and interactive testing Gaining advantage through assessment Growing Talent In Europe (R)Evolution? Paper. Computer Touch- Screen Tablet/Ph ablet Virtual- Reality? 9/25/2014 8
- 9. Growing Talent In Europe What’s the Impact? Gaining advantage through assessment 9/25/2014 9
- 10. Gaining advantage through assessment Growing Talent In Europe Where’s the research? ▪ Pushing products? ▪ Research required ▪ What’s the impact? ▪ And is it intentional… ▪ Case sample: shift towards tablets ▪ Math Garden Research ▪ Short Comparison Study Tablet/PC delivery 9/25/2014 10
- 11. Gaining advantage through assessment Growing Talent In Europe Math Garden Research on Tablet Use ▪ “a fun and stimulating online environment in which children can train and develop their mathematical skills playfully at their own level” ▪ Research by Matthieu Brinkhuis, University of Amsterdam 9/25/2014 11
- 12. Growing Talent In Europe Gaining advantage through assessment Results & Trends 9/25/2014 12
- 13. Gaining advantage through assessment Growing Talent In Europe Comparison PC/Tablet – First results ▪ Short comparison study ▪ 1 school in primary education (PE) ▪ 25 candidates ▪ Average age: 11 ▪ 2 schools in secondary education (SE) ▪ 62 candidates ▪ Average age: 13/14 ▪ Each candidate 1 PC, 1 tablet test, 1 survey ▪ Consisting of 5 items and types ▪ Research by Saskia Wools, Cito 9/25/2014 13
- 14. Gaining advantage through assessment Growing Talent In Europe Let’s take the test! 9/25/2014 14
- 15. Gaining advantage through assessment Growing Talent In Europe Results: Device Usage 60 50 40 30 20 10 0 Totaal Computer Laptop Tablet 60 50 40 30 20 10 25 20 15 10 5 25 20 15 10 5 9/25/2014 15 Number of students DEVICES @ HOME (PRIMARY EDUCATION) Owned Shared Total 0 Totaal Computer Laptop Tablet Number of students DEVICES @ HOME (SECONDARY EDUCATION) Owned Shared Total 0 Computer Laptop Tablet Aantal leerlingen DEVICE USAGE (PRIMARY EDUCATION) < once a week once a week few times a week every day 0 Computer Laptop Tablet Aantal leerlingen DEVICE USAGE (SECONDARY EDUCATION) < once a week once a week few times a week every day unknown
- 16. Gaining advantage through assessment Growing Talent In Europe Results: Device Preference PO VO Preference rating Itemtype Expected Result Preference rating 1Drag & Drop Tablet Tablet -30 2Hotspot Tablet Tablet -11 3Ordering Tablet Tablet -13 3 4 3 3 3 4Essay 2 Laptop Laptop 22 1 2 2 5Short answer Laptop Laptop 0 18 2 5 4 6 2 11 8 10 4 5 3 4 -5 -4 -3 -2 -1 0 1 2 3 4 5 < Tablet Laptop >
- 17. Gaining advantage through assessment Growing Talent In Europe Results: Average Response Time 17 ▪ Short answer item (multiple gaps): 9/25/2014 30000 25000 20000 15000 10000 5000 0 PE_PC PE_Tablet SE_PC SE_Tablet Average response time in ms
- 18. Gaining advantage through assessment Growing Talent In Europe First conclusions/confirmations ▪ Tablet-use is on the rise ▪ Children with tablets at home prefer tablets ▪ Could lead to Differential Item Functioning (DIF) ▪ Construction for tablet requires thought ▪ Keyboard popup, auto-complete, etc. ▪ Not all types are equal ▪ Hotspot/drag&drop preferred on tablet ▪ Short/long answer preferred on PC ▪ Response times can differ greatly 9/25/2014 18
- 19. Growing Talent In Europe So What’s Next? Gaining advantage through assessment 9/25/2014 19
- 20. Gaining advantage through assessment Growing Talent In Europe Current & Coming Developments 9/25/2014 20
- 21. Growing Talent In Europe Gaining advantage through assessment New technology? 9/25/2014 21
- 22. Growing Talent In Europe Gaining advantage through assessment Dangers? 9/25/2014 22
- 23. Growing Talent In Europe Gaining advantage through assessment Possibilities! 9/25/2014 23
- 24. Growing Talent In Europe Gaining advantage through assessment More possibilities…
- 25. Gaining advantage through assessment Growing Talent In Europe Takeaways ▪ The future is coming fast… ▪ But there is still time… ▪ Use it to add value ▪ Not because it is cool… ▪ Not doing the same tricks differently… ▪ Further research is needed! ▪ Especially on comparison & equivalence 9/25/2014 25
- 26. Growing Talent In Europe Gaining advantage through assessment 9/25/2014 26
- 27. Gaining advantage through assessment Questions & Discussion Growing Talent In Europe What possibilities do you see?
- 28. Gaining advantage through assessment Growing Talent In Europe Thank you very much for your attendance ▪ Come see us at our booths ▪ Contact: ▪ jason.fagan@trifork.nl ▪ mark.molenaar@cito.nl ▪ saskia.wools@cito.nl (PC/Tablet research) ▪ m.j.s.brinkhuis@uva.nl (MathGarden research) ▪ Twitter #EATP2014 Please leave your name or business card to receive the presentation and stay informed 9/25/2014 28
Editor's Notes
- Description IT is driving innovation in many fields of industry, including assessment. But how do these innovations impact the processes we know? What is the real added value and when is it only distracting from original purpose? When does cutting edge become bleeding edge? In this session we will discuss the possibilities and pitfalls of innovations which could leverage mobility of talent. We will discuss one example more extensively: tablets have become a common platform for many test drivers and can benefit (employee) mobility to great extent. But very little research has been done in comparing the results of “classical” Computer (PC/Mac) based Testing (CBT) and tablet-based assessment. Many comparison-studies have been performed on similar content using CBT and PPT (Paper & Pencil Testing) deliveries, but empirical data on tablets is lacking. During this session, the first results of a comparison-study will be presented and discussed. Other innovations in mobility are already underway; augmented reality (Google Glass), next generation Virtual Reality (Oculus Rift, Sony Project Morpheus) and other wearable devices (smart watches like Pebble and more health-oriented devices) are on the horizon. What are the possibilities for these innovations in IT for employee mobility and what would their impact be? The session will conclude with an outlook on these developments and suggestions for further research.
- Hand over to Jason
- Switch to Jason CvTE = College voor Toetsen en Examens, Exam board for nationwide exams
- Dutch examination body. Move away from paper-based exams to purely digital content with a view to adaptive testing. Our first implementation using the new QTI Assessment Delivery Engine. High and low stake exams. Distributed examination system Schools host the system. Variety of computer hardware and operating systems. Users take exams using a secure browser.
- Institute for Educational Evaluation of the University of Zurich. Eliminate paper-based exams Provides teachers with the ability to easily create practice tests. Runs on a wide range of devices, BYOD High and low stake exams and practice exams. Adaptive testing algorithms.
- Dutch Driving License Agency Multiple centres across the country. Windows touch screen computers (no keyboard or mouse). Our first 100% touch based implementation.
- Switch to Mark
- Mark: polish thi slide
- Most have computer, some own tablet; everyone has computer Laptop and tablet are used more frequently a week
- Daft punk video
- Training for doctors,etc.?
- Osmo Play: https://www.youtube.com/watch?v=CbwIJMz9PAQ Matrix: https://www.youtube.com/watch?v=6vMO3XmNXe4