10 Years of E-assessment & Math


Published on

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

10 Years of E-assessment & Math

  1. 1. 10 Years of E-assessment & MathM. Antonia HuertasComputer Science DepartmentUOC04/04/13 e-math 2013, UNED, Madrid 1
  2. 2. Introduction: E-assessment today• The term ‘e-assessment’ began to be used widely fromthe late 1990s• Ofqual (Office of the Qualifications and ExaminationsRegulator UK) defines e-assessment (2009) as:“the use of electronic systems for thedevelopment, operation and delivery of accreditedqualification assessment or the collection of performanceevidence, which contributes to the awarding of a unit or anaccredited qualification”http://www.ofqual.gov.uk/64.aspx• E-assessments are tests which learners take onscreenor online using computer technology. They are mainlyformed by multiple choice questions (MCQ)04/04/13 e-math 2013, UNED, Madrid 2
  3. 3. Types of e-assessment in formaleducation• Summative, ‘high stakes’ assessments ­– examinationswhere the results matter to the candidate and theinstitution, and hence are both carefully regulated andof public interest.• Formative (or assessments for learning) – thoseembedded within learning activities, are designed forlearners and teachers to monitor and plan their work.• Technologies employed in these two settings may besimilar, approaches to risk and innovation are verydifferent.04/04/13 e-math 2013, UNED, Madrid 3
  4. 4. Today use and benefits• Benefits:– Immediate feedback– Assessment validity– Space and time flexibility– More efficient administration• Not all benefits apply to all e-assessment use• Poorly designed or implemented e-assessment may achieve no benefits at all• Benefits are not ‘automatic’04/04/13 e-math 2013, UNED, Madrid 4
  5. 5. E-Learning technologies today• Education is influenced by a lot of emergingtechnologies• http://c4lpt.co.uk/top-100-tools-2012/• http://www.edtechmagazine.com/k12/article/2012/11/6-hot-trends-educational-technology-infographic04/04/13 e-math 2013, UNED, Madrid 5
  6. 6. Features in e- assessment formats• Question types: Short answers, Essay, Multiple Choice Questions(MCQ), Multiple Response Questions (MRQ, Fill in the Blanks(FIB), Matchin, Crossword puzzles.04/04/13 e-math 2013, UNED, Madrid 6
  7. 7. Beyond multiple-choice questionsAutomated scoring & feedback• Automated scoring– Mathematics marking Moodle + Wiris use in UOC experimental “Introduction of Maths forEngineering”– Programming languages & formal notations– Short free text answer• Automated feedback– Intelligent Tutoring Systems (ITSs)– Example for mathematics: ‘Carnegie Learning’s Cognitive Tutor’ at secondary and HE in USA ‘eFit’ for mathematics at lower secondary schools in Germany04/04/13 e-math 2013, UNED, Madrid 7
  8. 8. Beyond multiple-choice questionsAdaptive assessment• The system chooses which questions to ask next, basedon the candidate’s responses to previous questions.(+) Better measure of student’s capabilities in a given time(- ) Difficulty to ensure that the adaptivity functions correctly• Technical methods of implementation commonly usedinclude dynamic item selection from a pool of itemsbased on statistical methods04/04/13 e-math 2013, UNED, Madrid 8
  9. 9. Beyond multiple-choice questionsSimulations and serious gaming• Used when students interaction with the real thingis impractical (danger, too time consuming, tooexpensive)• Simulation technology is quite varied.– Generic simulation engines (such as ExpertSystems), tailored for educational purpose– But also computationally relatively simplesoftware, relying rather on rich and varied media• Serious gaming refers to games with an educationalpurpose. Rigorous assessment is a key area ofdevelopment in serious gaming04/04/13 e-math 2013, UNED, Madrid 9
  10. 10. E-assessement generations• We are in the Computer-based Testing Era:– To improve the efficiency of testing procedures.– E-assessment is centred on testing• Generation 1 (1994-2002):– Computerised conventional testing– Automated scoring• Generation 2 (2002-2012):– Computerised adaptive testing– MCQ & Moodle kind of systems– Automated feedbak04/04/13 e-math 2013, UNED, Madrid 10
  11. 11. Current and future e-Assessment strategiesSource : IPTS on the basis of Bennett, 2010; Bunderson, et al., 1989; Martin, 2008).04/04/13 e-math 2013, UNED, Madrid 11
  12. 12. E-assessement generations(the future)• We are going to the Embedded Assessment Era– Embedding assessment within the learning process– Based on ‘Learning Analytics’ (Data analysis + beheavouraltracking)– Intelligent tutoring systems– Automated feedback• Generation 3:– Continuous measurement to monitoring student’s achievementtrajectory• Generation 4:– Personalised feedback and tutoring– Intelligent monitoring by means of knowledge engeeniering• Requires technological advances04/04/13 e-math 2013, UNED, Madrid 12
  13. 13. A Generation 2-3 case study• Logic is an introductory course offered under theComputer Science Degree programme at UOC.• Looking for an e-assessment system to implementformative e-assessment• Design and implementation of an e-assessmentsystem for both knowledge and skill• Design and implementation of a methodology for apilot in a real course in UOC04/04/13 e-math 2013, UNED, Madrid 13
  14. 14. Architecture of the system04/04/13 e-math 2013, UNED, Madrid 14
  15. 15. Formalization of the pilot04/04/13 e-math 2013, UNED, Madrid 15
  16. 16. Formalization of the pilot04/04/13 e-math 2013, UNED, Madrid 16
  17. 17. Results• Formative continuous assessment is more correlated withsummative final assessment in the pilot group than in thecontrol group• Helping to implement formative e-assessment, when skilland knowledge acquisition have to be assessed• From the statistical study it is not possible to state that thestudent skills were improved (the ones evaluated in thefinal examination), but they are not worsened although inthe pilot group students had a higher workload• Teachers have the realistic chance to switch the previousassessment model for a complete formative assessment• Needing improvements: technologically andmethodologically.04/04/13 e-math 2013, UNED, Madrid 17
  18. 18. My 10 years of e-assessment & Math2002• Logic• MCQ tests(automated scoring)• Common test for allstudents2006• Algebra• Automated scoring(Wiris)• Individual test (nkinds of exerciseswith parameters)2009• Logic• ITS for Learning Logic• Automated feedbackstep by stepexcercises2012• Logic• E-assessment system(Moodle + ITS)• Automated scoringand feedback forknowledge and skills04/04/13 e-math 2013, UNED, Madrid 18
  19. 19. ObrigadaThank youGracias04/04/13 e-math 2013, UNED, Madrid 19
  20. 20. 04/04/13 e-math 2013, UNED, Madrid 20
  21. 21. 04/04/13 e-math 2013, UNED, Madrid 21
  22. 22. 04/04/13 e-math 2013, UNED, Madrid 22
  23. 23. 04/04/13 e-math 2013, UNED, Madrid 23
  24. 24. 04/04/13 e-math 2013, UNED, Madrid 24
  25. 25. 04/04/13 e-math 2013, UNED, Madrid 25
  26. 26. 04/04/13 e-math 2013, UNED, Madrid 26
  27. 27. 04/04/13 e-math 2013, UNED, Madrid 27