Successfully reported this slideshow.
Your SlideShare is downloading. ×

Technology-based Assessment: Challenges and Promises

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
160407 eua open education
160407 eua open education
Loading in …3
×

Check these out next

1 of 27 Ad

More Related Content

Slideshows for you (20)

Similar to Technology-based Assessment: Challenges and Promises (20)

Advertisement

More from EDEN Digital Learning Europe (20)

Recently uploaded (20)

Advertisement

Technology-based Assessment: Challenges and Promises

  1. 1. Gyöngyvér Molnár University of Szeged http://www.staff.u-szeged.hu/~gymolnar Technology-based Assessment: Challenges and Promises 6th EDEN Research Workshop Budapest, Hungary, 24-27 October, 2010 Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged „Developing Diagnostic Assessments” Project
  2. 2. Outline • Need for change • Possibilities and promises of TBA • Implementing TBA in international projects Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  3. 3. The need for change • Technology abounds • Change - Live, learn, communicate • New insight into the learning process • Potential to transform education Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  4. 4. The need for change • New expectation, lifestyle - the Net Generation “digital natives” => calls for different type of teaching • Schifting from generation gap to generation lap • New demands on the education Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  5. 5. The need for change • Existing models of assessment fail • New assessments are required • Multi-sensory feature • Retention rates • Technology alone will not result in higher achievement Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged • Technology is not the issue - > catalyst
  6. 6. The need for change • Data about education quality and its impact on economic output • After 4 cycles of PISA • Economic modeling • E.g. how the GDP would change according to the change in educational output • Result: relatively small improvements – large impact on future Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  7. 7. Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged 0 200 400 600 800 1000 1200 Mexico Turkey Greece Portugal Italy Luxembourg Spain UnitedStates Poland Norway SlovakRepublic OECD Hungary Denmark Germany Iceland France Ireland Sweden Austria Switzerland Belgium CzechRepublic UnitedKingdom Australia NewZealand Canada Netherlands Japan Korea Finland Catching up with Finland (in percent of GDP) % current GDP
  8. 8. Why is assessment so important? „You can not improve what you can not measure” (Schleicher, 2010) Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  9. 9. The feedback in education • ICT – growing in every segment of education • International tendencies (OECD, ETS, NCES) in educational assessment • PPT reached its barrier • Human resources, cost of money and time • TBA solve this problem • NEED to use technology to improve the efficiency of assessment • Improved quality of data and increased speed of assessment • Better conditions to improve the educational system Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  10. 10. Possibilities and promises of TBA (1) • Reach stimulus material (multimedia tools) • New item formats • New construct (PS in technology reach environment) • New areas (dynamic assessment) – assessing dynamic – MicroDYN approach – example Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  11. 11. Possibilities and promises of TBA (2) • TBA rises new issues in assessment: – background data – testing prozess - e.g. educational data mining – log file analyses - issues of reaction time, sequence – annotations regarding eye and face tracking Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  12. 12. Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  13. 13. Possibilities and promises of TBA (3) • offers new assessment methods (adaptive testing) – the difficulty of the test tailors dynamically to the student’s ability level – items are selected from an item bank dependent at a difficulty appropriate for the student – provides more time-efficient and accurate assessments. Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  14. 14. -4 -3 -2 -1 0 1 2 3 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 logitscale item Adaptive testing Person ability level Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  15. 15. (4) • increases motivation - students with high and low ability level - flow experience (5) • changes the whole assessment process (item generation, scoring, data-processing, information flow, feedback and the speed of assessment) • rapid and precise feedback (6) • poses new questions and problems – if TBA – replace PP ∑ improved quality and increased quantity of data and increased speed of assessment Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged Possibilities and promises of TBA
  16. 16. Implementing TBA in international projects • Europe: growing attention in several European countries and the European Union also facilitates these attempts • At national level Luxemburg leads the way: – able to assess an entire cohort simultaneously – automatic generation of the feedback to the teachers – University of Luxembourg and the Public Research Center Henri Tudor – TAO (Testing Assisté par Ordinateur) Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  17. 17. Implementing TBA in international projects • Germany – DIPF – TBA project : – adapts and further develops the TAO platform – establish a national standard for technology- assisted testing Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  18. 18. Implementing TBA in international projects • In Hungary – 2008 – first – media effect study: – Inductive reasoning – 5th grader (PP + CB) – TAO platform – Result: global achievement highly correlate – items, persons – behave differently • 2009 - a large-scale project was launched by the Center for Research on Learning and Instruction - University of Szeged - develop an online diagnostic assessment system for the first six grades of primary school in reading, mathematics and science Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  19. 19. „Developing Diagnostic Assessments” Project • the development of assessment frameworks • devising a large number of items both on paper and on computer - building item banks • research on comparing the achievements on the tests using different media. • long-term project aims: an online formative assessment system for the first six grades of primary school • First phase: pilot the system in 200 schools • study the related technological and methodological issues in detail Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  20. 20. Implementing TBA in international projects • US - many instances – CBA and online assessment so wide spread - testing industry - engaged in writing and developing task and different testing software – Despite of this – in 2009 – Microsoft, Cisco and Intel - transforming global educational assessment and improving learning outcomes – Teaching of 21st Century Skills project Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  21. 21. Teaching of 21st Century Skills project • Aim: resolve the gap - exists between what goes on in schools and what goes on in the real world • better measure the skills - needed for the 21st- century Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  22. 22. Teaching of 21st Century Skills project • Specify high-priority skills, competencies, - 21st century - turn these into measurable standards and assessment framework • Examine innovative learning environments and formative assessments - draw implications for ICT-based summative assessments • Identify methodological and technological barriers – support solutions • implementation of these standards and methodologies, pilot test, make recommendations for broader educational assessment reform. Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  23. 23. OECD PISA • Started to implement CBA • 2006 – CBAS • 2009 - ERA • 2012 - problem solving • 2015 - completely to CBT • still a lot of open questions Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  24. 24. • Despite of the advantages - in large-scale national and international assessment - faces several technological challenges • complexity of issues - no simple and clear solution • Several technological means available in schools • diversity, compatibility, connectivity require further considerations Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  25. 25. But the future is more promising… Research Group on the Development of Competencies, Hungarian Academy of Sciences Institute of Education, University of Szeged
  26. 26. Thank you for your kind attention! http://www.staff.u-szeged.hu/~gymolnar Research Group on the Development of Competencies, Hungarian Academy of Sciences … and remember: „Without data, you are just another person with an Opinion” (Schleicher, 2010)

×