Seminario eMadrid


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Seminario eMadrid

  1. 1. Open Standards and Specifications for Outcome-based Assessment<br />Israel Gutiérrez<br />Universidad Carlos III de Madrid<br />Madrid, April 12th 2010<br />
  2. 2. Standards and Specifications<br />
  3. 3. Computer-based assessment<br />Reduce effort and costs: permit continuous assessment<br />Assessment re-use: interoperability needed<br />Variety of assessment formats<br />Proprietary: MoodleXML, HotPotatoes, Blackboard <br />Ad hoc: OpenMark<br />IMS QTI<br />Open Standards and Specifications for Assessment<br />
  4. 4. Outcome-based Assessment<br />Universities developing an outcome-based learning strategy<br />Adaptation to European Space for Higher Education<br />Improve course structure for accreditation (ABET)<br />Creation of new courses<br />Learning outcomes<br />Assessment as a key process<br />
  5. 5. Outline<br />Analysis of assessment standards and specifications<br />Usage of assessment formats in Europe<br />Interoperability issues<br />Next generation assessment scenarios<br />Best practices<br />
  6. 6. Methodology<br />Analysis of Assessment Standards and Specifications<br />Metamodels (UML diagrams) from MDA<br />Understand better studied formats<br />Abstraction from syntax (XML binding)<br />Focus on content<br />
  7. 7. Studied Assessment Formats<br />Assessment purpose<br />General purpose<br />IMS QTI<br />FML<br />OpenMark<br />SuML<br />QAML<br />
  8. 8. Qualitative Comparison<br />Key features<br />Multiple response allowance<br />Correct response indication<br />Metadata capabilities<br />Hybrid question management<br />Question types (from FREMA)<br />Constructed: essay, short answer, FIB, etc.<br />Constrained: MCQ, MRQ, TF, matching, etc.<br />Miscellaneous: simulations, practical sessions<br />
  9. 9. Assessment Formats Comparison<br />
  10. 10. Project results: WP6<br />Assessment Formats Usage in Europethe current picture – methodology and sample description<br /><ul><li>Data collection
  11. 11. Personal interview
  12. 12. In-depthreport
  13. 13. Qualitativeanalysis
  14. 14. Samplecharacteristics
  15. 15. Highereducation
  16. 16. Multicultural
  17. 17. Multilingual</li></ul>Sample size<br />
  18. 18. Geographical Distribution<br />
  19. 19. Assessment Formats Study: Resultsthe current picture – key conclusions<br /><ul><li>Scarce re-use of assessment resources
  20. 20. Scarce acceptance of IMS QTI specification
  21. 21. High technological diversity
  22. 22. LMS, ad-hoc tools, etc.
  23. 23. Diversity of assessment methodologies
  24. 24. formative/summative, various types of activities, face-to-face/blended/elearning settings, etc.</li></li></ul><li>Interoperability Issues: IMS QTI<br />Scarce use of the specification<br />Incomplete support in LMS<br />i.e. adaptive material<br />Flexibility handicaps<br />QTI profiles<br />Versions instability: 1.2.1 or 2.1?<br />Versions are not backwards compatible<br />
  25. 25. Assessment Interoperability<br />Moodle<br />QTIlrn<br />QTIclix<br />QTIexport<br />QTIMoodle<br />QTI 1.2.1<br />QTI 1.2.1<br />QTI 1.2.1<br />.LRN<br />Clix<br />
  26. 26. Next Generation Assessment Scenarios<br /><ul><li>Hints
  27. 27. Adaptation
  28. 28. Instructional design </li></li></ul><li>Assessment Resources Interoperability Proposal<br />Assessment resources<br />
  29. 29. Objective: Gather and develop best practices about interoperable content as support to higher education and learning outcome oriented<br />Results<br /><ul><li>ICOPER Reference Model
  30. 30. Gathering of recommendations
  31. 31. Open ICOPER Content Space
  32. 32. One of the most successful eContent+ projects since 2004</li></li></ul><li>Open ICOPER Content Space<br /><ul><li>Federated repository of educational resources</li></li></ul><li>Outcome-based Assessment App<br />Use cases<br />Search for assessment resources in a repository<br />Filter by intended learning outcomes<br />Filter by assessment methods<br />Import/re-use of assessment resources<br />Publish/share assessment resources<br />Online demo:<br />
  33. 33. Outcome-based Assessment App<br />Use cases (cont.) <br />Annotate assessment resources<br />Peer review<br />Teacher reflection<br />Student feedback<br />Update learner’s profile<br />Stakeholders<br />Faculty members<br />Course developers<br />Online demo:<br />
  34. 34. Outcome-based Assessment App<br />Learningoutcome-based search<br />Implementation for dotLRN<br />based on ICOPER LOM profile<br />Learningoutcome profile update<br />
  35. 35. Outcome-based Assessment App: Evaluation<br />Engineering evaluation <br />Implementers<br />Tool developers<br />End user evaluation<br />Faculty members<br />Course developers<br />Epistemological evaluation <br />Researchers<br />Standardization bodies<br />
  36. 36. ICOPER Assessment app<br />Assessment resources should include information about<br />Intended learning outcomes<br />Recommended assessment methods<br />Annotations about resources<br />Usage of integrated environment for outcome-based assessment<br />
  37. 37. Best Practices for Assessment<br />
  38. 38. Thank you for your attention!<br /><br /><br />