Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Seminario eMadrid


Published on

  • Be the first to comment

  • Be the first to like this

Seminario eMadrid

  1. 1. Open Standards and Specifications for Outcome-based Assessment<br />Israel Gutiérrez<br />Universidad Carlos III de Madrid<br />Madrid, April 12th 2010<br />
  2. 2. Standards and Specifications<br />
  3. 3. Computer-based assessment<br />Reduce effort and costs: permit continuous assessment<br />Assessment re-use: interoperability needed<br />Variety of assessment formats<br />Proprietary: MoodleXML, HotPotatoes, Blackboard <br />Ad hoc: OpenMark<br />IMS QTI<br />Open Standards and Specifications for Assessment<br />
  4. 4. Outcome-based Assessment<br />Universities developing an outcome-based learning strategy<br />Adaptation to European Space for Higher Education<br />Improve course structure for accreditation (ABET)<br />Creation of new courses<br />Learning outcomes<br />Assessment as a key process<br />
  5. 5. Outline<br />Analysis of assessment standards and specifications<br />Usage of assessment formats in Europe<br />Interoperability issues<br />Next generation assessment scenarios<br />Best practices<br />
  6. 6. Methodology<br />Analysis of Assessment Standards and Specifications<br />Metamodels (UML diagrams) from MDA<br />Understand better studied formats<br />Abstraction from syntax (XML binding)<br />Focus on content<br />
  7. 7. Studied Assessment Formats<br />Assessment purpose<br />General purpose<br />IMS QTI<br />FML<br />OpenMark<br />SuML<br />QAML<br />
  8. 8. Qualitative Comparison<br />Key features<br />Multiple response allowance<br />Correct response indication<br />Metadata capabilities<br />Hybrid question management<br />Question types (from FREMA)<br />Constructed: essay, short answer, FIB, etc.<br />Constrained: MCQ, MRQ, TF, matching, etc.<br />Miscellaneous: simulations, practical sessions<br />
  9. 9. Assessment Formats Comparison<br />
  10. 10. Project results: WP6<br />Assessment Formats Usage in Europethe current picture – methodology and sample description<br /><ul><li>Data collection
  11. 11. Personal interview
  12. 12. In-depthreport
  13. 13. Qualitativeanalysis
  14. 14. Samplecharacteristics
  15. 15. Highereducation
  16. 16. Multicultural
  17. 17. Multilingual</li></ul>Sample size<br />
  18. 18. Geographical Distribution<br />
  19. 19. Assessment Formats Study: Resultsthe current picture – key conclusions<br /><ul><li>Scarce re-use of assessment resources
  20. 20. Scarce acceptance of IMS QTI specification
  21. 21. High technological diversity
  22. 22. LMS, ad-hoc tools, etc.
  23. 23. Diversity of assessment methodologies
  24. 24. formative/summative, various types of activities, face-to-face/blended/elearning settings, etc.</li></li></ul><li>Interoperability Issues: IMS QTI<br />Scarce use of the specification<br />Incomplete support in LMS<br />i.e. adaptive material<br />Flexibility handicaps<br />QTI profiles<br />Versions instability: 1.2.1 or 2.1?<br />Versions are not backwards compatible<br />
  25. 25. Assessment Interoperability<br />Moodle<br />QTIlrn<br />QTIclix<br />QTIexport<br />QTIMoodle<br />QTI 1.2.1<br />QTI 1.2.1<br />QTI 1.2.1<br />.LRN<br />Clix<br />
  26. 26. Next Generation Assessment Scenarios<br /><ul><li>Hints
  27. 27. Adaptation
  28. 28. Instructional design </li></li></ul><li>Assessment Resources Interoperability Proposal<br />Assessment resources<br />
  29. 29. Objective: Gather and develop best practices about interoperable content as support to higher education and learning outcome oriented<br />Results<br /><ul><li>ICOPER Reference Model
  30. 30. Gathering of recommendations
  31. 31. Open ICOPER Content Space
  32. 32. One of the most successful eContent+ projects since 2004</li></li></ul><li>Open ICOPER Content Space<br /><ul><li>Federated repository of educational resources</li></li></ul><li>Outcome-based Assessment App<br />Use cases<br />Search for assessment resources in a repository<br />Filter by intended learning outcomes<br />Filter by assessment methods<br />Import/re-use of assessment resources<br />Publish/share assessment resources<br />Online demo:<br />
  33. 33. Outcome-based Assessment App<br />Use cases (cont.) <br />Annotate assessment resources<br />Peer review<br />Teacher reflection<br />Student feedback<br />Update learner’s profile<br />Stakeholders<br />Faculty members<br />Course developers<br />Online demo:<br />
  34. 34. Outcome-based Assessment App<br />Learningoutcome-based search<br />Implementation for dotLRN<br />based on ICOPER LOM profile<br />Learningoutcome profile update<br />
  35. 35. Outcome-based Assessment App: Evaluation<br />Engineering evaluation <br />Implementers<br />Tool developers<br />End user evaluation<br />Faculty members<br />Course developers<br />Epistemological evaluation <br />Researchers<br />Standardization bodies<br />
  36. 36. ICOPER Assessment app<br />Assessment resources should include information about<br />Intended learning outcomes<br />Recommended assessment methods<br />Annotations about resources<br />Usage of integrated environment for outcome-based assessment<br />
  37. 37. Best Practices for Assessment<br />
  38. 38. Thank you for your attention!<br /><br /><br />