Advancing Online Assessment in Medical Education


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Advancing Online Assessment in Medical Education

  1. 1. Advancing Online Assessment in Medical Education<br />Greg Doyle, Faculty of Health Sciences <br />Stephen Marquard, Centre for Educational Technology<br />(Advised and enthused by Rachel Weiss, Faculty of Health Sciences)<br />University of Cape Town<br />June 2010<br />
  2. 2. Overview<br />Context <br />About Extended Matching Items<br />EMIs in Tests & Quizzes<br />Managing MCQs in a Question Bank<br />Sharing MCQs between institutions<br />2<br />11th Sakai Conference - June 15-17, 2010<br />
  3. 3. UCT Health Science Initiatives<br /><ul><li>Improve assessment practices
  4. 4. Introducing the use of Extended Matching Item (R-type) MCQs
  5. 5. A Question Bank application
  6. 6. Author, manage, reuse and share MCQs
  7. 7. Collaborating to develop and share MCQs</li></ul>3<br />11th Sakai Conference - June 15-17, 2010<br />
  8. 8. What is an EMI?<br />Extended Matching Item, aka R-type MCQ<br />EMI = R-Type MCQ (vs A-Type MCQ)<br />An EMI has:<br />A theme<br />A list of options (text or labelled image)<br />A lead-in statement<br />A list of items<br />11th Sakai Conference - June 15-17, 2010<br />4<br />
  9. 9. Sample EMI<br />11th Sakai Conference - June 15-17, 2010<br />5<br />This example shows options labelled1, 2, 3… but more conventionally should be labelled a, b, c …<br />
  10. 10. EMI variants<br />Items may require choosing:<br />One option<br />Multiple options (e.g. a, e, g) independent of order [in previous example, options 1,5,7]<br />Options in a specific order (e.g. g, a, e)<br />Implications for grading<br />Limits the ability to use older question types (e.g. fill-in-the-blank) for EMIs<br />11th Sakai Conference - June 15-17, 2010<br />6<br />
  11. 11. EMIs<br />Advantages<br />More discriminating<br />Reproducibility using one-third fewer items<br />Lower probability of guessing the correct answer<br />Functional distractors<br />Reusable by just writing a new stem/item<br />Disadvantages<br />Analysis, cannot test all content<br />Not much software support yet<br />
  12. 12. EMIs in Tests & Quizzes<br />What we used to do<br />Work around using fill-in-the-blank<br />So… :<br /> Work underway to add support for EMIs as a new question type in Tests & Quizzes, including statistics and analysis.<br /> When complete and reviewed by Stanford, should become available in a future T&Q release.<br />
  13. 13. Beyond Question Pools<br />Lifecycle of a question<br />Now: authored, lives in a Question Pool, delivered in assessments<br />Should be: created, reviewed, tagged (metadata), selected for assessments, delivered, updated with delivery history (e.g. difficulty, discrimination), updated, shared, reused <br />11th Sakai Conference - June 15-17, 2010<br />9<br />
  14. 14. We imagine a Question Bank which…<br />Supports A-type (traditional) MCQs, EMIs, F-format questions (progressive sequence) <br />Supports embedded rich media (image/video) <br />Supports authoring/revision/approval workflow<br />Includes flexible question metadata<br />Enables blueprinting of assessments by selecting sets of questions to a target level of difficulty<br />Records statistics from the delivery history of the question<br />Enables internal and external reuse and sharing<br />11th Sakai Conference - June 15-17, 2010<br />10<br />
  15. 15. Sharing MCQs<br />11th Sakai Conference - June 15-17, 2010<br />11<br />Question BankUniversity A<br />Question Bank<br />UCT<br />Public OER Question Bank<br />Assessment engine (e.g. Sakai T&Q)<br /> << Questions<br /> >> Delivery history <br /> and statistics<br />Question Bank University B<br />
  16. 16. We’d love to hear from you!<br />If you are now or anticipate using Extended Matching Items for online assessment<br />If you’d like to find ways to increase sharing and reuse of assessment items.<br />If the type of Question Bank we’ve envisaged sounds attractive to you.<br />If you want to help build a Question Bank tool / application.<br />11th Sakai Conference - June 15-17, 2010<br />12<br />
  17. 17. Contact us<br /><br /><br />11th Sakai Conference - June 15-17, 2010<br />13<br />