Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Advancing Online Assessment in Medical Education


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Advancing Online Assessment in Medical Education

  1. 1. Advancing Online Assessment in Medical Education<br />Greg Doyle, Faculty of Health Sciences <br />Stephen Marquard, Centre for Educational Technology<br />(Advised and enthused by Rachel Weiss, Faculty of Health Sciences)<br />University of Cape Town<br />June 2010<br />
  2. 2. Overview<br />Context <br />About Extended Matching Items<br />EMIs in Tests & Quizzes<br />Managing MCQs in a Question Bank<br />Sharing MCQs between institutions<br />2<br />11th Sakai Conference - June 15-17, 2010<br />
  3. 3. UCT Health Science Initiatives<br /><ul><li>Improve assessment practices
  4. 4. Introducing the use of Extended Matching Item (R-type) MCQs
  5. 5. A Question Bank application
  6. 6. Author, manage, reuse and share MCQs
  7. 7. Collaborating to develop and share MCQs</li></ul>3<br />11th Sakai Conference - June 15-17, 2010<br />
  8. 8. What is an EMI?<br />Extended Matching Item, aka R-type MCQ<br />EMI = R-Type MCQ (vs A-Type MCQ)<br />An EMI has:<br />A theme<br />A list of options (text or labelled image)<br />A lead-in statement<br />A list of items<br />11th Sakai Conference - June 15-17, 2010<br />4<br />
  9. 9. Sample EMI<br />11th Sakai Conference - June 15-17, 2010<br />5<br />This example shows options labelled1, 2, 3… but more conventionally should be labelled a, b, c …<br />
  10. 10. EMI variants<br />Items may require choosing:<br />One option<br />Multiple options (e.g. a, e, g) independent of order [in previous example, options 1,5,7]<br />Options in a specific order (e.g. g, a, e)<br />Implications for grading<br />Limits the ability to use older question types (e.g. fill-in-the-blank) for EMIs<br />11th Sakai Conference - June 15-17, 2010<br />6<br />
  11. 11. EMIs<br />Advantages<br />More discriminating<br />Reproducibility using one-third fewer items<br />Lower probability of guessing the correct answer<br />Functional distractors<br />Reusable by just writing a new stem/item<br />Disadvantages<br />Analysis, cannot test all content<br />Not much software support yet<br />
  12. 12. EMIs in Tests & Quizzes<br />What we used to do<br />Work around using fill-in-the-blank<br />So… :<br /> Work underway to add support for EMIs as a new question type in Tests & Quizzes, including statistics and analysis.<br /> When complete and reviewed by Stanford, should become available in a future T&Q release.<br />
  13. 13. Beyond Question Pools<br />Lifecycle of a question<br />Now: authored, lives in a Question Pool, delivered in assessments<br />Should be: created, reviewed, tagged (metadata), selected for assessments, delivered, updated with delivery history (e.g. difficulty, discrimination), updated, shared, reused <br />11th Sakai Conference - June 15-17, 2010<br />9<br />
  14. 14. We imagine a Question Bank which…<br />Supports A-type (traditional) MCQs, EMIs, F-format questions (progressive sequence) <br />Supports embedded rich media (image/video) <br />Supports authoring/revision/approval workflow<br />Includes flexible question metadata<br />Enables blueprinting of assessments by selecting sets of questions to a target level of difficulty<br />Records statistics from the delivery history of the question<br />Enables internal and external reuse and sharing<br />11th Sakai Conference - June 15-17, 2010<br />10<br />
  15. 15. Sharing MCQs<br />11th Sakai Conference - June 15-17, 2010<br />11<br />Question BankUniversity A<br />Question Bank<br />UCT<br />Public OER Question Bank<br />Assessment engine (e.g. Sakai T&Q)<br /> << Questions<br /> >> Delivery history <br /> and statistics<br />Question Bank University B<br />
  16. 16. We’d love to hear from you!<br />If you are now or anticipate using Extended Matching Items for online assessment<br />If you’d like to find ways to increase sharing and reuse of assessment items.<br />If the type of Question Bank we’ve envisaged sounds attractive to you.<br />If you want to help build a Question Bank tool / application.<br />11th Sakai Conference - June 15-17, 2010<br />12<br />
  17. 17. Contact us<br /><br /><br />11th Sakai Conference - June 15-17, 2010<br />13<br />