Advancing Online Assessment in Medical Education

  • 613 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
613
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
11
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Advancing Online Assessment in Medical Education
    Greg Doyle, Faculty of Health Sciences
    Stephen Marquard, Centre for Educational Technology
    (Advised and enthused by Rachel Weiss, Faculty of Health Sciences)
    University of Cape Town
    June 2010
  • 2. Overview
    Context
    About Extended Matching Items
    EMIs in Tests & Quizzes
    Managing MCQs in a Question Bank
    Sharing MCQs between institutions
    2
    11th Sakai Conference - June 15-17, 2010
  • 3. UCT Health Science Initiatives
    • Improve assessment practices
    • 4. Introducing the use of Extended Matching Item (R-type) MCQs
    • 5. A Question Bank application
    • 6. Author, manage, reuse and share MCQs
    • 7. Collaborating to develop and share MCQs
    3
    11th Sakai Conference - June 15-17, 2010
  • 8. What is an EMI?
    Extended Matching Item, aka R-type MCQ
    EMI = R-Type MCQ (vs A-Type MCQ)
    An EMI has:
    A theme
    A list of options (text or labelled image)
    A lead-in statement
    A list of items
    11th Sakai Conference - June 15-17, 2010
    4
  • 9. Sample EMI
    11th Sakai Conference - June 15-17, 2010
    5
    This example shows options labelled1, 2, 3… but more conventionally should be labelled a, b, c …
  • 10. EMI variants
    Items may require choosing:
    One option
    Multiple options (e.g. a, e, g) independent of order [in previous example, options 1,5,7]
    Options in a specific order (e.g. g, a, e)
    Implications for grading
    Limits the ability to use older question types (e.g. fill-in-the-blank) for EMIs
    11th Sakai Conference - June 15-17, 2010
    6
  • 11. EMIs
    Advantages
    More discriminating
    Reproducibility using one-third fewer items
    Lower probability of guessing the correct answer
    Functional distractors
    Reusable by just writing a new stem/item
    Disadvantages
    Analysis, cannot test all content
    Not much software support yet
  • 12. EMIs in Tests & Quizzes
    What we used to do
    Work around using fill-in-the-blank
    So…http://jira.sakaiproject.org/browse/SAK-17243 :
    Work underway to add support for EMIs as a new question type in Tests & Quizzes, including statistics and analysis.
    When complete and reviewed by Stanford, should become available in a future T&Q release.
  • 13. Beyond Question Pools
    Lifecycle of a question
    Now: authored, lives in a Question Pool, delivered in assessments
    Should be: created, reviewed, tagged (metadata), selected for assessments, delivered, updated with delivery history (e.g. difficulty, discrimination), updated, shared, reused
    11th Sakai Conference - June 15-17, 2010
    9
  • 14. We imagine a Question Bank which…
    Supports A-type (traditional) MCQs, EMIs, F-format questions (progressive sequence)
    Supports embedded rich media (image/video)
    Supports authoring/revision/approval workflow
    Includes flexible question metadata
    Enables blueprinting of assessments by selecting sets of questions to a target level of difficulty
    Records statistics from the delivery history of the question
    Enables internal and external reuse and sharing
    11th Sakai Conference - June 15-17, 2010
    10
  • 15. Sharing MCQs
    11th Sakai Conference - June 15-17, 2010
    11
    Question BankUniversity A
    Question Bank
    UCT
    Public OER Question Bank
    Assessment engine (e.g. Sakai T&Q)
    << Questions
    >> Delivery history
    and statistics
    Question Bank University B
  • 16. We’d love to hear from you!
    If you are now or anticipate using Extended Matching Items for online assessment
    If you’d like to find ways to increase sharing and reuse of assessment items.
    If the type of Question Bank we’ve envisaged sounds attractive to you.
    If you want to help build a Question Bank tool / application.
    11th Sakai Conference - June 15-17, 2010
    12
  • 17. Contact us
    stephen.marquard@uct.ac.za
    gregory.doyle@uct.ac.za
    11th Sakai Conference - June 15-17, 2010
    13