Quality of Medical Examination Questions - Sanjoy Sanyal
•Download as PPTX, PDF•
4 likes•890 views
This workshop was conducted by Dr Sanjoy Sanyal in Texila American University, Georgetown, Guyana, on 24 July 2015. it deals with the Dos and Dont's of creating good-quality USMLE-style examination questions.
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Quality of Medical Examination Questions - Sanjoy Sanyal
Similar to Quality of Medical Examination Questions - Sanjoy Sanyal (20)
More from Sanjoy Sanyal
More from Sanjoy Sanyal (20)
Recently uploaded
Recently uploaded (20)
Quality of Medical Examination Questions - Sanjoy Sanyal
- 1. { Quality of Medical Examination Questions Dr. Sanjoy Sanyal Associate Dean and Professor Texila American University, Guyana
- 2. Scramble the questions (either manually or through exam software) Number of questions vis-à-vis chapter length Number of questions vis-à-vis chapter importance Standard Format (Heading, Font size, Style, Pagination etc) Signed by Faculty, HoD, Chair of Examination Quality Control, Dean General Aspects
- 4. True-False MCQ Family Options 1, 2, 3 cannot be judged as absolutely true or false A group of experts will not agree with the answer Only option 4 is unambiguous and clearly true
- 5. True-False MCQ Family ‘May’ may be true ‘Usually’ is harder to define Most experts will not agree whether 4th option is true or false
- 6. True-False MCQ Family Each of these options may or may not be associated with VSD So, which is / are the correct answer(s)?????
- 7. Either Stem is unclear And / or Options contain vague terms And / or Options are partially correct A group of experts will have difficulty in arriving at a consensus on the correct answer Flawed True / False MCQ
- 8. T/F MCQ on a Lighter Note!!! It all depends on individual perspective
- 9. Parts of an A-type MCQ Item (1-Best Answer (OBA)) The whole thing is called an Item
- 10. Previous OBA MCQ Analysis Note, Stem is long, Options are short (next slide) This is a Front-loaded Question Item Lead-in Question asks ‘most likely’ diagnosis Other options are not totally wrong It should be possible to diagram all the options in a single continuum from ‘most likely’ to ‘least likely’ This is called Uni-dimensional Question Item
- 11. Item Shape
- 12. Flawed OBA MCQ Inherently flawed: Long Options, Verbs in Options, Small Stem It is a Back-loaded Question Item Too many adjectives – Which is ‘more true’ than the other? Trying to compare apples with oranges Options are heterogenous and Multi-dimensional Options cannot be rank-ordered in a single dimension Item cannot be answered without looking at the options
- 13. T/F vs. OBA MCQ TFA Easy to create Very few items in the real-world are absolutely true or false Correct answer depends on individual perspective Tests recall of isolated fact - Knowledge OBA More difficult and time- consuming to create Most commonly used When structured in a linear rank-ordered fashion, from most likely to least likely, there is no ambiguity in the correct option Tests higher levels of Bloom’s Cognitive Domain – Comprehension, Application, Analysis (next
- 14. OBA MCQ Based on Cognitive Domain of Bloom’s Taxonomy
- 15. OBA MCQ Based on Cognitive Domain of Bloom’s Taxonomy
- 16. 1. Grammatical Errors Options A and C do not follow grammatically or logically from the Stem
- 17. 2. Logical Errors Stem too short Options too long Options A, B, C are collectively exhaustive and include all possibilities Options D and E are partially correct and confusing fillers Options, D, E cannot be rank-ordered in the same dimension as Options A, B, C Back-loaded question
- 18. 3. Absolute Terminology Stem is inconclusive and un-focused Options Are long Contain verbs Are Multi-dimensional (cannot be rank-ordered in a Single- dimension) Contain Absolute terms (Test-wise student will immediately eliminate options C and D)
- 19. 4. Long Correct Answer Option C is longer than others It contains double options Examiners tend to write the correct answers first, and make them long to ensure they are comprehensive and correct Test-wise student immediately notices that
- 20. 5. Word Repeats Types of Word-repeats Actual: ‘Unreal’ in Stem and ‘Derealization’ in Option Metaphorical: The word ‘Bone’ in Stem, and the pre-fix ‘Osteo’ in the Option Both serve as cues to the test-wise student
- 21. 6. Options Long, Complicated Stem may be long or short – depending on purpose Here Stem is superfluously long Options are too long, complicated, double Tests reading speed rather than Cognition
- 22. 7. Inconsistent Numeric Data Options A, B, C are ranges; D, E are specific figures Should be either ranges or figures; not mixed Option C range includes options D and E Test-wise student will eliminate D and E Such Question Items test recall of facts rather than understanding of concepts
- 23. 8. Vague – Poorly Constructed Back-loaded question Vague frequency terms in Options – ‘Usually’, ‘Often’ Multi-dimensional
- 24. 9. Poor Item Construction Correct way to write the Options in this example: A. No follow up was made of non-vaccinated children B. Number of cases was too small C. The trial involved only boys D. Write a new option D Language is not parallel Mismatch between Stem and Options Options are in illogical order
- 25. 10. ‘None of the Above’
- 27. Summary of Technical Item Flaws
- 28. Summary of Technical Item Flaws
- 29. Item construction in General Focus on Important Concepts Don’t waste time on trivial facts Avoid ‘tricky’ or overly complex Items Issues Pertaining to Stem Maximum information should be in the Stem Should be long, in general But, avoid superfluous information Should be answerable without looking at the Options Avoid Negative terms – ‘Except’, ‘Not’ Guidelines for Item Construction
- 30. Issues Pertaining to Options Options should be short Should grammatically consistent and logically compatible with Stem Avoid Absolute terms – ‘Always’, ‘Never’, ‘All’ Avoid – ‘All of the above’, ‘None of the above’ Avoid Vague terms – ‘Usually’, ‘Frequently’ Distractors should be plausible Distractors should be same relative length as correct Option List Options in logical or alphabetical order Guidelines for Item Construction
- 31. Stem: Focused yet sufficiently comprehensive Student should be able to arrive at the answer without looking at the options Lead-in: In the form of a question Keep verbs in the Lead-in question Options: Short Approximately equal length Uniform between each other Uni-dimensional Capable of being rank-ordered from ‘Most likely’ to ‘Least likely’ Do’s of OBA MCQs
- 32. Stem: Too short, inconclusive, illogical, grammatically flawed, contradictory No Stem at all Lead-in In the form of a half-sentence or statement Not in the form of a question Options: Long and / or complex and / or contains verbs Contains Absolute terms – ‘Always’, ‘Never’ Unequal length Non-uniform between each other Multi-dimensional Cannot be rank-ordered from ‘Most likely’ to ‘Least likely’ Shares a common word from the Stem Don’ts of OBA MCQs
- 33. In Stem or Lead-in Each of the following is correct EXCEPT Which of the following statements is NOT correct? Which of the following statements is CORRECT? In the Options All of the above None of the above A and C above B and D above Other Don’ts in OBA MCQs
- 34. Basic Rules – Revisited
- 35. Basic Rules – Revisited
- 36. Options