Writing MCQs: traps to avoid and tips to help you write better MCQs, Institute of Veterinary, Animal and Biomedical Sciences Teaching and Learning Workshop, July 3, 2014, Massey University, Palmerston North, New Zealand

1. Writing MCQs
Liz Norman
Massey University

2. Why worry about item writing flaws?
• Affect 36-65% of questions
• 10-15% are serious enough to influence pass-
fail decisions
• high-achieving students were more likely than
borderline students to be penalised by flawed
items.
• reducing item flaws does improve the
performance of MCQs

3. Avoiding irrelevant difficulty

4. Focus on one thing per Q
• One thing
• Something important – common or catastrophic
• Application not recall
• Something that would happen in real life
• NOT something tricky or overly complex

5. Avoid double options
2-year-old male neutered Border Collie is presented
with the following history and neurologic signs: ……..
Which one of the following neuroanatomic localizations
and diagnoses is CORRECT?
A. Left C1-C5 myelopathy, intervertebral disk rupture
B. Right C6-T2 myelopathy, fibrocartilagenous
embolism
C. Right C1-C5 myelopathy, intervertebral disk rupture
D. Left C1-C5 myelopathy, fibrocartilagenous
embolism

6. Keep options short
Iris prolapse is a common sequel to penetrating corneal
wounds or ruptured corneal ulcers. Which of the following
steps is NOT appropriate for the treatment of iris prolapse?
A. primary closure of the corneal laceration with 8-0 vicryl
and treatment with topical antibiotics to control
infection.
B. placement of a nictitans flap and treatment with
systemic antibiotics to control infection.
C. placement of a corneal graft with an overlying
conjunctival pedicle graft and treatment with systemic
antibiotics to control infection.

7. A 7-year-old mare has had intermittent signs of moderately
severe colic for the past 48 hours. Heart rate is 56
beats/min. Hydration, acid-base balance, and electrolyte
parameters are near normal. On rectal examination, the left
dorsal and ventral colon feels distended and is felt coursing
in a dorsocranial direction. The spleen is displaced
caudomedially. Which of the following is the most likely
diagnosis?
A. Cecocolic intussusception or cecal inversion
B. Displacement of the left colon over the nephrosplenic
ligament
C. Ileocecal intussusception
D. Infarction of the large colon
E. Volvulus of the large colon and the cecum

8. Pose a clear task
Chyle is :-
A. The semifluid mass which empties from the
stomach into the duodenum.
B. The lymph containing fat droplets found in the
lacteals of the small intestine.
C. The contents of the gall bladder.
D. The rounded piece of chewed food which passes
down the oesophagus when the animal swallows.

9. Make all distractors plausible and
homogenous
Which of the following statements regarding hepatic
encephalopathy is true?
A. Patients typically present with asymmetrical neurological deficits
B. The most effective and appropriate anticonvulsant to use for a patient
that is seizuring due to hepatic encephalopathy is phenobarbital
C. Abdominal radiographs of dogs with portosystemic shunts will often
show an enlarged liver
D. Cats with portosystemic shunts often exhibit ptyalism as a clinical
sign
E. An appropriate treatment for hepatic encephalopathy is intravenous
neomycin

10. Include only ONE best answer
Which of the following is true?
A. Bananas are green
B. Motorbikes are faster than cars
C. Boys are taller than girls

11. Include only ONE best answer
Least
correct
Most
correct
D E AC B

12. Avoid none of the above
Which of the following is true regarding ligament injuries?
A. Ligament injuries are appropriately referred to as
“strains” or “sprains”.
B. Surgical intervention is indicated for treatment of
second-degree sprains with demonstrable instability.
C. The elastic nature of ligaments allows 30% elongation
before permanent deformation.
D. Following surgical repair of ligaments, immobilization
via ESF or external coaptation is contraindicated, as
range of motion is critical to successful repair.
E. None of the above.

13. Avoid negative framing
Which of the following statements is false regarding
arthrotomies?
A. When detachment of a ligament is necessary, this should be
performed by osteotomy of the bony origin rather than
transection of the ligament.
B. Complete closure of the synovium is necessary to prevent
synovial fluid leakage into subcutaneous tissue.
C. Surgical removal of osteophytes is often followed by their
relatively rapid regrowth, and has questionable value.
D. Monofilament absorbable suture material has a lower risk for
long term infection than does braided nonabsorbable suture.
E. None of the above.

14. Three options are enough
• Most 4- and 5-option MCQs actually function as
3-option anyway
• 3 options questions:
– discriminate as well or better
– question difficulty changes but this can be
compensated for
– reliability is as good or better
– may allow more questions per testing time –
increasing reliability and validity
– reduces exposure to “incorrect” options

15. But what about guessing?
• Possible significance not been borne out empirically:
– multiple studies and meta-analysis
– across many disciplines including medicine and nursing
• Would be detectable in post-test analysis if it
happened
• Probability modelling doesn’t apply because
guessing is not truly random
• Students do not do substantially better on tests
using three-option formats.

16. 3 options is enough
– but sometimes you do need more
A horse suffering from an acute intestinal accident
is MOST likely to have
A. primary respiratory acidosis
B. primary respiratory alkalosis
C. primary metabolic alkalosis
D. primary metabolic acidosis

17. Avoiding irrelevant ease
“testwiseness”

18. Flaws to watch out for include
• Grammatical cues:
– one or more distractors don’t follow grammatically
from the stem
• Logical cues:
– a subset of the options are collectively exhaustive
• Absolute and vague terms:
– terms such as “always”, “never” or “all” used in
options.
– terms such as “usually” and “frequently” used in
options.

19. Flaws to watch out for include
• Long correct answer:
– correct answer is longer, more specific, or more
complete than other options
• Word repeats:
– a word or phrase is included in the stem and in the
correct answer
• Convergence:
– the correct answer includes the most elements in
common with the other options

20. Flaws to watch out for include
• Numeric data not stated consistently
• Options are in an illogical order
• Stems (lead-ins) are tricky or unnecessarily
complicated

21. Eurodollars refers to dollar deposits when the
depository bank is located in
a) Europe.
b) Europe, and the Caribbean.
c) Outside the United States.
d) United States.

22. The gold standard still has ardent supporters who
believe that it provides
a) an effective hedge against price inflation.
b) fixed exchange rates between all currencies.
c) monetary policy autonomy.
d) the world economy can be subject to
inflationary pressure without changes in the
supply of monetary gold.

23. During a foot series the shoes should _____ be
removed
a) always (apart from laminitics)
b) never
c) sometimes
d) remain in place until the foot balance views are
taken then
e) if due for shoeing

24. Floating-rate notes (FRN)
a) always trade at par value.
b) have contractually specified coupon payments,
therefore they are fixed rate bonds.
c) appeal to investors with strong need to preserve
the principal value of the investment should they
need to liquidate prior to the maturity of the bonds.
d) They experience very volatile price changes
between reset dates.

25. Sedation for equine radiography is
a) never needed
b) only needed for foals
c) usually needed
d) only needed for adult horses

26. Which of the following statements about hedging a
stock portfolio with stock index futures is NOT true?
a) Futures contracts magnify gains (or losses) on the
stock portfolio
b) In a declining market, futures contracts help offset
losses on the portfolio
c) A risk-taker would probably not hedge the entire
portfolio with stock index futures
d) All of the above are true

27. Writing guidelines

28. Item writing guidelines
• Test application of knowledge not recall
• Don’t test “trivial” knowledge
• Focus on real life problems
• Use novel material - clinical vignettes are good
as long as they are realistic
• Use a blueprint

29. Item writing guidelines
• Minimize the amount of reading in each item
• Avoid unnecessary words
• Use correct grammar, punctuation,
capitalization, and spelling.
• Edit and proof items

30. Item writing guidelines
• Include the central idea in the stem instead of
the choices.
• Ensure that the directions in the stem are very
clear.
• Word the stem positively, avoid negatives such
as NOT or EXCEPT. If negative words are used,
use the word cautiously and always ensure that
the word appears capitalized and boldface

31. Item writing guidelines
• Place choices in logical or numerical order.
• Keep choices independent; choices should not
be overlapping.
• Keep choices homogeneous in content and
grammatical structure.
• Keep the length of choices about equal.
• Make sure that only one answer is right or
clearly better than any other
• Make all distractors plausible

32. Sample lead in and option lists for
clinical vignettes
• Which of the following is (abnormal)?
Options sets could include sites of lesions; list of nerves;
list of muscles; list of enzymes; list of hormones; types
of cells; list of neurotransmitters; list of toxins,
molecules, vessels, spinal segments.
• Which of the following findings is most likely?
Options sets could include list of laboratory results; list
of additional physical signs; autopsy results; results of
microscopic examination of fluids, muscle or joint tissue;
DNA analysis results; serum levels.
Constructing Written Test Questions For the Basic and Clinical Sciences, 3rd edn, 2001, National Board of Medical
Examiners (NBME), USA http://www.nbme.org/publications/item-writing-manual.html

33. Sample lead in and option lists for
clinical vignettes
• Which of the following is the most likely cause?
Options sets could include list of underlying mechanisms of the
disease; medications that might cause side effects; drugs or drug
classes; toxic agents; hemodynamic mechanisms, viruses,
metabolic defects.
• Which of the following should be administered?
Options sets could include drugs, vitamins, amino acids, enzymes,
hormones.
• Which of the following is
defective/deficient/nonfunctioning?
Options sets could include list of enzymes, feedback mechanisms,
endocrine structures, dietary elements, vitamins.
http://www.nbme.org/publications/item-writing-manual.html

34. Reviewing MCQs

35. Reviewing MCQs
• Peer review
– Both by subject experts and non-subject experts
• Evaluate Q performance after the exam
– Q difficulty – how many got the Q right
– Discrimination - How high performing and low
performing students answered each Q
– Utility of distracters

36. Evaluate Q
performance

38. “Effective item writers are trained,
not born”
Steven M Downing (2006) Twelve steps for effective test development, In: Downing
SM and Haladyna TM (Eds), Handbook of Test Development, Lawrence Erlbaum
Associates, Mahwah, NJ, USA, p.11

39. http://www.slideshare.net/liznorman