The document provides guidelines for constructing objective written exam questions, specifically focusing on multiple choice questions (MCQs), short answer questions (SAQs), and true/false questions. Key tips include ensuring clarity and relevance in stems, creating plausible distractors, and basing questions on learning objectives to assess various levels of cognitive understanding. The document also discusses the advantages and limitations of different question types, emphasizing the importance of structured marking schemes for assessment reliability.

1. TIPS FOR CONSTRUCTING OBJECTIVE WRITTEN EXAM QUESTIONS Presented by:Dr. Soha RashedMember of Medical Education DepartmentProfessor of Community MedicineFaculty of Medicine- Alexandria University12.1.2009

2. OBJECTIVE WRITTEN EXAM QUESTIONSMultiple Choice Questions (MCQs) Short Answer Questions (SAQs) Modified Essay Questions (MEQs)True/False Questions (TF) Matching Questions

3. MCQs

4. Guidelines for writing MCQs

5. How MCQ items are constructed?Incomplete statement, direct question, problem situation (clinical scenario), diagram, graph, image, radiologic image, histopathological section, laboratory findings, etc.Lead-in (question)Alternatives (options or choices):Onecorrect answer, and several plausible incorrect/wrong answers. The incorrect responses are called distractors.divert unsure students from the correct answerTo distract those students who are uncertain of the answer

6. Tips for writing clear MCQsWrite simple stems that clearly state the central problem:Present the problem as precisely as possible, and include only the necessary/relevant information.Ensure accuracy in grammar and sentence construction.Use familiar words, and avoid uncommon terminology and abbreviations.Avoid double negatives.

7. The stem should contain as much of the item as possible. If a phrase can be stated in the stem, it should not be repeated in the options.Ideally, the item should be answerable without having to read all of the options (cover test).

8. Be sure that the content of the question is based on a learning objective:What level of cognitive learning outcomes you want to measure?Recall of an isolated fact

9. Understanding

10. Application of knowledge

11. Problem solvingExamplesIn erythropoietin deficiency, you are expected to see the following pattern in RBC morphology:(a) Normocyticnormochromic(b) Microcyticnormochromic(c) Macrocyticnormochromic(d) MicrocytichypochromicWhich of the following hematopoietic factors is produced by the kidney?(a) Rennin(b) Angiotensin(c) Erythropoietin(d) Aldosterone(e) CortisolNote that both examples test knowledge that is important but in an isolated manner. Students only need to have recall-type knowledge to answer correctly (Isolated fact version= Non-contextual MCQ).

12. Contextual MCQ with the same themes (better example)A 55-year-old patient with chronic renal failure undergoing dialysis. He appears to be pale. A full blood count shows the following red cell indices:Hemoglobin: 8.7 gm/dl Hematocrit: 26%MCV: 92 fL (expected range 80-100 fL)MCH: 33 pg (expected range 27-31 pg)MCHC: 33 gm/dl (expected range 32-36 gm/dl)Reticulocyte count: 0.2%Which of the following is the most appropriate therapy?(a) Erythropoietin(b) Ferrous sulphate(c) Folic acid(d) Vitamin B 6(e) Vitamin B 12 (cyanocobalamin)Note the following features in this MCQ:• Combines basic science knowledge with clinical science knowledge• Students need to connect multiple themes (in this case the role of the kidney in erythropoiesis and the changes in RBC morphology)• The lead-in (question) focuses on only one aspect of the condition• The MCQ can be answered without looking at the options (cover test)

13. Mrs. B. is 20 years old and had an IUD inserted a month ago. She came to the health center 2 days ago with vaginal discharge and abdominal and pelvic pain. She reports that she does not have any fevers or chills. What is your plan?A. Gather history, send vaginal cultures, remove the IUD.B. Gather history, send vaginal cultures, follow up in 3 days.C. Gather history, treat with antibiotics for presumptive PID.D. Gather history, send vaginal cultures, treat with antibiotics for PID, and remove the IUD if the woman wishes.

14. Several multiple-choice questions can be created for the same scenario, but these questions must be independent of each other—the answer to one cannot be dependent on the answer to the others.This type of multiple-choice question requires a higher level ofthinking.

16. To test application of knowledge:Clinical scenarios can provide the basis for the question, beginning with:the presenting problem of a patient, followed by:

17. the history (duration of signs and symptoms),

18. physical findings,

19. results of diagnostic studies,

20. initial treatment,

21. subsequent findings, etc.At a dinner, the menu included fried chicken, home fried potatoes, peas, chocolate eclairs, and coffee. Within 2 hours, most of the diners became violently ill, with nausea, vomiting, and abdominal pain. Analysis of the contaminated food is most likely to yield large numbers of which of the following organisms?A. Escherichia coliB. Salmonella typhimuriumC. Staphylococcus aureusD. Streptococcus faecalisE. Enterococcus

22. AnatomyFor each patient with the following neurologic abnormalities, select the artery(ies) that is (are) most likely to be involved.1. A 72-year-old right-handed man has weakness and hyperreflexia of the right lower limb, an extensor plantar response on the right, normal strength of the right arm, and normal facial movements. *A. Left anterior cerebral arteryB. Right anterior cerebral arteryC. Left middle cerebral arteryD. Right middle cerebral artery2. A 68-year-old right-handed man has right spastic hemiparesis, an extensor plantar response on the right, and paralysis of the lower two-thirds of his face on the right. His speech is fluent, and he has normal comprehension of verbal and written commands. A. Left posterior cerebral arteryB. Right posterior cerebral artery*C. Left lenticulostriate arteriesD. Right middle cerebral artery E. Right lenticulostriate arteriesWhat area is supplied with blood by the left anterior cerebral artery?

23. AnatomyA middle-aged male complains of difficulty climbing the stairs. He describes weakness without pain in his right lower limb. He is able to place his right leg on each step without experiencing any problem, but has difficulty climbing the step, and must grasp the hand-rail to pull himself up. Climbing the next step with his left leg occurs normally. You also notice that his gait on a flat surface appears nearly normal; there is no weakness in extending the right knee against a considerable load. You suspect damage and/or malfunction in theA. Obturator nerveB. Tibial nerveC. Superior gluteal nerveD. Femoral nerveE. Inferior gluteal nerve (correct option)

24. Avoid using negative terms in the stem (such as no, never, none, and not) because students sometimes misread or misunderstand them. If you must use them, bold-face and/or underline the term to draw attention to it.Example: “Which of the following is not the. . . .”Example: “Which of the following is NOT the. . . .”

25. Provide a coherent list of possible answers:Ideal number is 4-5 options.Use similar style and keep the responses similar in length. The list of responses should be grammatically consistent in form. Use an “a/an” at the end of your stem to ensure that each of the responses will complete the statement correctly, regardless of whether or not the first word of each response begins with a vowel.

26. Write reasonable distractors: Avoid obviously wrong or silly distractors. Use distractors that are clear⎯they are not meant to be trick questions or confuse the student. All distractors should be of similar length to avoid giving clues to the correct response. Numerical values should be: listed in order, of the same format , with no overlap.You may get ideas for good distractors by looking at previous tests and using students’ most frequent errors.

27. Distractors (Options) should be homogenous:SymptomsSignsDiagnosisRisk factors Treatments

28. Avoid items with more than one correct answers Time consuming and confusing for the student.

29. Too complicated, requiring the examinee to constantly keep the answer code in mind.

30. The possible response combinations introduced a cueing effect that reduced item discrimination and lowered test reliability. Use of “All” and “None of the above” in responses should be avoided as much as possible.

31. Write clear directions that indicate how and where the students are to respond. Should they indicate the correct responses on the test or on a separate form? Is there only one correct answer or one best answer? What is the point value of each item?Example:Directions: Each of the items on this examination is followed by four possible responses. For each item, select the best response. Indicate your answer by circling the appropriate letter next to your answer. Each correct response is worth one point.

32. Recommended practiceUse of blueprint*Context or clinical scenario-based MCQUse a standard checklist prior to submission of MCQInvite peers to review the questionAnalyze MCQ by difficulty and discriminatory indicesEffect and rationaleImproves content validityAssessment of higher order knowledgeEfficient in identifying the problem and providing feedback to the item-writerPeer review will detect fine hidden problemsQuality assurance*Blueprinting refers to the process where test content is carefully planned against the learning objectives. The examination blueprint specifies the objectives that are to be tested in the given examination as well as their relative weight on the examination.

33. Imprecise and Difficult Terms to Avoid in MCQsSeveral terms should be avoided while writing MCQ as they are imprecise, difficult to quantify, and give away clue to the correct answer. Examples of such words are:• Never• Always• Sometime• Generally• Commonly• Usually• Same as• Can be• May be• Can appear• Possible (possibly)

36. Short Answer Questions (SAQ)DescriptionA practical alternative to the long essay question.SAQ is an open ended, semi-structured question format. They require the student to spontaneously respond (provide one or several responses to a question or situation) rather than choose from a selection of responses. A structured, pre-determined marking scheme improves objectivity.

37. Short-Answer Questions SAQThe questions can incorporate clinical scenarios.This type of question assesses higher-level thinking, but is less reliable because a teacher must interpret the responses.They are not as easy to complete or score as MCQ or true-false questions.A similar format is also known as modified essay question (MEQ)

38. Short Answer Questions SAQ

39. Tips for writing short-answer questionsMake the questions clear and easy to understand. Write precise questions so that students know exactly what response is expected.Prepare a structured marking sheet. For each question, list all of the acceptable responses, and be prepared to consider other answers that may be equally acceptable. Make the marking sheet easy to understand so that other teachers can use it if necessary.

40. Write some questions that involve possible multiple responses. This demands more from students and can address broader content.Example:Family planning methods most appropriate for adolescents are:a._______________, b. _______________, c._______________.Write clear directions for short-answer questions. Clearly state if a point will be given for each correct response or if the student must have all the correct responses to obtain one point.

41. Modified essay question (MEQ)e.g., Case vignette (Clinical scenario)You are a medical officer in pediatrics. You are asked to review a one-hour-old baby for increasing respiratory rate and sub-costal retraction. The baby was born at 35 weeks to a 29-year-old mother via elective LSCS. The indication for LSCS was uncontrolled BP. The mother had regular follow-up during her antenatal period. She had gestational diabetes and pre-eclamsia.Question 1: What are the most likely diagnoses? (Name two)Question 2: What are the preliminary investigations that you would like to perform at this point? (Name three)Question 3: For each of the diagnoses list one primary pathophysiological mechanism.Model answer:Question 1: Hyaline membrane disease; transient tachypnea of newborn (TTNB); (two marks)Question 2: Full blood count; chest X-ray; and arterial blood gas (three marks)Question 3: deficiency of surfactant; failure to reabsorb lung fluid (two marks)

42. Advantages• Better content coverage as compared to long essay question• Improved objectivity as the marking scheme can be structured andpredetermined• Less laborious to mark• Higher chance for assessment of clinical reasoningLimitation• If a large amount of knowledge needs to be tested, it is more efficient to use MCQs

43. Practical Tips in Writing SAQChoose a relevant clinical scenario Link the questions directly to the clinical scenario Ensure that questions cannot be answered without the caseSpecify the number of responsesSpecify the mark assigned to each questionIncorporate basic science principles (e.g. pathophysiological , mechanism) in the case scenario

44. True-False Questions

45. Tips for writing good true-false questionsThe language of true-false test items should be clear, concise, and understandable:Avoid words such as more, few, large, and good, because these are relative and may confuse the students. Avoid using negatively stated items (e.g., It is not recommended that. . . .) because students may not read the question accurately and may misunderstand the meaning. Use short statements that contain only one idea.

46. Use statements that are either completely true or completely false: Avoid a statement that is partially true and partially false. True-false items should focus on one idea and should challenge, but not trick, the students.Make the true statements equivalent in length and number to the false statements.

47. Another type of true-false item requires the student to correct false statements to make the statements true. This can be accomplished by underlining one or two key terms and asking the students to change the underlined term(s) if they decide the statement is false.With this type of item, you will need to determine if the revised terms are correct, which adds some effort and subjectivity to the scoring process.

48. Example:Read the following statement. Circle “T” if the statement is true. Circle “F” if the statement is false and write in the blank space the word(s) that would replace the underlined word(s) to make the statement true.T F Spermicide is the family planning method recommended for preventing sexually transmitted infections and HIV. ________ ________

49. Write clear directions that indicate how and where the students are to respond. Should they indicate the correct responses on the test or on a separate form? What is the point value of each item?Example:Directions: For each of the following statements, write a capital T in the block to the left of the number if the statement is true or a capital F if the statement is false. You will receive one point for each correct answer.

50. Matching QuestionsTwo lists of words and phrases that are to be matched. The first list is known as a list of premises, the second, as a list ofresponses. Items within each list should be similar (e.g., possible diagnoses, medications, etc.).

52. Tips for writing good matching questionsFocus the matching question on one subject.Keep the list of statements brief (no more than 5 to 7 statements). If the list is long, students may spend considerable time on one matching item even if they have a rather clear idea of what the response should be. Keep the matching exercise on one page; it is confusing when it runs onto the next page.

53. Arrange the premises and responses in a logical order: If the premises or responses are names or titles, they may be arranged alphabetically. If they are numbers, they may be arranged sequentially. Any logical order can be used.

54. Write clear directions that indicate the basis on which the lists are to be matched and how the student is to respond. Can answers be used more than once? Are the answers to be indicated on the test or on another sheet? Are there distractors? What is the point value of each item?