How to create multiple choice questions


Published on

This powerpoint goes over Bloom's Taxonomy and how to write multiple choice questions that fit each of the taxonomies

Published in: Education, Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Some people simplify this taxonomy into 3 subcategories: 1) Understanding (knowledge and comprehension) 2) Application 3) Higher-order cognitive objectives (analysis, synthesis, and evaluation)
  • Answer to #1: D – Providence
  • Correct Answer: C – I think, therefore I am
  • Correct Answer: D – 62.5%
  • Correct Answer: A – internal, external
  • Correct Answer: C – Both A & B
  • Correct Answer: B - Great
  • How to create multiple choice questions

    2. 2. What is a multiple choice question(MCQ)? <ul><li>According to Clegg and Cashin (1986), a multiple choice question contains three elements: </li></ul><ul><ul><li>Item stem : this is the problem statement. It contains all the information necessary to answer the multiple choice question. </li></ul></ul><ul><ul><li>Correct option : this is the correct answer to the multiple choice question. </li></ul></ul><ul><ul><li>Distractors : these are the incorrect answers presented as alternatives to the correct answer in a multiple choice question. </li></ul></ul>
    3. 3. Sample Multiple Choice Question <ul><li>What is the name of the presenter today? </li></ul><ul><ul><li>A. Dr. Marrow </li></ul></ul><ul><ul><li>B. Dr. Martin </li></ul></ul><ul><ul><li>C. Dr. Morrow </li></ul></ul><ul><ul><li>D. Dr. Morton </li></ul></ul><ul><li>Item stem: “What is the name of the presenter today?” </li></ul><ul><li>Correct answer: C. Dr. Morrow </li></ul><ul><li>Distractors: A, B, & D </li></ul>
    4. 4. Advantages of MCQs <ul><li>Easy to score </li></ul><ul><li>Can tap in to different cognitive levels (see Bloom’s Taxonomy) </li></ul><ul><li>Useful for diagnosing student difficulties (e.g., when you use common errors as distractors) </li></ul><ul><li>Provides an excellent basis for post-test discussion (you should discuss why distractors are wrong) </li></ul><ul><li>Requires relatively little time to answer </li></ul>
    5. 5. Disadvantages of MCQs <ul><li>Items are open to misinterpretation by students </li></ul><ul><li>It can be time-consuming to construct “good” test items </li></ul><ul><li>It is difficult to create questions that tap in to higher levels of learning (i.e., Evaluation) </li></ul><ul><li>Creativity cannot easily be tested </li></ul><ul><li>Students may find that items are too picky </li></ul><ul><li>They encourage guessing </li></ul>
    6. 6. Bloom’s Taxonomy <ul><li>B.S. Bloom and colleagues (1956) created a taxonomy which divides cognitive objectives into a hierarchy of subdivisions from the easiest to the most complex. </li></ul><ul><li>Bloom’s Taxonomy </li></ul><ul><ul><li>Knowledge </li></ul></ul><ul><ul><li>Comprehension </li></ul></ul><ul><ul><li>Application </li></ul></ul><ul><ul><li>Analysis </li></ul></ul><ul><ul><li>Synthesis </li></ul></ul><ul><ul><li>Evaluation </li></ul></ul>
    7. 7. Knowledge <ul><li>This is defined as the remembering of previously learned material. This is the lowest level of learning.These are the easiest types of MCQs to create. </li></ul><ul><li>Learning objectives at this level </li></ul><ul><ul><li>Know common terms, Know specific facts </li></ul></ul><ul><ul><li>Know methods and procedures, Know basic concepts </li></ul></ul><ul><ul><li>Know principles </li></ul></ul><ul><li>Question verbs: define, list, state, identify, label, name, who? when? where? what? </li></ul>
    8. 8. MCQ: Knowledge <ul><li>What is the capital of Rhode Island? </li></ul><ul><ul><li>Central Falls </li></ul></ul><ul><ul><li>Middletown </li></ul></ul><ul><ul><li>Portsmouth </li></ul></ul><ul><ul><li>Providence </li></ul></ul>
    9. 9. Comprehension <ul><li>This is defined as the ability to grasp the meaning of material.You can have respondents do the following:Translate material from one form to another, Interpret material, Estimate future trends </li></ul><ul><li>Learning objectives at this level </li></ul><ul><ul><li>Understand facts and principles </li></ul></ul><ul><ul><li>Interpret verbal material </li></ul></ul><ul><ul><li>Translate verbal material to mathematical formulae </li></ul></ul><ul><ul><li>Estimate future consequences applied in data </li></ul></ul><ul><li>Question verbs: Explain, predict, interpret, infer, summarize, convert, translate, give example, account for, paraphrase x ? </li></ul>
    10. 10. MCQ: Comprehension <ul><li>Explain what is meant by the phrase “cogito ergo sum”? </li></ul><ul><ul><li>I know, therefore I am </li></ul></ul><ul><ul><li>I am, therefore I know </li></ul></ul><ul><ul><li>I think, therefore I am </li></ul></ul><ul><ul><li>I am, therefore I think </li></ul></ul>
    11. 11. Application <ul><li>This refers to the ability to use learned material in new and concrete situations. Here you have to apply rules, methods, concepts, principles, laws, and/or theories. </li></ul><ul><li>Learning objectives at this level </li></ul><ul><ul><li>Apply concepts and principles to new situations </li></ul></ul><ul><ul><li>Apply laws and theories to practical situations </li></ul></ul><ul><ul><li>Solve mathematical problems </li></ul></ul><ul><li>Question verbs: How could x be used to y ? How would you show, make use of, modify, demonstrate, solve, or apply x to conditions y ? </li></ul>
    12. 12. MCQ: Application <ul><li>If you have 8 $100 bills and you spend $200 on shoes and $100 on books what % of the original amount of money do you have left? </li></ul><ul><ul><li>24.5% </li></ul></ul><ul><ul><li>37.5% </li></ul></ul><ul><ul><li>62.5% </li></ul></ul><ul><ul><li>86.5% </li></ul></ul>
    13. 13. Analysis <ul><li>This refers to the ability to break down material into its component parts so that its organizational structure may be understood. </li></ul><ul><li>Learning objectives at this level </li></ul><ul><ul><li>Recognize unstated assumptions </li></ul></ul><ul><ul><li>Recognize logical fallacies in reasoning </li></ul></ul><ul><ul><li>Distinguish between facts and inferences </li></ul></ul><ul><ul><li>Evaluate the relevancy of data </li></ul></ul><ul><li>Question verbs: Differentiate, compare / contrast, distinguish x from y, how does x affect or relate to y? why? how? What piece of x is missing / needed? </li></ul>
    14. 14. MCQ: Analysis <ul><li>Compared to laboratory settings, field settings have more ______ validity and less ______ validity. </li></ul><ul><ul><li>internal, external </li></ul></ul><ul><ul><li>construct, content </li></ul></ul><ul><ul><li>external, internal </li></ul></ul><ul><ul><li>content, construct </li></ul></ul>
    15. 15. Synthesis <ul><li>This refers to the ability to put parts together to form a new whole. </li></ul><ul><li>Learning objectives at this level </li></ul><ul><ul><li>Write a well organized theme </li></ul></ul><ul><ul><li>Propose a plan for an experiment </li></ul></ul><ul><ul><li>Integrate learning from different areas into a plan for solving a problem </li></ul></ul><ul><ul><li>Formulates a new scheme for classifying objects </li></ul></ul><ul><li>Question verbs: Design, construct, develop, formulate, imagine, create, change, write a short story and label the following elements: </li></ul>
    16. 16. MCQ: Synthesis <ul><li>Change the below research scenario to make it a design that will allow you to show cause and effect and be able to generalize to a larger population. </li></ul><ul><li>Dr. Smith is conducting a study looking a differences in sense of belonging of first-year students. He is comparing two groups of students: Learning community and Traditional students. Students self-select into the above groups. </li></ul><ul><ul><li>Use random assignment to groups </li></ul></ul><ul><ul><li>Use random selection of participants </li></ul></ul><ul><ul><li>Both A & B </li></ul></ul><ul><ul><li>None of the above </li></ul></ul>
    17. 17. Evaluation <ul><li>This is concerned with the ability to judge the value of material for a given purpose. This is the highest level of learning. </li></ul><ul><li>Learning objectives at this level </li></ul><ul><ul><li>Judge the logical consistency of written material </li></ul></ul><ul><ul><li>Judge the value of a work </li></ul></ul><ul><ul><li>Judge the adequacy with which conclusions are supported by data </li></ul></ul><ul><li>Question verbs: Justify, appraise, evaluate, judge x according to given criteria. Which option would be better/preferable to party y ? </li></ul>
    18. 18. MCQ: Evaluation <ul><li>Evaluate the strength of research design described below: </li></ul><ul><li>Dr. Arnold randomly assigns 100 students who have elected to be in her study on pilot training. She controls for various confounding variables in her study. She utilizes a pretest- posttest control group design. </li></ul><ul><ul><li>Excellent, it has high internal and external validity </li></ul></ul><ul><ul><li>Great, it has high internal validity and moderate external validity </li></ul></ul><ul><ul><li>Good, it has moderate internal validity and moderate external validity </li></ul></ul><ul><ul><li>Poor, it has low levels of both internal and external validity </li></ul></ul>
    19. 19. Rules for Writing MCQs <ul><li>Haladyna & Downing (1989) developed a set of rules one should follow when constructing MCQs. Some of them are: </li></ul><ul><li>Balance the key – this refers to where you place the correct answer in the response choices. You should vary where you place the correct answer. </li></ul><ul><li>Avoid the use of “None of the Above” – research has shown that these type of questions are more difficult and less reliable. </li></ul>
    20. 20. Rules for Writing MCQs (cont.) <ul><li>Avoid complex multiple choice items – you should not have complex response options that combine many smaller items or phrases (e.g., a. 1, 2, 3; b. 1 and 3; c. 2 and 4…). </li></ul><ul><li>Keep the length of responses options fairly consistent – all of your response options should be approximately the same length so students can’t easily figure out the correct answer just by looking at the length of the response. </li></ul>
    21. 21. Rules for Writing MCQs (cont.) <ul><li>Avoid grammatical clues – don’t use any grammatical clues that will let students know which is the correct answer (e.g., use a/an and not just the one for the correct answer). </li></ul><ul><li>Avoid the use of humor when developing options – you should not use humor when constructing your distractors. This eliminates a response right away. </li></ul>
    22. 22. Rules for Writing MCQs (cont.) <ul><li>Keep the stem simple – don’t add extra information to the stem to fill up space and/or confuse students. </li></ul><ul><li>Avoid using negative phrasing in the stem – don’t use negative phrases such as never, not, etc. in the stem. If you have to use negative phrasing then bold or capitalize the negative word. </li></ul>
    23. 23. Rules for Writing MCQs (cont.) <ul><li>Use plausible distractors – all of you options should be logical and related to the material you are trying to test. Don’t put in foolish or nonsense distractors. </li></ul><ul><li>Avoid using “All of the Above” – research shows that this type of question is more difficult and less discriminating. </li></ul><ul><li>Avoid the option “I don’t know” – research shows that this is not a valid option. </li></ul>
    24. 24. Rules for Writing MCQs (cont.) <ul><li>Each item should ask only one question – keep the stem simple. Don’t make it possible for students to correctly check more than one answer. </li></ul><ul><li>Use simple terminology or terminology that students should know – spell out acronyms and/or define difficult-to-understand terms. </li></ul><ul><li>Underline, italicize, or use bold print to draw attention to important terms – use this especially if you include negative words in the stem. </li></ul>
    25. 25. Additional Tips for Creating MCQs <ul><li>Limit the number of options – use 3 or 4 options. Avoid using 2 or more than 4. </li></ul><ul><li>Don’t lift phrases directly from the text or lecture – this encourages simple recall and not understanding. </li></ul><ul><li>Make sure questions are not culturally biased or contain sexist terminology – avoid using phrases that could be construed as racist or sexist (e.g., calling women girls), or which have a cultural bias. </li></ul>
    26. 26. Additional Tips for Creating MCQs <ul><li>Write the correct answer before creating the distractors . </li></ul><ul><li>Include only one correct or best answer – don’t have multiple correct answers that students can choose from. </li></ul><ul><li>Put responses vertically below the stem – this makes it easier to read. </li></ul><ul><li>Avoid unnecessary repetition in the response options – if every response starts with the phrase “in the” then put “in the” as part of the stem. </li></ul>
    28. 28. Assignment for 9/22 <ul><li>Create 6 multiple questions based on Dr. Whittecar’s theme readings. Create 2 questions that could be classified as either Knowledge or Comprehension questions, 2 questions that could be classified as Application questions, and 2 questions that could be classified as Analysis, Synthesis, or Evaluation questions. </li></ul><ul><li>Bring a typed copy of these to class with you on 9/22. </li></ul>