HOW TO CREATEHOW TO CREATEMULTIPLE CHOICE QUESTIONSMULTIPLE CHOICE QUESTIONSJENNIFER ANN MORROW, PH.D.
09/15/04 Jennifer Ann Morrow, Ph.D. 2What is a multiple choice question(MCQ)?What is a multiple choice question(MCQ)?• Acc...
09/15/04 Jennifer Ann Morrow, Ph.D. 3Sample Multiple Choice QuestionSample Multiple Choice Question• What is the name of t...
09/15/04 Jennifer Ann Morrow, Ph.D. 4Advantages of MCQsAdvantages of MCQs• Easy to score• Can tap in to different cognitiv...
09/15/04 Jennifer Ann Morrow, Ph.D. 5Disadvantages of MCQsDisadvantages of MCQs• Items are open to misinterpretation by st...
09/15/04 Jennifer Ann Morrow, Ph.D. 6Bloom’s TaxonomyBloom’s Taxonomy• B.S. Bloom and colleagues (1956) created ataxonomy ...
09/15/04 Jennifer Ann Morrow, Ph.D. 7KnowledgeKnowledge• This is defined as the remembering of previouslylearned material....
09/15/04 Jennifer Ann Morrow, Ph.D. 8MCQ: KnowledgeMCQ: Knowledge1. What is the capital of Rhode Island?A. Central FallsB....
09/15/04 Jennifer Ann Morrow, Ph.D. 9ComprehensionComprehension• This is defined as the ability to grasp the meaning ofmat...
09/15/04 Jennifer Ann Morrow, Ph.D. 10MCQ: ComprehensionMCQ: Comprehension1. Explain what is meant by the phrase “cogitoer...
09/15/04 Jennifer Ann Morrow, Ph.D. 11ApplicationApplication• This refers to the ability to use learned material in newand...
09/15/04 Jennifer Ann Morrow, Ph.D. 12MCQ: ApplicationMCQ: Application1. If you have 8 $100 bills and you spend $200 onsho...
09/15/04 Jennifer Ann Morrow, Ph.D. 13AnalysisAnalysis• This refers to the ability to break down material into itscomponen...
09/15/04 Jennifer Ann Morrow, Ph.D. 14MCQ: AnalysisMCQ: Analysis1. Compared to laboratory settings, field settingshave mor...
09/15/04 Jennifer Ann Morrow, Ph.D. 15SynthesisSynthesis• This refers to the ability to put parts together to form anew wh...
09/15/04 Jennifer Ann Morrow, Ph.D. 16MCQ: SynthesisMCQ: Synthesis1. Change the below research scenario to make it a desig...
09/15/04 Jennifer Ann Morrow, Ph.D. 17EvaluationEvaluation• This is concerned with the ability to judge the value ofmateri...
09/15/04 Jennifer Ann Morrow, Ph.D. 18MCQ: EvaluationMCQ: Evaluation1. Evaluate the strength of research design describedb...
09/15/04 Jennifer Ann Morrow, Ph.D. 19Rules for Writing MCQsRules for Writing MCQs• Haladyna & Downing (1989) developed a ...
09/15/04 Jennifer Ann Morrow, Ph.D. 20Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Avoid complex multiple...
09/15/04 Jennifer Ann Morrow, Ph.D. 21Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Avoid grammatical clue...
09/15/04 Jennifer Ann Morrow, Ph.D. 22Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Keep the stem simple –...
09/15/04 Jennifer Ann Morrow, Ph.D. 23Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Use plausible distract...
09/15/04 Jennifer Ann Morrow, Ph.D. 24Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Each item should ask o...
09/15/04 Jennifer Ann Morrow, Ph.D. 25Additional Tips for Creating MCQsAdditional Tips for Creating MCQs• Limit the number...
09/15/04 Jennifer Ann Morrow, Ph.D. 26Additional Tips for Creating MCQsAdditional Tips for Creating MCQs• Write the correc...
09/15/04 Jennifer Ann Morrow, Ph.D. 27MOST IMPORTANT RULEMOST IMPORTANT RULEHAVE OTHERS REVIEW YOURQUESTIONS – IF POSSIBLE...
09/15/04 Jennifer Ann Morrow, Ph.D. 28Assignment for 9/22Assignment for 9/22• Create 6 multiple questions based on Dr.Whit...
09/15/04 Jennifer Ann Morrow, Ph.D. 29
Upcoming SlideShare
Loading in...5
×

Howtocreatemultiplechoicequestions

111

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
111
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D.
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D. Some people simplify this taxonomy into 3 subcategories: 1) Understanding (knowledge and comprehension) 2) Application 3) Higher-order cognitive objectives (analysis, synthesis, and evaluation)
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D.
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D. Answer to #1: D – Providence
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D. Correct Answer: C – I think, therefore I am
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D. Correct Answer: D – 62.5%
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D. Correct Answer: A – internal, external
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D. Correct Answer: C – Both A & B
  • How to Create Multiple Choice Questions 09/15/04 Prepared by Jennifer Ann Morrow, Ph.D. Correct Answer: B - Great
  • Howtocreatemultiplechoicequestions

    1. 1. HOW TO CREATEHOW TO CREATEMULTIPLE CHOICE QUESTIONSMULTIPLE CHOICE QUESTIONSJENNIFER ANN MORROW, PH.D.
    2. 2. 09/15/04 Jennifer Ann Morrow, Ph.D. 2What is a multiple choice question(MCQ)?What is a multiple choice question(MCQ)?• According to Clegg and Cashin (1986), amultiple choice question contains three elements:– Item stem: this is the problem statement. It containsall the information necessary to answer the multiplechoice question.– Correct option: this is the correct answer to themultiple choice question.– Distractors: these are the incorrect answers presentedas alternatives to the correct answer in a multiplechoice question.
    3. 3. 09/15/04 Jennifer Ann Morrow, Ph.D. 3Sample Multiple Choice QuestionSample Multiple Choice Question• What is the name of the presenter today?– A. Dr. Marrow– B. Dr. Martin– C. Dr. Morrow– D. Dr. Morton• Item stem: “What is the name of the presentertoday?”• Correct answer: C. Dr. Morrow• Distractors: A, B, & D
    4. 4. 09/15/04 Jennifer Ann Morrow, Ph.D. 4Advantages of MCQsAdvantages of MCQs• Easy to score• Can tap in to different cognitive levels (seeBloom’s Taxonomy)• Useful for diagnosing student difficulties (e.g.,when you use common errors as distractors)• Provides an excellent basis for post-testdiscussion (you should discuss why distractorsare wrong)• Requires relatively little time to answer
    5. 5. 09/15/04 Jennifer Ann Morrow, Ph.D. 5Disadvantages of MCQsDisadvantages of MCQs• Items are open to misinterpretation by students• It can be time-consuming to construct “good” testitems• It is difficult to create questions that tap in tohigher levels of learning (i.e., Evaluation)• Creativity cannot easily be tested• Students may find that items are too picky• They encourage guessing
    6. 6. 09/15/04 Jennifer Ann Morrow, Ph.D. 6Bloom’s TaxonomyBloom’s Taxonomy• B.S. Bloom and colleagues (1956) created ataxonomy which divides cognitive objectives intoa hierarchy of subdivisions from the easiest to themost complex.• Bloom’s Taxonomy– Knowledge– Comprehension– Application– Analysis– Synthesis– Evaluation
    7. 7. 09/15/04 Jennifer Ann Morrow, Ph.D. 7KnowledgeKnowledge• This is defined as the remembering of previouslylearned material. This is the lowest level oflearning.These are the easiest types of MCQs tocreate.• Learning objectives at this level– Know common terms, Know specific facts– Know methods and procedures, Know basic concepts– Know principles• Question verbs: define, list, state, identify,label, name, who? when? where? what?
    8. 8. 09/15/04 Jennifer Ann Morrow, Ph.D. 8MCQ: KnowledgeMCQ: Knowledge1. What is the capital of Rhode Island?A. Central FallsB. MiddletownC. PortsmouthD. Providence
    9. 9. 09/15/04 Jennifer Ann Morrow, Ph.D. 9ComprehensionComprehension• This is defined as the ability to grasp the meaning ofmaterial.You can have respondents do thefollowing:Translate material from one form to another,Interpret material, Estimate future trends• Learning objectives at this level– Understand facts and principles– Interpret verbal material– Translate verbal material to mathematical formulae– Estimate future consequences applied in data• Question verbs: Explain, predict, interpret, infer, summarize,convert, translate, give example, account for, paraphrase x?
    10. 10. 09/15/04 Jennifer Ann Morrow, Ph.D. 10MCQ: ComprehensionMCQ: Comprehension1. Explain what is meant by the phrase “cogitoergo sum”?A. I know, therefore I amB. I am, therefore I knowC. I think, therefore I amD. I am, therefore I think
    11. 11. 09/15/04 Jennifer Ann Morrow, Ph.D. 11ApplicationApplication• This refers to the ability to use learned material in newand concrete situations. Here you have to apply rules,methods, concepts, principles, laws, and/or theories.• Learning objectives at this level– Apply concepts and principles to new situations– Apply laws and theories to practical situations– Solve mathematical problems• Question verbs: How could x be used to y? Howwould you show, make use of, modify, demonstrate,solve, or apply x to conditions y?
    12. 12. 09/15/04 Jennifer Ann Morrow, Ph.D. 12MCQ: ApplicationMCQ: Application1. If you have 8 $100 bills and you spend $200 onshoes and $100 on books what % of the originalamount of money do you have left?A. 24.5%B. 37.5%C. 62.5%D. 86.5%
    13. 13. 09/15/04 Jennifer Ann Morrow, Ph.D. 13AnalysisAnalysis• This refers to the ability to break down material into itscomponent parts so that its organizational structure maybe understood.• Learning objectives at this level– Recognize unstated assumptions– Recognize logical fallacies in reasoning– Distinguish between facts and inferences– Evaluate the relevancy of data• Question verbs: Differentiate, compare / contrast, distinguishx from y, how does x affect or relate to y? why? how? Whatpiece of x is missing / needed?
    14. 14. 09/15/04 Jennifer Ann Morrow, Ph.D. 14MCQ: AnalysisMCQ: Analysis1. Compared to laboratory settings, field settingshave more ______ validity and less ______validity.A. internal, externalB. construct, contentC. external, internalD. content, construct
    15. 15. 09/15/04 Jennifer Ann Morrow, Ph.D. 15SynthesisSynthesis• This refers to the ability to put parts together to form anew whole.• Learning objectives at this level– Write a well organized theme– Propose a plan for an experiment– Integrate learning from different areas into a plan forsolving a problem– Formulates a new scheme for classifying objects• Question verbs: Design, construct, develop, formulate,imagine, create, change, write a short story and label thefollowing elements:
    16. 16. 09/15/04 Jennifer Ann Morrow, Ph.D. 16MCQ: SynthesisMCQ: Synthesis1. Change the below research scenario to make it a designthat will allow you to show cause and effect and beable to generalize to a larger population.Dr. Smith is conducting a study looking a differences in sense ofbelonging of first-year students. He is comparing two groups ofstudents: Learning community and Traditional students.Students self-select into the above groups.A. Use random assignment to groupsB. Use random selection of participantsC. Both A & BD. None of the above
    17. 17. 09/15/04 Jennifer Ann Morrow, Ph.D. 17EvaluationEvaluation• This is concerned with the ability to judge the value ofmaterial for a given purpose. This is the highest level oflearning.• Learning objectives at this level– Judge the logical consistency of written material– Judge the value of a work– Judge the adequacy with which conclusions aresupported by data• Question verbs: Justify, appraise, evaluate, judge x accordingto given criteria. Which option would be better/preferable toparty y?
    18. 18. 09/15/04 Jennifer Ann Morrow, Ph.D. 18MCQ: EvaluationMCQ: Evaluation1. Evaluate the strength of research design describedbelow:Dr. Arnold randomly assigns 100 students who have elected tobe in her study on pilot training. She controls for variousconfounding variables in her study. She utilizes a pretest-posttest control group design.A. Excellent, it has high internal and external validityB. Great, it has high internal validity and moderate externalvalidityC. Good, it has moderate internal validity and moderate externalvalidityD. Poor, it has low levels of both internal and external validity
    19. 19. 09/15/04 Jennifer Ann Morrow, Ph.D. 19Rules for Writing MCQsRules for Writing MCQs• Haladyna & Downing (1989) developed a set ofrules one should follow when constructingMCQs. Some of them are:• Balance the key – this refers to where you placethe correct answer in the response choices. Youshould vary where you place the correct answer.• Avoid the use of “None of the Above” –research has shown that these type of questionsare more difficult and less reliable.
    20. 20. 09/15/04 Jennifer Ann Morrow, Ph.D. 20Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Avoid complex multiple choice items – youshould not have complex response options thatcombine many smaller items or phrases (e.g., a.1, 2, 3; b. 1 and 3; c. 2 and 4…).• Keep the length of responses options fairlyconsistent – all of your response options shouldbe approximately the same length so studentscan’t easily figure out the correct answer just bylooking at the length of the response.
    21. 21. 09/15/04 Jennifer Ann Morrow, Ph.D. 21Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Avoid grammatical clues – don’t use anygrammatical clues that will let students knowwhich is the correct answer (e.g., use a/an and notjust the one for the correct answer).• Avoid the use of humor when developingoptions – you should not use humor whenconstructing your distractors. This eliminates aresponse right away.
    22. 22. 09/15/04 Jennifer Ann Morrow, Ph.D. 22Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Keep the stem simple – don’t add extrainformation to the stem to fill up space and/orconfuse students.• Avoid using negative phrasing in the stem –don’t use negative phrases such as never, not, etc.in the stem. If you have to use negative phrasingthen bold or capitalize the negative word.
    23. 23. 09/15/04 Jennifer Ann Morrow, Ph.D. 23Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Use plausible distractors – all of you optionsshould be logical and related to the material youare trying to test. Don’t put in foolish or nonsensedistractors.• Avoid using “All of the Above” – researchshows that this type of question is more difficultand less discriminating.• Avoid the option “I don’t know” – researchshows that this is not a valid option.
    24. 24. 09/15/04 Jennifer Ann Morrow, Ph.D. 24Rules for Writing MCQs (cont.)Rules for Writing MCQs (cont.)• Each item should ask only one question – keepthe stem simple. Don’t make it possible forstudents to correctly check more than one answer.• Use simple terminology or terminology thatstudents should know – spell out acronymsand/or define difficult-to-understand terms.• Underline, italicize, or use bold print to drawattention to important terms – use thisespecially if you include negative words in thestem.
    25. 25. 09/15/04 Jennifer Ann Morrow, Ph.D. 25Additional Tips for Creating MCQsAdditional Tips for Creating MCQs• Limit the number of options – use 3 or 4options. Avoid using 2 or more than 4.• Don’t lift phrases directly from the text orlecture – this encourages simple recall and notunderstanding.• Make sure questions are not culturally biasedor contain sexist terminology – avoid usingphrases that could be construed as racist or sexist(e.g., calling women girls), or which have acultural bias.
    26. 26. 09/15/04 Jennifer Ann Morrow, Ph.D. 26Additional Tips for Creating MCQsAdditional Tips for Creating MCQs• Write the correct answer before creating thedistractors.• Include only one correct or best answer – don’thave multiple correct answers that students canchoose from.• Put responses vertically below the stem – thismakes it easier to read.• Avoid unnecessary repetition in the responseoptions – if every response starts with the phrase“in the” then put “in the” as part of the stem.
    27. 27. 09/15/04 Jennifer Ann Morrow, Ph.D. 27MOST IMPORTANT RULEMOST IMPORTANT RULEHAVE OTHERS REVIEW YOURQUESTIONS – IF POSSIBLE GETFEEDBACK FROM MORE THAN ONEPERSON. WORK IN TEAMS ANDCRITIQUE EACH OTHERSQUESTIONS
    28. 28. 09/15/04 Jennifer Ann Morrow, Ph.D. 28Assignment for 9/22Assignment for 9/22• Create 6 multiple questions based on Dr.Whittecar’s theme readings. Create 2 questionsthat could be classified as either Knowledge orComprehension questions, 2 questions that couldbe classified as Application questions, and 2questions that could be classified as Analysis,Synthesis, or Evaluation questions.• Bring a typed copy of these to class with you on9/22.
    29. 29. 09/15/04 Jennifer Ann Morrow, Ph.D. 29
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×