Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Design of multiple choice questions


Published on

MCQ Crafting skills

Published in: Education
  • Be the first to comment

Design of multiple choice questions

  1. 1. L P U Resource Person Er. Vivek SRIVASTAVA Assistant Professor School of Agriculture Lovely Professional University Workshop on Design of Multiple Choice Questions
  2. 2. Learning Outcomes • To exemplify the principles of designing good Multiple Choice Questions for assessing Higher Order Thinking Skills (HOTS) • To Identify the pitfalls in MCQ setting • To prepare sample MCQs for testing memory, understanding, application, analysis, evaluation and creativity © LPU :: Vivek Srivastava
  3. 3. Points to Ponder… 1. Recap on principles of designing MCQs 2. Identification of pitfalls in MCQ design and rectification of errors 3. Design of sample Multiple Choice Questions © LPU :: Vivek Srivastava
  4. 4. Well, that’s the whole point! •In order to ask a question, you must know something about the subject and so you must study it. •By studying and forming practice questions in your mind and on paper, you’ll begin to know that subject from a different point of view and better retain what you learn. You may be wondering how you can design up a classic MCQ’s? © LPU :: Vivek Srivastava
  5. 5. Guidelines for Designing MCQ’s “ Written properly, multiple choice exams correlate strongly with assessments by descriptive tests ” (Brown, Robert, “Multiple Choice Versus Descriptive Tests”, Frontiers in Education Conference, Reno NC 2001.) © LPU :: Vivek Srivastava
  6. 6. What is a multiple choice question(MCQ)? • According to Clegg and Cashin (1986), a multiple choice question contains three elements: – Item stem: this is the problem statement. It contains all the information necessary to answer the multiple choice question. – Correct option: this is the correct answer to the multiple choice question. – Distractors: these are the incorrect answers presented as alternatives to the correct answer in a multiple choice question. © LPU :: Vivek Srivastava
  7. 7. Sample Multiple Choice Question • Which of the following pH values represents a neutral condition ? a) 5.0 b)1.0 c) 6.0 d)7.0 • Item stem: “Which of the following pH values represents a neutral condition ? • Correct answer: d • Distractors: a,b,c © LPU :: Vivek Srivastava
  8. 8. Advantages of MCQs • Easy to score • Can tap in to different cognitive levels • Useful for diagnosing student difficulties (e.g., when you use common errors as distractors) • Provides an excellent basis for post-test discussion (you should discuss why distractors are wrong) • Requires relatively little time to answer © LPU :: Vivek Srivastava
  9. 9. Limitations of MCQs • Items are open to misinterpretation by students • It can be time-consuming to construct “good” test items • It is difficult to create questions that tap in to higher levels of learning (i.e., Evaluation) • Creativity cannot easily be tested • They encourage guessing © LPU :: Vivek Srivastava
  10. 10. Use of Multiple Choice Exams Not limited to Factual Knowledge – Making inferences – Application of models – Evaluation © LPU :: Vivek Srivastava
  11. 11. Multiple Choice vs. Descriptive Examinations DescriptiveDescriptive QuestionsQuestions MultipleMultiple ChoiceChoice Setting the examSetting the exam EasyEasy DifficultDifficult Grading TaskGrading Task DifficultDifficult EasyEasy Grading TimeGrading Time LongLong ShortShort Grade ConsistencyGrade Consistency VariesVaries HighHigh © LPU :: Vivek Srivastava
  12. 12. MCQ design principles and purpose(AV 1) © LPU :: Vivek Srivastava
  13. 13. Structure of a Multiple Choice Question Stem Answers © LPU :: Vivek Srivastava
  14. 14. PRINCIPLES FOR MULTIPLE- CHOICE QUESTIONS A. The stem* of the question should present a clear and complete problem, (e.g., “Who wrote the Declaration of Independence?”) B. The questions should not clue the students to the correct answer*: 1. The stem of the question should agree grammatically with each choice. 2. The options* should be similar in length and amount of detail. 3. All options should seem plausible to the uninformed. © LPU :: Vivek Srivastava
  15. 15. C. If the question is controversial, an authority or theory should be referred to as support, (e.g., “According to most German historians, which of the following caused World War II?”) D. It is generally suggested that negative words be avoided in the question stem. If included, any negative word should be capitalized and/or underlined and bolded. © LPU :: Vivek Srivastava
  16. 16. E. Each option should concisely state only one idea. F. The option “All of the above/None of the above” should not be used, © LPU :: Vivek Srivastava
  17. 17. G. All options should be as homogenous or similar as possible. H. Avoid complex multiple-choice formats, (e.g., both A and B, both B and C, etc.) I. Order the options to each question alphabetically J. List options vertically, not horizontally. K. The stem of a question should be written as a question, not as an incomplete sentence © LPU :: Vivek Srivastava
  18. 18. • 1. Which of the following is not a mammal? • a. Horse • b. Whale • c. Iguana • d. Human © LPU :: Vivek Srivastava
  19. 19. S. No Principles Violated Possible Corrections 1 ????? ?????? © LPU :: Vivek Srivastava
  20. 20. S. No Principles Violated Possible Corrections 1 Use of negative words, Options are not homogenous , Order of the options to each question are not alphabetically arranged Which of the Following is a Mammal? a.Sea Horse b.A Shark c.A Whale © LPU :: Vivek Srivastava
  21. 21. A verbal question: The forbidden fruit that Eve offered Adam was an: (a) apple (b) pear (c) banana (d) melon (e) carrot © LPU :: Vivek Srivastava
  22. 22. Scientific name of maize is? a) Zea zea b) Xea mea c) Zea mays d) Pommace e) Prunello © LPU :: Vivek Srivastava
  23. 23. Use related, but different terms…. Instrument use to measure intensity of rainfall is: (a) Rain meter (b) Rain stick (c) Rain scale (d) Rain gauge (e) Rainimeter © LPU :: Vivek Srivastava
  24. 24. Formulating Good Multiple Choice Questions • Wording the stem so that there is a single correct answer • Finding plausible wrong answers © LPU :: Vivek Srivastava
  25. 25. Stems Statements “A river that flows through a desert is called..” Questions “ Which of the following descriptions best fits the character of asian grapes?” Negative voice “ Which of the following vitamins is not present in guava? © LPU :: Vivek Srivastava
  26. 26. Stems Conditional Voice “ If you want a to acquire vitamin C that will assist your body requirement which fruit you will select in your diet:” © LPU :: Vivek Srivastava
  27. 27. Answer Options • Single correct answer options • Collective answer options: “ (a) and (c) are both correct ” “ all of the above are correct ” “ none of the above are correct ” © LPU :: Vivek Srivastava
  28. 28. Plausible Answer Strategies • Good questions contain several plausible answer options • Grey area between designing “tricky” questions and providing answer options that test knowledge thoroughly. • Inadvertently giving away the answers © LPU :: Vivek Srivastava
  29. 29. MCQs…..Classifying (AV 2) © LPU :: Vivek Srivastava
  30. 30. Creating variance in MCQs © LPU :: Remembering & Understandin g Applying & Analysing Evaluating & Creating Facts Remembering of previously learned material Ability to use learned material in new and concrete situations Ability to break down material into its component parts Concepts Ability to grasp the meaning of material Apply rules, methods, concepts, principles, laws, and/or theories To understand Organisational Structure Principles Memory based Apply laws and theories to practical situations Distinguish between facts and inferences Procedures Know common terms, Know specific facts Apply Knowledge and mathematical calculations Evaluate the relevancy of data
  31. 31. Creating variance in MCQs- Stems © LPU :: Remembering & Understanding Applying & Analysing Evaluating & Creating Facts Who is the father of genetics? Referring to the chart below which continent is most populous? Which scientist has greatest role in green revolution? Concepts When it is most appropriate to use optical sensor over NIR sensor for crop stress modelling? To set FCC in a satellite image which band combinations has to be used? Which of the following is an essential plant nutrient? Principles Principle of triangulation is used in this precision agriculture tool? How does gravitational pull of sun and moon affects the ocean? How can you optimize shutter speed and aperture when taking a picture in low light? Procedures In precision farming for acquiring maximum information on crop water content what wavelength range a farmer has to choose. How would providing excess irrigation affect rise in the levels of electrical conductivity? Which amendment is best suited for reclamation of saline sodic soils?
  32. 32. MCQs…..Classifying further (AV 3) © LPU :: Vivek Srivastava
  33. 33. MCQs… Writing Stems (AV 4) © LPU :: Vivek Srivastava
  34. 34. Evaluating Multiple Choice Questions Difficulty Factor = R/N R = # of times the correct answer is selected N = # of people taking the exam 0.5 + 10 Too EasyToo Difficult © LPU :: Vivek Srivastava
  35. 35. Difficulty Factor Values greater than 0.8: – The question is too easy – Stem wording gives away the answer – Not enough plausible incorrect answer options Values less than 0.2: – Guessing effect can be a factor – The question is defective – Material wasn’t covered – Stem wording is unclear, confusing – Answer options need work © LPU :: Vivek Srivastava
  36. 36. Evaluating Multiple Choice Questions Discrimination Index = Proportion of capable students that select an answer option – proportion of weak students that select the answer option i.e. D = Ftopquarter – Fbottomquarter 0 + 1- 1 Good Discrimination Reverse Discrimination © LPU :: Vivek Srivastava
  37. 37. Discrimination Index For the Correct Answer Option Values ~ 0.5 give good discrimination < 0.2 consider dropping negative… drop the question For Incorrect Answer Options Values should be negative. © LPU :: Vivek Srivastava
  38. 38. Knowledge or Successful Guessing? Multiple Choice Exam Strategies – Improve odds by eliminating 1 or more infeasible or unlikely answer options © LPU :: Vivek Srivastava
  39. 39. Adjustment for Guessing Negative Marking… – Elimination strategy reduces odds of wrong answer penalty – Subtracting a percentage of the number of wrong answers obtained from the final grade – Give a grade of 4 for a correct answer and a score of -1 for a wrong answer on a 4 choice question © LPU :: Vivek Srivastava
  40. 40. MCQs…. Designing Options (AV 5) © LPU :: Vivek Srivastava
  41. 41. Sample Verbs Knowledge – Cite, define, identify, label, list, match, name, recognize, reproduce, select, state Comprehension – Classify, convert, describe, distinguish between, explain, extend, give examples, illustrate, interpret, paraphrase, summarize, translate Application – Apply, arrange, compute, construct, demonstrate, discover, modify, operate, predict, prepare, produce, relate, show, solve, use © LPU :: Vivek Srivastava
  42. 42. Sample Verbs continued Analysis – Analyze, associate, determine, diagram, differentiate, discriminate, distinguish, estimate, infer, order, outline, point out, separate, subdivide Synthesis – Combine, compile, compose, construct, create, design, develop, devise, formulate, integrate, modify, organize, plan, propose, rearrange, reorganize, revise, rewrite, tell, write Evaluation – Appraise, assess, compare, conclude, contrast, criticize, discriminate, evaluate, judge, justify, support, weigh © LPU :: Vivek Srivastava
  44. 44. The challenge… Major Challenges •Effective Question Design •Errors/Special Characters etc. •Wrong Question Types •Question Duplication •Disproportionate Question Framing w.r.t marks © LPU :: Vivek Srivastava
  45. 45. Design of sample test papers… • All participants to design sample test paper of the course assigned and on the format provided • Time Allotted : © LPU :: Vivek Srivastava
  46. 46. Get Set Go!!! Gear up Fasten your seat belts Explore your question designing abilities Build futuristic assessment solutions… © LPU :: Vivek Srivastava
  47. 47. Queries if any… © LPU :: Vivek Srivastava