Recommended
More Related Content
Similar to Writing challenging multiple choice questions
Similar to Writing challenging multiple choice questions (20)
Recently uploaded
Recently uploaded (20)
Writing challenging multiple choice questions
- 1. Sophie Scott SJL Staff Conference 13th March 2020 “Challenging Students, Challenging Ourselves”
- 4. 1. All distractors must be plausible 1.What is the capital of Eritrea? London Paris Rome Asmara 1.What is the capital of Eritrea? Dakar Funafuti Asmara Papeete
- 5. 1. All distractors must be plausible 1.What is the capital of Eritrea? Abuja Asmara Apia Accra
- 6. 1. All distractors must be plausible Non-volatile memory describes memory that... ... loses its contents when the power is turned off, e.g. RAM. ... loses its contents when the power is turned off, e.g. ROM. ... retains its contents when the power is turned off, e.g. ROM.
- 7. 1. All distractors must be plausible
- 8. 2. Intentionally target misconceptions 1.Why should we include comments in our code? • So that the computer can run the program faster • So that other people can understand our code • To make it easier for the computer to run
- 9. 2. Intentionally target misconceptions
- 10. 3. More options can reduce the chance of a lucky guess
- 11. 3. More options can reduce the chance of a lucky guess 1.Antibiotics can be used to treat COVID19 • True – antibiotics can be used to treat all viruses • True – antibiotics can be used to treat some viruses, including COVID19 • False – antibiotics can be used to treat some viruses, but not COVID19 • False – antibiotics can not be used to treat any viruses
- 12. 4. “Tick all that apply” means that students have to carefully consider each option
- 13. 4. “Tick all that apply” means that students have to carefully consider each option
- 14. 5. “Which is the best answer?” removes the need for all distractors to be completely wrong
- 15. 5. “Which is the best answer?” removes the need for all distractors to be completely wrong
- 16. 6. Ask questions that require more than simple recall Example: Comparisons 1.How did the Soviet totalitarian system under Stalin differ from that of Hitler and Mussolini? • It built up armed forces • It took away human rights • It made trade unions illegal • It abolished private land ownership
- 17. 6. Ask questions that require more than simple recall Example: Consequences 1.Which of the following was a consequence of Stalin’s collectivisation of agriculture in the 1930s? • The production of iron, coal and steel increased significantly • The number of cattle in the USSR decreased significantly • The amount of grain produced increased significantly • The NEP did not give the government sufficient control over the grain supply • Millions of Ukrainians died in a famine
- 18. 7. Include images / graphs / tables / text to involve higher-order thinking skills
- 21. Which component is represented by the orange box? RAM Cache Hard disk drive
- 22. Unintended cues – to check for at the end • More detail in the correct answer • Correct answer always in the same position • Sentence stem doesn’t work in conjunction with all options
Editor's Notes
- How/why do you use MCQs at the moment? Why did you choose this workshop? Multiple choice questions are often thought of as being "easy", but written well they can provide high levels of challenge for students whilst also providing valuable assessment data. In this workshop we will look at how to write challenging and useful multiple choice questions. Can be very precise and focus on one small aspect of a topic Versatile – can be used as starters, plenaries, hinge point questions within a lesson, homework tasks, class tests… Quick to mark (or can be automated) and provide teachers with easy-to-analyse results and the opportunity to identify misconceptions – allowing us to more regularly implement retrieval practice without heavy impact on workload They can feel “low risk” to the students – they know the correct answer is there! Less ambiguity Can be gamified Not just about basic recall of content – can be used to test higher-order skills as well One study (Bjork – mentioned in this morning’s presentation – theory of disuse) has suggested that well-written MCQs aid recall of related information in subsequent cued recall tests more than cued recall practice does Writing the questions requires the teacher to think carefully about misconceptions – useful to do in collaboration with colleagues Unfortunately, whilst the MCQs we might see on exam papers are usually well-constructured by expert writers, many of the MCQs that we find online are not written well at all! ….
- Clearly not a good question – and it’s one of a large number of examples! This is why, in many cases, it’s better to write your own MCQs, so that you can ensure a high quality of question. It’s also excellent professional development, and allows you to tailor the questions carefully to exactly to your students. We’ll look now at a series of things that we can do to ensure that the questions we write are challenging and generate meaningful data about student understanding Of course we may not *always* want MCQs to be too challenging – as we heard from David Didau this morning, it is important that students have a sense of success first before being challenged!
- Terminology: Distractor – incorrect answer option Example taken from AQA Inside Exams podcast It is crucial that the distractors are plausible to students who lack the relevant subject knowledge - ensure that someone who hadn’t studied the subject would only have 25% chance of getting it right (if four options) #Method 1 –– the options need to be similar - in this case all capital city names that are not familiar to most people in the UK
- #Method 2 – choose options that look/sound similar – in this case, the student can’t just rely on “Oh, I remember it began with an A….” Caveat: Even this is limited in its power because students may recognise the correct answer but still not be able to recall it correctly out of thin air later on – Illusion of knowledge – this is why the most powerful MCQs are the ones which are more complex than remembering one-word answers – we will come on to this later!
- #Method 3 (longer options) – choose options which include the same / similar vocabulary
- #Method 3 – distractors all related to the correct answer (they are all organs)
- It can be helpful when writing questions to use a possible misconception as a starting point to determine the question, rather than devising the question and then considering the misconceptions
- Taken from AQA Inside Exams show notes – Annotated MCQs If a learner gets the answer wrong, their answer should show how or why they got it wrong. In other words, what misconception led the student to answer in that way, and furthermore, how can that misconception be addressed? Calculation questions should only have one step, so that if students get the answer wrong you know where they have gone wrong and how. When considering what misconceptions a learner might have, educators can draw on a number of sources, including their own experience of what learners often get wrong, the experience of other educators, or even academic research.
- … but ONLY if all distractors are plausible. Image 1 taken from Taken from AQA Inside Exams show notes – Annotated MCQs
- Can add reasons to True/False questions to check understanding more thoroughly
- Four answer choices – 1/16 chance of answering correctly without the required knowledge
- Five answer choices – 1/32!
- Useful in subjects where right/wrong is more subjective!
- Also useful for developing exam technique
- Taken from Making Good Progress? By Daisy Christodoulou We’ve already seen that we can use MCQs to assess students’ understanding of reasons Comparisons can also be made “If this question had been set as an essay… it would have been much harder to make such inferences. A pupil with a shaky understanding of the role of trade unions in the Soviet union might have written an essay which didn’t mention trade unions or private property at all… it would therefore have been impossible for the teacher to have inferred much about their understanding of two vital concepts.
- From @mrmountstevens
- Final strategy - involves presenting learners with novel information alongside the question, so that they are having to apply their knowledge rather than just remember it
- https://www.educationquizzes.com/gcse/english-literature/an-inspector-calls-extract-1/
- Time to work on assessing/improving your existing questions and/or write some new ones