This document provides guidance on writing effective multiple choice questions (MCQs) for assessment. It discusses the benefits of MCQs, such as wide topic coverage and fast feedback, as well as disadvantages like guessing and lower-order thinking. Tips are provided for writing high-quality question stems and plausible distractors that avoid clues. The document emphasizes shifting focus from recall to application and using novel contexts to make questions less "googleable." Strategies are presented for engaging students with formative assessment through varied feedback, social learning, and emphasis on intrinsic motivation.
This presentation help those student who wants to write and publish their research work in any research journal and it also help to student in selection of specific journal for their research paper.
How to do a Literature search for your research and scientific publication BhaskarBorgohain4
In the age of information boom it may be challenging task to find relevant information for your research work. its like finding a needle in a haystack. After initial readings from textbooks and library journals you may want to first search in Wikipedia, google, google scholar and then go to Pubmed, Medline, science direct , wileyonline, science.gov, cochrane library etc to formulate your keywords based on your research question. read a medical dictionary to find synonyms of the keywords and brainstorm with your supervisor, peers, friends etc to get more key words to search again and find the right search strategy. do not forget to look for Grey literature like unpublished Thesis works from reputed universities, proceedings of conferences of reputed professional associations as well. keep records using a software like end-note, Rayyan etc. References of authors must be recorded as you go along.
This presentation help those student who wants to write and publish their research work in any research journal and it also help to student in selection of specific journal for their research paper.
How to do a Literature search for your research and scientific publication BhaskarBorgohain4
In the age of information boom it may be challenging task to find relevant information for your research work. its like finding a needle in a haystack. After initial readings from textbooks and library journals you may want to first search in Wikipedia, google, google scholar and then go to Pubmed, Medline, science direct , wileyonline, science.gov, cochrane library etc to formulate your keywords based on your research question. read a medical dictionary to find synonyms of the keywords and brainstorm with your supervisor, peers, friends etc to get more key words to search again and find the right search strategy. do not forget to look for Grey literature like unpublished Thesis works from reputed universities, proceedings of conferences of reputed professional associations as well. keep records using a software like end-note, Rayyan etc. References of authors must be recorded as you go along.
This presentation shows how to use ENDNOTE software for citation management in different databases.
See also Biomedical Databases Handout and
Engineering Databases Handout in http://www.library.drexel.edu/services/refengineer.html
As a researcher, you are expected to start publishing early in your career. But original research could take years to complete! This does not mean you that you cannot publish a paper until you complete your research. You can disseminate your research in many other ways. These slides will help you learn more about the different types of scholarly literature so that you are able to choose the most suitable format for publishing your study.
This document, created in March 2019, describes how the Journal Impact Factor is calculated, specifically in relation to the June 2019 release of the Journal Citation Reports.
this is one of the main ingredients or topics which is added by the National Commission of India(NMC). It holds a one-hour important lecture in the revised basic course workshop. The feedback register is to be maintained by the student of MBBS. which is mandatory according to NMC guidelines
Comments by peer reviewers and journal editors are meant to help authors improve their manuscript. Follow the three golden rules when responding to reviewer comments.
Writing the results section for scientific publicationAshok Pandey
To introduce participants to the details of communication and writing scientific papers.
To guide researchers in the writing of scientific paper to increase its acceptability for publication in a journal; and
To upgrade the pre-existing knowledge of writing skills in a scientific manner.
Scientific Writing: Road and Tips (Part 1)Omar Abo Shady
This presentation shows the ways and steps to start writing to reach publication.
Omar A. Abo Shady ( Menoufia Faculty of Medicine - Egypt) 14 May, 2013.
scientific protocol.the Idea and Process.Mona Quenawy
In a clinical trial, the protocol is carefully designed to safeguard the health of the participants as well as answer specific research questions. A protocol describes what types of population may participate in the trial; the schedule of tests, procedures, medications, and dosages; and the duration of the study.means time frame.statistical analysis and outcome measurement.
This presentation shows how to use ENDNOTE software for citation management in different databases.
See also Biomedical Databases Handout and
Engineering Databases Handout in http://www.library.drexel.edu/services/refengineer.html
As a researcher, you are expected to start publishing early in your career. But original research could take years to complete! This does not mean you that you cannot publish a paper until you complete your research. You can disseminate your research in many other ways. These slides will help you learn more about the different types of scholarly literature so that you are able to choose the most suitable format for publishing your study.
This document, created in March 2019, describes how the Journal Impact Factor is calculated, specifically in relation to the June 2019 release of the Journal Citation Reports.
this is one of the main ingredients or topics which is added by the National Commission of India(NMC). It holds a one-hour important lecture in the revised basic course workshop. The feedback register is to be maintained by the student of MBBS. which is mandatory according to NMC guidelines
Comments by peer reviewers and journal editors are meant to help authors improve their manuscript. Follow the three golden rules when responding to reviewer comments.
Writing the results section for scientific publicationAshok Pandey
To introduce participants to the details of communication and writing scientific papers.
To guide researchers in the writing of scientific paper to increase its acceptability for publication in a journal; and
To upgrade the pre-existing knowledge of writing skills in a scientific manner.
Scientific Writing: Road and Tips (Part 1)Omar Abo Shady
This presentation shows the ways and steps to start writing to reach publication.
Omar A. Abo Shady ( Menoufia Faculty of Medicine - Egypt) 14 May, 2013.
scientific protocol.the Idea and Process.Mona Quenawy
In a clinical trial, the protocol is carefully designed to safeguard the health of the participants as well as answer specific research questions. A protocol describes what types of population may participate in the trial; the schedule of tests, procedures, medications, and dosages; and the duration of the study.means time frame.statistical analysis and outcome measurement.
Salam
Meeting & Workshop : Testing & Examiner Guide 2018
Today's points were:
1) defining testing
2) Testing vs assessment
3) Teachers vs testing
4) Why testing ?
5) Principles of testing
6) Bloom taxonomy and testing
7) How to plan tests and exams?
8) Types of tests
9) Importance of the examiner guide ( BEM guide ) in the teacher's daily teaching process
10) Why must teachers take into considerations this guide
11) From which level must this guide be used
12) what's new in the Examiner guide 2018?
13) The Examiner guide 2018 vs the one of 2013
14) Recommendations for national exam designers
15) Typology of the new Examiner guide 2018
16) Tips for designing exams
17) How to devise and test ?
18) The situation of integration its characteristics and evaluation criteria
19) The out off topic learners' productions
20 ) Test report and remedial work
N.B : I would like to thank Mr. Hachemi Irid superviors of ALgiers East for the invitation and all his teachers for their great welcome and large contribution during the delivery of the meetings
Mr.Samir Bounab ( teacher trainer)
The link of the presentation
An evidence-based model to enhance programme-wide assessment using technology: TESTA to FASTECH . Presented by Tansy Jessop and Yaz El-Hakim (University of Winchester) and Paul Hyland (Bath Spa University). Facilitated by Mark Russell (University of Hertfordshire).
Jisc conference 2011
Questioning is a central part of student assessment and quizzing, but it can also be a powerful learning tool. In this interactive workshop, we’ll explore research-based tips and ideas for achieving the full benefit of questioning. Effective use of common questioning tools -- clickers and discussion boards -- will be discussed as a means to achieve student engagement and deep learning.
Three Fundamental Principles For Crafting Assessment Tasks
Six Important Guidelines For Developing Multiple Choice Items
Five Guidelines For Developing Essay Items
From the CALPER/LARC Testing and Assessment Webinar Series
Download the handouts and ppt: https://larc.sdsu.edu/archived-events/
View the recording: http://vimeo.com/58413470
Presentation Description
The word assessment comes from the Latin assidere, meaning to sit beside. This notion evokes the image of learner and teacher working together to improve learning and teaching. Involving learners in assessment helps them reflect on their learning, set goals, monitor progress, and regularly evaluate their goals. In the case of listening, learners become aware of the cognitive processes and develop greater metacognitive awareness of listening to help them better regulate their comprehension processes. This leads to greater learner investment and motivation and, ultimately, autonomous language learners.
Comprehension, the product of listening, can be assessed by a variety of informal and formal methods. In this webinar, we will examine and discuss a number of examples of formative assessment of listening. We will then discuss some issues related to these examples, as well as some well-known examples of summative assessment, in light of five important criteria: 1) validity; 2) reliability; 3) authenticity; 4) washback; and 5) practicality.
Webinar Date: 2/23/2012
Similar to Science-Writing-Effective-MCQs-Workshop-Nov2020.pptx (20)
This is a presentation by Dada Robert in a Your Skill Boost masterclass organised by the Excellence Foundation for South Sudan (EFSS) on Saturday, the 25th and Sunday, the 26th of May 2024.
He discussed the concept of quality improvement, emphasizing its applicability to various aspects of life, including personal, project, and program improvements. He defined quality as doing the right thing at the right time in the right way to achieve the best possible results and discussed the concept of the "gap" between what we know and what we do, and how this gap represents the areas we need to improve. He explained the scientific approach to quality improvement, which involves systematic performance analysis, testing and learning, and implementing change ideas. He also highlighted the importance of client focus and a team approach to quality improvement.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
How to Split Bills in the Odoo 17 POS ModuleCeline George
Bills have a main role in point of sale procedure. It will help to track sales, handling payments and giving receipts to customers. Bill splitting also has an important role in POS. For example, If some friends come together for dinner and if they want to divide the bill then it is possible by POS bill splitting. This slide will show how to split bills in odoo 17 POS.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
1. Get chatting
Before the workshop starts at 14:00,
briefly introduce any previous
experiences using Moodle quizzes
•Contexts
•Positives
•Negatives
Join TEL Community of Practice on Teams
This session is being recorded
Mute Chat
3. Today’s Session
At the end of this workshop you will learn:
The benefits of using MCQs both for both
assessment & learning
How to write effective MCQs and pitfalls to avoid
How to construct, analyze and alter MCQs
questions
The experiences, ideas and tips of others!
Recording session & will make slides (with links) available
Please get in touch afterwards tel@bath.ac.uk for any follow-up queries
4. Good quiz practices = link to learning
Empower students as self-regulated learners
Activity linked to learning objectives
Aims for learners
• Impact through quality
feedback = students act on
feedback
• Bitesize, low stakes build
motivation & self-esteem
• Combine with other learning &
teaching strategies
D. J. Nicol & D. Macfarlane‐Dick (2006)
For authors
Planning • Learning objectives + cognitive skills
• Feedback next steps, dialogue (e.g.
tutorials, signposting, peer
support, forums, problems classes)
Writing • See following guidance...
• Continuously
Managing • Build question bank over time
• Use data to inform
• Review questions
5. Multiple Choice Questions
Advantages Disadvantages
• Different cognitive levels – fact
recall/apply/evaluate
• Finding “good” distractors hard
• Dominance of “recall” questions?
• “Higher order” vs. surface learning?
• Wide coverage (less “sampling”)
• Fast feedback for students
• Easy to mark/score - can apply
frequently
• Test analytics
• Different Uses (diagnostic, revision,
Assessment for/as Learning)
• Many accessible features “built-in”
• Difficult to “show working”
• Not well suited to maths
• Accuracy/suitability of assessment
• Engagement
• Students guessing
• Poor questions -> teachers work
harder than students
• Fairness/accuracy
6. "Non-googleable MCQs"
Learning objectives + cognitive skill = MCQ test
1. Shift focus from recall to application
– Understand (explain)
– Apply (recall in context)
– Analyse (explain in context)
– Evaluate (best fit)
2. Use novel context
3. Use common mistakes/misconceptions as
distractors
4. Multiple select (1 or 2 correct)*
5. Embrace open book exams…
Bloom’s Taxonomy by Nicoguaro is licensed under CC BY-SA
7. Constructing Effective Questions
Stem should be
meaningful on
its own
Don’t include
irrelevant
material
Use negative
phrasing
carefully
1
2
3
Writing Good Multiple Choice Test Questions – C. Brame (2013)
8. Constructing Effective Distractors
Plausible
Clear and concise
Mutually exclusive
Homogenous in nature
Free from having clues
to correct answer
All the distractors should be:
The distractors aren’t functional if they are not plausible, as students who don’t
understand the learning outcomes can still be rewarded. Use common student errors
– these make the best distractors.
Students who know the correct answer should not have to worry about unclear and
wordy answers and grasping what the answer is actually trying to say
Writing Good Multiple Choice Test Questions – C. Brame (2013)
A ‘trick’ question can sometimes lead to multiple answers being correct. Don’t have
this ambiguity – make the answers mutually exclusive
This means the answers should all be in the same topic area. If one answer is focused
on a different subject, it can lead students to the correct answer.
Savvy test takers can spot clues to correct answers such as differences in grammar,
length, formatting, and language choices.
9. MCQ pitfalls to avoid
1) Select the correct option
MCQs are not…
A) Unlikely to be never
ineffective
B) somewhat easy
C) Difficult to write, particularly
to make them valid
D) All of the above
2) Select the errors
demonstrated in question 1 from
the list
(Answers: 2,4,6,7,8,9)
Cognitive load
1. Superfluous information (minimise reading)
2. Unfocused question stems (avoid lots of text or open ended) and blanks
3. Unimportant content/minutia – instead test main points
4. Negative stem or answer options (if necessary, be clear to emphasise the
negative element “Which of this is NOT…”)
5. Confusingly ordered options (e.g. by size)
Wording Clues
6. Absolute terms (easy to dismiss) or vague terms (open to interpretation)
7. Grammatical clues (options deduced from stem)
Systematic
8. Unequal option length (correct answers often take more info)
9. “All/none of the above” option (e.g. trivial if 2+ match)
10. Logical clues (avoid mutually exclusive pair options, use multi select question
instead)
11. Convergent option – correct answer has similarities with others
Webb et al., 2015
10. Limiting student collusion
Some strategies (with collusion disincentive in brackets), listed from more extrinsic to intrinsic student motivation:
A. Police/enforce/invigilate (the “stick” approach)
B. Restrict duration (limit opportunities)
C. Don’t have marks associated with assessment – separate formative/summative assessment (remove incentive)
D. Question randomisation: mix up the order in the test or set question variants (remove ease of colluding)
E. Use confidence based marking to score students and encourage students to reflect (dissuade blind copying)
F. Follow up assessment e.g. create consequences: pick students to explain answer in tutorial, random to provide
written worked solution justification to accompany course material (create incentives to be honest & seek help)
G. Use variable/context randomisation – this could be used to promote peer collaboration/discussion (avoid simple
copying)
H. Educate and promote ethos, purpose of assessment– make it clear to students (from all backgrounds) what
collusion is, aims of assessment and disadvantages to learning (educate learners)
I. Broaden assessment aims to allow collaboration and group/peer instruction (remove incentive)
J. High intrinsic value of the activity and feedback (make motivation to collude counterproductive)
Disclaimers!
• This list isn’t exhaustive and the exact and extent of the solution(s) will be context specific (the above is not a hierarchy), but may be useful to consider!
• Many of these issues are not unique to e-Quizzes, but digitising does give some solutions!
12. Improve an MCQ question
Open WORKSHOP ACTIVITY
Powerpoint
Write an improved question
• Improve question stem?
• Improve distractors?
• Shift question emphasis?
One person per group >
Feedback on what you did
Groups: 3-4
10 mins
Feel free to add text box
comments to your own slides!
13. MCQ Experiences Dicussion
• Think about past experiences both taking and
delivering MCQs
What went well?
Do you refine and revisit questions?
Did you vary question types?
Were then un-googleable?
Any tips which you found helped?
Did it help students’ understanding?
14. Key points
• Start with learning objectives & the focus
for the quiz
• Write effective questions!
• Review and revisit your question bank
15. References and further reading
• Quick read: Writing Good Multiple Choice Test Questions – C. Brame (2013)
• Good summary with links to literature: The Importance of Writing Effective Distractors – C Malamed (2020)
• Nicol, D. J & Macfarlane‐Dick, D., 2006. Formative assessment and self‐regulated learning: a model & 7
principles of good feedback practice, Studies in Higher Education 31(2) p199
• Carneson J, Delpierre G, Masters K., 2016. Handbook: Designing and Managing Multiple Choice Questions
(2nd Ed) [online only]
• Gierl, M et al., 2017. Developing, Analyzing, and Using Distractors for Multiple-Choice Tests in Education: A
Comprehensive Review. Review of Educational Research, 87: 6, pp. 1082 –1116.
• Rodriguez, M. (2005). Three Options Are Optimal for Multiple‐Choice Items: A Meta‐Analysis of 80 Years of
Research, Educational Measurement: Issues and Practice, 24:2.
• Haladyna, T.M. et al., 2002. A Review of Multiple-Choice Item-Writing Guidelines for Classroom
Assessment. Applied Measurement in Education 15(3) p309
• Webb, E et al., 2015. Does Educator Training or Experience Affect the Quality of Multiple-Choice
Questions? Academic Radiology 22(10) pp. 1317
• McCoubrie, P. 2004. Improving the fairness of multiple-choice questions: a literature review. Medical
Teacher 26(8) p709
Moodle Quiz Settings and Setup FAQ on the Teaching Hub
16. How to get engagement in formative
assessment and feedback?
Link to general engagement strategies: employability & real world context, develop skills, student voice/ownership, peer
assisted, student generated feedback, stakeholder engagement, social learning, diversify/integrate with other teaching
activities, educate students on benefits, combine with summative, students reflect on learning, reflect & adapt teaching strategy
….
Strategies specific to online quizzes:
• Vary quiz behaviour
• low stakes & “fast fail” approach: multiple attempts (with penalties), instant feedback
• Use social aspects as a lever with technology
• Variable/context randomisation to promote peer collaboration
• Social learning – forums, MS Teams etc to support discussion
• Student generated feedback – students as assessors (e.g. Moodle workshop)
• Vary feedback delivery
• instant timely feedback, selective feedback (reveal feedback only, hide score etc), blended feedback (in tutorial)
• Stress High intrinsic value of the activity and feedback
• Assess aspects that would be take to long to assess traditionally (e.g. wide coverage, calculations with real data set
etc).
• Timely feedback can be more effective
• Create quality bespoke feedback (this becomes scalable)
17. Before you start
• Guidelines for setting up
quizzes have been created
for the current assessment
period
• Read the guidance (even if
familiar with Moodle quizzes)
1. Dos and Don'ts of online
assessments PDF
2. Quiz settings Setup and FAQ
3. Creating a Quiz Assessment in
Moodle as an alternative
assessment (PDF)
Some settings will be different
from the ones you normally use
for formative quizzes e.g.
visibility of feedback
18. Moodle quiz settings
Step by step
instructions with
screenshots:
Creating a Quiz
Assessment in Moodle
as an alternative
assessment
(PDF)
Quiz settings (rubric,
availability, timing etc)
Extra time
Edit question set
View, amend,
export grades
Quiz settings Setup and FAQ
19. Setting up the Moodle quiz
Key things to remember:
• Read the guidance and follow carefully
• Use default settings unless otherwise specified
• Name your quiz clearly
• Give clear instructions to students (University Rubric,
Academic Integrity Declaration)
• Consider using grade transfer
• Adjust availability or duration for students with DAPs
• Test: Preview your quiz, check availability