Developing short answer questions (sa qs)


Published on

The outline of the standard method to develop an SAQ.

Published in: Education, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Developing short answer questions (sa qs)

  1. 1. DEVELOPING SHORT ANSWER QUESTIONS (SAQs) Presented in a workshop at Quaid-e- Azam Medical College, Bahawalpur Facilitator: Prof. Javed Iqbal
  2. 2. Objective To write quality Short Answer Questions
  3. 3. Overview • Brief introductory presentation • Presentation and critique of submitted SAQs • Tea break • Develop new SAQs in groups • Presentation and critique
  4. 4. Criteria of a good item • Objective • Valid (Content validity) • Reliable • Feasible • Positive educational impact
  5. 5. Levels of Cognition • Evaluation • Synthesis • Analysis • Application • Comprehension • Knowledge
  6. 6. Two basic questions • Why CPSP has decided to switch over from Short Essay Questions (SEQs) to Short Answer Questions? • Why CPSP does not have both MCQ papers?
  7. 7. Answer to Q.1 • The very word ‘Essay’ invites to write more and tests: – Organization of thought – Knowledge of English language – Writing skills Instead the purpose is to test: – Interpretation, Reasoning & Problem-solving Skills
  8. 8. Answer to Q.2 • Very presence of ‘Best Answer’ among the options of an MCQ has a cueing effect • An MCQ tests (or provide solution to) only one aspect of a problem and that too without justification for arriving to that solution Whereas We may need answers for different aspects of same problem with reasons!
  9. 9. LEVEL OF KNOWLEDGE TESTED MCQs SAQs Breadth of knowledge Depth of knowledge
  10. 10. SHORT ANSWER QUESTIONS (SAQs) • Supply type item (a form of open-ended question) with no cueing • Response restricted to point-wise answers • More objective as point-wise answers make agreement on key easier Norman GR, Van der Vleuten CPM, Newble DI (eds.) International Handbook of Research in Medical Education. UK. Kluwer Academic Publishers; 2002. Chap. 20 , Assessment of Knowledge with Written Test Forms: pp.664-6
  11. 11. LIMITATIONS OF SAQs • Appropriate selection of words is essential • Construction of vignette and the key require careful deliberation. • If breadth of knowledge is to be tested, MCQs are a better option. Amin Z, Seng CY, Eng KH. Practical Guide to Medical Student Assessment. Singapore. World Scientific,2006. Chap.8, Short Answer Questions (SAQs): pp. 34-6 Norman GR, Van der Vleuten CPM, Newble DI (eds.) International Handbook of Research in Medical Education. UK. Kluwer Academic Publishers; 2002. Chap. 20, Assessment of Knowledge with Written Test Forms: pp.664-6
  12. 12. COMPARISON BETWEEN SEQ AND SAQ S# Characteristic SEQ SAQ 1. Type 2. Division of question 3. Response 4. Tests 5. Objectivity 6. Content Validity 7. Inter-rater reliability 8. Key construction 9. Educational impact
  13. 13. COMPARISON BETWEEN SEQ AND SAQ S# Characteristic SEQ SAQ 1. Type Supply type Supply type 2. Division of question May be divided Divided into 3-4 small tasks 3. Response Greater freedom Restricted point-wise answer 4. Test Higher levels of cognition Reasoning & Problem solving skills 5. Objectivity Low High 6. Content Validity Low High 7. Inter-rater reliability Low High 8. Key construction Difficult Easier 9. Educational impact Writing more will earn high scores Write to the point answers
  14. 14. STEPS FOR WRITING SAQs • Identify topic/concept that requires exploring of different aspects such as diagnosis along with reasons to arrive to the diagnosis. • Construct clinical vignette on the concept and link questions to the vignette • Frame 3-4 questions on specific tasks you want candidates to perform • • Select appropriate wordings for each question
  15. 15. WORDINGS FOR GOOD SAQs The wording of each question should: • Be simple, clear and tests an important aspect • Select appropriate words to restrict the response e.g. Name 3 preliminary investigations instead of saying ‘what investigations you will order?’ Give one reason for each investigation.
  16. 16. WORDS TO BE AVOIDED Avoid words that invite lengthy answers, like: – Discuss – Describe – Explain – Outline – Comment – Write a short note
  17. 17. DEVELOPMENT OF KEY •Instructions should include a mechanism to award marks to the required number of answers • Develop a key according to the key features of the answer for each question • Assign weight (marks) to each component answer according to its importance
  18. 18. Example of an SAQ You are a medical officer in OBGY. You have been asked to review a 28 years old woman , para 2 who has presented with a 6 weeks’ amenorrhea & slight vaginal bleeding. She is haemodynamically stable. Ultrasound shows a complex right adnexal mass, 3x2 cms in size. There is no pregnancy sac in the uterus. Case Vignette Exam: Final fellowship Importance: Essential
  19. 19. Example No.1 of an SAQ (contd…) Q1. What is the most likely diagnosis? (2 marks) Q2. Name three preliminary investigations you will like to perform at this stage. (3 marks) Q3. What will be immediate steps in the management of this patient? (5 marks) Questions
  20. 20. Example No.1 of an SAQ (contd…) Ans (1) Tubal pregnancy 2 marks Ans (2) 3 preliminary investigations are: • Complete blood count • Blood group and RH factor • Serum B-HCG. 3 marks Ans (3) lmmediate steps in management are: • Save I/V line 5 marks • Arrange two units of blood Key
  21. 21. • Medical management by methotrexate 50 mgm I/M if white cell and platelet count are normal, and serum B- HCG are <15000mIU/ml. • Serial monitoring by B-HCG level every 48 hrs. Expect 15% drop from day 4-7 after injection • Laparoscopy/laparotomy if initial B-HCG is above 15000mIU/ml or if the patient becomes haemo- dynamically unstable. Ref:Ectopic Pregnancy.Washington:American College of Obstetricians and Gynecologists.19900ACOG Technical Bulletin No 150. Example No.1 of an SAQ (contd…) Key