The document discusses quality assurance in medical student assessments. It notes that assessments typically involve two internal and two external examiners, with question papers developed by department heads and finalized by an external person. Some universities have theory exams evaluated by one examiner and practicals by another. Marks are sometimes recorded in a booklet rather than on the exam paper itself. The document emphasizes that assessments need quality assurance to prevent mistakes and ensure solutions meet customer needs. It provides examples of assessment planning, question types like MCQs and modified essay questions, and structured assessment methods like objective structured clinical exams. Overall it focuses on standardizing, monitoring, analyzing and improving medical assessment quality.

1. ASSESSMENTDr Hironmoy Roy
MD, MBA, ACME, FAIMER Fellow
Dip. Hospital Management
Copyright preserved Hironmoy Roy

2. Why?
Usually in a 100 students MBBS examination there are Two (2) Internal
Examiners & Two (2) External Examiners
Usually all HODs are requested to make the question paper for Prof Exam
and at the end it gets decided by an External person to finalize the paper
In some university, theory paper is evaluated by one set of Examiner and
Practical exam is taken by another set of examiner
In some university, the marks of a theory answer script is noted in a
booklet, instead of in the paper. That paper gets evaluated by a second
examiner and the marks gets made average
Ideally, among four “Groups”, two Groups are checked by Internal
Examiners and two Groups are checked by External Examiners
QA- IQA & EQA
Copyright preserved Hironmoy Roy

3. Quality Assurance
Quality assurance (QA) is a way of preventing mistakes
and defects in manufactured products and avoiding
problems when delivering solutions or services to
customers
Indian Medical Graduate
Society
in Medical Education
Needs QA in Assessment
Copyright preserved Hironmoy Roy

4. Copyright preserved Hironmoy Roy

5. While to plan an assessment programme
Whom to assess ? MBBS/Par medics/Nursing
What to assess ? Learning Objective, Learning Domain,
Chapters to be covered
Which level of MP? K/KH/SH/D
Which tool is liked
to be used ?
Theory Q/ Long case/ Short case/spot case
Copyright preserved Hironmoy Roy

6. Copyright preserved Hironmoy Roy

7. Planning our theory question
Group A
SEQ =1(2) X 12
Group B
SEQ= 2 (3) X 7
Group C
Short Note
3 (4) x 4=12
Group D
Explain why
4 (4) x 3= 12
50% is the Pass marks
75% is Honors marks
Copyright preserved Hironmoy Roy

8. LQ (LAQ) = EQ, MEQ
Supply type
Copyright preserved Hironmoy Roy

9. An “Essay Question”
Traditional
Easy to frame
Easy to mark
Fascinated by old-mass
Describe Protein-Energy Malnutrition [20]
Copyright preserved Hironmoy Roy

10. An “essay question”……
• Requires examinees to compose rather than select their
response.
• Elicits student responses that must consist of more than one
sentence.
• Allows different or original responses or pattern of responses.
• Requires subjective judgment by a competent specialist to
judge the accuracy and quality of responses.
John M Stalnaker 1951
Copyright preserved Hironmoy Roy

11. • An effective essay question will align with each of the four
criteria given in Stalnaker’s definition and provide students
with an indication of the types of thinking and content to use
in responding to the essay question.
Explain in what ways the “Protein energy Malnutrition”
affects the Anthropometric measurements on an under 5
child . Why the features in Kwashiorkor & Marasmus
differs?. [10+10]
Copyright preserved Hironmoy Roy

12. Modified Essay Question (MEQ)
• MEQs are a series of short- questions (sequence of questions)
based on a case study and designed to test higher order
thinking.
• Students are usually presented with a scenario, and then
given a series of questions based on it that they answer with a
short text.
• “Essay” is misnomer
Copyright preserved Hironmoy Roy

13. In a MEQ……
• A student shall identify the problem, define it, interpret the
situation by analysing the given data, organise/ paraphrase
the arguments/reflections by applying his/her thinking skills.
• Approaches higher level of cognitive domain
• Have small markings of each part/portion.
• In totally replace an traditional essay Q.
Copyright preserved Hironmoy Roy

14. Give an account of Fallopian Tube. [10]
Traditional Essay Q
Copyright preserved Hironmoy Roy

15. A 25 yrs old lady reports at obstetrics emergency with severe
abdominal pain and bleeding per vagina. On examination the
resident doctor explores that she missed her period of last
one month. USG reveals rupture ectopic pregnancy.
Which organ is affected in ectopic pregnancy? Enlist its parts.
Name the embryological precursor of the organ. Name the
blood vessels supplying this organ. Provide a histological
diagram of the affected organ with clear labeling. [1+ 2+ 1+
2+ 4= 10]
Modified Essay Q (MEQ)
Copyright preserved Hironmoy Roy

16. SQ/ SAQ
Copyright preserved Hironmoy Roy

17. A Short-answer Q (SAQ)
• Questions may require answers such as complete the
sentence, supply the missing word, short descriptive
or qualitative answers, diagrams with explanations
etc
Copyright preserved Hironmoy Roy

18. For Example:-
 One may simply ask the student to write in a missing word or
phrase:
The current Prime Minister of India is _____? (1)
• Or one may ask for brief listing:
List the three types of plastic surgery procedures
(1 mark for each)
• Or for a one sentence answer:
What is the basic principle of Gram’s staining ? (2)
Copyright preserved Hironmoy Roy

19. Items clearly needed to be specified how the question
should be answered.
e.g.
• Student should answer it briefly and concisely using a single
word or short phrase?
• Is bullet point format acceptable, or does it have to be an
essay format?
• Is the question given a specific number of blanks for
students to answer?
Copyright preserved Hironmoy Roy

20. • Example:
Family planning methods most appropriate for adolescents
are:
a._______________,
b. _______________,
c._______________.
• Questions that involve possible multiple responses demands
more from students and addresses broader content.
Copyright preserved Hironmoy Roy

21. Our MEQ= Group A
Our LQ= Group B
Our SAQ= ??
Copyright preserved Hironmoy Roy

22. Copyright preserved Hironmoy Roy

23. Give an account on the Middle ear cavity.
Describe the formation of brachial plexus.
Discuss the modifications of spinal pia mater.
Write briefly on spinal pia mater.
Short note - Fornix
Copyright preserved Hironmoy Roy

24. Enumerate the muscles of Middle ear cavity & their
nerve supply.
Enlist the modifications of spinal pia mater with
pictorial presentation.
Short note-
Cerebral Fornix- which type of white fibre is it and
why?
Copyright preserved Hironmoy Roy

25. We MUST AVOID using vague action-
verbs;
Describe
Discuss
Give an account
Write an essay
Write briefly
…….as they are not at all SMART to assess.
Copyright preserved Hironmoy Roy

26. “Short note” Group need to be revisited.
Example……
ORS- Its composition & Indication of use
[1.5+1.5=3 marks]
ORS [3 marks]
Copyright preserved Hironmoy Roy

27. Copyright preserved Hironmoy Roy

28. Copyright preserved Hironmoy Roy

29. Copyright preserved Hironmoy Roy

30. MCQ
Copyright preserved Hironmoy Roy

31. Copyright preserved Hironmoy Roy

32. Copyright preserved Hironmoy Roy

33. General strategies
• To write questions throughout the term.
• To instruct students to select the “best answer”
rather than the “correct answer”.
• To use familiar language.
• To avoid giving verbal association clues from the
stem in the key.
• To avoid trick questions.
• To avoid negative wording.
Copyright preserved Hironmoy Roy

34. Designing stems
• Express the full problem in the stem.
• Put all relevant material in the stem.
• Eliminate excessive wording and irrelevant
information from the stem.
Copyright preserved Hironmoy Roy

35. Designing alternatives
• Limit the number of alternatives. Max 5
• Make sure there is only one best answer.
• Make the distracters appealing and plausible.
• Make the choices grammatically consistent with the
stem.
• Place the choices in some meaningful order.
• Randomly distribute the correct response.
Copyright preserved Hironmoy Roy

36. Designing alternatives….. (contd.)
• Avoid using “all of the above”.
• Avoid using “none of the above”.
• Refrain from using words such as always, never, all,
or none.
• Avoid overlapping choices.
• Avoid questions of the form “Which of the following
statements is correct?”
Copyright preserved Hironmoy Roy

37. Phases of making MCQ
• Try out phase
• Item analysis phase
• Item revision phase
Copyright preserved Hironmoy Roy

38. How to analyse the “item”?
Copyright preserved Hironmoy Roy

39. Item- Difficulty
Number of the students who are
able to reply the item correctly
Total number of the students
%
Difficulty Index
Copyright preserved Hironmoy Roy

40. Range of
Difficulty
Index
Interpretation Action
0-25% Difficult Revise or
discard
26-75% Okay Retain/accept
76%+ Easy Revise or
discard
Copyright preserved Hironmoy Roy

41. fallacy
• Difficult item easily discriminate between the
students “who know” and “who doesnot know”
• Easy item cannot do this
• But, when a student not knows the subject matter,
cannot respond correctly, when even the question
is easy
• We need, a measure, which discriminates the good
knowledged student with the un-knowledged
student
Copyright preserved Hironmoy Roy

42. Top 25
Bottom
25
How difficult
the item was?
How difficult
the item was?
D U
D L
Copyright preserved Hironmoy Roy

43. Copyright preserved Hironmoy Roy

44. Copyright preserved Hironmoy Roy

45. Structured Viva-voce Exam (SVVE)
• Sets of equitable question-cards are prepared
• Each having equal numbers of questions
• With equal numbers of parts,
• Arranged successively with increasing difficulty and
equal marks.
All are packed & sealed in envelope.
• Students are to pickup cards as lottery and needs to
reply.
Copyright preserved Hironmoy Roy

46. Objective Structured Viva Exam (OSVE)
Station – based
Student needs to move from one station to another
In each station one question card is there,
 written with one structured question,
 what he needs to reply on-spot to the station-
attendee/observer;
 who will match the reply with checklist.
Copyright preserved Hironmoy Roy

47. The questions shall must be-
• Clear
• Short
• Precise
• Unequivocal
• In proper sequence of increasing difficulty
Copyright preserved Hironmoy Roy

48. Copyright preserved Hironmoy Roy

49. Standardize
Monitor
Analyze
Report
Verify
Track
Audit
Train
Copyright preserved Hironmoy Roy