Choosing the right
assessment method
Dr. Hironmoy Roy
MD, MBA, DHM, FAIMER (2015)
Sr. Asst. Professor (Anat)
Dept of Medical Education
Assessment vs Evaluation
Assessment
• Formal action to
measure the
competence or
performance of the
students.
• Used in context of
students (where the
student is now?)
Evaluation
• Value judgment, on a
quality or worth
• Used in context of any
educational/national
programmes
We need to choose right assessment method
• For a given---
(1) Competency
(2) Objective
(3) Learning methodVALID
RELIABLE
STANDARDISED
Choosing a tool…
A tool will be
• Valid: Does it really measure, what it is to measure?
• Reliable: Does it measure consistently?
• Educational impact: Does it has effect on teaching-
learning?
• Acceptable: Is it really acceptable to all stakeholders/
staffs/ teachers-students?
• Doable in the infrastructure
• cheap
Some well known tools…
• Essay type question (EQ)
• Structured essay type question (SEQ)
• Short answer questions (SQ)
• Multiple choice questions (MCQ)
Some well known tools…
• Traditional Long case/ short case
• OSCE/OSPE
• Mini CEX
• Case based discussion (CBD)
• 360 degree assessment
• Simulation (real/virtual)
• DOPS
OSCE
KSAC (mainly basic clinical skill)
• Different stations= discrete activities=
competencies
• Each students is exposed in all stations
• “Checklist” is the life of it.
• 8 to 18 stations are ideal
CBD
KSAC
• Case record is picked up
• Student justifies the treatment
360 degree assessment
KSAC
a multi-source feedback to individual’s
competency from multiple perspective, which is
objectively collected by a predefined tool.
Who are the evaluators?
Peers
Subordinate
Student
Patient
Patient party
administrator
Mini CEX (Mini Clinical Examination)
• KSAC
• 15-20 minutes shot to assess a component
(competencies) of a long case
• Immediate & structured feedback
• Competencies those are usually assessed-
1. History taking
2. Physical examination
3. Counseling
4. Communication
5. Professionalism
DOPS
• KSAC
• Same as Mini CEX
• Focuses only on procedural skill
Does
Shows
How
Knows how
Knows
Mini CEX, DOPS, WPBA
360 degree assessment
Long case, short case,
OSCE/OSPE
PBL, TBL
Essay Q, Mod EQ,
Short AQ, MCQ,
Oral-Viva
TIME
Given
Traditi
onal
Oral
Viva
Traditi
onal
Long
case
CBL OSCE Mini
CEX
PBL MCQ
1 hr 0.50 0.60 0.68 0.47 0.73 0.36 0.62
2 hr 0.69 0.75 0.73 0.64 0.84 0.53 0.76
4 hr 0.82 0.86 0.84 0.78 0.92 0.69 0.93
8 hr 0.90 0.88 0.82 0.96 0.93 0.82 0.93
0
0.2
0.4
0.6
0.8
1
1.2
1 hr 2 hr 4 hr 8hr
Traditional Oral Viva Traditional Long case CBL OSCE Mini CEX PBL MCQ
Chossing right assessment method

Chossing right assessment method

  • 1.
    Choosing the right assessmentmethod Dr. Hironmoy Roy MD, MBA, DHM, FAIMER (2015) Sr. Asst. Professor (Anat) Dept of Medical Education
  • 2.
    Assessment vs Evaluation Assessment •Formal action to measure the competence or performance of the students. • Used in context of students (where the student is now?) Evaluation • Value judgment, on a quality or worth • Used in context of any educational/national programmes
  • 4.
    We need tochoose right assessment method • For a given--- (1) Competency (2) Objective (3) Learning methodVALID RELIABLE STANDARDISED
  • 5.
    Choosing a tool… Atool will be • Valid: Does it really measure, what it is to measure? • Reliable: Does it measure consistently? • Educational impact: Does it has effect on teaching- learning? • Acceptable: Is it really acceptable to all stakeholders/ staffs/ teachers-students? • Doable in the infrastructure • cheap
  • 8.
    Some well knowntools… • Essay type question (EQ) • Structured essay type question (SEQ) • Short answer questions (SQ) • Multiple choice questions (MCQ)
  • 9.
    Some well knowntools… • Traditional Long case/ short case • OSCE/OSPE • Mini CEX • Case based discussion (CBD) • 360 degree assessment • Simulation (real/virtual) • DOPS
  • 10.
    OSCE KSAC (mainly basicclinical skill) • Different stations= discrete activities= competencies • Each students is exposed in all stations • “Checklist” is the life of it. • 8 to 18 stations are ideal
  • 11.
    CBD KSAC • Case recordis picked up • Student justifies the treatment
  • 12.
    360 degree assessment KSAC amulti-source feedback to individual’s competency from multiple perspective, which is objectively collected by a predefined tool. Who are the evaluators? Peers Subordinate Student Patient Patient party administrator
  • 13.
    Mini CEX (MiniClinical Examination) • KSAC • 15-20 minutes shot to assess a component (competencies) of a long case • Immediate & structured feedback • Competencies those are usually assessed- 1. History taking 2. Physical examination 3. Counseling 4. Communication 5. Professionalism
  • 14.
    DOPS • KSAC • Sameas Mini CEX • Focuses only on procedural skill
  • 15.
    Does Shows How Knows how Knows Mini CEX,DOPS, WPBA 360 degree assessment Long case, short case, OSCE/OSPE PBL, TBL Essay Q, Mod EQ, Short AQ, MCQ, Oral-Viva
  • 17.
    TIME Given Traditi onal Oral Viva Traditi onal Long case CBL OSCE Mini CEX PBLMCQ 1 hr 0.50 0.60 0.68 0.47 0.73 0.36 0.62 2 hr 0.69 0.75 0.73 0.64 0.84 0.53 0.76 4 hr 0.82 0.86 0.84 0.78 0.92 0.69 0.93 8 hr 0.90 0.88 0.82 0.96 0.93 0.82 0.93
  • 18.
    0 0.2 0.4 0.6 0.8 1 1.2 1 hr 2hr 4 hr 8hr Traditional Oral Viva Traditional Long case CBL OSCE Mini CEX PBL MCQ