Assessment in CBME Competency Based Medical Education by Dr Girish .B, Associate Professor, Department of Community Medicine, Chamarajanagar Institute of Medical Sciences (CIMS), Chamarajanagar, Karnataka
Valle Egypt Illustrates Consequences of Financial Elder Abuse
Â
Assessment in CBME Competency Based Medical Education Dr Girish .B CISP 2 MCI
1. Assessment in CBME
By
Dr Girish .B
M.B.B.S, M.D, M.B.A
Associate Professor
Department of Community Medicine
Chamarajanagar Institute of Medical Sciences (CIMS)
An Autonomous Institute of Govt of Karnataka,
Chamarajanagar
2. Objectives
At the end of the session the participant should be
able to:
ďUnderstand the differences between the traditional
assessment & Competency Based Assessment (CBA).
ďUnderstand the changes in assessment as per new
competency based curriculum .
ďUnderstand the components & tools of competency
based assessment.
ďUnderstand the role of feedback in assessment.
ďPlan, develop and implement CBA.
4. ďSubjective
ďMore of knowledge based
ďMore of summative, norm referenced
ďNot reliable or repeatable
ďUnidirectional
ďNo scope for feedback
ďNo much importance to affective domain
ďPrecipitates students dissatisfaction
Flaws in current methods of assessment
5. How does assessment differ ?
Conventional curricula
Competency
based curricula
Fragmented,
mostly summative,
norm-references
Integrated,
mostly formative,
criterion-referenced
7. Features of Competency Based Assessment
(CBA)
⢠CBA operates within the framework of
competencies. Assessment tools should align
with competencies/objectives.
⢠CBA should help to acquire competencies /
objectives (assessment for learning) & their
certification (assessment of learning)
8. Features of Competency Based Assessment
(CBA) ( 2 )
⢠CBA is continuous and ongoing process with
opportunities for providing developmental
feedback.
⢠Direct observation of students improves utility
of CBA and feedback.
⢠Multiple assessors, multiple tools & multiple
assessments improve the validity & reliability
of CBA.
12. Informal opportunities
⢠Provide the much needed feedback to the learner,
helping her to improve
⢠Dissociate assessment and decision making,
allowing students to âopen upâ
⢠Help to take away the stress of assessment
18. Formal Assessments
⢠Internal assessment: How he learnt
⢠University examinations: How much he learnt
⢠Both test different aspects
⢠One is not a replacement for other
19. Internal assessment
⢠Range of assessments conducted by the
teachers teaching a particular subject with
the express purpose of knowing what is
learnt and how it is learnt.
⢠Internal assessment can have both formative
and summative functions.
20. Internal assessment: Extract from GMER
⢠Regular periodic examinations shall be conducted
throughout the course.
⢠There shall be no less than 3 internal assessment
examinations in each Pre-clinical / Para-clinical
subject & no less than two (2) examinations in each
clinical subject in a professional year.
⢠An end of posting clinical assessment shall be
conducted for each clinical posting in each
professional year.
21. Components of Internal assessment:
⢠Theory IA : Written tests, should have essay
questions, short notes and creative writing
experiences.
⢠Practical / Clinical IA : OSPE, OSCE, DOPS, mini-CEX
⢠Assessment of Log-book : record all activities like
seminar, symposia, quiz, etc. upto 20% of IA.
⢠Internal Assessment for Professional development
programme (AETCOM) : a. Written tests
b. OSCE based clinical scenarios
22. Internal assessment:
⢠In subjects that are taught at more than one phase,
proportionate weightage must be given for internal
assessment for each Phase.
For example,
⢠General Medicine must be assessed in
second Professional, third Professional Part I &
third Professional Part II, independently.
23. Internal assessment: (cont)
⢠Day to day records and log book should be given
importance in internal assessment.
⢠Internal assessment should be based on
competencies and skills.
24. Internal assessment: Eligibility
⢠Made mandatory before a student is allowed to
appear for the final university examination.
⢠Need at least 35% separately in theory and
practicals.
28. Feedback in Internal assessment:
⢠Feedback should be provided to students
throughout the course so that they are aware of
their performance and remedial action can be
initiated well in time.
⢠The feedbacks need to be structured & the faculty
and students must be sensitized to giving &
receiving feedback.
29. Feedback in Internal assessment:
⢠The results of IA should be displayed on notice board
within two (2) weeks of the test and an opportunity
provided to the students to discuss the results and get
feedback on making their performance better.
⢠It is also recommended that students should sign with
date whenever they are shown IA records in token of
having seen and discussed the marks.
⢠Internal assessment marks will not be added to
University examination marks and will reflect as a
separate head of passing at the summative examination.
30. University examinations
⢠University examinations are to be designed with a
view to ascertain whether the candidate has
acquired :
â the necessary knowledge,
â minimal level of skills,
â ethical and professional values,
â with clear concepts of the fundamentals,
⢠which are necessary for him / her to function
effectively & appropriately as a physician of first
contact.
34. Theory examinations
⢠Use multiple tools like
âLong essay questions ( structured )
âShort notes
âReasoning questions
âShort notes Applied aspects
âShort notes AETCOM
âMultiple choice questions (MCQs)
âMCQs not more than 20%
35. MCQs
⢠MCQs can test well in the cognitive domain at all
levels.
⢠It tends to assess only recall of knowledge,
however, it can test any higher level of the cognitive
domain.
⢠It can also discriminate well between students, if it
is well constructed.
⢠It improves the validity of tests.
37. MCQs
⢠MCQs should be scenario based, single response
with 4 options in answers.
⢠Avoid one liner and negative terms in stem of
question.
⢠Avoid âall of aboveâ and ânone of aboveâ in options.
39. Practical / Clinical examinations
⢠To assess proficiency in skills, data interpretation
and logical conclusions
⢠Clinical cases should match what a practitioner is
likely to see in actual practice
⢠Avoid rare cases / syndromes.
⢠Focus on data gathering, physical examination,
writing records and management plans.
40. Assessing Skills in Competency
Based Assessment
ďPractical skills
ďOSPE
ďClinical skills
ďOSCE
ďOSLER
ďWork place based assessment methods
41. What is OSPE / OSCE?
It is an assessment tool for
assessment of practical / clinical skill.
42. Characteristics of the OSPE / OSCE
ďIt is an assessment approach primarily used to
measure practical/clinical competence.
ďShould be planned or structured (predetermined
practical/clinical competences).
ďDifferent types of test method can be
incorporated into it.
43. ďIn most stations students are observed
(by one or more examiners).
ďScored as they carry out the task or interpret
clinical materials (e.g. Laboratory data, X-
rays), write notes or answer question.
44. Hardenâs 12 Tips for Organizing an OSPE/ OSCE
ďWhat is to be
assessed?
ďDuration of station
ďNumber of stations
ďUse of examiners
ďRange of approaches
ďNew stations
ďOrganization of the
examination
ďAssigning priority
ďResource requirements
ďPlan of the examination
ďChange signal
ďRecords
45. Advantages of the OSPE/OSCE
ďValid examination
ďThe examiners can control the complexities
of the examination
ďUsed as summative as well formative
ďCan be used with larger number of students
ďReproducible
46. Advantages of the OSPE / OSCE (cont)
ďThe variable of the examiner and the patient
are to a large extent removed
ďFun activity within the department or college,
which promotes team work
47. Disadvantages of the OSCE/ OSPE
ďKnowledge and skills are tested in
compartments
ďThe OSCE may be demanding for both
examiners and patients
ďMore time in setting it up
ďShortage of examiners
ďMight be quite distressing to the student
48.
49. Cover wide range of skill
ďCommunicative
ďDecision
ďKnowledge
ďManual performance
50. Superiority of OSPE/ OSCE
ďSame content
ďSame criteria
ďSame examiner
ďHigh degree of validity & reliability
51. Station
ďWhat is a station?
ďStation organization
ďTypes of station
ďNumber of stations
ďTiming in one station
56. Simulation
⢠Going to occupy major place in T-L and
assessment.
⢠Assess students - Psychomotor domain
including soft skills (Communication ).
⢠Team skills also can be assessed - using Tele-
simulation.
⢠Skill Lab â Mandatory for all medical colleges (
GMER â 2019 ).
58. Objective Structured Long case Examination and
Record - OSLER
I. History taking (4)
ďˇ Pace & clarity of presentation
ďˇ Communication process
ďˇ Systematic approach
ďˇ Establishment of core facts
II. Physical examination (3)
ďˇ Systematic approach
ďˇ Examination technique
ďˇ Establishment of correct physical findings
III. Formulation of appropriate investigations in a logical sequence (1)
IV. Appropriate management (1)
I. Clinical acumen (1)
- Draws on previous nine items to assess candidates ability to identify and
solve problems
59. Advantages
⢠Involves examiner from the stage of History
taking till completion of case.
⢠Valid
⢠Reliable due to structured checklist
⢠Student becomes more methodical in his
approach to the students.
⢠Higher student satisfaction in assessment.
61. What is WPBA?
⢠Assessment of performance of the
student at real life situation
â work place.
âDirect observation
âConducted at workplace
âFollowed by feedback
63. Rationale for WPBA
ď Conforms to the highest level of Millerâs Pyramid
ď Focus on clinical skills including the necessary soft skills
(communication, behavior, professionalism, ethics,
attitude)
ď Context and content specificity
ď Alignment of learning with actual working
66. Mini Clinical Evaluation Exercise (Mini CEX)
⢠A faculty member observes a trainee as he/she interacts
with a patient around a focused clinical task.
⢠Each aspect of the clinical encounter is scored by a faculty
member using a 9âpoint rating scale
â 1â3 is unsatisfactory
â 4â6 is satisfactory
â 7â9 is superior.
⢠Each patient encounter takes roughly 15 minutes followed
by 5â 10 minutes of feedback.
⢠Trainees are assessed several times throughout the year of
training with different faculty and in different clinical
situations
67.
68. Direct Observation of Procedural Skills (DOPS)
⢠Trainee is assessed through out the day ,as he /she
performs the routine procedures on real patients
â Technical skill is observed,
â Way the trainee behaves,
â Way he interacts with fellow colleague, patients and
their bystanders,
â Professional behavior is also observed
⢠Feedback so received everyday can be used by the trainee
to better himself as days progress
69.
70. Case Based Discussions (CBDs)
ďIt is similar to the one practiced by us..
ďIdeal CBD includes patient-student
interaction
ďStudent describes his encounter with the
patient with the facilitator
ďFacilitator gives feedback to the student
73. ďChecklist with 16 aspects will be used for
assessment
ďDiagnosis and appropriate application of
available investigative tools
ďManagement of time
ďManagement of stress, fatigue, and workload
ďEffective communication
ďKnowledge of oneâs own limitations
ďQualitative feedback..
ďNeeds 15 minutes to complete the
assessment
74. Patient Satisfaction Questionnaire
ďPatient feedback tool to gather
patients assessment of residents â
communication skill / listening skills
or ability to answer queries, etc.
75. Challenges associated with WPBA tools
ďStandardization of tools
ďTime constraints
ďBusy OPDs
ďStudents strength in medical colleges
ďLess number of trained faculty
ďStudents are not oriented for these methods
77. Covid 19 Pandemic situation
ďWhen Covid 19 Pandemic situation is going to end ?
ďWhen does students return back ?
ďOnline assessment may be reality ?
78. Take Home Message
ďPrepare multiple assessment tools in department &
use according to the need.
ďHelp students develop critical thinking & clinical
reasoning skills while acquiring competencies.
ďThis compliments high state examination.
79. References :
ď Medical Council of India. Assessment Module for Undergraduate
Medical Education Training Program, 2019: pp 1-29.
ď Sharma R, Bakshi H, Kumar P. Competency-based undergraduate
curriculum: A critical view. Indian J Community Med 2019;44:77-80.