Assessment of clinical skills
Joseph Cacciottolo Josanne Vassallo
UNIVERSITY OF MALTA
ANNUAL CONFERENCE - OSLO - MAY 2007
Assessing clinical skills
1. WHY do we need to assess ?
2. WHAT do we want to measure ?
WHY do we assess ?
• To ensure safety of patients
our responsibility to the public
• Achievement of a minimum standard
res...
WHY do we assess ?
• To ensure competence
• Often as a means of academic competition
In practice: the scope
WHY do we assess ?
• Formative: to give feedback and advice
• Summative: to grade
• Qualificative or licensing
In practice...
WHAT do we measure ?
To test not only presence of knowledge
…but also the application of knowledge
In principle…
Several types of clinical assessment
One model presented in some detail to
provide a framework for later discussion
One model of clinical assessment
• Certification of competence - pass / fail
a state (and legal) requirement
• Grading in ...
One model of clinical assessment
Measurement of:
• adequacy of basic clinical skills
• ability to interpret clinical findi...
Practical steps for assessment
• 6 encounters with different clinical situations
• Two examiners at every encounter, each
...
Practical steps for assessment
• Encounters with at least six real patients
• Ability to interpret and discuss clinical da...
Set-points for testing
• Attitude to patient
• Actual examination skills
• Presentation of findings
• Clinical judgment
Cl...
Set-points for testing
• Evaluation of data
• Significance of data
• Clinical reasoning
Interpretation of clinical data
Set-points for testing
• Ability to solve problems
• Ability to discuss logically
• Clinical judgment and prioritization
•...
Set-points for testing
• Attitude to ‘patient’
• Ability to communicate well
• Clinical judgment
• General medical knowled...
Evaluation
• Subjective evaluation: looking in the mirror
• Objective evaluation of outcome
Evaluating ourselves
“Examinations are
formidable even to the
best prepared…
…for the greatest fool
may ask more than the
...
Evaluating outcome
• Pattern of results
• Consistency of results
• Patterns of marking
• Process shortcomings
• Basis for ...
What happens to candidates who fail ?
• Review of performance – a formative exercise
• Counselling at a personal level
• S...
Points for discussion
• Competence versus performance
• Exam based versus continuous assessment
• Methodology related issu...
Competence versus Performance
• In relation to methods used
• Use of core curriculum with additional
modules as required
•...
Competence versus Performance
• In relation to methods used
• Use of core curriculum with additional
modules as required
•...
Competence versus Performance
• In relation to methods used
• Use of core curriculum with additional
modules as required
•...
Competence versus Performance
• In relation to methods used
• Use of core curriculum with additional
modules as required
•...
Exam based vs continuous assessment
• Assessment of a modular curriculum
• Written exams: MCQs, SAQ, Essays
• Real-time ex...
Exam based vs continuous assessment
• Assessment of a modular curriculum
• Written exams: MCQs, SAQ, Essays
• Real-time ex...
Methodology-related issues
• Reliability
• Validity
• Practicality
• Transparency
Organisational issues
• Resources available
• Student numbers
• Patient availability
• Invasive procedures
• Scheduling
Role of grading
• Is there need for grading ?
• Selection process for employment
• Selection for postgraduate training
• Q...
Conclusions
There is wide diversity among European medical schools
regarding methods of assessment of clinical skills:
• s...
UNIVERSITY OF MALTA MEDICAL SCHOOL
1676 - 2007
EU UNIVERSITY PARTNER
● joseph.cacciottolo@um.edu.mt ● josanne.vassallo@um....
Thank you
ANNUAL CONFERENCE - OSLO - MAY 2007
Upcoming SlideShare
Loading in …5
×

13josanne

295 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
295
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The Breast: the good the bad and the beautiful Title slide
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • Plan for presentation Breasts over the years The breast and politics The medical breast The breast and commerce
  • 13josanne

    1. 1. Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007
    2. 2. Assessing clinical skills 1. WHY do we need to assess ? 2. WHAT do we want to measure ?
    3. 3. WHY do we assess ? • To ensure safety of patients our responsibility to the public • Achievement of a minimum standard responsibility to the candidate and University In principle…
    4. 4. WHY do we assess ? • To ensure competence • Often as a means of academic competition In practice: the scope
    5. 5. WHY do we assess ? • Formative: to give feedback and advice • Summative: to grade • Qualificative or licensing In practice: the purpose
    6. 6. WHAT do we measure ? To test not only presence of knowledge …but also the application of knowledge In principle…
    7. 7. Several types of clinical assessment One model presented in some detail to provide a framework for later discussion
    8. 8. One model of clinical assessment • Certification of competence - pass / fail a state (and legal) requirement • Grading in rank order for employment / placement purposes • A competition for the award of a prize In principle…a three-fold aim
    9. 9. One model of clinical assessment Measurement of: • adequacy of basic clinical skills • ability to interpret clinical findings • facility of communication in practical settings • ability to think analytically about diagnosis • ability to discuss management logically In practice…
    10. 10. Practical steps for assessment • 6 encounters with different clinical situations • Two examiners at every encounter, each examiner giving an individual assessment • Highly structured examination and detailed assessment of skills • Examiners from other Universities for process evaluation and quality control
    11. 11. Practical steps for assessment • Encounters with at least six real patients • Ability to interpret and discuss clinical data • Management of an emergency scenario • Appraisal of communication skills and attitude
    12. 12. Set-points for testing • Attitude to patient • Actual examination skills • Presentation of findings • Clinical judgment Clinical examination of patients
    13. 13. Set-points for testing • Evaluation of data • Significance of data • Clinical reasoning Interpretation of clinical data
    14. 14. Set-points for testing • Ability to solve problems • Ability to discuss logically • Clinical judgment and prioritization • General medical knowledge Management of emergency situations
    15. 15. Set-points for testing • Attitude to ‘patient’ • Ability to communicate well • Clinical judgment • General medical knowledge Communication skills
    16. 16. Evaluation • Subjective evaluation: looking in the mirror • Objective evaluation of outcome
    17. 17. Evaluating ourselves “Examinations are formidable even to the best prepared… …for the greatest fool may ask more than the wisest man can answer” Charles Caleb Colton 1780 - 1832
    18. 18. Evaluating outcome • Pattern of results • Consistency of results • Patterns of marking • Process shortcomings • Basis for improvement Analysis of data to assess effectiveness
    19. 19. What happens to candidates who fail ? • Review of performance – a formative exercise • Counselling at a personal level • Specific attention and individual training • Repeat assessment after a period of time
    20. 20. Points for discussion • Competence versus performance • Exam based versus continuous assessment • Methodology related issues • Organisational issues
    21. 21. Competence versus Performance • In relation to methods used • Use of core curriculum with additional modules as required • Eventual medical practice in different environment • Impact on mobility
    22. 22. Competence versus Performance • In relation to methods used • Use of core curriculum with additional modules as required • Eventual medical practice in different environment • Impact on mobility
    23. 23. Competence versus Performance • In relation to methods used • Use of core curriculum with additional modules as required • Eventual medical practice in different environment • Impact on mobility
    24. 24. Competence versus Performance • In relation to methods used • Use of core curriculum with additional modules as required • Eventual medical practice in different environment • Impact on mobility
    25. 25. Exam based vs continuous assessment • Assessment of a modular curriculum • Written exams: MCQs, SAQ, Essays • Real-time exams • Orals: clinical / table viva, OSCEs • Clinical assessment on site: mini-CEX, DOPS, multi-source feedback • Log books, portofolios, CATs
    26. 26. Exam based vs continuous assessment • Assessment of a modular curriculum • Written exams: MCQs, SAQ, Essays • Real-time exams • Orals: clinical / table viva, OSCEs • Clinical assessment on site: mini-CEX, DOPS, multi-source feedback • Log-books, portfolios, CATs
    27. 27. Methodology-related issues • Reliability • Validity • Practicality • Transparency
    28. 28. Organisational issues • Resources available • Student numbers • Patient availability • Invasive procedures • Scheduling
    29. 29. Role of grading • Is there need for grading ? • Selection process for employment • Selection for postgraduate training • Quality of assessment method and performance of candidates
    30. 30. Conclusions There is wide diversity among European medical schools regarding methods of assessment of clinical skills: • some schools aim at pass/fail outcomes, others use systems that lead to grading • several quality assurance mechanisms are used to varying degrees There is place for widespread application of agreed standard methods to: • assess clinical competences in core curricula • assess additional competences essential to individual practice
    31. 31. UNIVERSITY OF MALTA MEDICAL SCHOOL 1676 - 2007 EU UNIVERSITY PARTNER ● joseph.cacciottolo@um.edu.mt ● josanne.vassallo@um.edu.mt
    32. 32. Thank you ANNUAL CONFERENCE - OSLO - MAY 2007

    ×