Osce presentation RSA May 2014
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Osce presentation RSA May 2014

on

  • 96 views

Workshop given to the Examiners of the College of Family Medicine in Pretoria. Odi District Hospital May 2014

Workshop given to the Examiners of the College of Family Medicine in Pretoria. Odi District Hospital May 2014

Statistics

Views

Total Views
96
Views on SlideShare
96
Embed Views
0

Actions

Likes
1
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • How suitabler <br /> Not too disruptive to service provision <br /> Quirt enough <br /> Enough space <br /> Not likely to be disturbed <br />
  • Is it available <br /> Standardisation <br /> All in correct position <br />
  • Who are the role players <br /> Patients – pos – real , but neg – sick , ? Ethics , may have to stop the exam , ? Consistancy , cheap <br /> Actors, standard , not going to be sick , expensive <br /> Doctors, nurses or students - ? Consistancy <br />
  • Who are your examiners ? <br /> Ow are examiners selected – how many years qualified, are there further assessmnets needed before selection <br /> Examiner training <br /> Maintainance of skills - ? Regualr training schedule <br /> Demographics of examiners – gender, race <br />
  • Double check arrnagements <br />

Osce presentation RSA May 2014 Presentation Transcript

  • 1. OSCE workshop Dr Ed Parry-Jones RCGP
  • 2. Who am I ?  edwardparry-jones@nhs.net
  • 3. Overview of workshop topics  What is an OSCE  Blueprinting  Case writing  Paperwork  Mark scheme  location  Equipment  Role players  Examiners  The big day  Standard setting  Case assurance / metrics and review  Building and maintaining case bank 
  • 4. OSCE Objective Structured Clinical Examination 
  • 5. Objective - all candidates presented with same test Structured - the marking scheme for each station is structured Clinical Examination - test of skills, behaviour, attitudes and application of knowledge
  • 6. Why use OSCEs in clinical assessment? • careful specification of content • observation of wide sample of activities • increased reliability • interactions between examiner and student are structured
  • 7.  Validity – a qualitative issue, testing the RIGHT things in the RIGHT way   Reliability- Technical aspects of the test, consistency, accuracy, ability to predict who is competent or incompetent . Same level of test wherever or whenever it is taken  
  • 8. OSCE stations 3 categories: Observed Examiner present Non-observed interpretation of clinical data e.g. video, CT scan - no examiner Rest or Preparatory station 
  • 9. Types of OCSE stations history taking/ counselling /explanations
  • 10. Types of OCSE stations Examination skills
  • 11. Types of OCSE stations Data interpretation
  • 12. Resources Simulated patients (trained) Real patients Volunteers (not trained) Models Video/ DVD of patients Films/ scans Equipment for tests/ procedures 
  • 13. Blueprinting  Blueprint written and clinical assessments against each other  Content of the assessment should align with curriculum learning objectives  Use appropriate assessment methods to test different aspects of curriculum  Create a matrix - competencies along one axis, system or specialism along the other  Sample widely across the whole matrix 
  • 14. ●Learning outcome ● ●2.1 Evaluate a patient according to the bio-psycho social approach ●2.2 Formulate and execute, in consultation with the patient, a mutually acceptable, cost- effective management plan ●2.3 Provide comprehensive, continuing care throughout the life cycle incorporating preventative, diagnostic, therapeutic, palliative and rehabilitative interventions ●TOTAL NO OF QUESTIONS ●Key skills to be examined ● ● Interpretation of investigations (e.g. x- rays, ECG, blood tests, etc.) x2 ● ● Diagnostic procedures (e.g. LP, lymph node biopsy etc.)x2 ● Therapeutic skill or procedures (e.g. injection joint) x2 ● ● Emergency management and skills (e.g. resuscitation, primary survey, airway management) x2 ● ● Anaesthetic skills (e.g. spinal anaesthetic, use of local anaesthetic, GA) x1 ● ● Brief counseling (e.g. breaking bad news, behavior change) x2 ● ● Question based on portfolio x1 ● ●Weight ●Stations ●4 ●5 ●3 ● ● Trauma & Orthopedics ● 2 ● ● ● ● ●HIV/AIDS, TB and Malaria ●2 ● ● ● ● ●Women’s and Child Health ●2 ● ● ● ● ●Surgery & Anaesthetics ●2 ● ● ● ● ●ENT, eye, skin, mental health ●2 ● ● ● ● ●General adult medicine ●2 ● ● ● ● ●TOTAL NO OF QUESTIONS ● ● ● ● ●
  • 15. Case writing n plan the construct that the station is testing and write it out as a single sentence e.g. to assess the ability of a candidate to explain colonscopy clearly, accurately and succinctly to a patient e.g. to assess the competence of a candidate in examining a patient’s thyroid gland n refer to your construct while you write 
  • 16. Paperwork scheme  Learning Outcome  Station Title  Skill tested  Context and focus for the station  What the author(s) feel the candidate has to do to pass the station 
  • 17. Marking Scheme n Think about whether to use 2/1/0 2 = done well 1 = adequate 0 = poor/ not done n …..or 1/0 1 = Performed competently 0 = poorly performed/ not done OR other schemes ? Check lists and / or Global score 
  • 18. Location
  • 19. Equipment
  • 20. Role Players
  • 21. Examiners
  • 22. The Big Day
  • 23. Standard setting  Standards are based on judgments about examinees’ performances against a social or educational construct e.g. competent practitioner or student ready for graduation 
  • 24. The Standard Setting Problem Test Result Pass Fail Competent Incompetent
  • 25. Classification Scheme  Relative methods based on judgments about groups of test takers  Absolute methods based on judgments about test questions based on judgments about the performance of individual examinees  Compromise methods Livingston, S.A. & Zeiky, M.J. (1982) Passing scores: a manual for setting standards of performance on educational and occupational tests Educational Testing Service, Princeton
  • 26. Types of Standards   Relative standards/norm referenced methods:  based on a comparison among the performances of examinees  a set proportion of candidates fails regardless of how well they perform e.g. the top 84% pass Eg Borderline group, regression  Absolute standards/criterion referenced methods:  based on how much the examinees know  candidates pass or fail depending on whether they meet specified criteria e.g. examinees must correctly answer 70% of the questions  Eg Angoff
  • 27. Norm-referenced standard Test score distribution 30 % 50 % 80 %
  • 28. Criterion referenced standard 50 % Test score distribution (average group) Test score distribution (good group) Test score distribution (poor group)
  • 29. Case assurance / metrics  Measures of reliability not validity  Cronbach Alpha co-efficient – Central to producing a reliable examination is the idea that every question (or OSCE station)should contribute in the same direction towards the final mark or result.  Thus the marks that candidates get on each item / question / case should broadly correlate positively with each other and with the total mark   Standard error of measurement – the confidence interval around a score 
  • 30. Case / question banks Benefit of saving time writing cases / questions Allow ‘development’ of cases / questions based on metrics Useful for developing a regional / national exam Problems of security
  • 31. Thanks and acknowledgements  Sandy Mather, Tom Owen and John Howard at the RCGP  Angela Hall and Richard Wakefield 