This document outlines the Pan-Canadian Practice Ready Assessment for IMG Physicians, which aims to establish competency-based standards for provisional licensure in family medicine. It discusses the background and challenges with integrating International Medical Graduates. The Practice Ready Assessment is presented as a process using Miller's pyramid to assess clinical competence through point-in-time and over-time evaluations. Standards are developed in collaboration with various stakeholders and focus on what candidates can do rather than how assessments are implemented. Assessments occur in practice environments over 12 weeks using multi-source feedback from patients and colleagues to determine practice readiness.

1. Pan-Canadian Practice Ready Assessment for IMG Physicians:
A competency-based assessment for provisional
licensure in family medicine
Cindy Streefkerk, Dan Faulkner, Lauren Copp, Sydney Smee, PhD, André De Champlain, PhD, Timothy Allen, MD
Ottawa Conference
27 April 2014 - Ottawa

2. Disclosure Statement
I have no actual or potential conflict of interest in relation to this
presentation.


3. © NAC / MCC 2014
1. Background
2. Practice-Ready Assessment as a
process
3. Family Medicine PRA standards
3
Overview

4. © NAC / MCC 2014
25 per cent of practising physicians in
Canada are International Medical
Graduates (IMGs)
Rural and remote areas are the most
underserviced
13 jurisdictions govern licensure
4
Background

5. © NAC / MCC 2014
Ensure public protection
Credentials and experience are often
“unknowns”
No equivalent processes across
jurisdictions
• Complex pathways
Limited capacity to integrate IMGs
5
IMG Physician Situation & Challenges

6. © NAC / MCC 2014
1. Entry into Canadian Residency
2. Entry into Practice through
provisional licensure
a) Direct – credentials only
b) Practice-ready Assessment
6
IMG Physician Routes to Practice

7. Today: Current Processes for Entry-to-Practice

8. © NAC / MCC 2014 8* Miller’s pyramid of competence
Miller’s Pyramid & PRA – Assessing
Clinical Competence
NAC PRA Type
In Practice Assessment
Over-Time
Assessment
Selection
(Interactions with trained patients and
assessors - OSCE)
Point-in-
Time
Assessment
Selection
(Therapeutics, CDM, short-answer)
Screening
(MCQ – MCCEE)
SHOWS
HOW
DOES
KNOWS
HOW
KNOWS

9. © NAC / MCC 2014
Medical Regulatory Authorities (9) and
Federation of Medical Regulatory
Authorities
PRA Programs (8)
Certifying College – College of Family
Physicians of Canada
Provincial Funders (8-9)
Federal Government (Health Canada as
a project funder)
9
Stakeholders – In Collaboration

10. © NAC / MCC 2014
Standards as a starting point
• Precluded judgment of what was currently happening
• Set a common goal and expectations of each other’s
processes
• Focused on “what” not “how”
Ultimately, to trust in each others rigour for
provisional licensure purposes
10
Approach to pan-Canadian PRA

11. © NAC / MCC 2014
Baseline of current practices and
processes
Reviewed and synthesized possible
processes and tools into standards
Integrated existing practices into the
common standards, with a view to
improve
11
An AIterative Process

12. © NAC / MCC 2014
Follow a sampling framework
Sample as many observations as possible
More assessors are always better
Target assessment tools to competencies of
interest
Provide ongoing, structured feedback to
candidates
12
Results – Over-Time Assessment
Process

13. 13
Results: What a Family Physician Does
Sentinel habits define essential, priority skills that
are comprehensive and easily recognizable in busy
clinical settings
1: Incorporates the patient’s experience
and context into problem identification
and management
5: Uses generic key features when
performing a procedure
2: Generates relevant hypotheses
resulting in a safe and prioritized
differential diagnosis
6: Demonstrates respect and/or
responsibility
3: Manages patients using available best
practices
7: Verbal or written communication is
clear and timely
4: Selects and attends to the appropriate
focus and priority in a situation
8: Seeks out and responds appropriately
to feedback

14. 14
Results: Who they see
Clinical domains define the various populations and
activities that physicians encounter in clinical
settings
1: Behavioural medicine/mental health
5: Care of the vulnerable and
underserviced
2: Care of adults 6: Maternity/newborn care
3: Care of children and adolescents 7: Palliative care
4: Care of the elderly 8: Procedural skills

15. © NAC / MCC 2014
Assessment occurs in a practice
environment (community-based)
• Rich in patient care opportunities
Allow candidates time to acclimatize
Allow adequate time to assess response
to feedback
Should not take longer than 12 weeks to
determine practice-readiness for
provisional licensure
15
Results: Assessment place & time

16. © NAC / MCC 2014
Experienced, competent family
physicians
Hold a licence to practise medicine &
be in good standing
At least three years of practice in
Canada
Receive
• Ongoing support & feedback
• Training & orientation
16
Results: Who is the Assessor

17. 17
Results: Over-Time Assessment Toolkit
Multi-Source Data
Chart-Based
Components
Continuous Clinical
Assessment
DEFINED
Focus is on communicator,
collaborator & professional
roles
• Chart stimulated recall
• Chart audits
• Case-based discussions
• Mini-CEX
• DOPS
• CBAS
• Field notes
STANDARD
• Feedback comes from
patients & professional
colleagues
• Feedback is documented
• Demonstrates ability to
meet regulatory standards
for charting
• Observation of chart-based
assessments are
documented
• Observations cover all sentinel
habits across all clinical domains
• Observations occur across time &
patient problems
GUIDELINE
Ideally, feedback comes from:
• Minimum of 15 patients
sampled as broadly as
possible across
demographics & problems
• 5-8 professional colleagues
(MD & non-MD)
Assessor judgement
determines the number of
charts for review
• More than one clinical setting
may be required to ensure
appropriate sampling
• Ideally,
• If field notes only, one/day
totaling 40-80
• If mini-CEX (or equivalent),
one/week totaling 8-12

18. © NAC / MCC 2014
Future standards as a first step
• In a complex environment – varying processes
• With multiple stakeholders with a perspective
• Opportunity to bring subject matter expertise to the
table
To ultimately ensure
• Jurisdictions trust each others rigour
• Candidates have a fair/consistent process
18
Summary

19. Thank you!
Questions, comments, concerns?
Cindy Streefkerk - cstreefkerk@mcc.ca