Innovation in medical education

Innovation in Residency Education
Author: M. Ladhani

Objectives:
• Identify common problems faced by program directors
and potential innovative solutions.
• Describe examples of innovations in medical education
from the pediatric residency program at McMaster
University.
3

Problem 1: Attendance at AHD
5

Mandatory Group Learning Activities (MGLA):
Getting Residents Ready for Lifelong Learning.
Author: Moyez Ladhani,
Saleem Razack

Background:
• The Royal College of
Physicians of Canada’s
CanMEDS project defines
the roles of a physician to
include seven important
competencies
• Postgraduate training
programs must
incorporate the teaching
and evaluation of the
CanMEDS roles.
7
Mandatory Group Learning Activities (MGLA): Getting Residents Ready for Lifelong Learning. | Moyez Ladhani,
Saleem Razack

Background
• Our curriculum for our residents includes
mandatory and other teaching sessions.
• There is a large resource of formal didactic,
interactive and case based sessions available for
residents to meet their learning objectives.
8
Saleem Razack

However..
Faculty:
“Moyez…I was at AHD and the attendance was
poor, a lot of effort went into my talk and if
residents aren’t going to show up…etc.”
Residents:
“ I was taking the time to get some personal
matters attended to..”
Banana Republic had their in-store only 40% sale
that I couldn’t miss….”
9
Saleem Razack

Background
10
Saleem Razack
However there was no accountability for
residents’ attendance at the numerous
activities available to them.

Scholar Role
• An important aspect of the Scholar role is that:
1. Physicians are expected to maintain and
enhance professional activities through on-going
learning
2. Describe the principals of maintenance of
competence, and
3. Document their own learning processes, to
name a few.
11
Saleem Razack

Manager:
• The Manager role expects physicians to
implement processes to ensure personal practice
improvement and balance patient care, practice
requirements and outside life.
12
Saleem Razack

RCPSC
• In specialty practice beyond postgraduate training,
the RCPSC expects its fellows to participate in the
Maintenance of Certification (MOC) process for
maintenance and renewal of their fellowship.1
• We also know that educational meetings improve
professional practice and healthcare outcome for
patients.2
1. Frank,
JR
(Ed).
2005
The
CanMEDS
2005
physician
competency
framework.
The
Royal
College
of
Physicians
and
Surgeons
Canada.
2. Forsetlund,
L
et
al.
Continuing
education
meetings
and
workshops:
effects
on
professional
practice
and
health
care
outcomes.
Cochrane
Database
Syst
Rev.
2009
Apr
15;(2).
13
Saleem Razack

The Objectives of the MGLA is to:
• Develop skills required of them for the Scholar
and Manager roles.
• Maintain and document minimum attendance at
various teaching sessions
14
Saleem Razack

Getting Ready for MGLA
1. We identified sessions that should be
mandatory and sessions that were optional.
Sessions were grouped into 5 categories. For
mandatory sessions residents are protected to
attend from their clinical duties.
2. Credits were determined for minimal
attendance for each of the session groupings;
adjustments were made for year, vacation time,
post-call days and time away on electives.
15
Saleem Razack

3. Residents were required to document and
maintain their attendance. This profile was also
maintained in their CanMEDS portfolios.
4. Failure to maintain their minimal requirements
put the resident at risk for promotion.
5. Residents maintain their activities on the
Resident Activity Log
16
Saleem Razack

17
Saleem Razack

18
Saleem Razack

19
Saleem Razack

20
Saleem Razack
Reflection is relevant during and after educational activities or
clinical encounters. In this way, specialists develop greater
understanding, integrating and translating new knowledge or
skills, and in forming future action.3
A Continuing commitment to lifelong learning. Guide to maintenance of certification. The Royal College of Physicians and Surgeons
Canada.

Conclusions:
• This process has not only helped improve attendance but
provides a realistic “dry run” for residents with respect to
what will be expected of them as future life long learners,
Scholars and Managers.
21
Saleem Razack

Comments for Former Residents
• “ I liked the system - it did give me some experience in
keeping track of learning. Mainly, I found it useful to keep
track of my teaching, which along with my notes, helped
me with my learning objectives”
• “The MGLA system was helpful for organization and
recognition of different physician roles…. MGLA prepared
me- even just for the concepts- of having to be a manager
and scholar”
22
Saleem Razack

Comments for Former Residents
• “Keeping lists of hours of educational events attended has
some value and is not too time consuming”
• “ This has helped both as a transition to mainport and,
given that everyone does not attend everything, to have a
record that as a resident you have attended a reasonable
amount of formal learning (i.e. gives some independence
and flexibility to the learning process)”
23
Saleem Razack

MGLA:
• Three Take Away Points:
1. Models the future accountability required of
residents with respect to the ongoing
maintenance of certification.
2. Improved attendance at protected and other
teaching activities.
3. Helps teach the CanMEDS role of Scholar and
Manager.
24
Saleem Razack

Update
• All our resident now have to be a resident affiliate with the
Royal College.
• This allows them to now log their activities with the
MAINPORT APP
• As of July 1, 2013, Resident Affiliates who document
learning activities in MAINPORT during their residency
program can transfer up to 75 credits (25 in each of the
MAINPORT categories of Group Learning, Self-Learning
and Assessment) into their first five-year Maintenance of
Certification (MOC) cycle following certification.
25
Saleem Razack

Problem 2: Teaching CanMEDS
26

Longitudinal CanMEDS Competencies (LCC): Using
experiential learning to implement CanMEDS roles in
post-graduate medical education
Author: Moyez B. Ladhani

Background:
• The Royal College of
Physicians of Canada’s
CanMEDS project defines
the roles of a physician to
include seven important
competencies
• Postgraduate training
programs must
incorporate the teaching
and evaluation of the
CanMEDS roles.
28
Longitudinal CanMEDS Competencies (LCC): Using experiential learning to implement CanMEDS roles in post-graduate
medical education | Moyez B. Ladhani

Background:
• Dr. Ladhani(PD)
• I have been been struggling with implementing a
curriculum for the so called non-medical expert (NME)
roles or intrinsic roles in my pediatric residency program.
The International Conference on Residency Education | La conférence internationale sur la formation des résidents
29

Background:
• The Residents:
• “having CanMEDS teaching at AHD takes away from our
medical expert teaching” “Sitting in a large group and
listening to a speaker talk on how to collaborate is not
useful to my learning”
• The Literature:
• While NME roles are highly valued, there are challenges
in terms of current strategies for teaching and assessment
of these roles.
• Words such as “frustrating”, “nebulous”, “poorly defined”
and “difficult” were used to describe efforts to translate
the NME competencies into curriculum
• Whitehead C, Martin D, Fernandez N, Younker M, Kouz R, Frank J, Boucher A. Integration of CanMEDS Expectations and
Outcomes. Members of the FMEC PG consortium; 2011
30

Background:
• There is strong support for the CanMEDS construct of a
‘good doctor’ as requiring qualities beyond biomedical
expertise.
• The CanMEDS definitions of these NME roles are highly
endorsed by both residents and faculty members as
appropriately capturing the essential elements of a
competent and socially responsible physician.
• Whitehead C, Martin D, Fernandez N, Younker M, Kouz R, Frank J, Boucher A. Integration
of CanMEDS Expectations and Outcomes. Members of the FMEC PG consortium; 2011
31

Experiential Learning
• is the process of making meaning from direct experience
• Aristotle once said, "For the things we have to learn
before we can do them, we learn by doing them”
32

Experiential Learning:
• According to David Kolb, knowledge is continuously gained
through both personal and environmental experiences. In order to
gain genuine knowledge from an experience, certain abilities are
required:
1. the learner must be willing to be actively involved in the experience;
2. the learner must be able to reflect on the experience;
3. the learner must possess and use analytical skills to conceptualize the
experience; and
4. the learner must possess decision making and problem solving skills in
order to use the new ideas gained from the experience.
33

LCC:
• The goal of the curriculum was to teach non-medical
expert CanMEDS competencies.
• Residents are taught the multi-facetted Roles they will be
called upon to play in their professional duties
• The Curriculum covers the 6 intrinsic domains
34

Format:
• The curriculum is a three year curriculum, one hour every
third Tuesday of the month.
• Residents are in small groups of 11-12 residents ( PGY
1-4) facilitated by two faculty members. “A safe place”
• Residents are provided advance-reading material and are
expected to come to the session prepared for discussion.
The sessions involve discussion, role play, video scenarios
and occasional lecture format.
35

Format:
• The facilitators are there to facilitate and are not
necessarily content experts. They may or may not lead
the discussions.
• Groups may consider rotating the residents to lead the
discussions
36

The Cases:
37

Case Example:
38

Case Example:
39

Case Example:
40

Reflections:
• Reflection: Residents complete two reflections per year.
The residents are expected to share their reflections
during sessions scheduled:
1. The residency journey
2. Physician well being
41

Evaluation:
• There are 4 key sets of skills and behaviours upon which
students are evaluated by one another and their
facilitators.
• Students are expected to demonstrate proficiency along
all four domains and to continue to maintain/improve over
time.
1. Accountability/Respect
2. Respectful Listening
3. Balancing Inquiry and Advocacy
4. Taking Experiential Education Seriously
42

Students Satisfaction:
100
90
80
70
60
50
40
30
20
10
43
0
Strongly Agree
Agree
Somewhat agree

Resident Comments:
• “Groups allowed for open and honest discussions.”
• “Good mix of medical expert, resident wellness,
communicator”
• “Interactive materials (like on line modules) are good
since they are engaging”
• “I think learning CanMEDS competencies in smaller groups
was more interactive”
• “The one good thing is the actual content. I feel it will be a
better environment to discuss such topics rather than a
half-day session”
• “I feet that this format will help us build skills on how to
work as a team”
44

Conclusions:
• It is important to find ways to help educators and trainees
appreciate the intricate associations between the expert
role and all other roles.
• Integration of other roles with that of Medical Expert helps
to highlight the fact that the competent physician draws
upon various roles simultaneously.
• Integration of roles teaching and assessment into clinical
contexts gives practical relevance to the roles.
45

LCC:Longitudinal CanMEDS
Curriculum
• 3 Take-Away Points
1. A traditional half day does not allow for discussion and
sharing.
2. The LCC curriculum uses experiential learning in small
group format allowing residents to discuss their
experiences
3. The LCC curriculum helps learners to incorporate the
CanMEDS competencies to patient and self-care
46

Discussion
47

Update
• Residents now suggesting topics and volunteering to write
the case scenarios often as part of their teaching block.
48

Problem 3: The Right Shift!
49

Mini-MAS: A Work Based Assessment Tool to Assess
Milestones
Author: Moyez B. Ladhani

• PD
• “I need a better assessment tool to help me assess the
residents”
• Residents
• “No one watches me during my day to day work”
51

Work Based Assessment
• Knowledge, skills and attitudes should be assessed using a
multi-faceted longitudinal approach (Sherbino & Frank, 2011).
• The use of a multi-modal assessment strategy can
overcome the limitations of any one-assessment format.
• Longitudinal assessment monitors ongoing development
and avoids excessive testing at any one point (Cox, Irby, &
Epstein, 2007).
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents
52
Mini-MAS/Ladhani

• Miller (1990) suggests the achievement of competence
progresses from “knows” to “knows how” to “shows how”
to “does”.
53
Mini-MAS/Ladhani

DOES
SHOWS
HOW
Faculty Observation
KNOWS HOW
KNOWS
Standardized Patients
MCQs
Impact on Patient
Clinical vignettes
Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9 Suppl), S63-7.

WBA
• Kogan and Holmboe (2013) define WBA:
• “WBA is the assessment of trainees and physicians across
the continuum of day to day competencies and practice in
authentic, clinical environments…..it enables the
evaluation of performance in context” (p. S68).
55
Mini-MAS/Ladhani

WBA
• The In-Training Evaluation Report (ITER):
» Does not discriminate (Gray, 1996; Holmboe & Hawkins, 1998 )
» Completed retrospectively (Turnbull et al., 1998).
» Often faculty who have not observed are completing the
form (Epstein, 2007)
» Halo effect (Wilkinson & Wade, 2007)
» Raters fail to use the entire scale (Gray, 1996)
56
Mini-MAS/Ladhani

Mini-CEX
• The scale used in the mini-CEX is designed for linear
gradations of performance. The scores do not give the
evaluators a point of reference to help align a trainee to a
category or score (Crossley & Jolly, 2012).
• Faculty assessors resort to norm-referencing.
57
Mini-MAS/Ladhani

Mini-CEX
• Hawkins et al. (2010) The raters did not use the full nine-point
scale, and the distribution is right shifted towards
the higher end of the scale, the use of the lower end of
the scale was infrequent raising concerns about identifying
weaknesses.
• Individual competencies tended to be highly correlated.
This phenomenon is likely related to the fact the rating
form had overlapping descriptors
58
Mini-MAS/Ladhani

Designing a Better Tool
• WBA assessment tools should have anchors linked to the
construct of clinical independence, measuring the trainees’
level of progression and development (Crossley et al.,
2011).
• Assessors make more reliable judgments of performances
they can see clearly in a particular context or activity.
• The tool should focus on the competence relevant to the
activity, and avoid having multiple competencies to assess
at the same time (Crossley & Jolly, 2012).
59
Mini-MAS/Ladhani

The Campaign for McMaster University
Figure 2 General Curve of skills Acquisition. (ten Cate et al., 2010)

61
Mini-MAS/Ladhani
2 a week, one competency at a time, 6 competencies 40-44

6 Competencies
The assessment of competencies was limited to six core
competencies:
» history taking,
» physical exam,
» clinical reasoning,
» communication with families,
» communication with physicians and other health care
professionals and
» collaboration
62
Mini-MAS/Ladhani

• A total of 21 residents were included in the study.
• 12 PGY 1 residents at McMaster Children’s Hospital were required to
have completed 40 unique observations in six domains over 22
weeks.
• In addition, nine PGY 4 residents were also required to be observed
for the same competencies over the same time period. This group
was required to complete 15-20 encounters as their general
pediatric exposure was less during this final year of training.
• Faculty and Senior residents assessors were trained-Process
63
Mini-MAS/Ladhani

Results
64
Mini-MAS/Ladhani

Mini-MAS/Ladhani
The International Conference on Residency Education | La Conférence internationale sur la formation des résidents 65

Mini-MAS/Ladhani

74
Mini-MAS/Ladhani

Resident Comments
• “ I do believe this will influence my learning positively”,
• “the Mini-MAS book is a good tool for assessing the
learners in their visual environment” and
• “The Mini-MAS is arguably a good tracking tool…given the
culture of medicine and the temptation for trainees to
avoid supervision”
75

Conclusion
• Work based assessment tool should achieve three
requirements;
» the competences expected as outcomes and the
assessment should be aligned;
» feedback is provided during and or after the assessment
» the assessment is used to guide a trainee towards a
desired outcome (Norcini & Burch, 2007).
• The Mini-MAS tool has achieved these three requirements
and has shifted the focus of assessment from assessment
of the trainee to learning of the trainee. (Driessen &
Scheele, 2013)
76
Mini-MAS/Ladhani

Conclusion
• The Mini-MAS added as a formative assessment mode to a
multi-modal assessment program will benefit the trainee,
informing them on where they stand compared to their
level of training, what competencies they can improve on
and how they can do that.
77
Mini-MAS/Ladhani

Features of Successful Change
• Educational institutions with a history of effective change
are more likely to implement new innovations successfully.
• Successful change efforts are characterized by:
» (a) having a strong, influential advocate at the forefront
of the change effort.
• Organizations with high interaction, connection, and
networks of participatory teams are better able to
accomplish broad change than ‘‘segmental,’’
‘‘departmentalized,’’ or ‘‘loosely coupled’’ organizations
• There needs to be widespread agreement that the
innovation is relevant to the real problems that potential
users currently experience. 80

• Avoid overly ambitious undertakings. A balanced approach
that engages the organization members’ commitment and
support will be most successful.
• The importance of a positive, respectful work climate to
successful curricular change cannot be overstated.
• Successful innovation is fostered by frequent, timely,
substantive, and forthright communication.
• Faculty development and training is vital
81

• Formative evaluation is useful in locating difficulties and
solving problems .
• Stable leadership is positively associated with successful
innovation.
• Bland et al, Curricular Change in Medical Schools: How to Succeed, ACADEMIC MEDICINE, VOL. 75, NO .6 / JUNE 2000
82

Innovation in medical education

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Innovation in medical education

Similar to Innovation in medical education (20)

Recently uploaded

Recently uploaded (20)

Innovation in medical education