Assessment & Instructional Strategies

1. Educator Development Program School of MedicineVANDERBILT
Assessment
&
Instruc-onal
Strategies
Advanced
Clinical
Elec-ves
(ACE)
or
Ac-ng
Internships
(AI)
at
Vanderbilt
Assembled by
Kim Lomis, MD & Lillian B. Nanney, Ph.D
March 2015

2. Educator Development Program School of MedicineVANDERBILT
Learning Objectives for this Module
After viewing materials in this presentation, a course
director should be able to. . . . .
1. Match learning objectives for an ACE or AI course with
appropriate instructional and assessment strategies for
learners.
2. Design an ACE or AI course that conforms to best
practices that have been established by Vanderbilt’s
School of Medicine.

3. Educator Development Program School of MedicineVANDERBILT
Why
our
emphasis
on
assessment?
• Assessment drives learning.
• Assessment is a 2-way street. It
allows both the learner and the
instructor to determine if the
learning objectives have been
achieved.

4. Educator Development Program School of MedicineVANDERBILT
Assessment
Serves
Dual
Func-ons
Provide
an
Assessment
OF
Learning
This
is
summa4ve
=
used
for
grading
Provide
an
Assessment
FOR
Learning
This
is
forma4ve
=
guides
the
learner
Vanderbilt’s Course Directors are expected to . . . . . .

5. Educator Development Program School of MedicineVANDERBILT
A
New
Compelling
Argument
for
Providing
Medical
Students
with
Rigorous
Assessments
• Years 1 and 2 in the School of Medicine are now
foundational. They are graded as pass/fail.
• Assessments are critical for appraising a
student’s readiness for residency.

6. Educator Development Program School of MedicineVANDERBILT
Responsibili-es
of
the
Assessor
• Assessment
is
one
of
your
professional
du4es.
• Feedback
must
be
based
on
OBSERVATIONS
of
PERFORMANCE

7. Educator Development Program School of MedicineVANDERBILT
Best
Prac-ces
Courtesy
of
the
Standing
Assessment
CommiKee
Category:
Communica4on
to
Learners/Course
Faculty:
• Learning
objec-ves
MUST
be
shared
with
students
and
par-cipa-ng
faculty,
residents,
fellows.
Distribute
your
materials
on
Day
1.
• Ensure
that
each
assessment
is
linked
to
a
learning
objec-ves.
• Disclose
your
grading
procedures
to
students
and
par-cipa-ng
assessors
in
print
and/or
electronic
form.
Also
a
Day
1
ac-vity.

8. Educator Development Program School of MedicineVANDERBILT
Category:
Assessment
Methods
• Grading
must
be
based
on
more
than
one
assessment
method.
Collect
different
perspec4ves
to
ensure
accuracy
and
fairness.
• Performance
data
about
an
individual
student
should
be
solicited
from
mul4ple
team
members
(faculty,
residents,
other
health
professionals).
Category:
Feedback
to
learners:
• Courses
must
provide
forma4ve
feedback,
4med
such
that
there
is
opportunity
for
the
learner
to
improve.
Best
Prac-ces
Courtesy
of
the
Standing
Assessment
CommiKee

9. Educator Development Program School of MedicineVANDERBILT
• Learners have a right to expect fairness and consistency from
course to course. One purpose of this module is to norm the course
directors/assessors.
• Vanderbilt has a responsibility to ensure that learners are
progressing upward along a developmental learning curve. This
determination can only be made if learners are observed and
assessed.
Our Milestones Module is coming soon . . . . .
• Learners have a right to know specific areas where they are not
progressing or are deficient.
To recap - Assessment/observations must be accurate,
well-documented, informative and frequent.
Assessment
Does
MaKer

10. Educator Development Program School of MedicineVANDERBILT
Ideally,
assessment
should
inform
future
learning:
• The
Assessment
should
describes
current
performance
“You
are
here”
• The
Assessment
should
ar-culate
behaviors
necessary
to
aKain
the
next
level
of
performance.
“Go
this
way”
Milestones
=
GPS
(Global
Posi-oning
System)

11. Educator Development Program School of MedicineVANDERBILT
All
3
aspects
of
course
design
must
be
though[ully
aligned
Learning
Objec-ves
Assessment
Strategies
Instruc-onal
Strategies

12. Educator Development Program School of MedicineVANDERBILT
Let’s
focus
on
Learning
Objec-ves
Learning
Objec-ves
Assessment
Strategies
Instruc-onal
Strategies
1. How
will
you
help
students
aKain
them?
This
is
your
instruc-onal
strategy.
2.
How
will
you
ensure
they
have
aKained
them?
This
is
your
assessment
method.

13. Educator Development Program School of MedicineVANDERBILT
A
Learning
Objec-ves
can
best
be
achieved
by
a
specific
type
of
instruc-onal
tool.
Learning
Objec-ves
Assessment
Strategy
Instruc-onal
Strategy
Each
learning
objec-ves
must
be
measurable
(assessable)
and
reasonable
given
the
learning
-me.

14. Educator Development Program School of MedicineVANDERBILT
Examine our example below linking learning objectives
with likely educational activities and then how the learner
will be assessed. Our sketch would be suitable for an
ACE course in Neuroradiology or Neurosurgery.
Learning
Objec-ve
Educa-onal
Ac-vity
Assessment
Create
a
differen-al
diagnosis
of
brain
tumors
based
on
tumor
loca-on
a.
par4cipate
in
weekly
brain
tumor
boards
b.
complete
a
comprehensive
pa4ent
evalua4on
(using
a
template)
for
one
pa4ent
of
interest
during
the
first
2
weeks
c.
view
a
prepared
module
on
this
topic
d.
seek
evidence
based
literature
(epidemiological,
guideline
ar4cles)
on
this
topic.
e.
par4cipate
in
the
radiology
reading
room
experience
when
reports
are
being
prepared.
f.
present
assigned
pa4ents
to
the
team
during
daily
reports.
1. Student
completes
a
second
write-­‐up
by
the
end
of
course
for
an
unknown
tes4ng
case.
2. A
publicized
scoring
rubric
will
be
used
by
Resident/
faculty
to
score
two
pa4ent
presenta4ons
during
each
week.
The
first
2
weeks
are
for
feedback
purposes.
The
last
2
weeks
will
be
summa4ve.

15. Educator Development Program School of MedicineVANDERBILT
Examine another example linking learning objectives with likely
educational activities and then with learner assessment. Our
sketch would be suitable for an ACE course in Endocrinology
(Diabetes).
Learning
Objec-ve
Educa-onal
Ac-vity
Assessment
Gain
proficiency
in
evalua-ng
and
trea-ng
in
pa-ents
with
diabetes
mellitus
and
in
formula-ng
or
modifying
a
treatment
regimen
to
control
blood
glucose
level.
a. Student
will
aWend
the
adult
endocrinology
fellows’
conference
b. Student
will
aWend
rounds
with
the
diabetes
consulta4ve
service
in
the
hospital
c. Student
will
provide
glycemic
care
in
4
seYngs
(periopera4ve,
post-­‐surgical
emergency
and
cardiac
pa4ents).
d. Students
will
prac4ce
educa4ng
pa4ents
regarding
how
to
test
blood
gluose,
dose/administer
insulin,
count
carbohydrate
grams,
use
devices
such
as
insulin
pens.
1. Assessment
of
the
quality
of
pa4ent
educa4on
will
be
assessed
by
direct
clinical
observa4ons
on
the
glucose
management
service
once
a
day.
2. Assessed
by
oral
exam
from
a
set
of
prepared
cases
and
graded
by
a
scoring
rubric
familiar
with
to
the
learner.
3. Quality
of
glycemic
care
decisions
in
daily
prac4ce
will
be
scored
once
a
day
by
residents/fellows
on
the
service.

16. Educator Development Program School of MedicineVANDERBILT
Now
its
your
turn
for
prac-ce.
Here
is
a
simple
table
to
get
you
started.
Write
3
plans
for
your
course.
This
VSTAR
module
also
contains
a
word
doc
with
a
table
you
can
download
and
use.
Learning
Objec-ve
Educa-onal
Ac-vity
Assessment
Strategy
1.
2.
3.

17. Educator Development Program School of MedicineVANDERBILT
Who
will
be
the
Assessors?
Consider:
Other Faculty
The Residents
The Staff
1) Will they need some training so they can be as standardized as possible?
2) Should you furnish them with a checklist or rubric?
3) Do you need to do some “faculty” development?
Do they need to see the linkage between learning objectives, instructional
strategies, and assessments so they have clarity about expectations
for learners at this level?
Answers: Yes, Yes, Yes

18. Educator Development Program School of MedicineVANDERBILT
Unconscious
Bias
Share
these
reminders
with
your
assessors
Halo Error
Outstanding performance in one area, particularly early in the rotation,
tends to blind observers to poorer performance in other domains.
Similarly, it can be hard for a student to recover from a poor start.
Wave Effect
If everyone else has felt that the learner is superior, it can be difficult to
give an unsatisfactory assessment. The student is ‘”waved” through the
rotation on reputation and not on performance.
Hawk or Dove Effect
Take a close look at the new grading scale that Vanderbilt will be using.
Are you too hard, too easy?
Use the rating tools appropriately to ensure fair grading
(Stay turned for the Advanced Assessment Module)
.

19. Educator Development Program School of MedicineVANDERBILT
Similarity/Dissimilarity Bias
Recognize something of yourself in the Learner?
- Likely a good assessment will result.
Likewise, it can be hard to give a high rating to someone with a
very different practice style.
Prejudices
Accents, beards, gender, lip rings, tattoos?
Be aware of your prejudicial tendencies (blindspots)
that can interfere with fair assessment.
Unconscious
Bias
(con-nued)
Share
these
reminders
with
your
assessors

20. Educator Development Program School of MedicineVANDERBILT
Learning
Objec-ves
Assessment
Strategy
Instruc-onal
Strategy
Learning Objective: Interpret clinical information to formulate
a prioritized differential diagnosis that guides the creation of a
patient-specific management plan.
Assessment Strategy: SNAPPS
S - summarize the case
N - narrow the differential
A - analyze the differential
P - probe the preceptor
P - plan management
S - select an issue for self directed learning

21. Educator Development Program School of MedicineVANDERBILT
Consider
these
Using
these
Tools
• 1
minute
preceptor
technique
hWp://www.prac4caldoc.ca/teaching/prac4cal-­‐prof/teaching-­‐nuts-­‐bolts/one-­‐
minute-­‐preceptor/
• SNAPPS
technique
(your
departmental
Master
Clinical
Teacher
can
host
a
teaching
session)
hWp://www.prac4caldoc.ca/teaching/prac4cal-­‐prof/teaching-­‐nuts-­‐bolts/snapps/
• Mini-­‐Clinical
Evalua4on
Exercise
(Mini-­‐CEX)
hWp://www.abim.org/program-­‐directors-­‐administrators/assessment-­‐tools/mini-­‐cex.aspx
• S4mulated
chart
review
hWp://www.prac4caldoc.ca/teaching/prac4cal-­‐prof/teaching-­‐nuts-­‐bolts/chart-­‐s4mulated-­‐
recall/
• Case
Discussions
• Conference
Presenta4ons

22. Educator Development Program School of MedicineVANDERBILT
Use Verbal Feedback Techniques
§ Continue (You are on track)
§ Do More (of “this”)
§ Do Less (of “this”)
§ Stop (Don’t do “this” again!)
Considerations:
• Keep portions small and feedback spaced out over time
• Students tend to selectively “hear” good bites and miss the critique.
• Best
prac4ce
assumes
that
feedback
is
delivered
in
a
rela4vely
private
seYng.

23. Educator Development Program School of MedicineVANDERBILT
Standing
Assessment
CommiKee
(recapping
the
baseline
requirements)
1) Use
Mul4-­‐Modal
Tools
-­‐
to
gain
many
perspec-ves
2) Distribute
the
plan
to
students
-­‐
remove
the
mystery
around
“What
am
I
supposed
to
learn?”
3) Distribute
the
plan
to
all
the
possible
assessors
-­‐
eliminate
all
mystery
around
“What
am
I
supposed
to
do
with
this
student.”
4) Provide
frequent
feedback
-­‐mid
and
end-­‐of-­‐course
at
minimum
4) Final
grades
due
<
6
weeks

24. Educator Development Program School of MedicineVANDERBILT
Upcoming
Modules
•
Milestones
are
coming
to
Clinical
Evalua4on
We
will
share
which
competencies
must
be
assessed.
• Master
Adap4ve
Workplace
Learner
Cycle
We
will
be
helping
students
improve
their
prac4ce-­‐based
learning
skills
and
develop
the
habits
necessary
for
lifelong
learning.
Courses
will
be
asked
to
include
assessment
of
the
student’s
progress
through
the
Master
Adap4ve
Workplace
Learner
cycle.
More
informa4on
to
come
on
this
soon.