Chapter 14 ppt eval & testing 4e formatted 01.10 mo checked

© 2013 Springer Publishing Company, LLC.
Chapter 14
Clinical Evaluation
Methods
&Oermann Gaberson
Evaluation and Testing in Nursing Education
4th edition

Steps in Clinical Evaluation Process
♦ Decide on purpose of evaluation–why are you
evaluating student?
♦ Formative or summative?
♦ Establish standards for evaluation.
– Letter grade or P/F?
♦ Select evaluator.
– Faculty? Preceptor? Self? Multiple?
2

Steps in Clinical Evaluation Process
♦ Decide on evaluation method based on
competencies to be achieved
– Use a variety of appropriate methods
♦ Collect and interpret data
♦ Use results to make decisions about students
♦ Evaluate your process—need for changes?
3

Selecting Clinical Evaluation
Methods
♦ Select based on clinical competencies
♦ Vary the methods
♦ Select realistic evaluation methods to use
♦ Decide if formative or summative
♦ Review purpose and number required of each
assignment
♦ Consider your time
4

Clinical Evaluation Methods
• Observation
• Notes about
performance
• Checklists
• Rating scales
• Simulations
• Standardized patients
• Objective Structured
Clinical Evaluation
• Media clips
• Written assignments
• Journal
5

Clinical Evaluation Methods (cont’d)
• Nursing care plans
• Concept maps
• Case method,
unfolding cases,
case study
• Papers
• Portfolios
• Conferences
• Group projects
• Self-assessment
6

Observation
♦ Predominant method of assessing clinical
performance
♦ Should be guided by clinical outcomes or
competencies
– Help the teacher focus the observation
7

Observation: Threats to Validity
and Reliability
♦ Teacher’s values and biases may influence:
– What is observed
– Inferences and judgments about quality of
performance
♦ Overreliance on first impressions
– Should observe over time and in different
situations before drawing conclusions
8

and Reliability (cont’d)
♦ Focus of observation may differ with each
instructor and time period
♦ Teacher may make incorrect judgments
about the observation
– Discuss observations with students, obtain
their perceptions, and modify inferences
when indicated
9

and Reliability (cont’d)
♦ Every observation in the clinical setting is only
a sample of the student’s performance
– Observation of the same student at another time
may reveal a different level of performance
– Same is true for observations of teacher
performance
10

Documenting Observations
♦ Methods for recording observations and
their context
– Notes about performance
– Checklists
– Rating scales
11

Notes About Performance
♦ Descriptions of observed student performance
– May include a judgment about the quality of
the performance
♦ Should be recorded as soon after the
observation as possible
♦ Methods of recording
– Handwitten notes, flow sheets, narratives
– Using technology such as tablet computer or iPad
12

Notes About Performance (cont’d)
♦ Share with students frequently to give them
feedback about their performance
– Incorporate student input when indicated
♦ Use conferences with students to review
pattern of performance over time
♦ Serve as documentation for ratings on the
clinical evaluation tool
13

Checklist
♦ List of specific behaviors or activities to be
observed and means of indicating whether or
not they were present
♦ Often lists steps of a procedure or skill
♦ Facilitate teacher’s observation of
performance
♦ Facilitate learners’ self-assessment prior to
assessment by the teacher
14

Developing a Checklist
♦ List steps or actions in order
– Focus on critical items and sequencing
♦ List errors that learners often make
– Alerts the assessor to observe for these
♦ Develop into a form to check steps or
behaviors as they are performed
– Allow flexibility of sequencing if multiple ways of
performing a procedure safely
15

Rating Scales
♦ Also called clinical evaluation tools or
instruments
♦ Means of recording judgments about
observed performance of learners
♦ Two parts
– List of outcomes or competencies learner is to
demonstrate in clinical practice
– Scale for rating students’ performance of them
16

Rating Scales (cont’d)
♦ Most useful for summative evaluation of
performance
♦ Also may be used to evaluate specific clinical
activities
– e.g., student’s presentation of a case in
postclinical conference
♦ Useful for giving specific feedback to students
about their performance
17

Types of Rating Scales
♦ Two-point scales
– Pass-fail
– Satisfactory-unsatisfactory
♦ Multiple-level scales
– Letters (e.g., A, B, C, D, F)
– Numbers (e.g., 5, 4, 3, 2, 1)
18

Types of Rating Scales
♦ Multiple-level scales
– Qualitative labels (e.g., “excellent,” “very good,”
“good,” “fair,” “poor”)
– Frequency labels (e.g., “always,” “often,”
“sometimes,” “never”)
♦ Matrix combining different qualities of the
performance
19

Issues With Rating Scales
♦ Interrater consistency (reliability)
– Do all users agree on meaning of ratings?
• e.g., difference between “good” and “excellent”
– Problem even when using descriptors with letters,
numbers, or labels used to rate outcomes or
competencies
20

Issues With Rating Scales (cont’d)
♦ Scales based on frequency labels
– e.g., “always,” “sometimes”
– Limited opportunities for students to practice and
demonstrate a high level of some skills
– How to rate a student who has had only one
opportunity to practice a skill?
21

Common Errors With Rating Scales
♦ Leniency error
– Tendency to rate all students toward the high end of
the scale
♦ Severity error
– Tendency to rate all students toward the low end of
the scale
♦ Central tendency error
– Hesitancy to mark either end of the rating scale
– Tendency to use only the midpoint of the scale
22

Common Errors With
♦ Halo effect
– Basing judgment on a general impression of the student
♦ Personal bias
– Teacher’s personal values or preferences influence ratings
♦ Logical error
– Giving similar ratings to items that are logically related to
one another
23

Common Errors With
♦ Rater drift
– Definition or interpretation of competencies to be
observed and assessed changes over time
♦ Reliability decay
– Over time, teachers may become less consistent
in their ratings
24

Improving Use of Rating Scales
♦ Teachers should have regular discussion of
outcomes or competencies to be rated
– Meaning of each outcome or competency
– Describe what student performance would look
like at each rating level
– May use simulation to facilitate discussion
• Observe a performance, assess it with the rating scale,
and discuss rationale for the ratings
25

Clinical Evaluation Tools
♦ Same tool for all courses or course-specific
tool?
– Most faculties use one tool for all courses
• Competencies adapted to each course
♦ Two-level or multilevel scales?
– Most faculties use pass-fail or satisfactory-
unsatisfactory rating scales
26

Simulation
♦ Activity that allows learners to experience a
clinical situation without the risks and
constraints of a real-life situation
♦ Does not replace actual experiences with
patients
♦ Useful for clinical evaluation as well as
instruction
27

Use of Simulation for
Clinical Evaluation
♦ Case scenarios that students analyze
♦ Computer simulations
♦ Models and manikins
♦ Standardized patients
♦ Human patient simulators
28

Incorporating Simulation Into
Evaluation Protocols
♦ Identify clinical outcomes to be assessed with
simulation
♦ Identify types of simulations needed to assess the
designated outcomes
♦ Determine if simulations are available or need to be
developed by the faculty
♦ Determine if simulation is for formative or
summative evaluation or both
29

Incorporating Simulation Into
Evaluation Protocols (cont’d)
♦ Determine need to develop or obtain
checklists or other methods for rating
performance
♦ Decide when the simulations will be used in
the course
♦ Decide how the effectiveness of the
simulations will be evaluated
30

Standardized Patients
♦ Individuals trained to accurately portray the role of a
patient with a specific diagnosis or condition
♦ Provides consistency in performance evaluation
– Recreate the same patient condition and clinical situation
with each student
♦ Provide written and oral feedback to the students on
their performance
31

Objective Structured Clinical
Examination (OSCE)
♦ Students rotate through a series of stations
♦ At each station they may complete an activity
or perform a task
♦ Performance evaluated with checklists or
rating scales
♦ Usually used for summative evaluation
32

Games
♦ Involve competition, rules (structure), and
collaboration
♦ Individual games and games played against
other students either individually or in teams
♦ Many require props or equipment
♦ Not appropriate for grading; use only for
instruction and formative evaluation
33

Media Clips
♦ Short segments of a digital recording, DVD, video
from YouTube, other forms of multimedia
♦ May be viewed by students as a basis for:
– Discussions (e.g., postclinical conference, online discussion
boards)
– Small-group activities
– Critique in a written assignment
♦ Students can visualize the patient and context
34

Written Assignments
♦ Use: Assess problem solving, higher level
thinking, understanding content relevant to
clinical practice
♦ Misuse: Students complete the same
assignments repetitively throughout a course
after outcomes have been met
35

Written Assignments: Journals
♦ Opportunity for students to describe and reflect on
events and experiences in clinical practice
♦ Means of engaging in dialogue with the teacher
♦ Print or electronic format
♦ Students need to be clear about the purpose of the
journal activity
36

Written Assignments:
Journals (cont’d)
♦ Written guidelines for journal entries
– How many entries?
– What types of entries?
– How frequently?
♦ Teacher gives prompt and meaningful feedback
♦ Decide if graded or ungraded
– Grading inappropriate if journal is used to express feelings,
values, beliefs
37

Nursing Care Plans
♦ Learn components of the nursing process
♦ Weaknesses when used for clinical evaluation
– Linear process—does not facilitate assessment of
complex thinking necessary for clinical practice
– Students often paraphrase from textbook
– Teachers should be cautious about the number of
care plans required in a course
38

Concept Maps
♦ Tool for visually showing relationships among
concepts
♦ Help students organize knowledge as they
plan for clinical activities
– Can “see” how aspects of care relate
♦ Best used for formative evaluation
– Can be graded with criteria established for
evaluation
39

Written Assignments: Case Method,
Unfolding Cases, and Case Study
♦ Methods for assessing higher-level learning
♦ Individual or group activities
♦ Useful for formative evaluation and student
self-assessment
♦ Short cases, unfolding cases, and case studies
can be graded
– Establish scoring criteria for responses to the
questions with the case
40

Written Assignments: Papers
♦ Short papers about clinical practice
– Useful for assessing higher level thinking and
other cognitive skills
– Often better than long papers in which students
may summarize textbook and other literature
– Can be assessed formatively or summatively
41

Papers (cont’d)
♦ Term papers
– Critique and synthesize relevant literature and
relate it to patient care
– Assess drafts formatively to improve writing skills
– Grade final product
42

Portfolio
♦ Collections of products that document
student achievement
– Demonstrate clinical competencies
– Show work completed over period of time
♦ Types—can be combined for clinical
evaluation
– Best work (graded)
– Growth and development (formative)
43

Using Portfolios for
Clinical Evaluation
♦ Identify purpose of the portfolio
– Best work or growth and learning
– Formative, summative, or both
♦ Identify the type of entries and content to be included
in the portfolio
– Content to be chosen by student, faculty member or both?
♦ Decide how portfolio will be evaluated
– Individual entries and the portfolio overall
– Scoring rubric?
44

Electronic Portfolio
♦ Developed and stored electronically
– Facilitates updating and revision
♦ Can be easily sent to others for feedback
or scoring
♦ Limited or no cost
♦ Can be modified for job search
♦ Can include a variety of multimedia
45

Conferences
♦ Method for evaluating competency in oral
communication skills
♦ Can be used to assess higher-level thinking
and problem solving
♦ Can be used for formative or summative
evaluation or both
– If summative, need specific criteria and
scoring rubric
46

Conferences
♦ Types of conferences
– Preclinical
– Postclinical
– Interprofessional
♦ Format
– Face-to-face
– Online
47

Criteria for Evaluating Conferences
♦ Ability of students to:
– Present ideas clearly and logically
– Participate actively in group discussion
– Offer relevant ideas
– Demonstrate knowledge of content
– Offer different perspectives or share reflections
– Assume a leadership role, if relevant, in group
process
48

Group Projects
♦ Short- or long-term work
♦ Can assess products developed by the group
and ability of students to work cooperatively
♦ Rubrics should be used for assessing
– Should be geared to the specific project
– Should reflect the goals of group work
• Students can assess participation and contributions
of peers
49

Grading Group Projects
♦ Group grade
– Each member of the group receives the same
grade
– Does not account for individual student effort and
contribution to the group product
♦ Combination of individual and group grades
– Students identify their contributions to the overall
product; teacher or self-assessment
– Students prepare individual and group projects
50

Self-assessment
♦ Ability of students to evaluate their own
clinical competencies and identify need for
further learning
♦ Difficult for some students to assess their own
performance
– Need supportive environment, guidance, practice
♦ Appropriate only for formative evaluation
51

Clinical Evaluation in
Distance Education
♦ Preceptors may observe and evaluate individual
students
♦ Adjunct or part-time faculty members may evaluate
a group of students at remote site
♦ Intensive on-campus or regional site skill acquisition
workshop
♦ Students may independently demonstrate clinical
skills and procedures in simulation setting
– Record performance for evaluation by teacher
52

Clinical Evaluation in
Distance Education (cont’d)
♦ Critical decision: Which clinical skills and
competencies need to be observed and the
performance rated?
– Determines evaluation methods to use
♦ Should use consistent methods across all clinical sites
♦ Need to orient all evaluators of student performance
– Promotes reliability
53

Chapter 14 ppt eval &amp; testing 4e formatted 01.10 mo checked

More Related Content

What's hot

Viewers also liked

Similar to Chapter 14 ppt eval &amp; testing 4e formatted 01.10 mo checked

More from stanbridge

Chapter 14 ppt eval &amp; testing 4e formatted 01.10 mo checked

Chapter 14 ppt eval & testing 4e formatted 01.10 mo checked

Similar to Chapter 14 ppt eval & testing 4e formatted 01.10 mo checked

Chapter 14 ppt eval & testing 4e formatted 01.10 mo checked