More Related Content
Similar to Chapter 18 ppt eval & testing 4e formatted 01.10 kg edits
Similar to Chapter 18 ppt eval & testing 4e formatted 01.10 kg edits (20)
More from stanbridge (20)
Chapter 18 ppt eval & testing 4e formatted 01.10 kg edits
- 1. © 2013 Springer Publishing Company, LLC. 1
Chapter 18
Program Assessment
&Oermann Gaberson
Evaluation and Testing in Nursing Education
4th edition
- 2. © 2013 Springer Publishing Company, LLC. 2
Program Assessment
♦ Process of judging the worth or value of an
educational program
♦ Purposes
– Provide data on which to base decisions about the
program
– Provide evidence of educational effectiveness in
response to internal and external demands for
accountability
- 3. © 2013 Springer Publishing Company, LLC. 3
Program Assessment Models
♦ Many models available
♦ Three commonly used models in nursing
education programs
– Accreditation
– Decision oriented
– Systems oriented
- 4. © 2013 Springer Publishing Company, LLC. 4
Accreditation Models
♦ Examples
– NLNAC
– CCNE
– CASN
– The Joint Commission
♦ Assess whether the program meets external
standards of quality
♦ Use combination of internal and external assessment
methods
- 5. © 2013 Springer Publishing Company, LLC. 5
Decision-Oriented Models
♦ Examples
– CIPP model
– CQI model
♦ Provide information to decision makers for program
improvement purposes
– No guarantee that corrective action will be taken if it is
indicated
♦ Usually focus on internal standards of quality, value,
and efficacy
- 6. © 2013 Springer Publishing Company, LLC. 6
CIPP Model
♦ Context: What needs to be done?
– Program needs, problems, strengths, weaknesses
♦ Input: How should it be done?
– System capabilities, work plans, budgets
♦ Process: Is it being done?
– Monitor implementation, make changes as needed
♦ Product: Did it succeed?
– Impact, effectiveness, sustainability
- 7. © 2013 Springer Publishing Company, LLC. 7
CQI Model
♦ Based on assumption that those who are
closest to a process know it best
♦ 4 phases
– Plan
– Do
– Study
– Act
- 8. © 2013 Springer Publishing Company, LLC. 8
CQI Model
♦ Plan
– Define problems and opportunities for
improvement
– Collect and organize data
– Decide on improvement initiatives
♦ Do
– Identify needed resources
– Implement planned initiatives
(cont’d)
- 9. © 2013 Springer Publishing Company, LLC. 9
CQI Model (cont’d)
♦ Study
– Monitor progress of improvement initiative
– Observe the effects of the change
♦ Act
– Modify improvement as needed
– Incorporate change into program structure and
policies
– Educate stakeholders about the change
- 10. © 2013 Springer Publishing Company, LLC. 10
Systems-Oriented Models
♦ Assessment of inputs into the program
– Characteristics of students, teachers, administrators, other
participants
– Program resources
♦ Assessment of program processes, operations
– Program implementation
– Context within which the program is implemented
♦ Assessment of program outcomes
– Intended outcomes being achieved
– Stakeholder satisfaction
– Program quality and cost-effectiveness
- 11. © 2013 Springer Publishing Company, LLC. 11
Stakeholders
♦ Judge worth or value of an educational
program
– Affected directly or indirectly by the decisions
made
– Key stakeholders of nursing education programs
• Students
• Faculty and staff members
• Partners (health care and community agencies)
• Consumers
- 12. © 2013 Springer Publishing Company, LLC. 12
Stakeholders
♦ Assessment purpose determines stakeholders to involve
– Formative assessment (program improvement)
• Students
• Teachers
• Administrators
– Summative assessment (whether program should be continued,
revised, closed)
• Program participants
• Graduates and their employers
• Prospective students
• Health care and community agencies
• Consumers
• Legislative and regulatory bodies
- 13. © 2013 Springer Publishing Company, LLC. 13
External vs. Internal Evaluators
♦ Some program assessment models use both
♦ External evaluator
– May be more objective
– May not know program or its context well enough
– Additional expense
♦ Internal evaluator
– Better understanding of program operations and context
– Can provide ongoing feedback
– May be biased
- 14. © 2013 Springer Publishing Company, LLC. 14
Program Outcome Assessment
Tools
♦ Teacher-made or standardized?
♦ Must possess qualities of effective
measurement instruments
♦ Teachers must be able to make valid and
reliable interpretations of standardized test
results
♦ Should be chosen based on the outcomes to
be measured
- 15. © 2013 Springer Publishing Company, LLC. 15
Limitations of Outcomes
Assessment
♦ Some intended outcomes may not be important or
related to stakeholders’ needs
♦ May provide feedback only at the end of a program
♦ To contribute to program improvement, program
assessment also should examine
– program goals
– structure
– process
– “side effects”
- 16. © 2013 Springer Publishing Company, LLC. 16
Curriculum Assessment
♦ Focus is more narrow than program
assessment
– Focus on elements central to the course of studies
taken by students
– Program assessment includes additional elements
• Institutional support for the program
• Administrative structure
• Faculty qualifications and productivity
• Student support services
- 17. © 2013 Springer Publishing Company, LLC. 17
Curriculum Assessment
♦ Focus of curriculum assessment has changed
over time
♦ Classification of assessment models
– First generation
– Second generation
– Third generation
– Fourth generation
- 18. © 2013 Springer Publishing Company, LLC. 18
Focus of Curriculum Assessment
♦ First-generation model
– Measure to obtain student scores representing curriculum quality
♦ Second-generation model
– Describe congruence between student performance and curriculum
objectives
♦ Third-generation model
– Judge the merit of performance in relation to standards
♦ Fourth-generation model
– Develop holistic understanding of needed improvements based on
input from stakeholders
- 19. © 2013 Springer Publishing Company, LLC. 19
Curriculum Assessment Elements
♦ Curriculum outcome statements
– Congruent with professional and institutional standards?
– Appropriate to program level?
– Relevant to healthcare context?
♦ Curriculum design
– Good fit of components?
– Reflect curriculum foundation and goals?
– Congruent with program environment?
– Courses logically sequenced?
(cont’d)
- 20. © 2013 Springer Publishing Company, LLC. 20
Curriculum Assessment
Elements (cont’d)
♦ Curriculum outcomes
– Evidence of program outcome achievement?
– Success of program graduates?
♦ Courses
– Course goals congruent with curriculum goals?
– Learning activities consistent with the philosophical
framework and goals?
– Course content current, evidence-based, logically
organized?
(cont’d)
- 21. © 2013 Springer Publishing Company, LLC. 21
Curriculum Assessment
Elements (cont’d)
♦ Teaching-learning methods
– Congruent with philosophical framework?
– Help students to achieve outcomes?
– Respect student diversity?
– Student satisfaction with the quality of teaching?
(cont’d)
- 22. © 2013 Springer Publishing Company, LLC. 22
Curriculum Assessment
Elements (cont’d)
♦ Student assessment methods
– Assess all relevant types of learning?
– Accommodate student input, diversity, feedback
needs?
– Accommodate faculty expertise, preferences,
workload?
(cont’d)
- 23. © 2013 Springer Publishing Company, LLC. 23
Curriculum Assessment
Elements (cont’d)
♦ Resources (human, physical, fiscal)
– Sufficient numbers of appropriately credentialed faculty
members?
– Adequate number and type of staff members to support
the curriculum?
– Classrooms, offices, labs—adequate, comfortable,
appropriately equipped?
– Library holdings appropriate and sufficient?
– Sufficient quality and quantity of clinical placements?
(cont’d)
- 24. © 2013 Springer Publishing Company, LLC. 24
Curriculum Assessment
Elements (cont’d)
♦ Learning climate
– Effect of social, emotional, intellectual atmosphere
on faculty, student, staff quality of life?
– Satisfaction of faculty and students with
relationships, academic freedom, sense of
community?
– Promote diversity of perspective and foster
responsibility and accountability?
(cont’d)
- 25. © 2013 Springer Publishing Company, LLC. 25
Curriculum Assessment
Elements (cont’d)
♦ Policies
– Appropriate, reasonable, applied consistently?
– Well understood by faculty and students?
– Need for additional policies?
- 26. © 2013 Springer Publishing Company, LLC. 26
Assessment of Online Courses
♦ One element of curriculum assessment
♦ Involves many of the same criteria used to assess
courses offered in traditional classrooms
♦ Additional elements specific to online environment
– Technology
– Accessibility
– Instructional design
– Content
– Interactive activities
- 27. © 2013 Springer Publishing Company, LLC. 27
Assessment of Online Courses
♦ Challenges
– Online courses hidden from view of faculty
members and administrators who are not directly
involved in teaching them
• Limits the role that these colleagues can play in course
evaluation
• Difficult for outside reviewer to assess elements of
asynchronous learning
– Pace of the learning activities
– Timing of instructor feedback to students
- 28. © 2013 Springer Publishing Company, LLC. 28
Assessment of Teaching
♦ Focuses on quality of classroom and clinical teaching
♦ Also focuses on other dimensions of the faculty role
– Depend on goals and mission of the nursing education
program
– Examples
• Scholarship and research
• Service to the academic institution, nursing program, community,
nursing profession
• Clinical practice
- 29. © 2013 Springer Publishing Company, LLC. 29
Assessment of Teaching
♦ Research in nursing education: Qualities of effective
teaching in nursing
– Knowledge of the subject matter
– Clinical competence
– Teaching skill
– Interpersonal relationships with students
– Personal characteristics
♦ Consistent with studies about teacher effectiveness
in other fields
- 30. © 2013 Springer Publishing Company, LLC. 30
Assessment of Teaching
♦ NLN’s "Core Competencies of Nurse Educators
with Task Statements" (2005)
– Knowledge, skills, and attitudes required for
effectiveness in the role of nurse educator
– Each competency statement followed by
description of tasks that exemplify it
– Three of the core competencies focus on effective
performance in the role of teacher
- 31. © 2013 Springer Publishing Company, LLC. 31
Core Competencies of
Nurse Educator
1. Facilitate learning
– Uses variety of teaching strategies
– Communicates effectively
– Shows enthusiasm for teaching and nursing
– Demonstrates interest in learners
– Demonstrates personal attributes that facilitate
learning
– Maintains professional practice knowledge base
- 32. © 2013 Springer Publishing Company, LLC. 32
Core Competencies of
Nurse Educator
2. Facilitate learner development and
socialization
– Identifies individual learning needs and styles
– Provides appropriate learning resources
– Creates learning environments that promote
socialization to the role of nurse
- 33. © 2013 Springer Publishing Company, LLC. 33
Core Competencies of
Nurse Educator
3. Use assessment and evaluation strategies
– Uses a variety of methods to assess learning
– Gives timely and constructive feedback to learners
– Demonstrates skill in design and use of
assessment tools
- 34. © 2013 Springer Publishing Company, LLC. 34
Qualities of Effective Teaching
♦ Knowledge of subject matter
– Content area expert
– Understands theories and concepts relevant to
nursing practice
• Can assist students to apply to patient care
– Keeps current with nursing and other interventions,
new developments in areas of expertise, and research
– Necessary but not sufficient; must be able to
communicate knowledge to students
- 35. © 2013 Springer Publishing Company, LLC. 35
Qualities of Effective Teaching
♦ Clinical competence
– If teaching in the clinical setting
– Make sound clinical judgments
– Expert clinical skills
– Can guide students in skill development
- 36. © 2013 Springer Publishing Company, LLC. 36
Qualities of Effective Teaching
♦ Teaching skill
– Identify students’ learning needs
– Plan instruction
– Present material effectively
– Explain concepts and ideas clearly
– Demonstrate procedures effectively
– Use sound assessment practices
• Especially important in clinical evaluation
- 37. © 2013 Springer Publishing Company, LLC. 37
Qualities of Effective Teaching
♦ Positive relationships with learners
– Shows respect for and confidence in students
– Is honest and direct
– Supports students
– Is approachable
– Treats students fairly
– Creates climate of mutual respect between
teacher and student
- 38. © 2013 Springer Publishing Company, LLC. 38
Qualities of Effective Teaching
♦ Personal characteristics
– Enthusiasm
– Patience
– Sense of humor
– Friendliness
– Integrity
– Perseverance
– Courage
– Willingness to admit mistakes
- 39. © 2013 Springer Publishing Company, LLC. 39
Teaching Effectiveness
Data Sources
♦ Student ratings
– Students can make valid and reliable judgments
about
• teaching methods.
• Fairness.
• interest in students.
• enthusiasm for the subject.
– Class size, course format can affect ratings
- 40. © 2013 Springer Publishing Company, LLC. 40
Teaching Effectiveness
Data Sources
♦ Student ratings
– Necessary but insufficient data source
• Students not the best source of data about accuracy,
depth, and scope of the teacher’s knowledge
– Can be obtained in classroom or online
• Assure anonymity and confidentiality
- 41. © 2013 Springer Publishing Company, LLC. 41
Teaching Effectiveness
Data Sources
♦ Peer review
– Perspective of another teacher who knows the
content and who has experience working with
similar students
– Best used for formative evaluation
• Reviewer may not be a content or clinical expert in
same area as teacher
• Observations can be influenced by personal feelings
about the colleague
(cont’d)
- 42. © 2013 Springer Publishing Company, LLC. 42
Teaching Effectiveness
Data Sources
♦ Peer review (cont’d)
– Should take place within a context of continuous
improvement of teaching–learning process
– Must be supported by adequate resources
• Faculty development
• Mentoring
• Modeling by master teachers
– Observation in classroom, clinical setting,
laboratory
(cont’d)
- 43. © 2013 Springer Publishing Company, LLC. 43
Teaching Effectiveness
Data Sources
♦ Peer review (cont’d)
– Online environment
• Visit as guest user, review course materials
• Evidence of effective instruction
– Promotes interaction among students and teacher, active
learning, student engagement
– Communicates appropriate performance expectations; gives
prompt, rich feedback
– Respects diversity of views, learning styles, abilities
– Uses effective instructional and graphic design
- 44. © 2013 Springer Publishing Company, LLC. 44
Teaching Portfolio
♦ Collection of materials related to teaching
activities in classroom, clinical practice, online
environment, simulation center, or other
settings
♦ Materials selected by the faculty member
should be highly selective and organized to
create a cohesive professional narrative
♦ Variety of formats
- 45. © 2013 Springer Publishing Company, LLC. 45
Teaching Portfolio
♦ May reflect specific purpose
– Teaching improvement (formative use)
– Promotion and tenure review (summative use)
– Seeking faculty position (career portfolio)
♦ Variety of formats
– Print
– Electronic
- 46. © 2013 Springer Publishing Company, LLC. 46
Teaching Portfolio
♦ Contents
– Teaching materials
• e.g., syllabi, sample tests, student assignments
– Philosophy of teaching
• Should be reflected in the teaching material
– Additional content for instructional improvement
• Areas of teaching that need improvement
• Description of efforts to develop teaching skills
– Peer and administrator evaluations of teaching
– Self-evaluation of strengths and weaknesses