Outcomes Evaluation: The Future for CMEPresentation Transcript
Educational Outcomes Evaluation: The Future for CME James C. Leist, EdD Staff Consultant Alliance for Continuing Medical Education
Part time CME consultant
This format and some of the content were developed by Donald Moore, Vanderbilt University, Robert Addleton, Medical Association of Georgia and Derek Dietze, Improve CME
Plan for Session
Small and Large Group Work
Further Reflection and Integration
In her recent meeting with the Hospital CEO, Maureen O. Measure, BSN, MEd, the CME Director for Hopewell General Hospital, was asked about the value of CME. Maureen noted that they have done many programs for all physicians in the hospital system.
The CEO asked whether CME improved physician practice and patient care.
Maureen said that she didn’t have any data.
The CEO asked if QI data were needed and agreed to appoint her to the QI Committee for the Hospital. She agreed anxiously.
In her third meeting with the QI Committee, Maureen listened to a report about the varied levels of physician prescriptions of ACE inhibitors for patients with CHF. How could the physician behavior be changed to assure that at least 80% of patients received the appropriate dosage of ACE I?
Challenge Scenario Questions
What can a CME provider contribute to this scenario?
If a CME intervention is provided, what outcomes should be evaluated and how?
Challenge Individual Reflection
For the next 5 minutes, reflect on the challenges listed below:
What are Outcomes?
How do you evaluate the outcomes of a CME activity for this scenario?
Challenge Small Group
In your small groups in the next 10 minutes, share the results of your individual reflection on the definition of outcomes and how you would address the outcomes of a CME activity for this scenario .
Designate one member of the small group to share your results with the large group, time permitting.
Outcomes Evaluation Mini-Lecture
Learning Objectives for session
Define Educational Outcomes
Describe how CME providers in the Hospital/Health Systems Venue can contribute to more sophisticated educational outcomes
Apply measurement of educational outcomes to one CME activity when you return to your venue
An outcome is defined as
the result, or effect, of an event
the consequences of an action
In CME, an educational outcome is the result, or effect, of CME activities
Evaluation is the process for measuring outcomes
Key to Outcomes Evaluation is to plan for the results that are desired – NEEDS ASSESSMENT DATA ARE CRITICAL
“ If you don’t know where you’re goin’, you ain’t gonna get there ”
“ Yogi Berra”
Process for Evaluating Outcomes
Conduct a needs assessment - practice gap
Determine if education can address the gap
Identify the results of the educational intervention
Develop/Implement the intervention/s - target audience and Faculty
Measure the results/outcomes
Share the results with appropriate stakeholders
Repeat and Improve the process
Practice Gap - Individual and/or Organizational
Current Practice(Problem/Need) - Current level
Desired Result - Practice/Performance level desired
Gap - The Difference between the desired result and the current practice
Will an educational intervention/interventions address the identified gap.
A Continuum, or Levels, of Educational Outcomes/Results
Learning - KSA (Knows)
Competence (Knows/Shows How)
Performance/Behavior in Practice - Individual and Organizational (Does)
Community Health Status
Evaluation - A Continuum of Educational Outcomes
Current Practice - Generally
Learning (Knowledge, Skills and Attitudes)
Most Desirable, More Difficult and Required
Performance/Behavior in Practice - Individual and Organizational
Outcome Evaluation by Venues Venues Results Office-based Practice Hospital/ Health System Specialty Societies Med. Ed. Comm. Co. Medical School Attendance/ Satisfaction X X X X X Learning - KSA X X X X X Competence X X X X X Practice X X X? X? X? Patient Outcomes X X X? X? Community X X X X?
How do we Evaluate Outcomes?
Accurate data-based needs assessment of desired/current practice required to document need for change and provide baseline for measurement of improvement in practice.
Data Collection Strategies
Indirect – Self-report by physicians
Direct – Actual performance data that describes performance, or result of the performance
Regularly Scheduled Series - RSS
Data Collection Strategies - Indirect
Outcome – Physicians will prescribe ACE inhibitors in the appropriate dosage for patients with CHF in every case when it is indicated