![Today’s Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Alliance 2007 "Best of Conference" Presentation and Webinar: Beyond Theory: Practical Tools to Tackle Educational Outcomes Evaluation, Wendy Turell
Similar to Alliance 2007 "Best of Conference" Presentation and Webinar: Beyond Theory: Practical Tools to Tackle Educational Outcomes Evaluation, Wendy Turell (20)
Alliance 2007 "Best of Conference" Presentation and Webinar: Beyond Theory: Practical Tools to Tackle Educational Outcomes Evaluation, Wendy Turell
- 1. Alliance for CME Webinar Tuesday, September 25, 2007 Beyond Theory: Practical Tools to Tackle Educational Outcomes Evaluation Wendy Turell, DrPH Senior Director Educational Design and Accreditation Services BCME
- 5. An Outcomes-Based Educational Model
- 6. Outcomes-Based Educational Model Design Program for Maximum Impact Needs Assessment Identify Optimal Outcomes (based on educational gap) Outcomes Assessment & Program Evaluation Outcomes-Based Learning Objectives Needs Assessment
- 9. Reach for Greater Heights in Measurement LEVEL 5 PATIENT HEALTH LEVEL 4 PERFORMANCE LEVEL 3 LEARNING LEVEL 2 SATISFACTION LEVEL 1 PARTICIPATION LEVEL 6 POPULATION HEALTH Moore DE. A framework for outcomes evaluation. In: Davis D. Barnes BE, Fox R, eds. The Continuing Professional Development of Physicians: From Research to Practice. Chicago: AMA Press: 2003.
- 13. Sample Study Plan Study Subjects Survey Administration Survey Design Experimental Group Control Group (optional) Case-Based Questions Knowledge- Based Questions Pre-Test Staggered Post-Test Different options to assess learning and behavior change
- 33. Questions and Answers ? ?
Editor's Notes
- I Measurement and CME An Outcomes-Based Educational Model Rationale Overview II Educational Outcomes Evaluation: Methodology Pre and Post-testing Experimental and Control Groups III Sample Approaches to Study Design Symposium,Journal Supplement,Online Activity,Podcast IV Questions and Answers
- Return on Education (ROE) Educational dollars should be spent on effective education Guide for future programming What formats and strategies work best? Protection within CME guidelines Proof of educational effectiveness
- Integrated program planning progress
- Levels 1: Participant Satisfaction and Program Quality Measures No longer enough per ACCME and many grantors Level 2: Change (or intent to change) in Knowledge, Attitudes, or Skills Intent to change- has been shown to correlate with actual behavior change Level 3: Self-Reported Behavior Change Can be Captured via “staggered” post-tests administered weeks to months after activity completion Note : just because something is subjective does not mean it’s wrong – We just have less confidence in these results than we would if we used objective variables
- Level 4: Change in Practice Ex: chart reviews – may not capture every behavior – record keeping varies by institution & practice Level 5: Change in Treatment Outcomes or Health Status of Patients Best matched with CME activity that has strong impact (multiple exposures, major intervention) -you won’t likely find a change in patient outcome via a one-hour live symposium with 100 participants. Best matched for a series, a larger sweeping initiative in a health-care system, etc.
- I would fix the visuals on this and put in a build have different colors for each box- make it a bit more interactive. Make sure you say that when you can recurring medical education initiatives- you must have the same evaluation-same outcomes analysis plan- that you can compare in terms of success to other educational initiatives
- mail, online link, snail mail)
- You don’t have to poll everyone! Effect Size – magnitude of effect under study Power Calculation How to Estimate without having a statistics degree Rules of Thumb: If you have <30 participants, try to sample all of them
- Encourage participation We want to reach our target “n” Recommended If you plan to use a control group Should be “medically relevant” They are permitted in CME AMA guidelines- Incentives not to exceed $100.00 Unwritten guidelines- Do not exceed $25.00/survey Avoid the perception of coercion SAME INCENTIVES TO BOTH GROUPS
- Obtaining contact information for pretest contact Live Can offer pre-tests as participants arrive in room, but this may be awkward and logistically difficult ARS Enduring Materials May be more practical to skip the pre-test (validity may suffer) Funding Limitations Grantor concerns Pre-tests Incentives Can always skip incentives (note: validity may suffer due to lower enrollment) Objectivity