5 Factors for Success in CME Outcomes Assessment


Published on

Jeremy C. Lundberg, CEO of EthosCE Learning Management System and DLC Solutions moderates this expert panel with Derek Dietze and Dr. Christopher Viereck of ImproveCME.com. Derek and Christopher shared their insights and expertise on what they consider the “5 Keys to Successful CME Outcomes Assessment.” The feedback from the program has been excellent and we hope that you will share this program with your colleagues.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

5 Factors for Success in CME Outcomes Assessment

  1. 1. Five Keys for Success in Outcomes Assessment Presented by: Jeremy C. Lundberg, MSSW Derek T. Dietze, MA, FACEHP, CCMEP Christopher Viereck, PhD, CCMEP
  2. 2. EthosCE LMS Enterprise-level learning management system. 2012 Brandon Hall Award-Winner Designed for CME and ACCME PARS compliance. Integrated assessment, evaluation, and certificate. Personal “My CME” transcript. RSS Management Automates and streamlines “back office” administration. Powerful 24/7 reporting engine.
  3. 3. Common Model in CME Meeting Logistics Assessments Online Courses Evaluations
  4. 4. A Strategic Model for CME Learning Management System HR Management System Association Management System 3rd Party Applications External Education Partners
  5. 5. Question 1. What are your challenges with outcomes assessment? (Check all that apply) 1. I don’t know where to start 2. I don’t understand the differences in outcomes levels 3. I can’t get the level of outcomes data I want 4. I have trouble linking gaps to learning objectives, to outcomes 5. I don’t know how to meet commercial supporter expectations
  6. 6. 1. Plan Ahead…Way Ahead • Create a detailed descriptive outcomes assessment plan at the earliest stages of grant request or activity development • Develop assessment questions in concert with content, not as an afterthought • Will you have enough resources available to create assessment questions, implement the plan, and analyze/report the results? • Plan with the end in mind • Examples
  7. 7. 2. Measure to Your Educational Design • What do you expect to change in your learners? • In what domains do these expected changes fall? • Knowledge/awareness, confidence, competence, performance, patient outcomes • Accurately represent the Moore level(s) • For higher outcomes levels, distinguish between subjective and objective assessment
  8. 8. 2. Measure to Your Educational Design (cont.) • If your learning objectives are focused on knowledge and competence change, why assess performance change or patient outcomes? • When you measure to your design you have: • Enhanced credibility • More effective use of resources
  9. 9. Question 2. Please rate your confidence in optimally aligning outcomes questions with learning objectives and gaps. 1. Not confident at all 2. Not very confident 3. Somewhat confident 4. Very confident 5. Extremely confident
  10. 10. 3. Align, Align, Align • Bring all of the following into alignment: • Gaps • Learning objectives • Desired results • Instructional design • Content • Format • Outcomes questions
  11. 11. 3. Align, Align, Align (cont.) • Demonstrate alignment in your planning documents • Examples: • Each learning objective is linked to a specific gap • Each outcomes assessment question is linked to a specific learning objective • When your alignment is good: much greater opportunities for positive results
  12. 12. Question 3. When you promise “Level 5 Outcomes Results”, what does this really mean to you? 1. Chart pull data on participant performance 2. Performance changes self-reported by participants 3. Planned performance changes 4. Patient outcomes reported by a participant 5. I have absolutely no idea!
  13. 13. 4. Keep the Plan Simple and Specific • Best option: Describe what you are going to measure, how you are going measure it, and when you plan to deliver on your promises. • Be sure to include: • Outcomes objectives and overall methods • Activity-specific outcomes details • Your approach to statistical analysis • What is in the results reports, and when they will be delivered • How you will share results?
  14. 14. 4. Keep the Plan Simple and Specific (cont.) • • • Your outcomes plan is the implementation guide Complex, overly ambiguous or ambitiousplans work against you: • Ambiguous plans create misunderstandings • Falling short of original planshurts your long-term credibility Reality check: Do you have the internal and external resources?
  15. 15. 5. Use Multiple Types of Assessment Questions • Don’t put all your eggs in one basket by using only one question type. • Consider using more than one question type to address your learning objectives. • Example: Use a combination of pre/post multiple-choice, case-based or agreement scale questions. • Mandatory: Align the assessment questions with the gaps and learning objectives.
  16. 16. Five Keys for Success in Outcomes Assessment • Plan Ahead…Way Ahead • Measure to Your Educational Design • Align, Align, Align • Keep the Plan Simple and Specific • Use Multiple Types of Assessment Questions
  17. 17. Leverage the Value of Your Outcomes Results • Summarize your results succinctly • Validate your needs assessment • Determine the degree to which your learning objectives were met • Relative effectiveness of different question types • Identify effective versus less effective content • Inform future content development and improvements
  18. 18. Leverage the Value of Your Outcomes Results (cont.) • Obtain more financial support: insert results in an upcoming grant request • Assist commercial supporters in selling the value of CME within their organization • Justify department budgets • Allocate resources more efficiently • Share results in clinical and CME community
  19. 19. Tools for Success Image of EthosCE report screen Collaborative partnerships
  20. 20. Question & Answer
  21. 21. Thank You Jeremy C. Lundberg, MSSW (267) 234-7401 jlundberg@dlc-solutions.com www.EthosCE.com derek.dietze@improvecme.com christopher.viereck@improvecme.com