Improving Your Performance in PI-CME Harvey C. Parker, Ph.D., CCMEP National Association for Continuing Education (NACE) Gregg Sherman, MD National Association for Continuing Education (NACE) B. Stephen Burton, MS CE Outcomes LLC Jeremy C. Lundberg, MSSW DLC Solutions, LLC Michael S. Adler myCME Alliance for Continuing Education in the Health Professions 38th Annual Conference • San Francisco, California January 31, 2013
Disclosures Harvey C. Parker, Ph.D., CCMEPNational Association for Continuing Education (NACE) Nothing to Disclose Gregg Sherman, MDNational Association for Continuing Education (NACE) Nothing to Disclose B. Stephen Burton, MS CE Outcomes LLC Nothing to Disclose Jeremy C. Lundberg, MSSW DLC Solutions, LLC Nothing to Disclose Michael S. Adler, BA myCME Financial Interest in Haymarket/myCME
Learning Objectives1. Identify barriers to successful implementation of PI-CME2. Discuss importance of linking performance measures to interventions3. Use multiple outcomes strategies to evaluate your PI-CME4. Design a customized IT platform for your PI- CME5. Summarize strategies to increase the success of your PI-CME
Learning Objective1. Identify barriers to successful implementation of PI-CME Harvey C. Parker, Ph.D., CCMEP Gregg Sherman, MD
Tailor Education to Fit the LearnerGeorge Bernard Shaw once said…..“The only man who behaves sensibly is my tailor; hetakes my measurements anew every time he sees me, whileall the rest go on with their old measurements and expect meto fit them.”Like a good tailor, the CME provider must offer activities that meetthe specific needs of learners and not use a one-size-fits-all modelof program design and delivery.Individualized, performance-based education is precisely the goalof PI-CME.
Making the Case for PI-CME2005AMA approved guidelines by which AMA PRACategory 1 CreditTM could be awarded—up 20AMA PRA credits for participation in structuredperformance improvement CME (PI CME)activities.
3 Stages of PI-CMEPerformance Improvement CME Credit for Performance ImprovementFind evidence-based measures Stage Arelevant to the identified Learning from current practice—practice problem (Plan) performance assessmentImplement interventions Stage Bidentified to improve Learning what PI to apply to currentperformance (Do) patient careRe-measure to assess Stage Cimprovement (Study) Learning from the evaluation of the PI effortStandardize the improvement Restart the process(Act)
Barriers to Successful Implentation of PI-CMELearner Barriers CME Professional BarriersTime TimeEffort Steep learning curve for methodology to deliver content and measure outcomesSelf-directed Tracking learner involvementSustained effort over months Recruitment and education of learners about PI CMEData collection required Expense
Activity Start and End Dates: August 1, 2010 to July 31, 2011 Collaborating Organizations National Association for Continuing Education DLC Solutions, LLC CE Outcomes LLC myCME Canadian ADHD ADD Research Association (CADDRA)Children and Adults with Attention Deficit Hyperactivity Disorder (CHADD) Supported by Eli Lilly Grant ID 100204775
Statement of Need and PurposeThe vast majority of adult patients with ADHD remains undiagnosed and,therefore, untreated.•Close to half of PCPs (48%) do not feel confident to diagnose ADHD in adults.•A large majority (85%) indicate that they would take a more active role in diagnosingand treating adult ADHD if there were an easy-to-use, validated screening tool fordiagnosing adults.•A large majority of PCPs (75%) indicate that they would take a more active role indiagnosing and treating adult ADHD if there were effective prescription medicines thatwere not stimulants or controlled substances.•Approximately 13% of PCPs refer adult patients with ADHD to specialists specificallybecause many of the existing treatment options include controlled substances, whichthey are uncomfortable prescribing.Kessler RC, Adler L, Barkley R, Biederman J, Conners CK, Demler O, Faraone SV, Greenhill LL, Howes MJ, Secnik K, Spencer T, Ustun TB, Walters EE,Zaslavsky AM. (2006). The Prevalence and Correlates of Adult ADHD in the United States: Results from the National Comorbidity Survey Replication. Amer.Jour. of Psychiatry, 163: 724-732.Barkley, R.A., Murphy, K.R., & Fischer, M. (2008). ADHD in Adults: What the science says. Guilford Press, New York.Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51st Annual Meeting of the American Academy of Child and AdolescentPsychiatry; October 19-24, 2004; Washington, DC.
Survey Results: Knowledge of Target ConditionsPCPs indicated that they are significantly more knowledgeable about depression than they areabout all other target conditions. They are also significantly more knowledgeable about GAD thanthey are about bipolar disorder, ADHD and OCD. 100% 92% % Very / Extremely Knowledgeable 83% 80% (top 2 box) 60% 40% 36% 34% 34% 20% Mean= 0% 4. 4. 3. 3. 3. 2a Depression (n=336) 0b GAD (n=331) 3c Bipolar disorder 2 d ADHD (n=137) 2e OCD (n=134) (n=144) Base: Qualified Respondents (n=400) Q400 Please indicate how knowledgeable you consider yourself to be about the following conditions? Scale: 1=Not At All Knowledgeable; 2=Not Very Knowledgeable; 3=Somewhat Knowledgeable; 4=Very Knowledgeable; 5= Extremely Knowledgeable Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51 st Annual Meeting of the American Academy of Child and Adolescent Psychiatry; October 19-24, 2004; Washington, DC.
Survey Results: Level of Clinical InstructionRelative to other mental health conditions, PCPs report having received the least thoroughclinical instruction regarding ADHD. They report receiving the most thorough instructingregarding the diagnosis and treatment of depression. 60% 57%% Very / Extremely Thorough 40% 40% (top 2 box) 20% 20% 15% 13% Mean= 0% 3. 3. 2. 2. 2. 6a Depression (n=227) GAD 2 b (n=159) 7 c Bipolar disorder (n=78) 6d OCD (n=58) 3e ADHD (n=53) Base: Qualified Respondents (n=400) Q410 Please indicate the level of clinical instruction you received regarding diagnosing and treating the following conditions. Scale: 1=Not At All Thorough; 2=Not Very Thorough; 3=Somewhat Thorough; 4=Very Thorough; 5= Extremely Thorough Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51st Annual Meeting of the American Academy of Child and Adolescent Psychiatry; October 19-24, 2004; Washington, DC.
Survey Results: Refer to Specialist for DxPCPs indicate that they most frequently refer to a specialist for the diagnosis of ADHD and bipolardisorder. The vast majority of PCPs diagnose depression (98%) and GAD (97%) themselves. OCD Depression 2% 47% ADHD 54% 98% 35% GAD Bi-polar Disorder 65% 3% 33% Diagnose Myself 68% 97% Refer to a specialist for diagnosis Base: Qualified Respondents (n=400) Q505 For each condition, please indicate whether you most frequently make the diagnosis yourself, or refer adult patients to a specialist for diagnosis. Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51 st Annual Meeting of the American Academy of Child and Adolescent Psychiatry; October 19-24, 2004; Washington, DC.
Survey Results: Reasons for Working with SpecialistsInexperience / Lack of Inexperience / lack of confidence 52%confidence (52%) isthe primary reason Difficult to treat & diagnose / No clear 22%why PCPs refer to diagnostic criteriaspecialists whendiagnosing and Second opinion / confirm diagnosis 20%treating adults withADHD. Many treatment medications are 13% controlled substancesMany physicians alsocite difficulty e Other 11%diagnosing andtreating adult ADHD(22%) and a general Time constraints 4%desire for secondopinions to confirm 0% 10% 20% 30% 40% 50% 60%ADHD diagnoses % Answering(20%). Base: Refer to Specialist for Diagnosis of ADHD (n=261) Q507 Please explain why you sometimes collaborate with or defer to specialists when diagnosing adult ADHD. Adler, L.A. (2004). ADHD: a survey of 400 primary care physicians. Presented at 51 st Annual Meeting of the American Academy of Child and Adolescent Psychiatry; October 19-24, 2004; Washington, DC.
NACE Convened a Panel of ADHD Experts to Develop Performance Measures for Adults ADHD Dx and Tx 1. Performs a comprehensive interviews to assess ADHD history and symptoms. 2. Uses adult ADHD rating scales to assess symptom frequency and severity. 3. Uses interviews or rating scales to corroborate patient self- report and assess functional impairment. 4. Initiates a pharmacologic treatment plan for adult ADHD patients when indicated 5. Recommends non-pharmacologic treatments and providing patient education for adult ADHD patients. 6. Provides timely and consistent follow-up care.
Baseline Self-Assessment of Utilization of Performance Measures by PCPsN=94 (45 Physicians(top row); 22 NPs, 24 PAs, 3 Others(bottom row))Performance Measures Not at Somewhat Moderately More Than Very All Moderately Much1. Conduct an interview 21% 32% 16% 21% 11% 33% 21% 18% 26% 3%2. Use ADHD Rating Scales 50% 24% 11% 16% 0% 69% 10% 10% 8% 3%3. Gather corroborating information 26% 39% 21% 11% 3% and assess functional impairment 44% 28% 5% 15% 8%4. Plan pharmacologic therapy to treat 18% 21% 16% 37% 8% ADHD symptoms 47% 8% 8% 21% 16%5. Provide or recommend psychosocial 20% 20% 28% 18% 15% interventions 47% 16% 5% 26% 5%6. Schedule regular follow-up visits to 16% 21% 21% 26% 16% monitor treatment safety and efficacy for ADHD interventions 24% 24% 18% 21% 13%
Baseline Self-Assessment of Confidence inthe Use of Performance Measures by PCPs N=94 (45 Physicians(top row); 22 NPs, 24 PAs, 3 Others(bottom row)) How confident are you in your ability to: Not at All Somewhat Moderately More Than Very Moderately Much 1. take a patient history that will identify the 15% 17% 34% 27% 7% essential components of an ADHD complaint? 33% 29% 16% 16% 6% 2. employ clinical assessment tools (e.g., 21% 24% 31% 21% 2% ADHD symptom checklists, structured diagnostic interview, etc.) in the diagnosis 31% 35% 20% 6% 8% of ADHD in your adult patients? 3. identify co-morbid psychiatric disorders 7% 17% 43% 29% 5% (e.g., depression, anxiety, substance use/abuse disorders) in your adult patients 27% 20% 20% 24% 8% presenting with symptoms of ADHD? 4. discuss appropriate ADHD medication 10% 33% 21% 21% 14% options with your adult ADHD patients? 39% 29% 12% 16% 4% 5. select the appropriate pharmacotherapy for 12% 24% 31% 31% 2% treating ADHD, taking into account your patient’s overall health status? 37% 29% 18% 14% 2% 6. determine when psychosocial interventions 19% 38% 26% 17% 0% such as cognitive behavioral therapy (CBT) or workplace or academic accommodations 33% 35% 20% 10% 2% are indicated as a part of an ADHD treatment plan for your patients?
Program Overview Stages A-B-C of Our PI-CME: Making Exam Room Decisions for Adults with ADHD Adults with ADHD: Making Exam Room Decisions Participants evaluated existing practices in providing care to Participants evaluated existing practices in providing care toStage A adults with ADHD by completing a self-assessment survey adults with ADHD by completing a self-assessment survey and answering chart-review questions. Participants selected and answering chart-review questions. Participants selectedPre-Assessment one or more Quality Improvement Plans (QIP) to improve care one or more Quality Improvement Plans (QIP) to improve care to adults with ADHD. to adults with ADHD. Participants implemented one or more of the QIP selected in Participants implemented one or more of the QIP selected inStage B Stage A by using the materials found in the NACE Adult ADHD Stage A by using the materials found in the NACE Adult ADHD Toolkit. Comprehensive educational and informational Toolkit. Comprehensive educational and informationalAction Stage resources were provided at this stage with online and printed resources were provided at this stage with online and printed materials. materials. Participants re-evaluated performance by again completing Participants re-evaluated performance by again completing the self-assessment survey and answering chart-review the self-assessment survey and answering chart-reviewStage C questions on aasecond set of 55patients. Feedback provided to questions on second set of patients. Feedback provided toPost-Assessment participants explained the improvements that were made in participants explained the improvements that were made in caring for adults with ADHD. caring for adults with ADHD.
Audience Generation and Participation Participants were recruited by invitations that were both surface and electronically mailed, recruitment at NACE Emerging Challenges in Primary Care symposia, and through online advertising (e-newsletters and banner on myCME.com)
Steps Taken To Increase Participation• Aggressive marketing for audience generation• E-newsletters to NACE members announcing PI CME activity• Mailed invitations to PCPs in cities throughout the US• Listing with myCME.com• Announcements and recruitment at NACE live symposia• Weekly postcard and e-mail reminders to move from one stage to another• Dedicated CME manager to call or email participants to remind them to continue activity• Sent hard copy of NACE Adult ADHD Toolkit when learners completed Stage A and moved to Stage B (Action Stage) and this motivated them to continue• After hours phone support
Learning Objective2. Discuss importance of linking measures to interventions Harvey C. Parker, Ph.D., CCMEP Gregg Sherman, MD
Our 6 Performance Measures1. Performs a comprehensive interviews to assess ADHD history and symptoms and co-morbid disorders.2. Uses adult ADHD rating scales to assess symptom frequency and severity.3. Uses interviews or rating scales to corroborate patient self- report and assess functional impairment.4. Initiates a pharmacologic treatment plan for adult ADHD patients when indicated5. Recommends non-pharmacologic treatments and providing patient education for adult ADHD patients.6. Provides timely and consistent follow-up care.
NACE Adult ADHD Toolkit The National Association for Continuing Education ( NACE) developed this Adult ADHD Toolkit to assist clinicians in the assessment, diagnosis and treatment of adults with Attention-Deficit/ Hyperactivity Disorder ( ADH D) . This toolkit was prepared as a resource for an accredited medical education per formance improvement activity entitled, Making Exam Room Decisions for Adults with ADH D.* The toolkit contains practical information and tools to help clinicians and members of the adult ADH D care team incorporate evidence-based and consensus-based guidelines into office practice. T he toolkit contains: www.lillygrantoffice.com.Available Online: http://www.naceonline.com/AdultADHDtoolkit/QI1.php
Performance Measures Linked to Interventions Contained in the NACE ADHD Toolkit ASSESSMENT TOOLS PROFESSIONALEDUCATION PATIENT EDUCATION TOOLS TOOLSAdult ADHD Self-Report Scale-V1.1 Canadian ADHD Practice Guidelines Diagnosis of ADHD in Adults PM 1, 2, 3(Symptom Checklist (English and PM 1, 2, 3, 4, 5, 6Spanish) PM 2Adult ADHD Self-Report Scale-V1.1 FDA Approved Medications for Adults Facts About Approved Medication forScreener PM 2 with ADHD PM 4 Adults with ADHD (Monograph) PM 4Barkley Quick Check for Diagnosis of ADHD Coding Fact Sheet Succeeding in the WorkplaceADHD in Adults (Monograph) PM 6PM1, 2Brief Semi-Structured Interview for DSM-IV TR ADHD Symptoms Managing Money (Monograph) PM 6ADHD in Adults PM 1, 2 PM 2Weiss Functional Impairment Rating Understanding ADHD in Adults Social Skills in Adults with AD/HD PM 6Scale Self-Report PM 3 (Webcast and Monograph) PM 1, 2, 3ADHD Medication Side Effects Checklist Assessment of Adult ADHD (Webcast A Guide to Organizing the Home andPM 4 and Monograph) Office PM 6 PM 1, 2Medication Response Form Pharmacotherapy of Adult ADHD and ADHD Coaching for Adults (Monograph)PM 4 Co-morbid Conditions PM 4, 5 PM 6CAGE Questionnaire PM 1 Maintenance and Follow-up Care for Legal Rights: Higher Education and the Adults with ADHD (Webcast and Workplace (Monograph) PM 6 Monograph) PM 4, 5Hamilton Depression Rating Scale PM 1 Psychosocial Interventions for ADHD in Women and ADHD (Monograph) PM 6 Adults (Webcast and Monograph) PM 6 ADHD in Adults—Differential and Adult ADHD Resources (Monograph) Coexisting Diagnosis (Webcast and PM 6 Monograph) PM 3
Stage A—Pre-Assessment• Participants evaluated existing practices in providing care to adults with ADHD by completing a self-assessment survey and answering chart-review questions on 5 patients.• Participants selected one or more Quality Improvement Plans (QIP) to improve care to adults with ADHD. • QIP 1 Assessment of ADHD in Adults • QIP 2 Treatment of ADHD in Adults • QIP 3 Patient Education• Toolkit was sent to all participants who completed Stage A.• 5 credits awarded.
Stage B—Action Stage• Participants implemented one or more of the QIPs selected in Stage A by using the materials found in the NACE Adult ADHD Toolkit.• Comprehensive educational and informational resources were provided at this stage with online and printed materials.• Participants must wait at least 45 days before starting Stage C.• 5 credits awarded.
Stage C—Post-Assessment• Participants re-evaluated performance by again completing the self-assessment survey and answering chart-review questions on a second set of 5 patients.• Feedback provided to participants explained the improvements that were made in caring for adults with ADHD.• Participants evaluated their performance in the PI- CME activity.• 5 credits awarded for completing Stage C and 5 more for completing final evaluation.
Follow Up Education for ADHD in AdultsTake a DVD of the NACE Adult ADHD Toolkitwith you today and share it with your primary care provider. In 2012 NACE educated an additional 1600PCPs on adult ADHD and introduced them to theNACE Adult ADHD Toolkit through our EmergingChallenges in Primary Care symposia held in 15 cities. We hope to extend this reach in 2013.
Learning Objective3. Use multiple outcomes strategies to evaluate your PI-CME B. Stephen Burton, MS
Outcome Measures for Adult ADHD PI CME1. Learner self-assessment at Stages A and C • Frequency with which each measure is used in practice • Degree of confidence learner has with the use of each measure • Case-based knowledge/competence for diagnosing and treating ADHD2. Chart Review Data Collection • Data collected from 5 patients at Stage A and 5 new patients at Stage C on use of each measure • Drill down on details regarding each measure3. Evaluation of program and self-assessment of learning
Self-Assessment Stages A and C Changes in the Use of Performance MeasuresN = 92 P < .001 for each item Graph displays those that selected “always” on a five point frequency scale from “never” to “always”
Self-Reported Changes in Confidence in Diagnosing and Treating Adults with ADHD from Stages A to CN = 92 P < .001 for each item Graph displays those that selected “very confident” on a five point confidence scale from “not at all confident” to “very confident”
Chart Abstraction ToolPatient History - as obtained by you at initial assessment or confirmedduring review of patients medical recordsWas a patient history obtained or confirmed that covered all of the following elements:developmental/academic history, past medical & psychiatric history, medication use,substance use, and family history? Yes NoWas the patient assessed for significant impairments in social, academic andoccupational functioning? Yes NoWas the patient assessed or screened for common comorbid conditions includinglearning disability, depression, anxiety and bipolar disorder? Yes No
Chart Review Data Changes in the Use of Performance Measures P < .001 P < .01 P = .078 P = .093 P = .384 P = .015Stage A, N = 464Stage C, N = 462
Learning Objective4. Design a customized IT platform for your PI- CME Jeremy C. Lundberg, MSSW DLC Solutions, LLC
Learning Objective• Design a customized IT platform for your PI- CME1. Leverage mature, CE-oriented learning management system (i.e., EthosCE.com)2. Design easy-to-use, step-wise learner workflow to complete module.3. Use email reminder triggers to encourage completion and mitigate against attrition.4. Share individual and aggregate benchmark reporting with learners to demonstrate variance.5. Pilot test to ensure engaging, intuitive learning experience.
Learning Objective5. Describe strategies to increase success in PI- CME Audience Generation Michael S. Adler, BA
DisclosureDisclosure The views expressed in this presentation are my own and do not represent those of my current employer. I do have a financial interest in Haymarket Medical Education/myCME. 59 59
myCME Partnership with NACEmyCME Partnership with NACE • Extended Reach of myCME--a Preferred Choice of NACE and Other Medical Education Providers – Experienced distribution site that delivers a cost-effective learner solutions for their educational offerings – Superior placement of educational offerings – Aggressive marketing/recruitment campaign – High guarantees on readership and certification • 2010: myCME Chosen to Partner with NACE on PI- CME Activity – Adults with ADHD: Making Exam Room Decisions – Promotion and distribution over an 8-month period To this day, NACE continues its partnership with myCME To this day, NACE continues its partnership with myCME 60 60
Adults with ADHD Activity onAdults with ADHD Activity onmyCMEmyCME 61
myCME RecruitmentmyCME RecruitmentE-Mail/Online CampaignsE-Mail/Online Campaigns • Executed throughout the myCME Network and tailored to the specific audience • E-mail – Featured spot in the myCME “Weekly Update” and “Monthly Update” eNewsletters – Dedicated e-mail blasts specific to the activity – CME Spotlight of The Clinical Advisor monthly newsletter - the #1 NP/PA medical journal – CME Spotlight of MPR Weekly News Brief eNewsletter • Online Recruitment – “Featured CME Course” image rotator on the myCME home page – “Latest Courses” section on the myCME home page – “Relevant Specialty” home pages (Psychiatry, Family Medicine, et al.): 24/7 fixed placement myCME delivered more than 4,800 referrals to the myCME delivered more than 4,800 referrals to the activity, which yielded a 29% participation rate! activity, which yielded a 29% participation rate! Proprietary and Confidential 62 62
New CME/CE Programs from myCMENew CME/CE Programs from myCMEWeekly UpdateWeekly Update 63 63
Learning Objective5. Summarize strategies to increase success in PI-CME Harvey C. Parker, Ph.D., CCMEP Gregg Sherman, MD
Strategies to Improve Your Performance in PI-CME• Collaborate with partners that have PI-CME experience• Have an aggressive marketing plan for audience generation• Develop systems to keep learners involved (i.e. e-mail reminders, postcards, phone contact, etc.)• Use evidence-based performance measures• If your activity is web-based use a platform that is easy to access and navigate
Strategies to Improve Your Performance in PI-CME• Make certain your PI CME platform can track learner data and can provide feedback to your learners so they can assess their performance• Develop meaningful interventions that provide value to learners and tools that can be immediately implemented into practice based on your learner’s specific needs• Keep data collection as simple as possibleFor more information on Making Exam Room Decisions for Adults with ADHD and outcome data see:Parker, H., et al, Improving the Diagnosis, Treatment and Follow- up of Adult ADHD Patients in Primary Care. CE Meas. 2012;6:3-12.Our thanks to Derek Dietz at Improve CME for his invaluableassistance in reviewing this manuscript.
Questions and DiscussionThank You For Your Attention!