Building Science

1,708 views

Published on

This presentation takes a detailed look at unmet needs in mental health services for children.

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,708
On SlideShare
0
From Embeds
0
Number of Embeds
78
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Building Science

    1. 1. Building a Science on Implementation of Evidence-based Practices in Children’s Mental Health Kimberly Eaton Hoagwood, Ph.D. Columbia University December 11, 2005
    2. 2. Turning Points in Child & Adolescent Services Research: A Very Brief History <ul><li>Unclaimed Children (Knitzer, 1982) documents lack of community-based care (soon to be updated) </li></ul><ul><li>Systems of Care Monograph published (Stroul & Friedman, 1986); CASSP established, 1986 </li></ul><ul><li>Tripling of funding for research on children’s mental health at NIMH (1989-2001) </li></ul><ul><li>Meta-analyses of psychotherapies document effect sizes (Weisz et al, 1995, 1998, 2003; Kazdin et al., 1998, 2003) </li></ul><ul><li>System of Care study results published (Bickman, 1996) </li></ul><ul><li>Healthcare reform in US and Britain spurs growth of evidence-based practice movement (1996-present) </li></ul><ul><li>Surgeon General’s Reports (1999; 2000; 2001) highlight disparities between research and practice </li></ul><ul><li>Identification of >550 psychosocial therapies (Kazdin, 2003); medication trials for ADHD, OCD, aggression, depression </li></ul><ul><li>Methods developed for assessing organizational context applied to youth mental health services and found to predict child outcomes (Glisson, 2002, 2005; Schoenwald et al, 2003) </li></ul><ul><li>EBPs spread into state and federal policy planning (2004-present) </li></ul>
    3. 3. Yet…sobering facts about mental health services for children <ul><li>Unmet need as high now as 20 years ago </li></ul><ul><li>Unmet need highest among minority youth </li></ul><ul><li>Receipt of mh services increased but only 1/5 of children with the most severe needs receive mh services (Kessler et al 2005; Foster, Rollefson, Doksum et al., 2005) </li></ul><ul><li>Onset by age 14 for 90% of adults with SMI (Kessler et al., 2005) </li></ul><ul><li>Lack of availability and access a major barrier for most families </li></ul><ul><li>There is no system: use and need don’t match up </li></ul>
    4. 4. Ringel &Sturm, 2000; NIMH, 2001 Unmet Need for Mental Health Services 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% White African- American Latino Other
    5. 5. National Averages of Use and Need Don’t Match National Average MH Need for Children at 6-17: 7.09% National Average MH Use for Children at 6-17: 7.45% Data Source: NSAF wave 1 and 2, Sturm, 2001
    6. 7. Beyond the Linear Model Basic Research Clinical Trial (Efficacy) Treatment Development Effectiveness Trial Treatment Deployment
    7. 8. Schoenwald & Hoagwood, 2001
    8. 9. The Rise in Popularity of the term “Evidence-Based” (Hoagwood & Johnson, 2003)       EBT EBP EBM   1900-1990 0 0 0   1990-1995 3 7 76   1995-2002 63 459 5,425
    9. 10. Psychotherapies provided in routine clinical care have little to no effect (Weisz et al., 1995) Mean Effect Sizes Weisz et al., 1995 Children & Adolescents Adults University Clinic settings
    10. 11. What is Evidence? Lonigan, Ebert & Johnson, 1998; Chambless et al., 1998 <ul><li>At least two controlled group design studies or a large series of single-case design studies </li></ul><ul><li>Minimum of two investigators (for well-established) </li></ul><ul><li>Use of a treatment manual </li></ul><ul><li>Uniform therapist training and adherence </li></ul><ul><li>Tested with clinical samples of youth </li></ul><ul><li>Tests of clinical and functional outcomes </li></ul><ul><li>Long-term outcomes beyond termination of treatment </li></ul>.
    11. 12. Grading the Quality of Evidence <ul><li>Biglan, Mrazek, Carnine, Flay (Am Psych 2003) </li></ul><ul><li>Grades 1-7 </li></ul><ul><ul><li>1 = multiple RCTs or multiple time series experiments by 2 or more indep teams + implementation effectiveness </li></ul></ul><ul><ul><li>2 = 1 without implementation effectiveness </li></ul></ul><ul><ul><li>3 = no indep teams </li></ul></ul><ul><ul><li>4 = 1 RCT or time series </li></ul></ul><ul><ul><li>5 = comparisons w/o randomization </li></ul></ul><ul><ul><li>6 = pre-post </li></ul></ul><ul><ul><li>7 = endorsement by authorities </li></ul></ul>
    12. 13. Kazdin (2004) criteria <ul><li>Not evaluated </li></ul><ul><li>Evaluated but unclear, no or possibly negative effects at this time </li></ul><ul><li>Promising (some evidence) </li></ul><ul><li>Well-established (parallel to well-established in conventional schemes) </li></ul><ul><li>Better/Best Treatments (treatments shown to be more effective than other evidence-based treatments) </li></ul>
    13. 14. 14 Major Reviews of Evidence-based Interventions for Children (1998-2004) <ul><li>Chambless & Hollon (1998) Defining empirically-supported therapies. Journal of Consulting & Clinical Psychology </li></ul><ul><li>Surgeon General’s Mental Health Report (1999) </li></ul><ul><li>Weisz & Jensen (1999) Mental Health Services Research </li></ul><ul><li>Journal of the Am. Academy of Child/Adol. Psychiatry , 1999 </li></ul><ul><li>Olds et al., (1999) Review of Preventive Interventions, Center for Mental Health Services </li></ul><ul><li>Burns, Hoagwood, & Mrazek (2000) Effective treatments for mental disorders in children and adolescents, Child Clinical and Family Psych Rvw </li></ul><ul><li>Rones & Hoagwood (2000) School based mental health services review. Clinical Child and Family Psychology Review </li></ul>
    14. 15. <ul><li>Webster-Stratton & Taylor (2001) Preventing violence in adolescence with interventions for young children </li></ul><ul><li>Greenberg, et al., (2001) Prevention of mental disorders in school-aged children. Prevention & Treatment </li></ul><ul><li>Surgeon General’s Youth Violence Report (2001) </li></ul><ul><li>Burns & Hoagwood (2002) Community treatments for youth : Oxford University Press </li></ul><ul><li>Kazdin & Weisz (2003) Evidence-based psychotherapy for children and adolescents </li></ul><ul><li>Weisz, JR. (2004) Psychotherapy for children and adolescent: Evidence-based treatments and case examples. Cambridge University Press. </li></ul><ul><li>Burns, BJ & Hoagwood, K (in press) Update on evidence-based practices. Two Volumes. Psychiatric Clinics of North America </li></ul>14 Reviews of EBPs
    15. 16. <ul><li>More than 1500 published clinical trials on outcomes of psychotherapies for youth </li></ul><ul><li>More than 550 different named psychotherapies (Kazdin, 2000; Kazdin & Weisz, 2003; Weisz, 2004) </li></ul><ul><li>6 meta-analyses of their effects: effects as robust as for adults </li></ul><ul><li>More than 300 published clinical trials on safety/efficacy of psychotropic medications and growing </li></ul><ul><li>Approx 50 field trials of community-based services </li></ul><ul><li>47 effective school-based interventions cited by Rones & Hoagwood (2000) </li></ul><ul><li>34 effective preventive interventions cited by Greenberg et al, 2001 </li></ul>Strength of the evidence
    16. 17. EVIDENCE-BASED PSYCHOSOCIAL TREATMENTS Well-Established Probably Efficacious <ul><li>DEPRESSION </li></ul><ul><li>None Self-Control (children) </li></ul><ul><li>Coping with Depression (adolescents) </li></ul><ul><li>IPT (adolescents) </li></ul><ul><li>ADHD </li></ul><ul><li>Behavioral Parent Training Behavioral Management Training </li></ul><ul><li>Behavioral Interventions in the Classroom Behavioral Modification in Classroom </li></ul><ul><li>ANXIETY </li></ul><ul><li>None Cognitive-Behavioral Therapy </li></ul><ul><li>Phobia </li></ul><ul><li>Participant Modeling Imaginal and In Vivo Desensitization </li></ul><ul><li>Reinforced Practice Live and Filmed Modeling </li></ul><ul><li>DISRUPTIVE BEHAVIOR </li></ul><ul><li>Living with Children Delinquency Prevention Program </li></ul><ul><li>Videotape Modeling Parent-Child Interaction Therapy Parent Training Program </li></ul><ul><li>Time-Out Plus Signal Seat Treatment </li></ul><ul><li>Anger Coping Therapy </li></ul><ul><li>Problem Solving Skills Training </li></ul><ul><li>Anger Control Training w/ Stress Innoc </li></ul><ul><li>Assertiveness Training </li></ul><ul><li>Multisystemic Therapy </li></ul><ul><li>Rational-Emotive Therapy </li></ul>Preschool Adolescent School Age Source: Journal of Clinical Child Psychology, Volume 27, Number 2, 1998 + additional studies for depression and PTSD
    17. 18. Evidence-Based Psychosocial Treatments Well-Established Probably Efficacious PTSD <ul><li> </li></ul><ul><li>None Cognitive-Behavioral Therapy for sexual & physical abuse </li></ul>Source: Journal of Clinical Child Psychology, Volume 27, Number 2, 1998 + additional studies for depression and PTSD
    18. 19. Ineffective Psychosocial Treatments <ul><li>Non-behavioral interventions for disruptive behavior disorders and/or ADHD (Weisz et al., 1995; Pelham et al., 1998) </li></ul><ul><li>Group, peer-based interventions for disruptive disorders </li></ul>
    19. 20. <ul><li>DARE (5th and 6th grade curriculum) </li></ul><ul><li>Gun Buyback programs </li></ul><ul><li>Boot Camps </li></ul><ul><li>Peer counseling programs </li></ul><ul><li>Summer job programs (at risk youth) </li></ul><ul><li>Home detention with electronic monitoring </li></ul><ul><li>Wilderness / challenge programs </li></ul><ul><li>Casework / counseling </li></ul><ul><li>What produces negative (iatrogenic) outcomes </li></ul><ul><li>Waivers to adult (criminal courts) </li></ul><ul><li>Scared Straight </li></ul><ul><li>Shock Probation / Parole </li></ul>Ineffective programs to prevent youth violence (Elliot, 2000)
    20. 21. Psychopharmacology Evidence for Childhood Disorders STRONG <ul><ul><li>ADHD Stimulants </li></ul></ul><ul><ul><li>TCAs </li></ul></ul>MODERATE WEAK <ul><ul><li>DEPRESSION SSRIs </li></ul></ul><ul><ul><li>AUTISM Antipsychotics </li></ul></ul><ul><ul><li>OCD SSRIs, TCAs </li></ul></ul><ul><ul><li>ODD/CD Antipsychotics, Mood stabilizers, Stimulants </li></ul></ul><ul><ul><li>ANXIETY SSRIs </li></ul></ul><ul><ul><li>AGGRESSION Atypical antipsychotics </li></ul></ul><ul><ul><li>BIPOLAR Lithium </li></ul></ul><ul><ul><li>TOURETTE’S Antipsychotics </li></ul></ul>
    21. 22. Evidence-based Home & Community-based Services Multisystemic Therapy (Henggeler et al., Schoenwald et al) Intensive Case Management (including Wraparound) (Evans; Burchard; Burns) <ul><ul><li>8 RCTs and 1 quasi-exper. </li></ul></ul><ul><ul><ul><li>fewer arrests </li></ul></ul></ul><ul><ul><ul><li>fewer placements </li></ul></ul></ul><ul><ul><ul><li>decreased aggression </li></ul></ul></ul><ul><ul><ul><li>cost-savings </li></ul></ul></ul><ul><ul><li>4 RCTs and 3 quasi-exper. </li></ul></ul><ul><ul><ul><li>less restrictive placements </li></ul></ul></ul><ul><ul><ul><li>some increased functioning </li></ul></ul></ul>
    22. 23. Evidence-based Home & Community-based Services Treatment Foster Care (Chamberlain) Nurse home visitation Program (Olds et al) Functional family therapy (Alexander & Sexton) <ul><ul><li>4 RCTs </li></ul></ul><ul><ul><ul><li>more rapid improvement </li></ul></ul></ul><ul><ul><ul><li>decreased aggression </li></ul></ul></ul><ul><ul><ul><li>better post-discharge outcomes </li></ul></ul></ul><ul><ul><ul><li>6 RCTs </li></ul></ul></ul><ul><ul><ul><li>Long term improvements in reducing child abuse </li></ul></ul></ul><ul><ul><ul><li>3 RCTs </li></ul></ul></ul><ul><ul><ul><li>Reduced recidivism 6-42 months out </li></ul></ul></ul>
    23. 24. Evidence-based Home & Community-based Services (cont’d) Parent Empowerment (fam ed) (Heflinger & Bickman) Mentoring (Vance) Respite Services (Bruns & Burchard) Crisis Services <ul><ul><li>1 RCT </li></ul></ul><ul><ul><ul><li>increased knowledge and self-efficacy </li></ul></ul></ul><ul><ul><li>1 RCT </li></ul></ul><ul><ul><ul><li>less substance use and aggression </li></ul></ul></ul><ul><ul><ul><li>better school, peer, and family func </li></ul></ul></ul><ul><ul><li>2 wait-list control studies </li></ul></ul><ul><ul><ul><li>fewer placements </li></ul></ul></ul><ul><ul><ul><li>reduced family stress </li></ul></ul></ul><ul><ul><li>0 controlled, 1 pre-post </li></ul></ul><ul><ul><ul><li>placement prevented in 60-90% of cases </li></ul></ul></ul>
    24. 25. Evidence for Institutionally-Based Care Hospital Residential Treatment Center Group Home Partial Hospitalization <ul><ul><li>3 RCTs </li></ul></ul><ul><ul><ul><li>findings in favor of community comparison conditions </li></ul></ul></ul><ul><ul><li>2 quasi-experimental </li></ul></ul><ul><ul><ul><li>Project Re-Ed: gains versus untreated </li></ul></ul></ul><ul><ul><ul><li>Gains in residential treatment center were equal </li></ul></ul></ul><ul><ul><ul><li>to treatment foster care (TFC @ one-half cost) </li></ul></ul></ul><ul><ul><li>2 quasi-experimental </li></ul></ul><ul><ul><ul><li>mixed findings -- gains and </li></ul></ul></ul><ul><ul><ul><li>deterioration (arrest rates) </li></ul></ul></ul><ul><ul><li>1 RCT </li></ul></ul><ul><ul><ul><li>partial hospital versus wait-list controls </li></ul></ul></ul><ul><ul><ul><li>benefits at 6 months for behavior symptoms, </li></ul></ul></ul><ul><ul><ul><li>and family </li></ul></ul></ul>
    25. 27. Challenges to Putting EBPs into Practice <ul><li>No science to guide implementation </li></ul><ul><li>Policies are way ahead of the knowledge base </li></ul><ul><li>Pressures to hold providers accountable for outcomes </li></ul><ul><li>Poorly trained workforce </li></ul><ul><li>Incentives misaligned </li></ul>
    26. 28. 5 Core Components for State System Change <ul><li>Train practicing clinicians to deliver clinical EBPs with proven effectiveness </li></ul><ul><li>Engage families in services by removing barriers to access: Target clinician outreach </li></ul><ul><li>Empower families with tools, guidelines, and support: Target families and advocates </li></ul><ul><li>Support core organizational processes </li></ul><ul><li>Create system-wide incentives to support change </li></ul>
    27. 29. New York State Implementation Model System & Policy Context Financial policies, methods of reimbursement, state policies Organizational Context Culture Climate Structure Clinical Care Improvement Training on EBP’s, supervision, consultation and support Engagement Empowerment Attitudes, Beliefs & Expectancies of Families & Youth Improved Child & Family Outcomes Attitudes, Beliefs & Expectancies of Clinicians and Supervisors Improved Implementation Efficiency & Effectiveness
    28. 30. Implementation of Trauma-focused CBT for Children: The CATS Study <ul><li>Developed in response to the World Trade Center Disaster </li></ul>
    29. 32. Children and Youth participating in the CATS research project ID’s Generated 1068 Assigned 650 Declined 50 Ineligible 165 CATS 450 Conferenced-In 51 Comparison 149 Children 287 Adolescents 163 Children 108 Adolescents 41 Unassigned 204
    30. 33. Acknowledgements CATS Consortium The CATS Consortium is a cooperative multi site treatment study performed by nine independent teams in collaboration with the New York State Office of Mental Health.  The New York State collaborators are Kimberly Eaton Hoagwood, Ph.D., Chip Felton, M.S.W., Sheila Donahue, Ph.D., Anita Appel, M.S.W., James Rodriguez, Ph.D., (NYSPI), Laura Murray, Ph.D., (NYSPI), David Fernandez, M.A., (NYSPI), Joanna Legerski, B.S. (NYSPI), Michelle Chung, B.A., Jacob Gisis, B.S., Jennifer Sawaya, B.A., Sudha Mehta, M.P.H. (OMH), Jessica Mass Levitt, Ph.D. (NYSPI).  The Principal investigators and co-investigators from the nine sites are Robert Abramovitz, M.D., (JBFCS),), Reese Abright, M.D., (St. Vincents Hospital), Peter D’Amico, (North Shore/Long Island Jewish), Giussepe Constantino, Ph.D., (Lutheran Hospital), Carrie Epstein, C.S.W.-R., (Safe Horizon), Jennifer Havens, M.D., (Columbia University), Sandra Kaplan, M.D., (North Shore/LIJ), Jeffrey Newcorn, M.D., (Mt. Sinai), Moises Perez, Ph.D., (Alianza Dominicana), Raul Silva, M.D., (NYU/Bellevue), Heike Thiel de Bocanegra, Ph.D., (Safe Horizon),), Juliet Vogel, Ph.D. (North Shore/LIJ). The Scientific Advisors to the project are:  Leonard Bickman, Ph.D., (Vanderbilt University), Peter S. Jensen, M.D., (Columbia University), Mary McKay, Ph.D., (Mount Sinai Medical School), Susan Essock, Ph.D., (Mount Sinai Medical School), Sue Marcus, Ph.D., (Mount Sinai Medical School), Wendy Silverman, Ph.D. (Florida International University), Robert Pynoos, M.D. (University of California, Los Angeles); Allan Steinberg, Ph.D. (University of California, Los Angeles); Lawrence Palinkas, Ph.D. (University of California at San Diego); and Joseph Cappelleri, Ph.D., (Pfizer Corporation).  The Treatment Developers and Scientific Consultants to the project are:  Judy Cohen, M.D., (University of Pittsburgh), Anthony Mannarino, Ph.D., (University of Pittsburgh), Christopher Layne, Ph.D., (Brigham Young University), William Saltzman, Ph.D ., (UCLA).  
    31. 34. CATS Design <ul><li>81 routine practice therapists working in schools and clinics trained in 2 EBP trauma models </li></ul><ul><li>Ongoing consultation/support provided by treatment developers </li></ul><ul><li>Children and families offered EBP trauma or treatment as usual </li></ul><ul><li>Baseline, 3, 6, 12 month follow-up </li></ul><ul><li>Assessments of PTSD, other anxiety, depression, behavior problems, strengths, school functioning </li></ul><ul><li>Regression discontinuity design + propensity analyses to assess predictors of outcome improvement </li></ul>
    32. 35. Child Trauma-focused Treatment <ul><li>Cohen, Judith; Mannarino, A, Deblinger, E. et al. (2002) Child and Parent Trauma-Focused Cognitive Behavioral Therapy Treatment Manual. </li></ul><ul><ul><li>Trauma Focused Interventions: </li></ul></ul><ul><ul><ul><li>Psychoeducation </li></ul></ul></ul><ul><ul><ul><li>Stress inoculation/Relaxation </li></ul></ul></ul><ul><ul><ul><li>Cognitive triangle </li></ul></ul></ul><ul><ul><ul><li>Trauma Narrative/Gradual exposure </li></ul></ul></ul><ul><ul><ul><li>Cognitive processing </li></ul></ul></ul><ul><ul><li>Parallel parent treatment sessions </li></ul></ul>
    33. 36. Adolescent Treatment <ul><li>Layne, Christopher M.; Saltzman, William R.; Pynoos, Robert S.; (2002) Trauma/Grief-Focused Group Intervention for Adolescents. </li></ul><ul><ul><li>Module I (6 sessions): </li></ul></ul><ul><ul><ul><li>Psychoeducation </li></ul></ul></ul><ul><ul><ul><li>Coping Skills </li></ul></ul></ul><ul><ul><ul><li>Cognitive work </li></ul></ul></ul><ul><ul><ul><li>Communication skills </li></ul></ul></ul><ul><ul><li>Module II (8-12 sessions): </li></ul></ul><ul><ul><ul><li>Trauma narrative/Exposure </li></ul></ul></ul><ul><ul><ul><li>Cognitive restructuring </li></ul></ul></ul><ul><ul><li>Module III (3 sessions): </li></ul></ul><ul><ul><ul><li>Resuming developmental progression </li></ul></ul></ul><ul><ul><ul><li>Problem-solving </li></ul></ul></ul>
    34. 37. Intensive consultation <ul><li>81 clinicians trained on these models </li></ul><ul><li>3 day training + 2 booster sessions + bi-weekly consultation calls for 1 year </li></ul><ul><li>Bi-weekly site visits </li></ul><ul><li>Weekly steering committee calls with PIs </li></ul><ul><li>Weekly site coordination meetings </li></ul>
    35. 38. Engaging Families in Treatment <ul><li>“ Triple threat condition”: poverty, single parent status, and stress </li></ul><ul><li>Rates of service use are lowest in low-income, urban communities. No show rates can be as high as 50% (Armbruster & Kazdin) </li></ul><ul><li>Trained all teams on McKay’s engagement protocol (McKay & Bannon, 2005) </li></ul>
    36. 39. Treatment as Usual Show Rates McKay et al., 2005
    37. 40. Empirically supported engagement interventions <ul><li>Reminders reduced missed appointments by 32% (Kourany et al., 1990; McLean et al., 1989; Shivack et al., 1989) </li></ul><ul><li>Intensive family-focused telephone engagement associated with 50% decrease in initial show rates and a 24% decrease in premature terminations (Szapocznik, 1988; 1997) </li></ul><ul><li>Combined telephone and first interview engagement interventions associated with attendance rates of 74%, representing a 16 to 25% increase above the clinic comparison families (McKay et al., 1998). </li></ul>
    38. 41. First Interview Results
    39. 42. Key elements of engagement training <ul><li>Help clinicians and intake staff examine their perceptions of barriers </li></ul><ul><li>Practice skills related to the initial face-to-face interview with a child and their family </li></ul><ul><li>Support clinicians and intake staff abilities to form collaborative working relationships with adult caregivers and youth </li></ul><ul><li>Help them identify an immediate and practical concern that can be addressed in the first interview </li></ul><ul><li>Learn skills related to the development of a shared commitment, language and understanding with the family </li></ul>
    40. 43. CATS Assessment vs. Treatment Show Rates
    41. 44. Assessment Rate by Site
    42. 45. Treatment Show Rate by Site
    43. 46. Treatment Settings
    44. 47. Average Number of Treatment Sessions for Treatment Completers
    45. 48. Treatment Sessions N=446
    46. 49. Implementation challenges: Matching EBPs to individual cases <ul><li>Strategy: Bruce Chorpita and State of Hawaii level of evidence </li></ul><ul><li>5 levels of evidence </li></ul><ul><li>Detailed information about sample demographics </li></ul><ul><li>Practical and flexible menu of options </li></ul>
    47. 50. Autism Conduct Depression Oppositional Substance None None CBT Parent/Teacher Training CBT None Multisystemic Therapy CBT + parents; IPT; Relaxation Anger Coping; Assertiveness; PSST Behavior Tx; Family Tx ABA; FCT None None None None Play Therapy; GIST Juvenile Justice; Individual Tx Family Tx; Individual Tx Relaxation; Individual Tx Individual Therapy ADHD Behavior Therapy None None Biofeedback; Play Tx; GIST None None Group Therapy None Group Therapy Group Therapy Anxiety CBT; Exposure; Modeling CBT+ parents; Ed support None EMDR; Play Tx; GIST None Problem Level 1 best support Level 2 good support Level 3 some support Level 4 no support Level 5 known risks Example: Chorpita (2002) EBT Analysis
    48. 51. Example: Clinical application <ul><li>14 year old </li></ul><ul><li>Depressed </li></ul><ul><li>Puerto Rican </li></ul><ul><li>Male </li></ul><ul><li>Late in semester </li></ul>
    49. 52. Level 2 CBT + parents Interpersonal Relaxation 88% 85% 100% MA; PhD MA; PhD; MD MA; PhD clinic clinic school CBT 94% MA; PhD Clinic; school 1.74 1.40 1.51 1.48 Level 1 Intervention Finish 14 to 18 12 to 18 11 to 18 9 to 18 Age Staff Setting Effect NS 49% PR; 41% HA; 10% C NS 84% NS; 18%PR; 3%AA Ethn 7 to 8 weeks 12 weeks 5 to 8 weeks 5 to 16 weeks Length Evidence: Interventions for Depression
    50. 53. Level 2 CBT + parents Interpersonal Relaxation 88% 85% 100% MA; PhD MA; PhD; MD MA; PhD clinic clinic school CBT 94% MA; PhD Clinic; school 1.74 1.40 1.51 1.48 Level 1 Intervention Finish 14 to 18 12 to 18 11 to 18 9 to 18 Age Staff Setting Effect NS 49% PR; 41% HA; 10% C NS 84% NS; 18%PR; 3%AA Ethn 7 to 8 weeks 12 weeks 5 to 8 weeks 5 to 16 weeks Length Evidence: Interventions for Depression
    51. 54. Level 2 CBT + parents Interpersonal Relaxation 88% 85% 100% MA; PhD MA; PhD; MD MA; PhD clinic clinic school CBT 94% MA; PhD Clinic; school 1.74 1.40 1.51 1.48 Level 1 Intervention Finish 14 to 18 12 to 18 11 to 18 9 to 18 Age Staff Setting Effect NS 49% PR; 41% HA; 10% C NS 84% NS; 18%PR; 3%AA Ethn 7 to 8 weeks 12 weeks 5 to 8 weeks 5 to 16 weeks Length Evidence: Interventions for Depression
    52. 55. Example <ul><li>16 year old </li></ul><ul><li>Female </li></ul><ul><li>Anxiety problems </li></ul><ul><li>Both parents available </li></ul>
    53. 56. Level 2 CBT + parents Edu support 93% 85% MA; PhD N/A clinic clinic CBT 95% UG; MA; PhD Clinic; school 1.05 1.78 N/A Level 1 Intervention Finish 14 to 18 6 to 17 2 to 17 Age Staff Setting Effect NS 92% C 54% NS; 33% C; 7% Arm; 6%AA Ethn 12 weeks 12 weeks 3 to 16 weeks Length Evidence: Interventions for Anxiety
    54. 57. Level 2 CBT + parents Edu support 93% 85% MA; PhD N/A clinic clinic CBT 95% UG; MA; PhD Clinic; school 1.05 1.78 N/A Level 1 Intervention Finish 7 to 12 6 to 17 2 to 17 Age Staff Setting Effect NS 92% C 54% NS; 33% C; 7% Arm; 6%AA Ethn 12 weeks 12 weeks 3 to 16 weeks Length Evidence: Interventions for Anxiety
    55. 58. <ul><li>Strategy: Distillation approach (Chorpita & Weisz, 2005; Dalaiden & Chorpita, 2005) </li></ul><ul><li>Cross tabulate studies with intervention elements </li></ul><ul><li>Use all studies; code each study </li></ul><ul><li>Yields a matrix demonstrating protocol overlaps </li></ul>Implementation Challenges: Too many models to choose among
    56. 59. Example (Chorpita et al., 2005) Internalizing Externalizing All Ext Int
    57. 60. Anxiety and Phobias (Chorpita et al) Depression All Dep A/P Ext Int
    58. 61. Strategies for making EBP implementation practical <ul><li>Integrated Psychotherapy Consortium (Center for the Advancement of Children’s Mental Health– P. Jensen, E. Goldman) </li></ul><ul><li>Michigan’s clinical outcome assessments (K. Hodges, J. Wotring) </li></ul><ul><li>Casey Blue Sky Project (P. McCarthy, S. Henggeler, S. Schoenwald, T. Sexton, P. Chamberlain) </li></ul><ul><li>Hawaii’s clinical decision-making system (B. Chorpita, E. Dalaiden) </li></ul>
    59. 62. New York State Implementation Model System & Policy Context Financial policies, methods of reimbursement, state policies Organizational Context Culture Climate Structure Clinical Care Improvement Training on EBP’s, supervision, consultation and support Engagement Empowerment Attitudes, Beliefs & Expectancies of Families & Youth Improved Child & Family Outcomes Attitudes, Beliefs & Expectancies of Clinicians and Supervisors Improved Implementation Efficiency & Effectiveness
    60. 63. Family-based services <ul><li>Hoagwood (2005) review of 4,000 articles since 1980 identified 41 rigorous studies of family-based services </li></ul><ul><ul><li>3 categories: </li></ul></ul><ul><ul><ul><li>Families as recipients of services </li></ul></ul></ul><ul><ul><ul><li>Families as co-therapist </li></ul></ul></ul><ul><ul><ul><li>Processes of involvement, engagement, empowerment </li></ul></ul></ul>
    61. 64. Conclusions from family-based services review <ul><li>Broader view of outcomes is needed </li></ul><ul><li>Absence of robust literature on process variables limits conclusions </li></ul><ul><li>Evidence ambiguous as to whether these services improve child outcomes </li></ul><ul><li>Linkage of these services to EBP implementation may be needed to amplify effects </li></ul>
    62. 65. Parent Empowerment in New York <ul><li>4 year process: scientific review of the literature </li></ul><ul><li>Identified one controlled trial of empowerment (Bickman et al, 1998) </li></ul><ul><li>Adapted for parent advocates and for multi-ethnic families </li></ul><ul><li>Added modules about EBPs for child mental health (ADHD, depression, conduct, treatment efficacy) </li></ul><ul><li>Collaborative partnership: Mental Health Assn., Columbia University, NYS Office of Mental Health </li></ul><ul><li>Added engagement strategies </li></ul><ul><li>Developed 4 manuals for advocates and parents </li></ul><ul><li>Conducting 2 NIMH-funded effectiveness trials to examine impact of program on knowledge, skills, self-efficacy and use of services (behavior) </li></ul>
    63. 67. PEP Manual Content <ul><li>Parent Advocate Manual </li></ul><ul><li>Introduction </li></ul><ul><li>Getting Ready </li></ul><ul><li>Building Engagement, Listening, and Boundary Setting Skills </li></ul><ul><li>Building Your Teaching and Group Management Skills </li></ul><ul><li>Developing Priority Setting Skills </li></ul><ul><li>Specific Disorders and Their Treatments </li></ul><ul><li>The Mental Health System of Care: What to Expect and How to Prepare </li></ul><ul><li>Services and Options Through the School System </li></ul><ul><li>Teaching Tools for Parent Advocates </li></ul><ul><li>Parent Handbook </li></ul><ul><li>Introduction </li></ul><ul><li>Knowing Yourself </li></ul><ul><li>Knowing Your Child </li></ul><ul><li>Treatment Management Skills: How to be Your Child’s Case Manager </li></ul><ul><li>Specific Disorders and Their Treatments </li></ul><ul><li>The Mental Health System of Care: What to Expect and How to Prepare </li></ul><ul><li>Services and Options Through the School System </li></ul><ul><li>Helpful Tools for Parents </li></ul>
    64. 68. Parent Empowerment Research Study Basic Design 40 Parent Advocates/Family Support Specialists (PA/FSS) 20 PA/FSS PEP Training 20 PA/FSS Training as Usual 120 Parent/Caregivers Receiving PA/FS Services 6 per PA/FSS 120 Parent/Caregivers Receiving PA/FS Services 6 per PA/FSS
    65. 69. Post Training Self Efficacy <ul><li>N=31 </li></ul><ul><li>=p< .05 </li></ul>5 = Greater Efficacy 1 = Less Efficacy
    66. 70. Post Training Self-Efficacy <ul><li>N=31 </li></ul><ul><li>+ = p< .10 </li></ul><ul><li>= p< .05 </li></ul><ul><li>** = p<.001 </li></ul>5 = Greater Efficacy 1 = Less Efficacy
    67. 71. Embedding effective clinical practices in settings and systems <ul><li>Family support services (e.g., engagement, empowerment) and effective clinical treatments are part of larger work environments </li></ul><ul><li>Studies of environmental contexts have identified characteristics that improve or interfere with service delivery </li></ul>
    68. 72. Key Constructs in Measurement of Organizational Contexts <ul><li>Organizational climate reflects perceptions of the work environment and has been linked with child outcomes in studies of child welfare agencies (Glisson & Himmelgarn, 1998) </li></ul><ul><li>Organizational culture refers to the ways things are done in a work environment—the norms and shared expectations </li></ul><ul><li>Organizational structure refers to the hierarchy of power </li></ul>
    69. 73. Organizational context affects uptake of EBPs and outcomes <ul><li>Three decades of studies by Glisson and colleagues </li></ul><ul><li>Glisson & Himmelgarn’s (1998) study of child welfare agencies found that the strongest predictor of child improvement was organizational climate </li></ul><ul><li>Organizational culture , not climate, explained variations in service quality (Glisson & James, 2002) </li></ul><ul><li>Organizational level interventions can improve climate and reduce staff turnover (Glisson, in press) </li></ul><ul><li>Organizational factors affect youth outcomes (Schoenwald et al., 2003) </li></ul>
    70. 74. Glisson & Himmelgarn (1998) Parameter Estimates for Hypothesized Six-Variable Model Service Quality County Demographics Service Outcomes (problem levels) -.13* .12* -.05 -.24* -.03 .02 -.36* .01 .06 -.20* * p < .05 Organizational Climate Interorganizational Services Coordination Interorganizational Relationships
    71. 75. Organizational Context: Implications for the Transport of Evidence-Based Treatments To Mental Health Provider Organizations Sonja K. Schoenwald, Ph.D. Family Services Research Center Psychiatry & Behavioral Sciences Medical University of South Carolina
    72. 76. Organizational structure & climate (Schoenwald et al., 2003) <ul><li>Multi-site study of 40+ community clinics delivering MST </li></ul><ul><li>Examine impact of organizational context on therapist adherence and outcomes </li></ul><ul><li>Organizational structure and climate factors were not associated with adherence scores </li></ul><ul><li>Organizational factors were associated directly with youth outcomes. </li></ul><ul><li>And some associations were in unexpected directions </li></ul>
    73. 77. Aims of Transportability Study <ul><li>To examine : </li></ul><ul><li>the association of MST adherence to outcomes in field sites </li></ul><ul><li>organization’s impact on adherence </li></ul><ul><li>extra-organizational factors’ impact on organizational factors affecting adherence </li></ul><ul><li>the impact of clinician training & experience on adherence </li></ul><ul><li>a mediation model of effectiveness </li></ul>
    74. 78. Social Ecological Model of TreatmentTransportability <ul><li>Extra-Organizational Context </li></ul><ul><li>(Referral, Reimbursement, Disposition) </li></ul><ul><li>Organization Clinician Child </li></ul><ul><li> Adherence Outcomes </li></ul><ul><li>Clinician Variables </li></ul><ul><li>Professional Training & Experience </li></ul>
    75. 79. Transportability of MST - Evidence of Multi-Level Treatments* <ul><li>First 666 referred youth (juvenile justice, child welfare, and mental health) </li></ul><ul><li>14.7 years old, 67% male, 61% Caucasian </li></ul><ul><li>57% one bio parent, 15% both bio parents </li></ul><ul><li>48% less than 20k/yr </li></ul><ul><li>*Schoenwald, Sheidow, Letourneau, & Liao (2003). Mental Health Services Research </li></ul>
    76. 80. Treatment Outcomes <ul><li>Significant pre-post reductions in child behavior problems and functioning </li></ul><ul><li>Discharge was based on achievement of treatment goals in 73% of cases </li></ul><ul><li>Discharge decisions were made by the therapist and family (versus external entity) in 64% of cases </li></ul>
    77. 81. Adherence-Outcomes Linkages <ul><li>Higher adherence predicted post-treatment decreases in child behavior problems and child functioning problems . </li></ul><ul><li>Higher adherence predicted positive discharge circumstances . </li></ul>
    78. 82. Therapist Effects Pre-Post Differences In CBCL Total Scores by Adherence
    79. 83. Organizational Structure & Climate Findings <ul><li>Organizational structure and climate factors were not associated with adherence scores </li></ul>
    80. 84. Organizational Factors Predicted Youth Outcomes <ul><li>Organizational factors were associated with youth outcomes. </li></ul><ul><li>And, some associations were in unexpected directions. </li></ul>
    81. 85. Adherence & Organization: Direct Effects on Outcomes
    82. 86. Moderation of Organizational Effects by Adherence Level (1) <ul><li>Sample of adherence scores was split into upper and lower adherence quartiles </li></ul><ul><li>Advancement & Reward x low adherence = increased child problems </li></ul><ul><li>Advancement & Reward x high adherence = unrelated to child problems </li></ul><ul><li>Greater Procedural Specification x high adherence = increased child problems </li></ul><ul><li>Greater Procedural Specification x low adherence = unsuccessful discharge </li></ul>
    83. 87. Adherence Moderates Organization Effects on Outcomes <ul><li>Opportunities for Advancement & Reward appear to matter little when adherence is high, but translates into poorer outcomes when adherence is low </li></ul><ul><li>Hierarchical Authority and Procedural Specification may interfere with positive outcomes when therapists are adhering to MST, but matters little when adherence is low. </li></ul>
    84. 88. Implications <ul><li>Need to better understand criteria used in organizations for advancement and reward, and to consider including adherence and outcome indicators in those criteria </li></ul><ul><li>Need to better understand how organizational hierarchy and procedures may interfere with adherence to a specific EBP </li></ul>
    85. 89. Key Factors Associated with Adoption and Diffusion (Greenhalgh et al, 2004): A Meta-Narrative Synthesis of Evidence <ul><li>Characteristics of the innovation </li></ul><ul><li>Characteristics of the individual adopter </li></ul><ul><li>Sources of communication and influence </li></ul><ul><li>Structural and cultural characteristics of potential organizational adopters </li></ul><ul><li>Characteristics of the external environment </li></ul><ul><li>Innovation uptake practices </li></ul><ul><li>Linkage among components of the model </li></ul>
    86. 90. Organizational change is personal <ul><li>Interpersonal influence through social networks is the dominant mechanism for diffusion (Valente, 1996) </li></ul><ul><li>Champion roles: </li></ul><ul><ul><ul><li>Organizational maverick </li></ul></ul></ul><ul><ul><ul><li>Transformational leader </li></ul></ul></ul><ul><ul><ul><li>Organizational buffer </li></ul></ul></ul><ul><ul><ul><li>Network facilitator </li></ul></ul></ul><ul><li>Boundary spanners (social networkers) </li></ul><ul><ul><li>Organizations that promote boundary-spanning roles are more likely to assimilate innovations (Barnsley, Lermieux-Charles & McKinnet, 1998; Ferlie et al., 2001) </li></ul></ul>
    87. 91. Sustaining organizational change (Gustafson et al., 2003; Rogers 1995; Plsek 2003; Champagne et al., 1991) <ul><li>Tension for change: Staff want a change </li></ul><ul><li>Innovation-system fit: Innovation fits norms and values of organization </li></ul><ul><li>Assessment of implications of innovation: Implications are thought about in advance </li></ul><ul><li>Support and advocacy—Existence of champions and boundary spanners </li></ul><ul><li>Dedicated time and resources </li></ul><ul><li>Capacity to evaluate the innovation: Ability to monitor and evaluate the impact of the innovation </li></ul>
    88. 92. Areas for further study <ul><li>What are the key factors that improve the uptake and sustainability of efficacious treatments? </li></ul><ul><li>What factors improve the fidelity of implementation efforts? </li></ul><ul><li>What are the most effective outcome measures and suitable methodologies for dissemination and implementation? </li></ul><ul><li>How do different stakeholder perspectives about EBPs affect organizational readiness to adopt new practices? </li></ul><ul><li>What are the mediators and moderators of organizational effects? </li></ul><ul><li>Can organizational context be changed to improve adoption of new practices? What are effective interventions for changing organizational culture and climate? </li></ul><ul><li>How do family and consumer perspectives affect organizational readiness to adopt new practices? </li></ul>
    89. 93. Important Contributing Fields <ul><li>Social Marketing—packaging EBPs? </li></ul><ul><li>Behavioral Change—why use an EBP? </li></ul><ul><li>Anthropology—fit in different communities? </li></ul><ul><li>Organizational behavior—can organizational environments be changed? </li></ul><ul><li>Finance/Economics—is there an economic argument for EBPs? </li></ul><ul><li>Technology development: EBPs are a kind of technology—how to efficiently incorporate new technologies in new environments? </li></ul>
    90. 94. Culture Structure Psychological Climate Organizational Climate Attitudes Social Norms Self-Efficacy Beliefs & Expectations Behavioral Intention Models of Diffusion, Organizational Implementation & Social Processes Systems Context Organizational Properties Individual & Shared Perceptions Behavior Structural Determinants of Organizational Innovation Social Determinants of Organizational Innovation Adapted from Glisson 2002
    91. 95. Concluding thoughts <ul><li>“There is no practice without theory, however much that theory is suppressed, unformulated, or perceived as obvious.” Northrup Frye/Belsey </li></ul>
    92. 96. Closing thought <ul><li>“New technologies alter the structure of our interests: the things we think about. They alter the character of our symbols: the things we think with. And they alter the nature of community: the arena in which thoughts develop.” </li></ul><ul><li>Neil Postman, Technopoly </li></ul>
    93. 98. Dimensions of Organizational Readiness <ul><li>What factors are considered important to the uptake of evidence-based practices? </li></ul><ul><li>Do all stakeholders agree on the relative importance of specific factors? </li></ul>
    94. 99. Service System Organization Service Delivery Client Practitioner Intervention Schoenwald & Hoagwood, 2001 Organiza-tional mandates Ethnicity/ cultural iden Salary level/ Criteria for increases Clarity of intervention Organiza-tional mission Gender Endorsement of intervention Complexity of intervention Interagency working relationship Organiza-tional climate Source of payment Age and dev- elopmental status Training of practitioner Similarity of int to std practice Legal mandate for referrals Organiza- tional culture Physical location of sessions Source of referral Supervisor/ Researcher Intervention specification Manual? Financing methods Personnel policies Length of sessions Family context Adherence monitoring Focus of intervention Policies of referral source, pay Structure, hierarchy Frequency of sessions Nature of referral problems Specialized training Nature of intervention theory
    95. 100. Intervention Characteristics Theoretical foundation, strength of research support, clinical foundation, precision, availability of manual, specificity of manual, clarify of model Practitioner Characteristics Clinical adherence to model, frequency of clinical supervision, structure of clinical supervision, type of clinician, treatment orientation of clinician Client Characteristics Referral problem(s), family context, client’s ethnicity/cultural identification Service Delivery Characteristics Referral source, frequency of treatment sessions, length of treatment sessions, setting/location of treatment sessions, setting/location of the clinic or school Service System Characteristics Salary incentives to adopt EBPs, policies and practices of referral sources, source of payments for the specific EBP, financing/payment mechanisms, legal mandates of referral sources, strength of interagency relationships DOMAIN 1 DOMAIN 2 DOMAIN 3 DOMAIN 4 DOMAIN 5 DOMAIN 6 Key Readiness Factors Service Agency Characteristics Endorsement by site leadership, structure of organization, size of organization, culture and climate of organization, policies and practices within the organization
    96. 101. Dimensions of Organizational Readiness (DOOR) Question: how important are the following factors: Frequency of clinical supervision required to deliver the EBT (e.g., consultations between clinicians & supervisors) Length of each treatment session required to deliver the EBT Strength of the research supporting the EBT           … outside agencies           … state mental health authorities           … consumer advocacy           … families or youth in the service setting           … clinical staff (e.g., therapists, social workers, psychologists, psychiatrists)           Support for the EBT by …. Clinic Directors Clinicians Consumers Researchers To Me
    97. 102. Mean Ratings of Readiness Factors Scale
    98. 103. Total Mean Ratings of Readiness Factors
    99. 104. Comparison of Respondent Group Ratings Scale

    ×