Evidence based practices


Published on

Presentation to the Board on evidence based programming

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Evidence based practices

  1. 1. The Case for Evidence-Based Programs Del Elliott, Ph.D. Center for the Study and Prevention of Violence University of Colorado, Boulder
  2. 2. Evidence: Something thatfurnishes or tends to furnish proof (Webster)
  3. 3. Reviewing the Scientific Evidence: The Blueprints Strategy A systematic review of individual program evaluations to identify violence, drug abuse and delinquency prevention programs that meet a high scientific standard of effectiveness Individual programs meeting this standard are certified as Model or Promising evidence-based programs Only Model programs are considered eligible for widespread dissemination
  4. 4. Scientific Standard for Certification as Evidence-Based* Experimental Design/RCT Statistically significant & substantive positive effect Effect sustained for at least 1 year post- intervention At least 1 external replication with RCT/QED RCT’s/QED’s adequately address threats to internal validity No known health-compromising side effects*Adapted from Blueprints and Working Group for the Federal Collaboration on What Works , 2004. www.ncjrs.gov/pdffiles1/nij/220889.pdf
  5. 5. Evidence-Based Classification* Model: Meets all standards I. Model: Meets all standards II. Effective: Internal rather than external RCT replication(s) III. Promising: Lacks any RCT replication IV. Inconclusive: Contradictory findings or non-sustainable effects V. Ineffective: Meets all standards but with no statistically significant effects VI. Harmful: Meets all standards but with negative main effects or serious side effects VII Insufficient Evidence: All others*Adapted from Blueprints and Hierarchical Classification Framework for Program Effectiveness, Working Group for the Federal Collaboration on What Works, 2004.
  6. 6. Review of Evaluation Evidence* Crime and Drug Prevention Programs Most Programs Have No Credible Evaluation Those With Credible Evaluations:  Most Don’t Work  40 to 45 Clearly Work or Have Promise  A Few Appear to be Harmful Most Model Programs Don’t Have Capacity to Go to Scale*Over 900 programs. Center for the Study and Prevention of Violence
  7. 7. What Doesn’t Work! *Negative Effects Waivers to Adult (Criminal Courts)* • Boot Camps • Traditional and New DARE* • GGI and Positive Peer Culture • Gun Buyback Programs* • Peer Counseling Programs • Summer Job Programs (At Risk Youth)Adapted from Sherman et al., 2002; Aos et al.,2004; Sloboda et al, 2009; Redding,2008; Sparks, 2010; James-Burdumy et al, 2005.
  8. 8. What Doesn’t Work! *Negative Effects• Neighborhood Watch• Arrests of Unemployed Domestic• Abusers* Scared Straight*• Shock Probation/Parole*• Home Detention with Electronic Monitoring• Casework/Counseling*• Character Education.
  9. 9. What Doesn’t Work! *Negative Effects Intensive Supervised Probation/Parole (A,J) Case Management Substance Abuse Program (A) Community Policing-Increasing Information Flow Strategy Diversion from Court to Job Training (A) Drug Market Arrests
  10. 10. What Doesn’t Work! *Negative Effects Juvenile Wilderness Programs Peer Counseling Programs Short-Term Non-Residential Training Urine Testing Job Training Partnership Act 21st Century Community learning Centers* Rehabilitation Programs –Non Directive Unstructured Counseling
  11. 11. Evidence-Based Programs Effective Prevention and Intervention Programs
  12. 12. Evidence-Based Model Programs (Blueprint Programs)Nurse Home Visitation Midwestern Prevention ProjectPATHS Functional Family TherapyBullying Prevention Program Multisystemic TherapyBig Brothers Big Sisters Project TNDLife Skills Training Treatment Foster CareIncredible Years
  13. 13. The Incredible YearsDescription: Parent, teacher, and child training to promote child emotional and social competenceTarget: Children ages 2-8Cost: $1,300 BASIC, $775 ADVANCE, $995 SCHOOL, $975 Child Training, $1,250 TeacherContact: Carolyn Webster-Stratton University of Washington, School of Nursing Department of Family and Child Nursing 1107 NE 45th St., Suite 305 Seattle, WA 98105-4631 (206) 543-6010
  14. 14. The Incredible Years Evidence of EffectParent Program:• Reduced conduct problems.• Increases in positive affect and compliance to parental commands.Teacher Program:• Reductions in peer aggression in the classroom.• Increases in positive affect and cooperation with teachers, positive interactions with peers school readiness and engagement with school activities.Child Program:• Reductions in conduct problems at home and school.• Improved cognitive problem-solving strategies with peers.
  15. 15. Multisystemic TherapyDescription: Multidimensional home-based family therapyTarget: Chronic and violent offenders, ages 12-17Cost: $4,500 per youthContact: Scott Henggeler, Ph.D. Program Designer Medical University of South Carolina For training and implementation questions: Keller Strother, MST, Inc., P.O. Box 21269 Charleston, SC 29413 843-843-856-8226
  16. 16. Multisystemic Therapy Evidence of Effect• Reductions in rearrests 25-70%.• Reductions in self-reported criminal behavior.• Decreased drug use.• Improvements in family functioning.• Decreased mental health problems.
  17. 17. Life Skills TrainingDescription: Drug preventionTarget: All middle/junior high school (grades 6/7) studentsCost: $625 per 30 students for 3 years (no training incl)Contact: Gilbert Botvin, Ph.D., Director Institute for Prevention Research Cornell University Medical College 411 East 69th Street, KB-201 New York, NY, 10021 212-746-1270
  18. 18. Life Skills Training Evidence of Effect• Reduces tobacco, alcohol, and marijuana use 50-75%.• Effects maintained through grade 12. • Cuts polydrug use up to 66%. • Reduces pack-a-day smoking by 25%. • Decreases use of inhalants, narcotics, and hallucinogens.
  19. 19. Promising Programs Programs Effective at ReducingDelinquency/Crime, Violence & Drug UseATLASBASICS (Brief Alcohol Intervention with College Students)CASASTART (formerly Children at Risk Program)Behavioral Monitoring and Reinforcement ProgramBrief Strategic Family TherapyCTC (Communities that Care)Guiding Good ChoicesIowa Strengthening Families ProgramProject NorthlandPreventive Treatment Program (Montreal)Raising Healthy ChildrenSeattle Social Development ProjectStrong African American Families
  20. 20. Promising Programs Programs Effective at Reducing ChildhoodPrecursors of Crime (Predelinquent Aggression)FAST TrackGood Behavior GameI Can Problem SolveLinking the Interests of Families and Teachers (LIFT)Perry PreschoolTriple P- Positive Parenting Program
  21. 21. Demonstrated Outcomes of EB Prevention Programs Improved Grades Higher Rates of Next Grade Promotion Increased GPA Higher Graduation Rates Better Reading, Math and Writing Skills Higher Standardized Test Scores Increased Credits Earned Higher Child Development Levels (young Children CSAP Model Programs’ Academic Outcome Measures and Proven Outcomes, 2002
  22. 22. Demonstrated Outcomes of EB Prevention Programs Con’td Lower Absenteeism Lower High School Dropout Higher Parent Involvement Fewer Suspensions Fewer Retentions Fewer Special Education Referrals Fewer School Behavioral Incidents Improved Social Competence Higher Participation in After School Activities CSAP Model’s Academic Outcome Measures and Proven Outcomes, 2002
  23. 23. Benefit-to-Cost Ratios: Selected Blueprint Programs* Program Ratio Outcome  BBBS $1.01 None  FFT $13.25 Crime reduction  LST $25.61 Drug reduction  MPP $5.29 Drug reduction  MST $2.64 Drug reduction  MTFC $10.88 Crime reduction  NFP $2.88 Crime reduction *Washington Institute for Public Policy (2004)
  24. 24. Benefit-to-Cost Ratios: Selected Other Programs* Program Ratio Outcome  DARE $0.0 None  Boot Camps $0.0 None  Head Start $0.23 None  Even Start $0.0 None  Healthy Families $0.34 None  Scared Straight $-203.51 Increases crime *Washington Institute for Public Policy (2004)
  25. 25. Annual Taxpayer Costs & Benefits:Forecast with Moderate Portfolio of Evidence-Based Programs $450 $400 $350 $300 $250 Benefits $200 Cost $150 $100 $50 $0 2008 2012 2016 2020 2024 2028 2030
  26. 26. Redirection Project* State of Florida Felony conviction/adjudication rates, 1 year post completion: Redirect 19% vs Comparison 25% Felony referral/arrest: Redirect 32% vs Comparison 43% Cost savings: Redirection costs $28,500 less per offender Cost savings: Nearly $40 Million [2004 -2008].* MST &FFT. Comparison involves current residential programs
  27. 27. Feasibility Example Cost to provide every student in U.S. a model drug prevention program like LST is $550 million per year Current national drug control spending is approximately $40 billion per year This represents 1.5% of the current drug control spending
  28. 28. “…both benefit-cost analysesand meta analyses have proven quite appealing in public policy: they lead to simple , quantified results of general application,can be readily remembered, and are not hindered by multiple caveats”. Shadish et al., 1991
  29. 29. The Ideal Evidence-Based Program* Addresses major risk/protection factors that are manipulatable with substantively significant effect sizes Relatively easy to implement with fidelity Causal and change rationales and services/treatments are consistent with the values of professionals who will use it Keyed to easily identified problems Inexpensive or positive cost-benefit ratios Can influence many lives or have life-saving types of effects on some lives *Adapted from Shadish, Cook and Leviton, 1991:445.
  30. 30. Summary: Benefits of Using Evidence-Based Programs and Practices Ethics-Avoiding Harmful Effects Stronger & More Consistent Positive Outcomes Cost Savings to Taxpayers Improving the Well-being of Our Children
  31. 31. THANK YOUCenter for the Study and Prevention of Violence Phone: 303-492-1032 Web Site: www.colorado.edu/cspv
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.