The Case for Evidence-Based Programs Del Elliott, Ph.D. Center for the Study and Prevention of Violence University of Colorado, Boulder
Evidence: Something thatfurnishes or tends to furnish proof (Webster)
Reviewing the Scientific Evidence: The Blueprints Strategy A systematic review of individual program evaluations to identify violence, drug abuse and delinquency prevention programs that meet a high scientific standard of effectiveness Individual programs meeting this standard are certified as Model or Promising evidence-based programs Only Model programs are considered eligible for widespread dissemination
Scientific Standard for Certification as Evidence-Based* Experimental Design/RCT Statistically significant & substantive positive effect Effect sustained for at least 1 year post- intervention At least 1 external replication with RCT/QED RCT’s/QED’s adequately address threats to internal validity No known health-compromising side effects*Adapted from Blueprints and Working Group for the Federal Collaboration on What Works , 2004. www.ncjrs.gov/pdffiles1/nij/220889.pdf
Evidence-Based Classification* Model: Meets all standards I. Model: Meets all standards II. Effective: Internal rather than external RCT replication(s) III. Promising: Lacks any RCT replication IV. Inconclusive: Contradictory findings or non-sustainable effects V. Ineffective: Meets all standards but with no statistically significant effects VI. Harmful: Meets all standards but with negative main effects or serious side effects VII Insufficient Evidence: All others*Adapted from Blueprints and Hierarchical Classification Framework for Program Effectiveness, Working Group for the Federal Collaboration on What Works, 2004.
Review of Evaluation Evidence* Crime and Drug Prevention Programs Most Programs Have No Credible Evaluation Those With Credible Evaluations: Most Don’t Work 40 to 45 Clearly Work or Have Promise A Few Appear to be Harmful Most Model Programs Don’t Have Capacity to Go to Scale*Over 900 programs. Center for the Study and Prevention of Violence
What Doesn’t Work! *Negative Effects Waivers to Adult (Criminal Courts)* • Boot Camps • Traditional and New DARE* • GGI and Positive Peer Culture • Gun Buyback Programs* • Peer Counseling Programs • Summer Job Programs (At Risk Youth)Adapted from Sherman et al., 2002; Aos et al.,2004; Sloboda et al, 2009; Redding,2008; Sparks, 2010; James-Burdumy et al, 2005.
What Doesn’t Work! *Negative Effects• Neighborhood Watch• Arrests of Unemployed Domestic• Abusers* Scared Straight*• Shock Probation/Parole*• Home Detention with Electronic Monitoring• Casework/Counseling*• Character Education.
What Doesn’t Work! *Negative Effects Intensive Supervised Probation/Parole (A,J) Case Management Substance Abuse Program (A) Community Policing-Increasing Information Flow Strategy Diversion from Court to Job Training (A) Drug Market Arrests
What Doesn’t Work! *Negative Effects Juvenile Wilderness Programs Peer Counseling Programs Short-Term Non-Residential Training Urine Testing Job Training Partnership Act 21st Century Community learning Centers* Rehabilitation Programs –Non Directive Unstructured Counseling
Evidence-Based Programs Effective Prevention and Intervention Programs
Evidence-Based Model Programs (Blueprint Programs)Nurse Home Visitation Midwestern Prevention ProjectPATHS Functional Family TherapyBullying Prevention Program Multisystemic TherapyBig Brothers Big Sisters Project TNDLife Skills Training Treatment Foster CareIncredible Years
The Incredible YearsDescription: Parent, teacher, and child training to promote child emotional and social competenceTarget: Children ages 2-8Cost: $1,300 BASIC, $775 ADVANCE, $995 SCHOOL, $975 Child Training, $1,250 TeacherContact: Carolyn Webster-Stratton University of Washington, School of Nursing Department of Family and Child Nursing 1107 NE 45th St., Suite 305 Seattle, WA 98105-4631 (206) 543-6010
The Incredible Years Evidence of EffectParent Program:• Reduced conduct problems.• Increases in positive affect and compliance to parental commands.Teacher Program:• Reductions in peer aggression in the classroom.• Increases in positive affect and cooperation with teachers, positive interactions with peers school readiness and engagement with school activities.Child Program:• Reductions in conduct problems at home and school.• Improved cognitive problem-solving strategies with peers.
Multisystemic TherapyDescription: Multidimensional home-based family therapyTarget: Chronic and violent offenders, ages 12-17Cost: $4,500 per youthContact: Scott Henggeler, Ph.D. Program Designer Medical University of South Carolina For training and implementation questions: Keller Strother, MST, Inc., P.O. Box 21269 Charleston, SC 29413 843-843-856-8226
Multisystemic Therapy Evidence of Effect• Reductions in rearrests 25-70%.• Reductions in self-reported criminal behavior.• Decreased drug use.• Improvements in family functioning.• Decreased mental health problems.
Life Skills TrainingDescription: Drug preventionTarget: All middle/junior high school (grades 6/7) studentsCost: $625 per 30 students for 3 years (no training incl)Contact: Gilbert Botvin, Ph.D., Director Institute for Prevention Research Cornell University Medical College 411 East 69th Street, KB-201 New York, NY, 10021 212-746-1270
Life Skills Training Evidence of Effect• Reduces tobacco, alcohol, and marijuana use 50-75%.• Effects maintained through grade 12. • Cuts polydrug use up to 66%. • Reduces pack-a-day smoking by 25%. • Decreases use of inhalants, narcotics, and hallucinogens.
Promising Programs Programs Effective at ReducingDelinquency/Crime, Violence & Drug UseATLASBASICS (Brief Alcohol Intervention with College Students)CASASTART (formerly Children at Risk Program)Behavioral Monitoring and Reinforcement ProgramBrief Strategic Family TherapyCTC (Communities that Care)Guiding Good ChoicesIowa Strengthening Families ProgramProject NorthlandPreventive Treatment Program (Montreal)Raising Healthy ChildrenSeattle Social Development ProjectStrong African American Families
Promising Programs Programs Effective at Reducing ChildhoodPrecursors of Crime (Predelinquent Aggression)FAST TrackGood Behavior GameI Can Problem SolveLinking the Interests of Families and Teachers (LIFT)Perry PreschoolTriple P- Positive Parenting Program
Demonstrated Outcomes of EB Prevention Programs Improved Grades Higher Rates of Next Grade Promotion Increased GPA Higher Graduation Rates Better Reading, Math and Writing Skills Higher Standardized Test Scores Increased Credits Earned Higher Child Development Levels (young Children CSAP Model Programs’ Academic Outcome Measures and Proven Outcomes, 2002
Demonstrated Outcomes of EB Prevention Programs Con’td Lower Absenteeism Lower High School Dropout Higher Parent Involvement Fewer Suspensions Fewer Retentions Fewer Special Education Referrals Fewer School Behavioral Incidents Improved Social Competence Higher Participation in After School Activities CSAP Model’s Academic Outcome Measures and Proven Outcomes, 2002
Benefit-to-Cost Ratios: Selected Blueprint Programs* Program Ratio Outcome BBBS $1.01 None FFT $13.25 Crime reduction LST $25.61 Drug reduction MPP $5.29 Drug reduction MST $2.64 Drug reduction MTFC $10.88 Crime reduction NFP $2.88 Crime reduction *Washington Institute for Public Policy (2004)
Benefit-to-Cost Ratios: Selected Other Programs* Program Ratio Outcome DARE $0.0 None Boot Camps $0.0 None Head Start $0.23 None Even Start $0.0 None Healthy Families $0.34 None Scared Straight $-203.51 Increases crime *Washington Institute for Public Policy (2004)
Redirection Project* State of Florida Felony conviction/adjudication rates, 1 year post completion: Redirect 19% vs Comparison 25% Felony referral/arrest: Redirect 32% vs Comparison 43% Cost savings: Redirection costs $28,500 less per offender Cost savings: Nearly $40 Million [2004 -2008].* MST &FFT. Comparison involves current residential programs
Feasibility Example Cost to provide every student in U.S. a model drug prevention program like LST is $550 million per year Current national drug control spending is approximately $40 billion per year This represents 1.5% of the current drug control spending
“…both benefit-cost analysesand meta analyses have proven quite appealing in public policy: they lead to simple , quantified results of general application,can be readily remembered, and are not hindered by multiple caveats”. Shadish et al., 1991
The Ideal Evidence-Based Program* Addresses major risk/protection factors that are manipulatable with substantively significant effect sizes Relatively easy to implement with fidelity Causal and change rationales and services/treatments are consistent with the values of professionals who will use it Keyed to easily identified problems Inexpensive or positive cost-benefit ratios Can influence many lives or have life-saving types of effects on some lives *Adapted from Shadish, Cook and Leviton, 1991:445.
Summary: Benefits of Using Evidence-Based Programs and Practices Ethics-Avoiding Harmful Effects Stronger & More Consistent Positive Outcomes Cost Savings to Taxpayers Improving the Well-being of Our Children
THANK YOUCenter for the Study and Prevention of Violence Phone: 303-492-1032 Web Site: www.colorado.edu/cspv