1
Using What Works
2
Today’s Presenters
3
Elizabeth
Gaines,
The Forum
for Youth
Investment
(Host and
moderator)
Dirk
Butler,
The Annie E.
Casey
Foundation
Kevin
Haggerty,
Social
Development
Research
Group
Jessie
Watrous,
The Annie E.
Casey
Foundation
4
Communicate with us using the Q&A window at the
right of the presentation window.
• Type questions for the panel at any time during the
webinar.
• Use the box to let us know if you are having technical
difficulties.
Communicating During the Webinar
Welcome and Overview
Dirk Butler
5
The Annie E. Casey Foundation
develops solutions to
build a brighter future for
children, families
and communities.
6
The Evidence2Success Framework
Governance
structure
including public
systems and
community
Strategic use of
data
Comprehensive
financing
strategies
Implementation
of tested,
effective
programs
Performance
measures
7
8
The Evidence2Success Process
Youth
Experience
Survey
Fund-
Mapping Tool
and
Strategies
Blueprints
Database of
Evidence-
Based
programs
Implemen-
tation
Dashboard
9
Preventing Problems Before They Start
Concepts of Prevention Science and Finding Proven Programs
Identifying Community Priorities
Gathering data from the Youth Experience Survey and working together
to choose priority outcomes
Financing Tested, Effective Programs
Using tools to map funding sources and establish work plans for
supporting programs that work
The Using What Works Webinar Series
How Prevention Science
Works
Kevin Haggerty
10
11
Why Prevention Science?
Heart
disease
High blood
pressure
Obesity
Diet high in calories,
sodium, saturated fat
Use a public health approach to positively affect outcomes
by:
• reducing community levels of elevated risk factors;
• increasing the levels of community suppressed protective factors;
and
• implementing tested, effective programs unique to the community
need
12
Three Elements of
a Prevention Science Approach
Evidence2Success:
Powered by Communities That Care
Large trial of Communities That 
Care produced reductions in drug 
use and delinquency
• 33% tobacco
• 32% alcohol
• 25% delinquent behavior
13
A system of tools and resources to
• decrease community risk factors
• increase community levels of
protection
• implement evidence-based
programs
• change systems
to IMPACT outcomes
Common Risk Factors Can
Predict Multiple Outcomes

14
What is Required for Young People to Be Successful?
Community
School
Individual and Peer
Family
Child Well-
being
Positive
Relationships
Behavior
EducationPhysical Health
Emotional
Well-being
15
Protective Factors: Five Elements
16
Interaction of Risk and Protection:
Foundation of Resilience
Risk
Factors
Protective
Factors
17
What Are They?
1. R__________
2. P___________
3. T___________________
Three Critical Elements
18
What Are They?
1. Risk Factors
2. Protective Factors
3. Tested Effective Programs
Three Critical Elements
19
Finding Tested,
Effective Programs
Jessie Watrous
20
What Is a Tested, Effective Program?
21
Impact
Evaluation
Quality
System
Readiness
Intervention
Specificity
 Positive impact on
child well-being
outcomes
 Absence of any
negative effects
 One randomized
controlled trial OR
two quasi-
experimental trials
 Population clearly
defined
 Risk and protective
factors identified
 Training materials
 Workforce capacity
 Cost-benefit analysis
• Stronger and more consistent positive outcomes
• Strong ethical argument – avoid potential harmful effects
• Potential cost savings to taxpayers and society
• Improving the well-being of our children at a population level
Why Tested, Effective
Programs?
A Six-Step Process
23
Assess risk
and
protection
Prioritize
risk and
protective
factors and
outcomes
to address
Assess
resources
and identify
gaps to
address
priorities
Search
Blueprints
for
programs
that
address
priorities
and gaps
Assess
program fit
on multiple
dimensions
Select
tested,
effective
program
Considerations for Program Fit
24
Type of
prevention
Potential return
on investment
Local capacity
required and
support
provided
Cost and
alignment with
funding
Setting
Population
characteristics
Three Types of Prevention Programs
25
Universal
Selective
Indicated
• All children/youth
• Example = Positive
Action
• Elevated risk/exposure
• Example= Nurse
Family Partnership
• Signs of early problem
behavior
• Example=
Multisystemic Therapy
Locating the Right Program
Through Online Databases
26
• Pew MacArthur Results First Clearinghouse Database
(database of databases):
http://www.pewtrusts.org/en/multimedia/data-
visualizations/2015/results-first-clearinghouse-database
• Child Trends What Works/LINKS database:
http://childtrends.org/what-works/
• SAMHSA National Registry of Evidence-Based Programs
and Practices: http://nrepp.samhsa.gov/01_landing.aspx
• Blueprints for Healthy Youth Development:
www.blueprintsprograms.com
The Blueprints database:
Your tool for finding tested, effective programs
27
www.blueprintsprograms.com
What Is a Tested, Effective Program?
28
Impact
Evaluation
Quality
System
Readiness
Intervention
Specificity
 Positive impact on
child well-being
outcomes
 Absence of any
negative effects
 One randomized
controlled trial OR
two quasi-
experimental trials
 Population clearly
defined
 Risk and protective
factors identified
 Training materials
 Workforce capacity
 Cost-benefit analysis
Blueprints Program Designations
29
Three Ways to Search
30
31
32
Please use the Q&A function on your screen to
submit questions.
Questions?
The Road to a “Proven” Distinction
33
Develop a
strong
program
design
Attain strong
evidence of
positive program
outcomes
• Carry out
evaluation with
a comparison
group
• Conduct
regression
analysis (quasi-
experimental
design )
• Perform
multiple pre-
and post -
evaluations
• Meta-analysis
• Carry out
evaluation with
a comparison
group
• Conduct
regression
analysis (quasi-
experimental
design )
• Perform
multiple pre-
and post -
evaluations
• Meta-analysis
Produce
indicators
of
positive
outcomes
• Conduct evaluation
with random
assignment
(experimental
design)
• Carry out multiple
evaluations with
strong comparison
group (quasi-
experimental
design)
• Conduct evaluation
with random
assignment
(experimental
design)
• Carry out multiple
evaluations with
strong comparison
group (quasi-
experimental
design)
• Conduct
pre- and
post-
interventi
on
evaluation
• Conduct
pre- and
post-
interventi
on
evaluation
• Evaluate
program
quality and
process
• Establish
continuous
improvement
system
• Evaluate
program
quality and
process
• Establish
continuous
improvement
system
Ensure
fidelity of
Implement-
ation
Obtain
evidence of
positive
program
outcomes
• Create logic
model and
replication
materials
• Create logic
model and
replication
materials
34
Identifying Community Priorities for Child Well-Being
1 p.m. — 2 p.m. ET
Thursday, July 21, 2016
Register at http://bit.ly/1XPGtPa
For additional information after this webinar:
Kate Shatzkin, kshatzkin@aecf.org
For supplemental materials and more information about
Evidence2Success tools: www.aecf.org
Next in Our Webinar Series

Preventing Problems Before They Start

  • 1.
  • 2.
  • 3.
    Today’s Presenters 3 Elizabeth Gaines, The Forum forYouth Investment (Host and moderator) Dirk Butler, The Annie E. Casey Foundation Kevin Haggerty, Social Development Research Group Jessie Watrous, The Annie E. Casey Foundation
  • 4.
    4 Communicate with ususing the Q&A window at the right of the presentation window. • Type questions for the panel at any time during the webinar. • Use the box to let us know if you are having technical difficulties. Communicating During the Webinar
  • 5.
  • 6.
    The Annie E.Casey Foundation develops solutions to build a brighter future for children, families and communities. 6
  • 7.
    The Evidence2Success Framework Governance structure includingpublic systems and community Strategic use of data Comprehensive financing strategies Implementation of tested, effective programs Performance measures 7
  • 8.
    8 The Evidence2Success Process Youth Experience Survey Fund- MappingTool and Strategies Blueprints Database of Evidence- Based programs Implemen- tation Dashboard
  • 9.
    9 Preventing Problems BeforeThey Start Concepts of Prevention Science and Finding Proven Programs Identifying Community Priorities Gathering data from the Youth Experience Survey and working together to choose priority outcomes Financing Tested, Effective Programs Using tools to map funding sources and establish work plans for supporting programs that work The Using What Works Webinar Series
  • 10.
  • 11.
    11 Why Prevention Science? Heart disease Highblood pressure Obesity Diet high in calories, sodium, saturated fat
  • 12.
    Use a publichealth approach to positively affect outcomes by: • reducing community levels of elevated risk factors; • increasing the levels of community suppressed protective factors; and • implementing tested, effective programs unique to the community need 12 Three Elements of a Prevention Science Approach
  • 13.
    Evidence2Success: Powered by CommunitiesThat Care Large trial of Communities That  Care produced reductions in drug  use and delinquency • 33% tobacco • 32% alcohol • 25% delinquent behavior 13 A system of tools and resources to • decrease community risk factors • increase community levels of protection • implement evidence-based programs • change systems to IMPACT outcomes
  • 14.
    Common Risk FactorsCan Predict Multiple Outcomes  14
  • 15.
    What is Requiredfor Young People to Be Successful? Community School Individual and Peer Family Child Well- being Positive Relationships Behavior EducationPhysical Health Emotional Well-being 15
  • 16.
  • 17.
    Interaction of Riskand Protection: Foundation of Resilience Risk Factors Protective Factors 17
  • 18.
    What Are They? 1.R__________ 2. P___________ 3. T___________________ Three Critical Elements 18
  • 19.
    What Are They? 1.Risk Factors 2. Protective Factors 3. Tested Effective Programs Three Critical Elements 19
  • 20.
  • 21.
    What Is aTested, Effective Program? 21 Impact Evaluation Quality System Readiness Intervention Specificity  Positive impact on child well-being outcomes  Absence of any negative effects  One randomized controlled trial OR two quasi- experimental trials  Population clearly defined  Risk and protective factors identified  Training materials  Workforce capacity  Cost-benefit analysis
  • 22.
    • Stronger andmore consistent positive outcomes • Strong ethical argument – avoid potential harmful effects • Potential cost savings to taxpayers and society • Improving the well-being of our children at a population level Why Tested, Effective Programs?
  • 23.
    A Six-Step Process 23 Assessrisk and protection Prioritize risk and protective factors and outcomes to address Assess resources and identify gaps to address priorities Search Blueprints for programs that address priorities and gaps Assess program fit on multiple dimensions Select tested, effective program
  • 24.
    Considerations for ProgramFit 24 Type of prevention Potential return on investment Local capacity required and support provided Cost and alignment with funding Setting Population characteristics
  • 25.
    Three Types ofPrevention Programs 25 Universal Selective Indicated • All children/youth • Example = Positive Action • Elevated risk/exposure • Example= Nurse Family Partnership • Signs of early problem behavior • Example= Multisystemic Therapy
  • 26.
    Locating the RightProgram Through Online Databases 26 • Pew MacArthur Results First Clearinghouse Database (database of databases): http://www.pewtrusts.org/en/multimedia/data- visualizations/2015/results-first-clearinghouse-database • Child Trends What Works/LINKS database: http://childtrends.org/what-works/ • SAMHSA National Registry of Evidence-Based Programs and Practices: http://nrepp.samhsa.gov/01_landing.aspx • Blueprints for Healthy Youth Development: www.blueprintsprograms.com
  • 27.
    The Blueprints database: Yourtool for finding tested, effective programs 27 www.blueprintsprograms.com
  • 28.
    What Is aTested, Effective Program? 28 Impact Evaluation Quality System Readiness Intervention Specificity  Positive impact on child well-being outcomes  Absence of any negative effects  One randomized controlled trial OR two quasi- experimental trials  Population clearly defined  Risk and protective factors identified  Training materials  Workforce capacity  Cost-benefit analysis
  • 29.
  • 30.
    Three Ways toSearch 30
  • 31.
  • 32.
    32 Please use theQ&A function on your screen to submit questions. Questions?
  • 33.
    The Road toa “Proven” Distinction 33 Develop a strong program design Attain strong evidence of positive program outcomes • Carry out evaluation with a comparison group • Conduct regression analysis (quasi- experimental design ) • Perform multiple pre- and post - evaluations • Meta-analysis • Carry out evaluation with a comparison group • Conduct regression analysis (quasi- experimental design ) • Perform multiple pre- and post - evaluations • Meta-analysis Produce indicators of positive outcomes • Conduct evaluation with random assignment (experimental design) • Carry out multiple evaluations with strong comparison group (quasi- experimental design) • Conduct evaluation with random assignment (experimental design) • Carry out multiple evaluations with strong comparison group (quasi- experimental design) • Conduct pre- and post- interventi on evaluation • Conduct pre- and post- interventi on evaluation • Evaluate program quality and process • Establish continuous improvement system • Evaluate program quality and process • Establish continuous improvement system Ensure fidelity of Implement- ation Obtain evidence of positive program outcomes • Create logic model and replication materials • Create logic model and replication materials
  • 34.
    34 Identifying Community Prioritiesfor Child Well-Being 1 p.m. — 2 p.m. ET Thursday, July 21, 2016 Register at http://bit.ly/1XPGtPa For additional information after this webinar: Kate Shatzkin, kshatzkin@aecf.org For supplemental materials and more information about Evidence2Success tools: www.aecf.org Next in Our Webinar Series