A webinar from the Annie E. Casey and William T. Grant foundations explores how partnerships between researchers and child welfare professionals can be a valuable resource for agencies serving kids and families.

1. Tools and Resources for Research-Practice
Partnerships
leading with
evidence
1

2. Welcome and Overview
Kimberly Dumont
2

3. The Leading With Evidence Webinar Series
3

4. Today’s Presenters
4
Kimberly
DuMont,
The William T.
Grant
Foundation
(Host and
moderator)
Scott
Burdick,
Orange
County Social
Services
Agency
Andrew
White,
New York City
Administration
of Children's
Services
Kristin
Bernard,
Stony Brook
University

5. 5
Communicate with us using the Q&A window at the
right of the presentation window.
• Type questions for the panel at any time during the
webinar.
• Use the box to let us know if you are having technical
difficulties.
Communicating During the Webinar

6. Partners in Data:
Lessons Learned
Scott Burdick
6

7. Orange County
7

8. Orange County
Child Welfare System
8
2,250
out of home
3,600
children in care
1,000
In home
350
non-minor
dependents

9. Social Worker Purpose:
Making a Difference
• Definition of insanity
• Going further tomorrow than we are today
requires change.
• Change for change sake is NEVER good.
• Change needs to be strategic.
• Strategic = evaluation
9

10. Lesson # 1
What Gets Watched, Gets Done
10

11. A LOT Going ON!
11

12. Regular Outcome Tracking
12

13. Lesson # 2
Collaboration is Key
Agency
Community
Outcomes
Research
13

14. Self-Evaluation Team (SET)
• Began in 2003 as part of Family to Family Initiative
• Co-chairs
– Children and Family Services deputy director
– Research manager
– Reports team manager
• Members
– Data Analysis and Reporting Team-(DART)
– Research team
– Program staff
– Policy staff
– Community partners
– Strategic communication
14

15. Orange County Internal Structure
• Self-Evaluation Team (SET)
• SET Technical subcommittee
• Data Analysis and Reporting Team (DART): 7 members
• Research team: 4 members
• Families and Communities Together(FaCT) data team: 2 members
• Child Welfare System Improvement Partnership (CWSIP)
• Deputy director/director meetings
• Research/data teams are immersed in our process
15

16. Social Services Agency Organizational Structure
SSA Director
Chief Deputy
Director
FSS/Adult
Services
Assistance
Programs
Administration
Research
Staff
Reports Team
Children and
Family
Services
Children and
Family
Services Staff
16

17. Funding Internal Data Staff
• Social Services Agency
– 4 divisions
– Administrative costs
– Over 4,350 staff
– Blending of multiple funding streams from different divisions: Time study
Department Federal State County
Research Team 31.85% 62.39% 5.76%
Reports Team 75% 17.5% 7.5%
17

18. 18
• “If you are looking at everything, you are looking at
nothing!”
• Evaluate whether reports are giving you valuable
information and are still relevant
• Teach a man to fish
– Avoid duplicative reports: SafeMeasures
o Make small adjustments
– Public resources
Lesson #3: Allow the Data and Research Team to
Do What They Need to

19. Lesson # 4:
Data Allow Leveraging of Partnerships
Neighbor to Neighbor
• 2/3 of all hotline calls were made on families where there had been a
previous call
• Differential Response Path 1-Calls do not rise to the statutory threshold for a
response
• Unfunded
• Thousands of families served preventatively
• Win-win
19

20. 20
 Pilot community agencies partnering with three Family
Resource Centers and the faith community to engage these
families in preventative services
 10 percent engagement rate
 Additional evaluation occurring to look at impact including:
– comparison to those that did not receive services
– whether families have any prior child welfare history
– whether families have subsequent calls, investigation,
substantiation?
Neighbor to Neighbor/Family Resource
Centers/Faith In Motion Pilot

21. Emergency Response Warm Handoff Pilot
• High/very high risk
• Substantiated allegations
• Safety threats mitigated and referral to be closed
• Warm handoff to Family Resource Centers
21

22. 2013 California Risk Validation Results
16%
5%
1%
30%
11%
4%
46%
19%
10%
58%
26%
16%
Subsequent Investigation Subsequent Substantiation Subsequent Investigation
with Removal
Low Moderate High Very High
Total = 11,444 substantiated/inconclusive investigations from July 1, 2010 – June 30,
2011; 18-month follow-up period 22

23. Orange County
Risk Level and Service Impact
Subsequent Substantiation Within 16 Months From Original Referral
0%
5%
10%
15%
20%
25%
30%
35%
Low and
Moderate
Risk
High Risk Very High
Risk
8%
18%
20%
8%
24%
33%
Opened
Closed
2003 OCSSA Research
23

24. Substantiated Closed Referrals
By Risk Level
2017
1%
12%
54%
33%
Very High-23
High-232
Moderate-1010
Low-623
24

25. Family Resource Center Outcomes
Differential Response Path 2 Families
25

26. Summary
Lessons Learned
• Teach a man to fish.
• What gets watched gets done. Prioritize!
• Research should be the first call, not an afterthought.
• Constant evaluation: Are the data being regularly produced still valuable or
are we doing it “because we always have?”
• It is important that those doing the research immerse themselves in the
process.
• Research is an ongoing discussion.
26

27. Summary
Challenges
• High number of requests to research
• Too many regular reports
• Research/data staff need time and experience to learn our complicated
system.
• Confidentiality
– Multidisciplinary Team Section Codes
– Data sharing between agencies - e.g., substance abuse
• Partners that have their agenda in mind as opposed to overall outcomes
• SACWIS system does not capture all necessary data fields.
• Delays in creating internal data bases
27

28. The Road to
Adaptation
Andrew White and Kristin
Bernard
28

29. 29
• Andrew White, deputy commissioner for policy, planning &
measurement at NYC Administration for Children’s Services
• Kristin Bernard, Ph.D., assistant professor of psychology,
Stony Brook University
• Anne Heller and Erasma Monticiollo, leadership team at
Power of Two, nonprofit community organization
Key Players in Our Research-Practice Partnership

30. 30
Goals and Perspectives: NYC ACS
• Invest in evidence-based practice, such as Attachment and
Biobehavioral Catch-up (ABC)
• Scale services to reach many children and families
• Support research that:
– builds the evidence base of ACS-supported programs
– demonstrates our impact on children and families
– informs future decision making

31. Child Welfare in New York City
• 60,000 investigations per year, including
more than 80,000 children
• 14.5% of all allegations are of abuse*
– 12% physical abuse
– 2% sexual abuse
• *Remainder are allegations of neglect
• 36%-42% investigations are indicated
– Indication rate has remained in that range over
past decade
• 3,647 children entered foster care last
year
31

32. 13,856
44,445
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
50,000
1996 2017
+221%
Children in Families
Receiving Prevention Services in
NYC
32
Prevention Services and Foster Care
in New York City
41,669
8,747
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
1996 2017
-79%
Children in Foster Care
in NYC

33. 33
Goals and Perspectives: Kristin Bernard
• Goals and perspectives of Dr. Kristin Bernard:
– Apply my research about ABC to the real world
– Conduct new research that:
o Examines novel outcomes of parenting programs for
vulnerable children
o Identifies what works best for whom, with the goal of
tailoring services
– Publish research and obtain grant funding

34. 34
Goals and Perspectives: Power of Two
• Goals and perspectives of Power of Two:
– Ensure that every child in New York City grows up with the
best possible start in life
– Disrupt the effects of intergenerational threats, like poverty
and trauma
– Deliver and scale ABC with fidelity
– Obtain ongoing funding to support large team of parent
coaches and reach many families

35. 35
How We Came Together
• In 2014, ACS was applying for Federal Title IV-E Foster Care
Waiver program funding to implement innovative child welfare
practices that would
– Increase permanency
– Prevent child abuse and neglect
– Promote positive outcomes
• Dr. Bernard (new faculty at Stony Brook) was seeking
partnerships to access vulnerable families for research
opportunities.
• Power of Two (new organization) was seeking funding and
partnerships to bring ABC to NYC.

36. 36
Where We Started: Early Planning
• Early discussions centered around ensuring successful
implementation, such as how we:
– maintain model fidelity
– train frontline staff to communicate with families
– maximize family engagement
• And designing a rigorous yet feasible evaluation
– Independent evaluation team (Chapin Hall)
– Reliance on Dr. Bernard’s expertise for selection of
measures to reliably and validly assess program
effectiveness

37. 37
Where We Are Now: Successes and Challenges
• Successes
– Citywide implementation of ABC for infants/toddlers
in foster care or returning home from foster care
(approximately 1,000 children served over three
years)
– Strong evidence of effectiveness: enhanced
parental sensitivity and reduced child
socioemotional risk
– High fidelity of model as scaled
• Challenges and areas for growth
– Consistently meeting referral targets, reaching all
children
– Sustainability and funding

38. 38
Day to Day
• A glance at the day to day operations
– Frequent communication:
o Monthly calls between ACS, research partner
(KB) and Power of Two
o Biweekly calls between ACS and Power of Two
leadership, and between Power of Two and
agencies
– Collaborative decision making and bidirectional
reporting:
o Reports regarding model fidelity (from KB to
ACS)
o Goal setting for target numbers (ACS and Power
of Two)
o Data sharing with regard to model outcomes
S M T W T F S
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31

39. 39
Leveraging Partnership to Meet Individual Goals
• Benefits of research partnership for ACS:
– Ongoing monitoring of program fidelity and
effectiveness
– Development of training
– Evidence-based decision-making (e.g., issues of
double-dosing for foster parents with multiple
children)
• Benefits of practice partnership for Dr. Bernard:
– Access to archival program evaluation data for
research questions related to effectiveness and
dissemination
– Infrastructure for grant proposals to seek funding

40. 40
Obtaining Funding for Research
• Infrastructure of RPP is attractive to external
research funding sources
• Example feedback to KB from recent NIH grant
review:
– “The well-established connection across the
investigative team, Power of Two, and NYC
Administration for Children’s Services is a critical
strength of the project.”
– “This effectiveness trial involves a strong
collaboration with community providers that will
help to ensure that the results have utility.”
– ”The environment is excellent, in terms of
integration with child welfare and community
service agencies, resources for data collection in
the field…”

41. 41
Challenges of Partnership
• Challenges of RPP:
– ACS’s need to prioritize enhancing child and family
well-being sometimes interferes with using most
rigorous research designs (e.g., randomized
control trials)
– Limited funding for research restricts ability to ask
novel research questions
– Empirical questions of immediate interest to research
partner (e.g., “Does ABC change parental brain
activity?”) may not align with practice-relevant
questions of immediate interest to practice partner
(e.g., “What families will benefit from ABC vs. a more
intensive intervention?)

42. 42
Lessons Learned
• Research-Practice Partnerships: Lessons Learned
– Identify shared research interests: What projects can
both inform decision-making in child welfare and
advance science in the research community?
– Give and take: What can the practice partner offer the
research partner and vice versa, while minimizing
burden on either side?
– Be patient and flexible: Developing a collaborative and
mutually-beneficial relationship takes time, and outputs
may not come quickly early on.

43. 43
William T. Grant Foundation Microsite on Research-
Practice Partnerships
http://rpp.wtgrantfoundation.org/

44. Please use the Q&A function on your screen to
submit questions.
A recording of this webinar will be sent to those who
registered and posted at www.aecf.org/webinar.
Questions?
44

45. Resources
• Bringing Evidence-Based Program Adaptations Into Child Welfare Systems
– https://www.aecf.org/blog/watch-our-webinar-adapting-evidence-based-programs-in-child-
welfare/
• Funding Evidence-Based Programs in Child Welfare: Implications of the Family First Act
– https://www.aecf.org/blog/funding-evidence-based-programs-in-child-welfare-with-the-family-
first/
• Assessing Readiness for Implementation: Using the Hexagon Tool
– https://www.aecf.org/blog/webinar-assessing-readiness-for-implementation-with-the-hexagon-
tool/
45
LEADING WITH EVIDENCE WEBINAR RECORDINGS
THE ORANGE COUNTY SELF-EVALUATION TEAM
WILLIAM T. GRANT FOUNDATION MICROSITE
http://rpp.wtgrantfoundation.org
http://www.oc4kids.com/ff/strategy/meetings/selfeval

46. 46