Wraparound Institute Presentation

1. Child and Adolescent Characteristics,
Outcomes, and Wraparound Fidelity:
Results from Eight California Programs
Abram Rosenblatt,
Michelle Coufal, Kate Cordell, Elisha Heruty,
Catherine Aspiras, Mary Ann Wong
EMQ FamiliesFirst

2. Presentation Overview
• Youth Profile and Core Outcomes
• Child and Adolescent Needs and Strengths
(CANS) Outcomes
• Wraparound Fidelity Index (WFI-4) Profile and
Outcomes
• Longitudinal Outcomes
2

3. Number of Youth Served
3%
21%
42% Total Number of Youth Served
in Wraparound: 4,432
11%
24%
Bay Area Capital
Central Inland Empire
Los Angeles
3

4. Number of Youth Served in CY11
6%
28%
24%
Total Number of Youth Served in
Wraparound in CY11: 1,158
19%
24%
Bay Area Capital
Central Inland Empire
Los Angeles
4

5. Youth’s Profile
• Average Age at Admission: 14 years
• 61% Male; 39% Female
• Youth’s Ethnicity
Native American Other African American
1% 2% 21%
Latin American
30%
Asian American/
Pacific Islander
3%
Caucasian
40%
5

6. Youth’s Profile in CY11
• Average Age at Admission: 14 years
• 60% Male; 40% Female
• Youth’s Ethnicity
Native
American African
Other
1% American
1%
Latin American 17% Asian
46% American/
Pacific
Islander
2%
Caucasian
34%
6

7. Primary Diagnosis at Admission
100%
80%
60%
48%
40% 35%
20%
6% 9%
2%
0%
Adjustment Mood Disruptive Psychotic Other
Disorder Behavior
7

8. Primary Diagnosis at Admission for Youths Served in CY11
100%
80%
60%
44%
39%
40%
20%
9% 7%
1%
0%
Adjustment Mood Disruptive Psychotic Other
Disorder Behavior
8

9. Living Situation at Admission
100%
80%
60% 55%
40%
24%
20% 16%
0.2% 2% 1% 2%
0%
Ho Fo Ho Ju Re Sh Ot
m st sp st si el her
e e rF it a ic de te
e nt r/ H
am l ia om
i ly l
el
es
s
9

10. Living Situation at Admission for Youths Served in CY11
100%
80%
60%
60%
40%
23%
20% 15%
0% 2% 0.3% 0.3%
0%
Ho Fo Ho Ju Re Sh Ot
m st sp st si el her
e e rF it a ice de te
l nt r/ H
am ia om
i ly l el
es
s
10

11. Outcomes for Discharged Youth
11

12. Profile of Discharged Youth
• Number of Youth Discharged from Wraparound:
3,893
• Average Length of Stay: 12 months
12

13. Profile of Youth Discharged in CY11
• Number of Youth Discharged from Wraparound:
607
• Average Length of Stay: 11 months
13

14. Living Situation: Admit vs. Discharge
100%
80% 73%
67%
60%
40% 29%
23%
20%
3% 3% 0.3% 1%
0%
Community Facility Other Unknown
Admit Discharge
14

15. Living Situation: Admit vs. Discharge for Youth
Discharged in CY11
100%
82%
80%
80%
60%
40%
17% 17%
20%
1% 1% 0.4% 2%
0%
Community Facility Other Unknown
Admit Discharge
15

16. Core Outcomes for Discharged Youth
100% 82% 82%
80%
80%
60%
40%
20%
0%
In Home In School Out of Trouble
16

17. Core Outcomes for Youth Discharged in CY11
100% 81% 83% 84%
80%
60%
40%
20%
0%
In Home In School Out of Trouble
17

18. Longitudinal Core Outcomes
100% 85%
77% 76%
80%
60%
40%
20%
0%
In Home In School Out of Trouble
18

19. Why the CANS?
• Item Level Tool
• Items translate immediately into action levels
• It is about the youth, not about the service
• Cultural and developmental contexts are considered
before establishing action levels
• It is about the ‘what’, not about the ‘why’
• 30 day window for rating unless otherwise specified
19

20. CANS Overall Reliable Change Index (RCI)
20

21. CANS Overall RCI by Program
21

22. CANS Life Domain Functioning Domain RCI by
Program
22

23. CANS Child Strengths Domain RCI by Program
23

24. CANS Caregiver Strengths and Needs Domain
RCI by Program
24

25. CANS Child Behavior and Emotional Needs
Domain RCI by Program
25

26. CANS Child Risk Behaviors Domain RCI by
Program
26

27. What is the WFI-4?
• Measures implementation of wraparound process
• Set of interviews – Facilitator, Caregiver, and Youth
– 40 items for Facilitator and Caregiver
– 32 items for Youth
• Confidential interviews w/multiple respondents  unique
perspectives
27

28. Summary of Respondents
Administration Time Frame: January 2008 – January 2012
1,498 interviews from 671 families
8 traditional wraparound Interview Detail:
programs in 7 counties: • 671 facilitators interviewed
•Santa Clara – 40.3 minutes average time
•Sacramento
•Nevada • 522 caregivers interviewed
•Yolo – 39.1 minutes average time
•Fresno • 305 youth interviewed
•San Bernardino – 34.3 minutes average time
•Los Angeles
28

29. Demographics of WFI Youth
• Average Age at Admission: 13.5 years
• 57% Male; 43% Female
• Youth’s Ethnicity
Native American Other
African American
Latin American 1% 1%
17%
41%
Asian American/
Pacific Islander
3%
Caucasian
38%
29

30. Primary Diagnosis at Admission for WFI Youth
100%
80%
60%
45%
39%
40%
20%
10%
6%
1%
0%
Adjustment Mood Disruptive Psychotic Other
Disorder Behavior
30

31. Living Situation at Admission for WFI Youth
100%
80%
60% 54%
40%
24%
20% 19%
0% 1% 0.3% 0.3%
0%
Ho Fo Ho Ju Re Sh Ot
m st sp st si el her
e e rF it a ic de te
e nt r/ H
am l ia om
i ly l
el
es
s
31

32. Outcomes for Discharged WFI
Youths
32

33. Living Situation: Admit vs. Discharge for WFI Youth
100%
81%
80% 70%
60%
40% 29%
17%
20%
1% 1% 0.3% 1%
0%
Community Facility Other Unknown
Admit Discharge
33

34. EMQ FF vs National Data: Overall Fidelity
100% 87%
83% 83% 80%
77% 75% 78%
80% 73%
60%
40%
20%
0%
Combined Facilitator Caregiver Youth
EMQ FF National Mean
34

35. EMQ FF vs National Data : Fidelity Scores by Phase
100% 88%
82% 86%
81%
80% 76% 76% 76%
69%
60%
40%
20%
0%
Engagement Plan Development Implementation Transition
EMQ FF National Mean
35

36. EMQFF Phase Scores by Program
100%
90% 90% 92%
89% 89%
86% 87% 87%
85% 85% 87% 87% 87%
87%
84% 83% 84%
81% 81% 81% 83%
80% 78% 80% 78% 78% 80% 78%
76% 76% 75% 75%
70%
60%
40%
20%
0%
Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 Program 8
Engagement Planning
Implementation Transition
36

37. EMQFF Principle Scores by Program
100% 94% 94% 95% 94% 96%
93%
91% 92% 92% 90% 92% 90% 91% 92% 91%
89% 87% 89%
82% 80% 82% 82% 82%
79% 79%
80% 73%
78% 76% 76%
70%
70% 72% 72%
72% 72%
67% 67% 69%
65% 64%
60%
40%
20%
0%
Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 Program 8
Family Voice and Choice Team-Based
Natural Supports Collaboration
Community-Based
37

38. EMQFF Principle Scores by Program continued
100% 97% 97% 95% 96% 98%
95% 96%
91% 90% 91% 90% 92%
89% 89%
85% 86%
84% 86% 85% 86%
82% 83% 83%
78% 79%
81% 80% 82% 81%
80% 78% 78%
74% 76%
78% 76% 76% 76%
72% 70%
66%
60%
40%
20%
0%
Program 1 Program 2 Program 3 Program 4 Program 5 Program 6 Program 7 Program 8
Cultural Competence Individualized
Strength-Based Persistence
Outcome- Based
38

39. Average Total WFI Scores By Program
EMQ Wraparound Program Average Total WFI Score
Program 1 84%
Program 2 85%
Program 3 83%
Program 4 82%
Program 5 85%
Program 6 85%
Program 7 83%
Program 8 78%
39

40. Core Outcomes for Discharged WFI Youth
90%
100% 84%
80%
80%
60%
40%
20%
0%
In Home In School Out of Trouble
40

41. Longitudinal Core Outcomes for WFI Youth
100% 85% 88% 83%
80%
60%
40%
20%
0%
In Home In School Out of Trouble
41

42. Data Implications
• Further studies employing the WFI and CANS
on Outcomes
42

43. WFI and CANS Outcomes
• Is our fidelity associated with our outcomes?
• Which fidelity elements are more strongly
associated with which outcomes?
43

44. WFI & CANS Outcomes
• 324 clients with two CANS Outcomes at least 6 months apart
and a WFI at 6 months of service
44

45. WFI & Life Domain Functioning CANS
• Improved or Declined CANS groups determined by Reliable Change Index (RCI)
45

46. WFI & Child Strengths CANS
• Improved or Declined CANS groups determined by Reliable Change Index (RCI)
46

47. WFI & Caregiver Strengths/Needs CANS
• Improved or Declined CANS groups determined by Reliable Change Index (RCI)
47

48. WFI & Child Behav. & Emo. Needs CANS
• Improved or Declined CANS groups determined by Reliable Change Index (RCI)
48

49. WFI & Child Risk Behaviors CANS
• Improved or Declined CANS groups determined by Reliable Change Index (RCI)
49

50. WFI and CANS Outcomes
• Which source of fidelity scores are more
strongly associated with improved
outcomes?
50

51. WFI-Facilitator Source & CANS
• Facilitator fidelity scores are strongly associated with CANS outcomes scores.
51

52. WFI-Caregiver Source & CANS
• Caregiver fidelity scores are lower than facilitator fidelity scores.
52

53. WFI-Youth Source & CANS
• Youth fidelity scores are lower than facilitator scores and exhibit reversal of
pattern in some elements for association between fidelity and outcomes.
53

54. WFI and CANS Outcomes
• How is fidelity at different phases of the
program associated with CANS outcomes?
54

55. WFI by Phase & CANS
The group of clients who only
declined in CANS domains had
significantly lower fidelity scores at
the:
• Planning (p=0.04),
• Implementation (p=0.005)
• Transition (p=0.001)
phases of the Wraparound program
as compared to the group of clients
who improved in at least one CANS
domain.
55

56. WFI and CANS Outcomes
• Focused Quality Improvement:
– In what fidelity elements should we focus our energies in order
to maximize improvement in Child Behavioral and Emotional
Needs outcomes?
• Measurable Quality Improvement
– How much improvement (i.e., what increased percentage of kids
would have reliable improvement) would we expect to see if we
brought all of these elements in our programs up to ‘high fidelity’
56

57. Child Behavior and
Emotional Needs
317 Improving Fidelity
and CANS Outcomes
Improved 45.1 %
No Change 40.1 %
Declined 14.8 %
High fidelity (>=85%) for Element 5 (Community Based) = YES
Child Behavior and • There were 317 clients with a
Emotional Needs
176 CANS Child Behavioral and
Improved
No Change
47.2 %
38.1 %
Emotional Needs score which
Declined 14.8 %
could be reliably improved from
High fidelity (>=85%) for Element 6 (Culturally Competent) = YES
Time 1 (GTE RCI of 2.2).
Child Behavior and
Emotional Needs
168
• Overall, 45.1% of clients improved
48.8 %
between Time 1 and Time 2.
Improved
No Change 36.9 %
Declined 14.3 %
High fidelity (>=85%) for Element 9 (Persistent) = YES
• Programs with higher fidelity in
Child Behavior and
Emotional Needs
109
certain elements resulted in better
Improved 52.3 %
outcomes.
No Change 36.7 %
Declined 11.0 %
Order of elements determined by classification and
High fidelity (>=85%) for Element 10 (Outcomes Based) = YES regression tree (CART).
Child Behavior and
Emotional Needs
46
Improved 56.5 %
No Change 37.0 %
Declined 6.5 %
57

58. WFI and CANS Outcomes Summary
• Our fidelity scores are associated with our CANS
outcomes
• Facilitator fidelity scores are more strongly associated
with CANS outcomes
• The data suggests that focused quality improvement in
fidelity will result in a measurable improvement in CANS
outcomes
58

59. Challenges
 New staff
 New Measure Implementations
 Difficult populations
 Language barriers
 Ratio of certified interviewers to interviewees
 Relatively low response rates
59

60. Lessons Learned
 Program buy-in
 Work with Wrap teams
 Use EMQ FF language certified employees
60

61. Future Directions
61

62. Future Directions
• Build on Initial Analyses
• Translate to Clinical Staff
• Consider Fidelity Intervention Options
• Ongoing WRAP Fidelity Feedback (Bruns)
• Build Ongoing Reporting Mechanisms
• Link to Quality of Care and Key Performance Indicators
62

63. Q&A
63

64. References
• Bruns, E. (nd). Wraparound Fidelity Assessment System. Retrieved
from Wrap Info website:
http://depts.washington.edu/wrapeval/WFI.html
• Lyons, J. Child and Adolescent Needs and Strengths. Retrieved from
Praed Foundation website:
http://www.praedfoundation.org/About%20the%20CANS.html
• Wraparound Evaluation & Research Team. The Wraparound Process.
Retrieved from http://depts.washington.edu/wrapeval/approach.html
64

65. Contact Information
Abram Rosenblatt
 arosenblatt@emqff.org (408) 364-4016
65