1. 1
MIDDLE SCHOOL SUPPORT PROJECT EVALUATION
Nesholm Family Foundation
College Spark Foundation
In Collaboration with
Seattle Public Schools, Sound Mental Health,
Seattle Children’s Hospital and the University of Washington
Summary Report
June 2012
Launched in the summer of 2006, the Middle School Support Project (MSSP) has served students in the
four Seattle public middle schools that also sponsor the Nesholm “Kids in the Middle” literacy
instructional improvement program. The goal of MSSP is to provide onsite child mental health experts,
called Care Coordinators (CC), at selected Seattle middle schools to assure that the mental health needs
of students who are struggling academically are being addressed in concert with their academic needs.
Phase One of the MSSP (the pilot phase) began in 2006 ended in the spring of 2009. By the end of
Phase One, three service levels had been clearly defined based on an assessment by a CC. Three
service levels are defined based on an assessment by a CC. The three MSSP service levels mirror the
tiered framework that has emerged from education sciences where Tier 1 services are those that are
offered to all youth within the school system, and Tier 3 services are designed to address the needs of
the smaller segment of the student population with more significant academic and behavioral challenges:
The goal of Phase Two of the project during the subsequent three school years (2009-2012) was to build
on the successful elements of Phase One, expand the model to include support of MSSP students as
they transitioned to high school, and conduct an expanded evaluation under the direction of Seattle
Children’s Hospital/University of Washington (SCH/UW) Division of Child and Adolescent Psychiatry,
This evaluation was funded by the Nesholm Family Foundation, the College Spark Foundation, and the
Bank of America. The following summary describes the results from the initial MSSP pilot program
evaluation completed from 2006-2009 and the results from the expanded evaluation conducted from
2009-2012.
Level/Tier Two:
Assessment and Referral. These students were determined to have mental
health needs, but not complex needs or multi-system involvement and were
referred to other mental health providers for providers for on-going treatment
with CC monitoring care.
Level/Tier One:
Consultation and Crisis Intervention. These students were seen at school
but did not require formal assessment or services outside the school setting;
they were provided crisis intervention or other supportive in-school services,
such as consultation with school staff.
Level/Tier Three:
MSSP Intensive. These students were determined to have
complex needs and multiple system involvement.

2. 2
Phase One: Program Evaluation of the MSSP 2006-2009
During the three years of Phase One of the Middle School Support Project, an annual evaluation was
designed and conducted under the leadership of the SCH/UW. Goals of the pilot evaluation included 1)
characterizing the students referred for services under the MSSP; 2) describing the services provided
through the MSSP; and 3) assessing effectiveness of MSSP, at both the system and student/family
levels. The following is a brief summary of the results of the progress on the core goals:
Goal 1: Characterizing students needing services (target population) and the referral process
 In years 1 through 3 significant efforts went into defining and prioritizing participating students into
the three general categories (Levels 1-3). Although fluctuating each year, the program served
roughly equal numbers of female and male students (49.7 female; 50.3 male). Approximately
80% were low income (measured by FRL). Table 1 depicts the racial composition of the group of
students enrolled in MSSP intensive.
 A written referral process was developed and refined to assist in the education of school staff
regarding the target population, ensure more appropriate referrals and reduce the number of
referrals “made on the fly” in the hallways.
Table 1. MSSP Intensive Students
Racial Group %
Black or African American 41.6%
Hispanic or Latino 18.1%
Asian 13.9%
Two or More Races 8.4%
White/Non-Hispanic 6.0%
American Indian or Alaskan Native 4.2%
Native Hawaiian or Other Pacific
Islander 3.0%
Missing 4.8%
Goal 2: Defining and describing the services provided
By the 2010-11 school year, 421 students were enrolled across the three program Levels, representing
an increase from 358 in 2009-10, with the largest increase coming in the number of students enrolled in
Level 3: Intensive MSSP.
Table 2. Students served in MSSP September 2010-May 2011
Aki Denny Mercer-
2 CCs
TOTAL
Total students served by
Care Coordinator
99 129 193 421
Level 3: MSSP Intensive 32 26 55 113
Level 2: MSSP Assess and
Refer to other services
22 20 54 96
Level 1: Crisis and
Consultation services
provided at school
45 83 84 212

3. 3
In addition, a major goal in the development of the program was defining the role of the CC in the school
setting and defining the scope of the services they were to provide. Services provided included the
following: Assessment; Case Management; Development of WrapAround Teams and Implementation of
Team Meetings; Referral and Linkage; Individual Support; Family Support; Group Treatment; Crisis
Intervention and Consultation. While the CC had set expectations of time devoted to Intensive caseload
as the program developed, principals were able to decide how to prioritize services for a portion of the
CC’ schedule (i.e. crisis work, group work, expanding caseload). Eventually, to augment program
funding, CC were also required to assess, tier and provide insurance-reimbursable therapy to a sub-set
of the students on their caseload. Table 3 summarizes the activities of CC in addition to their
MSSP intensive responsibilities during the 2010-2011 school year.
Table 3. Students receiving other services from the CC (2010-2011 school year)
Aki Denny Mercer-
2 CCs
TOTAL
MSSP Students Enrolled in Group 4 7 18 29
Number of Group hours provided 24 56 173 253
Total MH Referrals made for
students in levels 1 + 2
46 48 99 193
Students tiered for mental health
services and provided therapy
10 10 11 31
In the 2009-2010 school year the program responded to a request from leadership to focus on assisting
MSSP Intensive students in transitioning from 8th
grade to high school. This was in part driven by
feedback the year before from parents and students who felt “abandoned” and unsupported as they
entered a new phase of their educational experience. Each Care Coordinator arranged to meet with 9th
grade students from the previous year’s caseload for one session in the fall. The goal of the intervention
continued to be to sustain gains made by students in the middle school program by directly building their
support network in the high school setting. Although feedback from students and parents was positive,
the intervention was scaled back during the 2010-2011 school year when it became clear that this effort
required too much time away from the middle school setting and was having a negative effect on ability
to implement the MSSP intervention as planned. In 2011 it was recommended that to ensure efficient
time budgeting, the 9th
grade extension activities would have to be listed among a principal’s priorities for
how CC devoted their time.
Goal 3: Assessing the project’s effectiveness
Qualitative Feedback from School Personnel, Students and Parents: During Phase One the evaluation
team conducted qualitative interviews with principals, teachers, Care Coordinators, and participating
students and parents to gather their input on MSSP program effectiveness and acceptability. The
themes identified via the qualitative data analysis indicated that the program was seen as:
 Increasing the schools' ability to deal with complex student needs.
 Increasing the schools' ability to coordinate services for students.
 Helping school teams realize the value of mental health collaboration and accessing multiple
systems to address student needs.
 Freeing up teachers to invest more time in teaching.
Parents and students reported that, through MSSP participation, students had gained competencies
across multiple areas such as:

4. 4
 Getting support: “how to form a support network outside the family;” “you can get support if you
ask;” “how to talk to teachers in a constructive way when there is a problem;” “resources are
available and people do care.”
 Setting and following though on goals: “how to keep school work organized;”
 Dealing with stress: “learned what to do when I get mad;”
 Communicating with family members and others: “a lot of stuff isn’t worth fighting over. I need to
stop and think before reacting.”
Parents consistently stated that their students needed to have support continue from middle school into
the high school years.
Student Functioning: Outcomes of MSSP participants were evaluated on the basis of repeated
assessments. Three questionnaires were administered: 1) the Columbia Impairment Scale (CIS) which
assessed impairment in students’ adaptive functioning across family, school, and extra-curricular
settings; 2) the Short Mood and Feelings Questionnaire (SMFQ), a 13-item scale that assesses child and
adolescent depressive symptoms and general emotional distress; and 3) the Needs Met Questionnaire,
which assessed areas of concern in multiple life domains. Data were analyzed yielding results
demonstrating that students served by the MSSP program experienced reduced depressive symptoms
and improved adaptive functioning. Questions remained as to how the outcomes of these students
would have compared to other similarly challenged students who were not participating in the MSSP
program.
Review of the WrapAround Process: A focus of MSSP Intensive is the application of the WrapAround
model, which means bringing multiple members of a child’s support network together to execute a
coordinated plan. Challenges to implementing a traditional mental health WrapAround model in the
school setting were identified early in the development of the MSSP. Although challenges remain,
significant progress had been made by Year 5 in implementing an adapted WrapAround Team Meeting
(WATM) in the school setting. The CC continue to approach each case with the goal of using the “spirit”
of the WrapAround model and then employ a common sense and creative approach to modifying the
traditional model in ways that are appropriate and necessary within the school setting. Creative
adaptations led over time to an increase in WATMs. Although the emerging MSSP model is similar to
the traditional mental health Wrap-around model, there are key elements which make the school-based
model unique:
1) Given that the ultimate goal of the MSSP program is to support academic performance, the CC are
based in the school and seen as members of the school teams, as opposed to outsider service
providers. They have been able to accomplish an integration of the mental health needs and the
academic needs of the students in a way that may be more difficult for the mental health professional
not based in the school setting.
2) Convening team meetings within the school system has been challenging, but the barriers have
led to creative adaptations. Even within the traditional mental health Wrap-around model where team
meetings can be scheduled in homes or community settings and at flexible times, it is an ongoing
challenge to ensure that team members regularly attend team meetings that are convened within the
regular school day. Adaptations have included:
 Team meetings incorporated into the existing structure of school staffings and other routine
school meetings where parents and others are already scheduled to attend on behalf of
students.
 Smaller teams or partial teams convened with the CC serving as “the hub of the wheel” to
communicate and link team members to ensure the coordination of care.

5. 5
Phase Two: Expanded Evaluation of the MSSP 2009-2012
Beginning in the 2009-2010 school year, the primary goal of the evaluation was to more systematically
assess the MSSP Program’s effectiveness in supporting positive outcomes for students and families.
Evaluation goals included:
1. Collect robust data on improvements in students’ behavioral health.
2. Collect detailed student data related to school engagement, academic success, and college-
readiness. These include data on general adaptive functioning, attendance, disciplinary problems
and actions, course completion, family involvement, and involvement in extra-curricular activities.
3. Compare the behavioral, functional and academic outcomes of students served by the MSSP
Intensive with outcomes of students with comparable needs who were not served by MSSP.
4. Measure long term outcomes. In particular, determine whether MSSP gains in behavioral health,
adaptive functioning, academic performance, level of school engagement, and connection to
supportive services were sustained over.
METHODS
The methods used for the MSSP Phase Two evaluation - including the evaluation study participants, the
array of outcome measures, and the long term follow up assessments- are described below.
Evaluation Participants: 118 students (54 intervention and 64 control) and their caregivers were enrolled
in the Phase Two evaluation. The intervention group was recruited from the three middle schools
providing MSSP services: Aki Kurose Academy; Denny Middle School; Mercer Middle School. The
control group was recruited from the three middle schools targeted as having the closest demographic
profiles to the intervention schools: Hamilton Middle School; Madison Middle School and Washington
Middle School. 74% of eligible MSSP intensive students and their caregivers enrolled in the study (54 of
73 eligible). Participants in the control group were referred by school counselors who used the MSSP
intake materials to guide them to identify students having the same high-risk indicators that would have
qualified them as eligible for Level 3 MSSP Intensive.
Recruitment Procedures: MSSP and comparison group students and parents were invited to learn about
the evaluation via a letter sent by their school principal. Caregivers who gave permission for the school to
share their contact information with evaluators were called by study staff and invited into the study.
Students/caregivers who agreed to participate were guided through an informed consent/assent process
that was approved by both the Seattle Public School District and by the Seattle Children’s Hospital
Institutional Review Board.
Evaluation Procedures: All participants completed an evaluation assessment in which a UW/SCH
evaluation field assistant administered a set of structured questionnaires in person to students and
parent/guardians at three time points over the course of three consecutive school semesters. Student
and parent/guardians were interviewed separately, and each received a $20 gift card as a thank you for
their participation.
Evaluation Measures: The questionnaires that were administered assessed demographic
characteristics, general adaptive functioning, behavioral health symptoms, school performance and

6. 6
attachment, service utilization, and Wrap Around fidelity (intervention group, only). We made minor
changes in measures at the start of the study to assess program impact.
Demographics. Student and caregiver demographic information was collected including age,
gender, ethnicity, education, and parent’s marital status. In addition to tracking student eligibility for free
and reduced price lunch programs, caregivers provided information about their education level (highest
grade/degree completed) and occupation to calculate the Four-Factor Index of Social Status
(Hollingshead, 1975).
General Adaptive Functioning. The 13-item Columbia Impairment Scale (CIS; Bird, 1993) was
administered to students and caregivers to measure overall level of adaptive functioning across domains
of school, family, and extra-curricular activities. This scale is well-validated and has been used
extensively in evaluations of child mental health programs.
Behavioral Health Symptoms/Distress.
 The Achenbach System of Empirically Based Assessment (ASEBA; Achenbach, 2001) was used to
evaluate student psychosocial functioning. The Youth Self Report (YSR) was administered to students
and the Child Behavior Checklist (CBCL) was administered to caregivers. Both forms yield a series of
internalizing and externalizing scales including anxiety, depression, aggression and rule-breaking. Both
have been used extensively in child psychiatric epidemiological studies and have well-substantiated
reliability and validity.
School Performance. We obtained official school records from the Seattle Public School District
for each semester of student participation. School records include courses failed, credits earned, days
absent, and disciplinary actions (incidents, suspensions, expulsions).
School Support/Attachment/Involvement/Goals:
These school outcomes were evaluated using subscales of the High School Questionnaire (HSQ;
Eggert, et al., 1995).
 School Support Scale (SSS). The SSS, is a 10-item measure of the extent to which youth experience
their school settings as encouraging, reliable, and fulfilling (e.g., “My teachers are people I can count
on to help me, and offer useful points of view”). Response options range “never” to “always” on a 7-
point Likert-style scale.
 School Attachment Scale (SAS). The SAS, also part of the HSQ, is a 6-item measure, which
assesses youth satisfaction and engagement in the schools and classrooms (e.g., “How involved
were you in school activities,” “My overall satisfaction with classes was…”). Response options for the
SAS are rated along a 7-point (0-6) Likert-style scale. Scale anchors vary by item.
 Engagement: We measured school attachment using a 3-item attachment scale that is a refinement
of the school connectedness scale used in the Add-Health Study, as well as three subscales
(attachment, Alpha=.63; commitment, Alpha = .71; and involvement, Alpha=.58) from the school
bonding scale. We also used the CBCL/YSR activities section to measure number and time involved
in activities including sports, organizations, and other activities.
Mental Health Service Utilization. The Services Assessment for Children and Adolescents
(SACA; Horwitz et al., 2001) was administered to parent and student. The SACA is a comprehensive
instrument about children’s use of mental health services. The parent and youth interview versions of the
SACA assesses mental health services that the youth received over three broad domains: inpatient,
outpatient, and school-based.

7. 7
WrapAround Fidelity. The Wraparound Fidelity Index (WFI; Bruns et al., 2004) is a 44-item
interview that was administered to parents of students in the MSSP intervention group, only. The WFI
asks whether the services and supports the student and parent receive adhere to WrapAround
principles. Parents were asked, for example, whether MSSP WrapAround team members included the
student, parent/guardians, and representatives of the student’s formal and informal support networks,
whether the parents’ and students’ voices were reflected in the team’s planning process, and whether
team members worked well together to coordinate supports for the student.
RESULTS
Demographic Comparability of Students in MSSP Intervention and Control Groups. Initially, we
compared the overall characteristics of intervention schools and control schools (Table 4). Once
students were enrolled in the evaluation, we also compared the characteristics of students in the
intervention group with students in the control group (Table 5). Despite our attempts to match MSSP
schools with comparable schools within the Seattle Public School District, intervention schools were
serving more students who qualified for free and reduced price lunch, had limited English proficiency,
lived with only one parent, and were from an ethnic/racial minority background. Within these schools, we
were generally able to recruit individual students from the control schools that were similar to individual
students in the MSSP intervention group on socio-economic indicators. The intervention group students
were more likely to be African American and less likely to be White than the control group students.
Overall, students across both groups were of low to moderate socioeconomic status and were at
considerable risk for school failure.
Table 4. Comparing Characteristics of MSSP Intervention and Control Schools
MSSP
Intervention
Schools (%)
Control
Schools (%)
Total Enrollment of
Seattle Public
Schools (%)
Free and reduced lunch 74.4 39.9 43.8
Limited English Proficiency 16.9 5.6 10.9
Not living with both parents 38.7 30.8 34.6
American Indian 1.8 1.9 1.7
Asian/Pacific Islander 35.9 21.9 24.0
Black/African American 30.9 19.3 19.8
Hispanic/Latino 17.6 10.0 12.3
White 19.7 46.6 41.3
Table 5. Comparing Characteristics of MSSP Intervention and Control Students
Intervention Students Control Students
Enrolled in Study Total: 54 Total: 64
Gender: Female : Male 48% : 52% 44% : 56%
Hispanic or Latino 31% 27%
American Indian/Alaska Native 3% 3%
Asian 6% 6%
Black or African American 33% 22%
White 4% 23%
Mean Socioeconomic Status Score 27 31
Baseline Symptom and Functional Comparability of MSSP Intervention and Control Students. Baseline
assessments revealed that MSSP students were showing signs of more severe mental health problems,
impairment in functioning and poor school performance than students in the control condition, despite our

8. 8
intentions and attempts to ensure comparability between students in the intervention and control groups.
Students in the intervention group were experiencing higher baseline levels of parent- and student-
reported internalizing (e.g., depression and anxiety) and externalizing (e.g., disruptive behavior)
problems (e.g., Figure 1). In addition students in the intervention group had a higher number of
unexcused absences in the baseline semester (mean = 3.45) than those in the control group (mean =
1.83).
Comparison of Improvements in Adaptive Functioning, Behavioral Health and Academic Performance
Indicators of MSSP Intervention and Control Students. Comparison of the intervention and control groups
revealed a number of differences in the progression of students’ adaptive functioning and behavioral
health problems over the course of the MSSP evaluation study. Specifically, students in the MSSP
Intervention group evidenced significantly greater improvements in parent-reported adaptive functioning
(across school, family, and extracurricular domains) (Figs. 1 and 2) and internalizing (depression and
anxiety) symptoms (Figure 3) and student-reported externalizing (disruptive and aggressive) behavior
(Figure 4). This means that while they began the study period with more symptoms and greater
impairment than students in the control group, students in the MSSP intervention group also experienced
greater declines in both functional impairment and in symptoms over time. Students participating in
MSSP either got better while students in the control group got worse (e.g., externalizing behavior, parent-
reported impairment in adaptive functioning), or students in the control group showed improvements, but
these improvements were not as dramatic as for students in the MSSP condition (e.g., internalizing
symptoms).
Figure 1. Impairment in Adaptive Functioning – Parent Report
Figure 2. Impairment in Adaptive Functioning – Student Report
10
11
12
13
14
15
16
17
Tim e 1 Tim e 2 Tim e 3
In te rve n tio n
C o n tro l
10
11
12
13
14
15
16
17
18
19
20
Time 1 Time 2 Time 3
Intervention
Control
10
11
12
13
14
15
16
17
Time 1 Time 2 Time 3
Intervention
Control

9. 9
When academic outcomes were evaluated by comparing indicators available from school records, few
differences between the intervention or control groups in changes in overall attendance, course failures,
or disciplinary actions were identified. However, because the MSSP intervention targeted the most
problematic students in schools with the highest needs in the District, we also determined whether there
were differential effects for a particularly high-risk subsample of both groups. Because failed courses in
earlier grades have been found to be highly predictive of later dropout and because the risk of dropout is
cumulative with higher numbers of failed courses (Celio, 2009), we examined whether Time 2 and Time
3 course failure rates differed between the subset of MSSP intervention and control students who had
already failed at least one course in the semester prior to Time 1. Findings indicated that, among youth
who had previously failed at least one course, only 38% of students in the MSSP intervention received
an additional failing grade, compared to 65% of the control youth who failed at least one additional
course during the follow-up period.
Figure 4. Externalizing Behavior – Student Report
Figure 3. Externalizing Behavior – Student Report
Figure 3. Internalizing Behavior – Parent Report
48
50
52
54
56
58
60
Time 1 Time 2 Time 3
Intervention
Control
49
50
51
52
53
54
55
56
Time 1 Time 2 Time 3
Intervention
Control
*

10. 10
Linkage to Support Services. A primary goal of MSSP Care Coordination and the WrapAround process
is to evaluate the student’s need for additional supports to promote academic success. At each
assessment we asked the student and parent/guardian to report on linkage to community and school-
based services. Table 6 below shows that in each of the service/support categories queried (community-
based mental health services, school-based mental health services, and academic support), a higher
proportion of MSSP participants had been linked. Particularly noteworthy are the differences in the
proportion who were receiving mental health counseling at school or in the community. The bottom rows
of Table 6 show only service linkages established after the baseline interview. Although the numbers
were smaller and did not reflect statistically significant between group differences, the odds of new
service linkages for MSSP students was about double that for control students in each of the service
categories.
Table 6. Comparing Linkage to Support Services for MSSP and Control Groups over Follow-up Period
Student or Parent Reported that Student Received These Services/Supports at Any
Assessment
Control
N=61
Intervention
N=52 OR (95% CI) P-value
Including those with services pre-study
Community Mental Health Center or Other
Outpt Mental Health Clinic or See a Mental
Health Professional
46% 62% 1.88 (0.89-4.00) 0.0960
Mental Health Counseling or therapy at
school
48% 81%
4.63 (1.97-
10.88)
0.0002
Special help (tutoring) in regular classroom 21% 33% 1.79 (0.77-4.17) 0.1724
Excluding those with services pre-study
Community Mental Health Center or Other
Outpt Mental Health Clinic or See a Mental
Health Professional
15% 25% 2.11 (0.83-5.38) 0.1117
Counseling or therapy at school 18% 30% 1.95 (0.81-4.68) 0.1306
Special help (tutoring) in regular classroom 12% 19% 1.83 (0.64-5.23) 0.2509
Implementation of the WrapAround Service for MSSP Participants. Care Coordinators facilitated school
setting-adapted WrapAround services for students in the MSSP intervention. For students in the
intervention group, only, we tracked the number who had WrapAround teams, the number of team
meetings conducted, and the number of team members in attendance at the meetings. At the second
and third interview, parents were asked about their experiences with the WrapAround services. Their
responses are highlighted in Table 8. Of the 54 intervention students in the evaluation study, 32 (59%)
had at least one formal WrapAround team meeting, with 14 students (26%) having multiple team
meetings, as depicted in Table 6. A total of 60 team meetings occurred during the evaluation period.
Consistent with WrapAround principles of family engagement (Principle 1: Family Voice and Choice),
parents were in attendance at 95% of the meetings. Another core principal of the WrapAround process
involves identification of a collaborative team of supportive individuals (Principle 2: Team Based). As
depicted in Table 7, the number of attendees at the 60 formal WrapAround team meetings ranged from 3
to 10 per meeting and totaled 364 over all the meetings.

11. 11
Table 9. Parent Indicators of Adherence to WrapAround Principles in MSSP Intensive
WrapAround Principle Question
Parent Responses
Yes No
Collaboration Did you and your team create a written care plan?
78.26 21.74
Strengths-Based Planning
Were services in the care plan connected to your child’s
strengths?
86.36 13.64
Community–Based Support
In a crisis do you feel your team can keep your child in the
current school placement?
63.64 36.36
Family and Student Voice
and Choice
Are important decisions made without asking you?
21.74 78.26
Culturally Competent
Do members of the team always use language you can
understand?
95.65 4.35
Culturally Competent
Did all members of the team demonstrate respect for you
and your family?
95.65 4.35
Family and Student Voice
and Choice
Did your child have the opportunity to contribute his/her
ideas when it came time to make decisions?
77.27 22.73
As shown in Table 9, caretivers reported that their MSSP WrapAround teams practiced strong adherence
to WrapAround principles. Cultural competence, as reflected in the team members demonstrating
respect for the family and talking in ways the family could understand, was rated very high, as was
planning of services that took into account the student’s strengths. Over 75% of caregivers reported that
their perspectives and those of their student were taken into account in decision-making
Summary of Results: The evaluation results are promising. They support the stated goal of the MSSP
program to improve functional outcomes of middle school students with behavioral health problems that
interfere with academic success. The trends across a one year follow-up period show that students who
participated in the MSSP intensive were doing better across domains of school, home, and
extracurricular activities and that their mental health symptoms improved more compared to middle
school students with similar challenges who did not participate in MSSP. These improvements were
demonstrated during a period of adolescent development when students’ behavioral health status would
otherwise be expected to decline. This translates into a “green light” for investing resources to provide
mental health professionals on site in middle schools to support a bridging of the gap between students’
behavioral health and academic needs.
A key ingredient of the MSSP program was the Care Coordinators convening WrapAround teams that
could consider the student’s needs across multiple domains and create a support plan with strong input
Table 7. Number of WrapAround Team
Meetings for 32 MSSP Students
Number of
meetings per
student
Number of
Students
1 meeting 18
2 meetings 8
3 meetings 0
4 meetings 5
6 meetings 1
60 total meetings 32
Table 8. Number of Attendees at Formal
WrapAround Team Meetings
Number of Meetings
Number of
Attendees
4 3
5 4
11 5
13 6
13 7
7 8
4 9
2 10
Total 60 Total 364

12. 12
from student and parent. Care Coordinators brought 364 people to 60 WrapAround meetings to work
together to weave a stronger network of support for students at high risk of school failure. The CC’s and
the WrapAround teams supported students and their families in accessing community and school-based
mental health and other services. Tying together Care Coordination, implementation of the WrapAround
approach and positive student outcomes, an experienced Care Coordinator reported that the Care
Coordinators, with their knowledge and experience of both educational and mental health systems, were
a unique and valued asset in the school setting. He noted that the WrapAround team was a powerful
force for change; that it took time to build the trust of parents to get the teams working effectively; and
that it was best when the team met consistently to strategize interventions or celebrate gains, not waiting
until the student happened to be doing poorly, but meeting also to build on successes at the time of a
meeting. In conclusion, the MSSP program developed a viable and valuable intervention model that
targets a subset of high needs students in challenging school settings, builds a team from the student’s
formal and informal support network, and facilitates positive outcomes.
Evaluation Study Limitations: The MSSP evaluation had several limitations, including lower than
anticipated study recruitment (74%), less than optimal comparability between intervention and control
student members and schools; attrition of study participants over time, good but not optimal research
design for attributing outcomes to program participation, and inability to observe gains beyond one-year
follow-up. Obstacles that impeded reaching our enrollment goals included study delays in getting
approval from and meeting administrative requirements of the SPS and SCH systems to conduct the
study, and turn over in and changes in responsibilities of Care Coordinators that led to a slowdown in
MSSP enrollment. To some extent, the low comparability of students in the intervention and control
groups was inevitable, given that the charge of the MSSP program was to be located at the most highly
stressed schools. Random assignment to MSSP or control conditions, while optimal for ensuring internal
validity and credibility of study results, was not an option we could exercise in conducting this evaluation.
Retention of study participants from baseline to second follow-up assessment was about 80%. The high
risk students served by the MSSP program are mobile. Of students enrolled in the evaluation, 15
transferred outside the Seattle Public School District, and seven transferred to other non-MSSP schools
within the SPS District within the follow-up time frame. This means that 20% of the total sample was
difficult to retain. It is likely that students who completed follow-up assessments were less challenged
than those who left the study. Countering potential bias was the fact that rates of attrition were similar
across the intervention and control groups. Finally, while study participants were followed for three
consecutive semesters, we suspect that for some students, academic gains may not appear until beyond
the time frame of this evaluation.
Recommendations: Emerging from this evaluation were signs that the MSSP program is meeting its
goals. In that promising context we would make the following recommendations as the program moves
forward:
1. We recommend that a further investment be made in training and ongoing supervision in the
philosophy and practice of WrapAround in the school context. It is difficult to form an effective
team, and Care Coordinators need ongoing support from experienced supervisors.
2. The progress made in defining and integrating the role of the Care Coordinator into the school
context should continue to be a high priority. While much focus in these initial six years has been
on defining the MSSP program itself, a slight shift in focus to defining the function of the program
distinct from the Individualized Education Program (IEP) process, the work of the Student
Intervention Teams (SIT), and other key groups within the complex educational setting - while
aligning the MSSP program with the goals and structure of the Seattle Public Schools may
enhance program value within the educational context.

13. 13
3. We recommend the continued systematic assessment of program and participant outcomes using
readily available tools. For example, the Columbia Impairment Scale - which measures student
functioning within salient contexts of school, family, and extracurricular activities and is easily
administered and scored - has proven to be sensitive to change over time and in response to the
MSSP program and is an excellent tool to use for ongoing assessment of student change and
program effects.
REFERENCES
Achenbach, T.M. & Rescorla, L.A. (2001). Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT:
University of Vermont, Research Center for Children, Youth, & Families.
Bird, H.R., Shaffer, D., Fisher, P., & Gould, M.S. (1993). The Columbia Impairment Scale (CIS): Pilot findings on a
measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric
Research, 3, 167-176.
Bisaga, K., Whitaker, A., Davies, M., Chuang, S., Feldman, S., & Walsh, B. T. (2005). Eating disorder and
depressive symptoms in urban high school girls from different ethnic backgrouns. Developmental and
Behavioral Pediatrics, 26, 257-266.
Bruns, E.J., Suter, J.C., Burchard, J.D., Force, M., & Leverentz-Brady, K. (2004). Assessing fidelity to a community-
based treatment for youth: the Wraparound Fidelity Index. Journal of Emotional and Behavioral Disorders, 12,
69-79.
Celio, M.B. (2009). Seattle School District 2006 Cohort Study. Presented to the Seattle Levy Oversight Committee,
June 16, 2009.
Cohen, P., & Brooks, J. S. (1985). Disorganized Poverty Questionnaire. Children in Community Study, Columbia
University.
Hollingshead, A. B. (1975). Four factor index of social status. Unpublished manuscript, Yale University, New
Haven, CT.
Horwitz, S. M., Hoagwood, K., Stiffman, A. R., Summerfeld, T., Weisz, J. R., Costello, E. J. et al. (2001). Reliability
of the services assessment for children and adolescents. Psychiatric Services, 52(8), 1088-1094.
Eggert, L. L., Herting, J. R., & Thompson, E. A. (1995). High school questionnaire; Profile of experiences. Seattle:
University of Washington School of Nursing, Reconnecting At-Risk Youth Prevention Research Program.
Messer, S. C., Angold, A., Costello, E. J., Loeber, R., Van Kammen, W., & Southamer-Loeber, M. (1995).
Development of a short questionnaire for use in epidemologic studies of depression in children and
adolescents: Factor composition and structure across development. International Journal of Methods in
Psychiatric Research, 5, 251-262.
Sund, A. M., Larsson, B., & Wichstrom, L. (2001). Depressive symptoms among young Norwegian adolescents as
measured by the Moods and Feelings Questionnaire (MFQ). European Child & Adolescent Psychiatry, 10,
222-229.