A RESOURCE GUIDE FORUNDERSTANDING COMMUNITY SCHOOLS      Community School Evaluations                October 2012         ...
Community School EvaluationsTable of ContentsA Resource Guide for Understanding Community Schools ...........................
A Resource Guide for Understanding Community SchoolsINTRODUCTIONUrban Strategies Council has collected and reviewed more t...
4. Collaborative Leadership   Addresses how to build, strengthen and expand the collaborative leadership structure at   co...
UPDATING THE RESOURCE GUIDEUrban Strategies Council will continue its efforts to update the Resource Guide with the mostcu...
Our Community School work with                       Oakland Unified School DistrictUrban Strategies Council has a long hi...
Community School Evaluations: Literature ReviewIntroductionEvaluations of community school initiatives are integral to und...
to better understand issues such as school climate, trust between students and adults, andfeelings of personal improvement...
achievement had a positive correlation with the number of days students attended communityschool programs.Other significan...
3. Concluding Remarks: Being Intentional about EquityThe findings of the evaluations are consistent with previously publis...
Community School Evaluations: Annotated BibliographyThe Community School Effect: Evidence from an Evaluation of the Tulsa ...
Evaluation of the San Francisco Beacon InitiativeSocial Policy Research Associates. Social Policy Research Associates, Nov...
Hartford Community Schools Evaluation: Findings from Year 2 (September 2009-November2010)HPS and ETO Data Analysis.” The O...
Final Report: 2009-2011: Hartford Community Schools EvaluationThe OMG Center for Collaborative Learning. The OMG Center fo...
7. Increased outreach and programs for families8. Increased family engagement9. Changes in family behavior10. Increased aw...
Community Learning Centers: Year in Review 2010-2011Mitchell, Dr. Monica. Cincinnati Public Schools, INNOVATIONS in Commun...
SUN Community Schools FY 2010-2011Schools Uniting Neighborhoods (SUN) Service System. Department of County Human Services,...
SUN Community Schools FY 2009-2010Schools Uniting Neighborhoods (SUN) Service System. Department of County Human Services,...
Social and Support Services for Educational Success FY 2009-2010Schools Uniting Neighborhoods (SUN) Service System. Depart...
Parent Child Development Services FY 2009-2010Schools Uniting Neighborhoods (SUN) Service System. Department of County Hum...
Executive Summary: SUN Service System Results - Fiscal Year 09/10Schools Uniting Neighborhoods (SUN) Service System. Depar...
Summary of the Children’s Aid Society Community School Results to DateChildren’s Aid Society. Children’s Aid Society, Janu...
Communities in Schools National Evaluation: Five Year Summary ReportICF International. Communities in Schools, October 201...
Comprehensive Evaluation of the Full-Service Community Schools Model in Pennsylvania:Lincoln and East Allegheny Middle Sch...
Comprehensive Evaluation of the Full-Service Community Schools Model in Maryland: GeneralSmallwood Middle SchoolLaFrance A...
Comprehensive Evaluation of the Full-Service Community Schools Model in Iowa: HardingMiddle School and Moulton Extended Le...
Comprehensive Evaluation of the Full Service Community Schools Model in Washington:Showalter Middle SchoolLaFrance Associa...
Three Years into Chicago’s Community Schools Initiative (CSI): Progress, Challenges, andLessons LearnedWhalen, Ph.D Samuel...
Report of the Evaluation of the Polk Bros. Foundation’s Full Service Schools InitiativeWhalen, Ph.D Samuel P. Chapin Hall ...
Achievement Plus Evaluation: 2009-2010Mueller, Dan, Katie Broton, and Edith Gozali-Lee. Wilder Research, September 2010. W...
Community Schools—Results That Turn Around Failing Schools: Test Scores, Attendance,Graduation and College-Going RatesCoal...
Raising Graduation and College Going Rates—Community High School Case StudiesAxelroth, Rita. Coalition for Community Schoo...
Evaluation of Community Schools: Findings to DateDryfoos, Joy. Coalition for Community Schools, 2000. Web. 19 December 201...
Community Schools Collaboration Evaluation ReportMurray, Mary, and Jessica Ganet. MEM Consultants, September 2010. Web. 28...
The Economic Impact of Communities in SchoolsESMI. ESMI, May 2012. Web. 5 July 2012.<http://www.communitiesinschools.org/a...
Positive Student Outcomes in Community SchoolsSebastian Castrechini and Rebecca A. London. Center for American Progress an...
Upcoming SlideShare
Loading in...5
×

Community School Evaluations

504

Published on

Reviewing community school evaluations helps those interested in or planning community school initiatives understand the successes, challenges and limitations of the community school strategy. This content area explores the methods and models used by community schools to conduct their evaluations, as well as each initiative’s outcomes and alignment with its vision. The information highlighted in this section includes the methodology, indicators of success, findings, trends, challenges and promising practices revealed in community school evaluations.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
504
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Community School Evaluations

  1. 1. A RESOURCE GUIDE FORUNDERSTANDING COMMUNITY SCHOOLS Community School Evaluations October 2012 Prepared by: Iris Hemmerich Urban Strategies Council
  2. 2. Community School EvaluationsTable of ContentsA Resource Guide for Understanding Community Schools .......................................................................... 2 Updating the Resource Guide ................................................................................................................... 4 Additional Community School Resources ................................................................................................. 4Our Community School work with Oakland Unified School District ............................................................. 5Community School Evaluations: Literature Review ...................................................................................... 6 Introduction .............................................................................................................................................. 6 Review ....................................................................................................................................................... 6 1. Methodology .................................................................................................................................. 6 2. Indicators and Measures of Success ............................................................................................... 7 3. Outcomes........................................................................................................................................ 7 Conclusion ................................................................................................................................................. 8 1. Methodological Flaws & Challenges .............................................................................................. 8 2. Promising Practices ........................................................................................................................ 8 3. Concluding Remarks: Being Intentional about Equity .................................................................... 9Community School Evaluations: Annotated Bibliography .......................................................................... 10 1 ©Urban Strategies Council, October 2012
  3. 3. A Resource Guide for Understanding Community SchoolsINTRODUCTIONUrban Strategies Council has collected and reviewed more than 175 evaluations, case studies,briefs and reports for use by those considering or planning a community school or communityschool district. Our intention is to provide interested individuals and stakeholders theresources they need to better understand the unique structure and core components ofcommunity schools. The promising practices, recommendations, tools and information sharedin this document have been culled from documents representing the last 20 years of researchand documentation of community schools across the United States.We highlighted 11 content areas that we believe to be the most foundational for understandingcommunity schools. Within each of the content areas, you will find: 1. A literature review: The literature reviews for each content area are organized around core questions and provide a synthesis of the most commonly identified solutions and responses to each question, as well as highlights, promising practices, challenges and recommendations. 2. An annotated bibliography: We gathered and annotated literature in each of the content areas to underscore key themes, some of which include: best practices, exemplary sites, models and tools. The annotations vary by content area in order to draw attention to the most pertinent information. For example, the Evaluations content area includes annotations of the evaluation methodology and indicators of success.The 11 content areas include the following: 1. Community School Characteristics Provides a general overview of the structure, function, core elements, programs and services of a community school. 2. Planning and Design Explores the general planning and design structures for community schools, and discusses the initial steps and central components of the planning and design process, as well as strategies for scaling up community schools. 3. Equity Frameworks and Tools Examines literature and tools that can be adapted to an equity framework for community schools. We included equity frameworks and tools that explore disproportionality and the monitoring of disparities and demographic shifts. 2 ©Urban Strategies Council, October 2012
  4. 4. 4. Collaborative Leadership Addresses how to build, strengthen and expand the collaborative leadership structure at community schools. Collaborative leadership is a unique governance structure that brings together community partners and stakeholders to coordinate a range of services and opportunities for youth, families and the community.5. Family and Community Engagement Explores how community and family engagement operates as well as the challenges for actualizing it at the site level. Family and community engagement is a unique component of community schools in which the school, families, and community actively work together to create networks of shared responsibility for student success.6. Data Collection and Analysis Addresses the outcomes measured at community schools, methods for collecting data at community schools, and short and long term indicators.7. Assessment Tools Includes tools used to measure outcomes at community schools.8. Community School Evaluations Provides evaluations of community school initiatives with special attention paid to methodology, indicators of success, findings and challenges.9. Community School Funding Explores how to leverage revenue streams and allocate resources at community schools.10. Budget Tools Includes tools that support the process of budgeting and fiscal mapping.11. Community School Sustainability Explores promising practices for creating sustainability plans, partnership development and leveraging resources for the future. 3 ©Urban Strategies Council, October 2012
  5. 5. UPDATING THE RESOURCE GUIDEUrban Strategies Council will continue its efforts to update the Resource Guide with the mostcurrent information as it becomes available. If you know of topics or resources that are notcurrently included in this guide, please contact Alison Feldman, Education Excellence Program,at alisonf@urbanstrategies.org. We welcome your ideas and feedback for A Resource Guide forUnderstanding Community Schools.ADDITIONAL COMMUNITY SCHOOL RESOURCESNational:The Coalition for Community Schoolshttp://www.communityschools.org/The National Center for Community Schools (Children’s Aid Society)http://nationalcenterforcommunityschools.childrensaidsociety.org/Yale University Center in Child Development and Social Policyhttp://www.yale.edu/21c/training.htmlRegional:The Center for Community School Partnerships, UC Davishttp://education.ucdavis.edu/community-school-partnershipsCenter for Strategic Community Innovationhttp://cscinnovation.org/community-schools-project/about-cscis-community-schools-project/community-school-initiative-services-coaching-and-ta/’ 4 ©Urban Strategies Council, October 2012
  6. 6. Our Community School work with Oakland Unified School DistrictUrban Strategies Council has a long history of working with the Oakland Unified School District(OUSD) to support planning for improved academic achievement. Most recently, we helpeddevelop and support the implementation of OUSD’s five-year strategic plan, CommunitySchools, Thriving Students. Adopted by the Board of Education in June 2011, the plan calls forbuilding community schools across the district that ensure high-quality instruction; developsocial, emotional and physical health; and create equitable opportunities for learning. UrbanStrategies Council has worked with the school district, community members and otherstakeholders to support this system reform in several ways: Community Schools Strategic Planning: Urban Strategies Council facilitated six School Board retreats over a 14-month period to help develop the strategic plan. As part of that process, the District created 14 task forces to produce recommendations for the plan, with Urban Strategies Council facilitating one task force and sitting on several others. Full Service Community Schools Task Force: Urban Strategies Council convened and co- facilitated the Full Service Community Schools and District Task Force, which created a structural framework and tools for planning and implementation, and produced a report with a set of recommendations that formed the foundation of the strategic plan. Community Engagement in Planning: Urban Strategies Council partnered with the district to educate and engage more than 900 school and community stakeholders on how community schools could best serve them. Planning for Community Schools Leadership Council: Urban Strategies Council has been working with OUSD’s Department of Family, School and Community Partnerships to lay the groundwork for building an interagency, cross-sector partnership body that will provide high-level system oversight and support, and ensure shared responsibility and coordination of resources towards the vision of healthy, thriving children supported through community schools. Convening Workgroups: Urban Strategies Council continues to partner with the District to convene and facilitate several workgroups developing specific structures, processes, and practices supporting community school implementation, as well as informing the eventual work of the Community Schools Leadership Council. African American Male Achievement Initiative: Urban Strategies Council is a partner in OUSD’s African American Male Achievement Initiative (AAMAI), a collaboration supporting efforts to close the achievement gap and improve other key outcomes for African American males in OUSD. Urban Strategies Council has developed data-based research; explored promising practices, programs and policies inside and outside the school district; analyzed the impact of existing system-wide policies; and developed policy recommendations to improve outcomes in various areas identified by the AAMAI Task Force. Boys and Men of Color: Urban Strategies Council is the Regional Convener for the Oakland Boys and Men of Color site, which adopted community schools as a vehicle to improve health, education and employment outcomes for boys and men of color. 5 ©Urban Strategies Council, October 2012
  7. 7. Community School Evaluations: Literature ReviewIntroductionEvaluations of community school initiatives are integral to understanding the impact of thecommunity school strategy to improve student learning while building stronger families andhealthier communities1. The evaluations expose the challenges and limitations of assessing thecommunity school strategy, which is critical for designing a more accurate evaluation in thefuture. Furthermore, they reveal the effectiveness of supports and operational elements andhelp inform future improvements for those areas. We used four central research questions toguide the literature review of community school evaluations: 1. What methodology was employed for the evaluation? 2. What were the indicators and measures of success? 3. What were the evaluation findings? 4. What were the challenges and limitations throughout the evaluation process?A compilation of recent research on community school evaluations published in 2009 showstrends in student improvement on reading and math standardized test scores2. Othersignificant trends include reduced drop-out rates and improved attendance3, improvedbehavior and youth development4, and greater parental engagement.5 While there is someresearch on the community schools evaluation process (most notably the “Evaluation Toolkit”compiled by the Coalition for Community Schools), what seems to be lacking in research andscholarship is an explicit focus on how to integrate equity into the evaluation process.Review 1. MethodologyA combination of qualitative and quantitative assessment methods were employed in all of thecommunity school evaluations. The most common qualitative approach was the use of surveys1 Coalition for Community Schools. “What is a Community School?” Coalition for Community Schools, 2012. Web.24 April 2012. <http://www.communityschools.org/aboutschools/what_is_a_community_school.aspx>.2 A study of Communities in Schools, a national community school model found: net increases of +6.0% in grade 8math and +5.1% in grade 8 reading scores for high-implementing community schools over their matchedcomparison group. Net increases in math scores for all grades over their comparison groups (+2.5% urban, +3.3%rural). Net increases in math for schools predominantly serving traditionally-low performing populations.3 Communities in Schools found net increases of +0.2% in elementary, +0.1% in middle, and +0.3% in high schoolfor high-implementing community schools over their matched comparison group.4 Blank, M., A. Melaville, and B. Shah. “Making the difference: Research and practice in community schools.”Washington, DC: Coalition for Community Schools, Institute for Educational Leadership, 2003. Web. 24 April 2012.5 Coalition for Community Schools. “Community Schools Research Brief: 2009.” Coalition for Community Schools,2009. Web. 24 April 2012.<http://www.communityschools.org/assets/1/AssetManager/CCS%20Research%20Report2009.pdf>. 6 ©Urban Strategies Council, October 2012
  8. 8. to better understand issues such as school climate, trust between students and adults, andfeelings of personal improvement among students. Interviews and student focus groups werealso commonly used to gain a better understanding of the aforementioned issues. In manycases, key documents were reviewed to assess the alignment of community school initiativeswith their shared vision and goals.All of the evaluations utilized a public database or other data management system to collectand analyze quantifiable data. The type of data collected ranged from student academicachievement and attendance to individual program participation. The most common evaluationdesign was the longitudinal study, in which the improvement of a cohort was documented overa period of time. Most frequently, a cohort of community school participants was comparedwith non-participants over a period of time. In a few cases, randomized controlled trials (RCTs)were employed and pre/post cohorts of participants and non-participants were examined. Afew evaluations used state benchmarks to measure the success of community schoolparticipants against other students in the district or state. The majority of the evaluations tookplace during a 1-3 year period. 2. Indicators and Measures of SuccessThe most common indicators of success were standardized test scores for math and readingsubject areas, attendance rates and survey results. Other frequently used indicators includesuspension and expulsion rates, disciplinary infractions and graduation credits (for high schoolstudents only). There were a few evaluations that looked at increases in parental engagementas an indicator of success; however, measurement of parental engagement was difficult andoftentimes done arbitrarily through voluntary surveys.One evaluation which stood out was the Cincinnati Community Learning Centers, whichemployed a “Learning Partner Dashboard” to measure student success. The aforementionedindicators of success were used but individual student data was disaggregated and studentimprovement was correlated with rates of participation in specific programs6. 3. OutcomesOf the 25 studies and over 70 school initiatives reviewed, the most common outcome amongcommunity schools was improved academic achievement on math and reading standardizedtests. Findings indicate that the vast majority of students participating in community schoolprograms outperformed free/reduced lunch students in state math and reading tests atcomparison schools. Community schools also increased the number of students meetingproficiency levels and state benchmarks for reading and math. In many evaluations, academic6 Mitchell, Dr. Monica. “Community Learning Centers: Year in Review 2010-2011.” Cincinnati Public Schools,INNOVATIONS in Community Research, 2011. Web. 19 December 2011.<http://news.cincinnati.com/assets/AB1820921121.PDF>. 7 ©Urban Strategies Council, October 2012
  9. 9. achievement had a positive correlation with the number of days students attended communityschool programs.Other significant findings from community schools include increased Academic PerformanceIndex scores, increased credits toward graduation and graduation rates, higher attendancerates, more schools meeting Annual Yearly Progress and lower rates of disciplinary infractions.Survey outcomes were positive and the majority of students who responded to surveys feltsupported academically, welcome and safe at school. Significant survey outcomes includeimproved student mental health, increased parental engagement and increased feelings ofcollective trust. It is important to note that only a few programs cited better access tohealthcare or improved general health in their outcomes.Conclusion 1. Methodological Flaws & ChallengesCollecting student data from school sites seemed to present one of the greatest challenges tothe evaluation process. Issues of parental consent likely contributed to inconsistencies orinaccuracies in student surveys aimed at obtaining demographic information or participantfeedback data. The students that did receive parental consent were not obligated to fill outsurveys and the voluntary responses may have skewed the results.Another challenge presented itself in the actual methodology of most evaluations. Manyevaluations did not have a control group and it was not certain that comparison cohortsmirrored the circumstances of one another. The lack of a control for the differences betweenparticipants and non-participants in community school programs indicates there may havebeen unexplored disparities in student data. Unexplored disparities among participants andnon-participants include factors such as prior and current student grade point averages, testscores, race and socioeconomic status, all of which could have altered the findings. 2. Promising PracticesThe “Learning Partner Dashboard” database used by the Cincinnati Community LearningCenters7 stood out as the most promising tool for collecting and analyzing individual studentdata and improvement. Data was disaggregated by multiple “priority factors”, some of whichinclude race, non-proficiency on standardized tests, five or more behavior referrals and five ormore absences. Individual student data was then assessed in relation to rates of participation inspecific programs. This allowed for the centers to not only evaluate individual student successbut the success of specific programs.7 Mitchell, Dr. Monica. “Community Learning Centers: Year in Review 2010-2011.” Cincinnati Public Schools,INNOVATIONS in Community Research, 2011. Web. 19 December 2011.<http://news.cincinnati.com/assets/AB1820921121.PDF>. 8 ©Urban Strategies Council, October 2012
  10. 10. 3. Concluding Remarks: Being Intentional about EquityThe findings of the evaluations are consistent with previously published research on communityschools and align with the community school vision. What is missing from the evaluations,however, is an intentional look at which students are benefitting from the community schoolstrategy and which students continue to experience disparate outcomes. The vast majority ofthe evaluations conclude that outcomes are improving for the overall student population, butfew evaluations address the experiences or outcomes of student subpopulations. Communityschools need to be more intentional about integrating equity into the evaluation process inorder to develop a complete understanding of who is benefitting from services and how theyneed to be targeted. A more comprehensive system of tracking and disaggregating student datawould produce more accurate findings and more equitable outcomes in the future. 9 ©Urban Strategies Council, October 2012
  11. 11. Community School Evaluations: Annotated BibliographyThe Community School Effect: Evidence from an Evaluation of the Tulsa Area CommunitySchool InitiativeAdams, Curt M. The Oklahoma Center for Educational Policy, November 2010. Web.19December 2011.<http://www.csctulsa.org/files/file/Achievement%20Evidence%20from%20an%20Evaluation%20of%20TACSI.pdf>.The 2008-2009 report evaluates the achievement effect of the community school model inTulsa, Oklahoma. The Tulsa Area Community Schools Initiative (TACSI) is a participant in thenational Coalition for Community Schools. TACSI programs intend to target low-incomestudents; however, the evaluation did not disaggregate data by socioeconomic status, race orother indicators. Instead, fifth grade student data was collected from 18 TACSI schools and 18non-TACSI schools with comparable demographics.The report suggests that the comparable school demographics (across indicators of poverty,average teacher experience, school size, and student ethnicity) in TACSI and non-TACSI schoolsreduced the probability that achievement differences were the result of confounding factors orselection bias. It also suggests that bringing the community school model to scale in TACSIschools enhances student achievement and narrows the achievement gap. When isolating thepoverty effect, results indicate that students in TACSI schools significantly outperformedfree/reduced lunch students in state math and reading tests than the comparison schools. Thestudy also found that collective trust mediated the relationship between student poverty leveland achievement. Methodology: 1. Cross-sectional and ex post facto data collection and analysis on state math and reading scores for a sample of fifth grade students at 18 TACSI and 18 non-TACSI schools 2. Surveys 3. Use of a poverty differential to account for individual achievement 4. Use of Optimal Design 2.0 to test the power of the sample in detecting significant achievement differences 5. Use of multilevel modeling with HLM 6.04 to evaluate the achievement effect 6. Use of Community School Development Scale to measure development of the community school model in TACSI schools (four levels of community school diffusion: inquiring, emerging, mentoring, and sustaining) Indicators & measures of success: 1. State math and reading exam scores 2. Survey results Tools: Sample Beacon scorecards (Appendix C) 10 ©Urban Strategies Council, October 2012
  12. 12. Evaluation of the San Francisco Beacon InitiativeSocial Policy Research Associates. Social Policy Research Associates, November 13, 2008. Web.19 December 2011.<http://www.sfbeacon.org/00_Evaluations/Evaluations/2008_Beacon_Evaluation_Report_SPR_Full_Report.pdf>.The 2007-2008 report evaluates the San Francisco Beacon Centers’ alignment with the Beaconvision to promote youth and family centers in public schools as beacons of activity for thesurrounding neighborhood. While the eight Beacon Centers in San Francisco are notcommunity schools, they similarly provide a myriad of services and programs and function ascommunity hubs. Beacon programs intend to target African American, English LanguageLearner, Latino, Pacific Islander, Samoan, and Special Education students; however, theevaluation did not disaggregate data by race or other indicators.The majority of youth surveyed felt that people at Beacon respected their culture and heritage,but few replied that Beacon helped them learn about their culture and heritage. Elementaryschool Beacon participants decreased their number of suspensions, while the number ofsuspensions for non-Beacon elementary school youth increased. However, the Average DailyAttendance (ADA) of middle and high school youth decreased across all levels of after schoolBeacon participation. There were also inconsistencies with Beacon youth California StandardsTest (CST) proficiency. Non-Beacon elementary school youth actually had higher percentages ofCST proficiency than Beacon youth for ELA and Math; Beacon middle school youth had higherpercentages of CST proficiency than non-Beacon youth; and non-Beacon high school youth hadhigher percentages of proficiency than Beacon youth on both tests.*Reviewer’s comment: The survey response that Beacon did not help students learn about theirculture and heritage suggests that Beacon Centers may need to pay more attention to issues ofrace, ethnicity and culture in programming. The inconsistencies in CST proficiency may be dueto differences in the student subpopulations that attend and don’t attend Beacon Centers. Methodology: 1. Key informant interviews 2. Surveys 3. Student focus groups 4. Contract Management System (CMS) program for collecting and analyzing data on attendance, suspension rates, and CST scores in ELA and Math Indicators & measures of success: 1. Attendance 2. Suspension rates 3. CST scores in ELA and Math 4. Survey results Tools: Sample Beacon scorecards (Appendix C) 11 ©Urban Strategies Council, October 2012
  13. 13. Hartford Community Schools Evaluation: Findings from Year 2 (September 2009-November2010)HPS and ETO Data Analysis.” The OMG Center for Collaborative Learning. The OMG Center forCollaborative Learning, February 2011. Web.19 December 2011.<http://www.communityschools.org/assets/1/AssetManager/Hartford%20Community%20Schools%20summary%20report%2009-10.pdf>.The 2009-2010 report evaluates the success of the community schools approach in HartfordCommunity Schools by the extent that the schools have realized student achievement gains.Hartford Community Schools provide an integrated approach to K-12 education, focusing onhigh academic standards and the provision of social and health supports to improve student-learning outcomes, and strengthen families and the community. The schools did not target aspecific student subpopulation and did not disaggregate data by race, income or otherindicators. Data was aggregated and compared between participants and non-participants.In 2010, a higher percentage of community school students scored proficient or above on theConnecticut Mastery Test (CMT) Reading test than other students in Hartford Schools. A greaterpercentage of students participating in the afterschool programs moved at least one level up onthe math and writing CMT tests than students who did not participate. Enrollment in theafterschool portion of the day reached 25 percent of the community schools’ population.Attendance rates averaged above 70 percent for all students enrolled in afterschool programsin any given month over the past academic year. Furthermore, the vast majority of studentswho responded to the 2009-2010 survey felt supported academically and welcome at school.*Reviewer’s comment: The lack of a control for the differences between participants and non-participants in afterschool programs implies that there are unexplored disparities in the data(ex: prior and current course grades, test scores, race and socioeconomic status). Methodology: 1. Data collection and analysis of CMT Math, Reading and Writing scores from Hartford Public Schools 2. Data collection and analysis of ETO data including afterschool participation, attendance, and activity data from the Hartford Office of Youth Services 3. School Climate Survey Indicators & measures of success: 1. Attendance 2. CMT Math, Reading and Writing proficiency levels 3. School Climate survey results 12 ©Urban Strategies Council, October 2012
  14. 14. Final Report: 2009-2011: Hartford Community Schools EvaluationThe OMG Center for Collaborative Learning. The OMG Center for Collaborative Learning,October, 2011. Web. 12 March 2012.<http://www.communityschools.org/assets/1/AssetManager/HCS%20Final%20Report%20(2-6-12).pdf>.The 2011 final report presents findings, three-year summary assessments andrecommendations for moving Hartford Community Schools forward. Systems-level communityschool implementation, school-level implementation and progress toward outcomes atHartford Community Schools are explored. Hartford Community Schools provide an integratedapproach to K-12 education, focusing on high academic standards and the provision of socialand health supports to improve student-learning outcomes, and strengthen families and thecommunity. The schools did not target a specific student subpopulation and data wascompared among participants and non-participants.Outcomes were explored at the school-level, systems-level, student, family and communitylevel. At the school-level, outcomes included: more robust community school programs;increased capacity of lead agency staff; demonstrated value to school leadership; and increaseddata quality. Systems-level outcomes included: increased clarity about the importance of asystems-level focus; district-wide emphasis on school-community partnerships; and expansionof funding and school partners in HCS. School, student, family and community outcomes werecharted and color-coded according to progress. Most outcomes were positive and met theirindicators of success. Increased interaction between school and lead agency staffs wasidentified as making strong progress. Improvements and increasing community schoolprogramming was also identified as making strong progress. Two outcomes, increased familyparticipation and changes in family behavior, proved to be immeasurable due to a lack of data.*Reviewer’s comment: The lack of a control for the differences between participants and non-participants in afterschool programs implies that there are unexplored disparities in the data(ex: prior and current course grades, test scores, race and socioeconomic status). Methodology: 1. In-person local leadership interviews 2. In-person focus groups with community school directors and lead agency managers 3. Follow-up phone interviews with the HCS director and HFPG program officer 4. Secondary data requests from Hartford Public Schools and Hartford Office of Youth Services School Climate Survey Indicators & measures of success: 1. Increased interaction between school and lead agency staffs 2. Increased use of data 3. Changes in community school programming 4. Changes in classroom/school environment 5. Increased student access to afterschool programs 6. Increased utilization/participation in afterschool programs 13 ©Urban Strategies Council, October 2012
  15. 15. 7. Increased outreach and programs for families8. Increased family engagement9. Changes in family behavior10. Increased awareness of Hartford Community Schools in the community11. Increased community access to Hartford Community Schools12. Increased community involvement in schools13. School Climate survey results14. Attendance15. CMT Math, Reading and Writing proficiency levelsTools:1. Appendix A: Hartford Community Schools Partnership Structure2. Appendix C: Key Milestones, Activities, and Contextual Changes 14 ©Urban Strategies Council, October 2012
  16. 16. Community Learning Centers: Year in Review 2010-2011Mitchell, Dr. Monica. Cincinnati Public Schools, INNOVATIONS in Community Research, 2011.Web. 19 December 2011.<http://news.cincinnati.com/assets/AB1820921121.PDF>.The 2010-2011 annual performance report evaluates the outcomes of individualized studentservices related to tutoring, mentoring, afterschool programs and college access at theCincinnati Public Schools Community Learning Centers (CLC). CLC is a district-wide initiativedesigned to provide academic reinforcements for students and develop community-centered“hubs” of services and resources through co-located partnerships. The centers specificallytarget students who have one or more “priority factors”. These factors include non-proficiencyin Ohio Achievement Assessments (OAA) Math scores or Reading scores, five or more absences,five or more tardies, or five or more behavioral referrals. Data is disaggregated by race andother indicators through the comprehensive “Learning Partner Dashboard” tool.Data for 2010-2011 show that academic achievement has improved in CLC schools since theonset of Resource Coordination. Students receiving student support services, includingtutoring, mentoring, college access and afterschool, all showed positive academic trends from2009-2010 to 2010-2011. “Priority factor” students with tutors made important gains on OAAMath and Reading scaled scores from 2009-2010 to 2010-2011. “Priority factor” students withCollege Access services made important gains on OAA Math scaled scores from 2009-2010 to2010-2011. Greater parent volunteerism was positively linked to fewer absences, fewerbehavioral referrals, and lower math and reading priority factors. Methodology: 1. Use of “Learning Partner Dashboard” database to track and analyze individual student needs and service partner coordination 2. Data collection and analysis of OAA Math and Reading scores from Cincinnati Public Schools 3. Surveys Indicators & measures of success: 1. OAA Math and Reading scores for “priority factor” students related to tutoring, mentoring, afterschool programs and College Access 2. Behavioral referrals for “priority factor” students related to tutoring, mentoring, afterschool programs and College Access 3. Tardies for “priority factor” students related to tutoring, mentoring, afterschool programs and College Access 4. Absences for “priority factor” students related to tutoring, mentoring, afterschool programs and College Access 5. Survey results 15 ©Urban Strategies Council, October 2012
  17. 17. SUN Community Schools FY 2010-2011Schools Uniting Neighborhoods (SUN) Service System. Department of County Human Services,Multnomah County, Oregon, 2012. Web. 14 May 2012.<http://web.multco.us/sites/default/files/sun/documents/suncsannual_outcomes1011.pdf>.The 2010-2011 report evaluates students who attended at least 30 days of Schools UnitingNeighborhoods Community School (SUN CS) programming, who had signed parental releasesand could be matched to district data. SUN CS provide school-based educational support,recreation, social and health services, and parent engagement to students and their families. In2010-2011, there were 60 SUN CS sites at 23 elementary schools, 13 middle schools, 6 highschools, and 18 schools serving grades K to 8 across Multnomah County, Oregon.Over 75% of SUN CS students showed improvement in state math and reading scores.Additionally, nearly 75% of SUN CS students were meeting their reading benchmarks or ontrack to reach their benchmarks in three years. Fewer students were on track to meet theirmath benchmarks, but the report suggests that this is largely due to the more rigorous cutoffscores in 2010-2011. SUN CS students attended an average of 94.5% of school days, which isabove the state benchmark of 92%. 17% of SUN CS students were classified as chronicallyabsent compared to the districts average of 32%. 79% of 12th grade SUN CS students graduated.*Reviewer’s comment: SUN CS did not employ an equity framework, according to informationobtained from an interview with Peggy Samolinski, the Division Director for SUN System inMultnomah County. SUN CS use 19 “risk factors” which they believe research has proven toimpact school success; they fall under the umbrella categories of children living in poverty,children of color, and English learners. Due to complications with parental consent, some stafflearned that students were at-risk only after the students started participating in the programs.This created difficulties in pre/post data collection and analysis. SUN CS measured the impact ofthe program based on participants versus non-participants. They also compared the CSparticipants’ outcomes with district and state benchmarks in attendance, reading, math, andcredits earned for graduation (if in high school). At the site level, SUN CS employ ServicePoint, aweb data base that collects information with an activity point module. Methodology: 1. Use of ServicePoint at site level 2. Collection and analysis of Oregon Assessment of Knowledge and Skills (OAKS) Math and Reading score data for SUN CS participants and non-participants 3. Comparison of SUN CS participants’ outcomes with district and state benchmarks in attendance, reading, math, and credits earned for graduation (if in high school) 4. Surveys Indicators & measures of success: 1. OAKS Reading and Math test scores related to individual improvement and state benchmarks 2. Attendance related to district level 3. Graduation credits (if in high school) 16 ©Urban Strategies Council, October 2012
  18. 18. SUN Community Schools FY 2009-2010Schools Uniting Neighborhoods (SUN) Service System. Department of County Human Services,Multnomah County, Oregon, 2011. Web. 19 December 2011.<http://web.multco.us/sites/default/files/sun/documents/suncsannualoutcomes0910.pdf>.The 2009-2010 report evaluates students who attended at least 30 days of Schools UnitingNeighborhoods Community School (SUN CS) programming, who had signed parental releasesand could be matched to district data. SUN CS provide school-based educational support,recreation, social and health services, and parent engagement to students and their families. In2009-2010, there were 60 SUN CS sites at 23 elementary schools, 14 middle schools, 7 highschools, and 16 schools serving grades K to 8 across Multnomah County, Oregon.Over 75% of SUN CS students showed improvement in state math and reading scores. Inaddition, almost three quarters of SUN CS students were meeting their benchmarks by eighthgrade in reading and math. SUN CS students attended an average of 94.3% of required schooldays, which is above the state benchmark of 92%. About 15% of SUN CS students wereclassified as chronically absent compared to 25% across the district. Of 12th grade SUN CSstudents, 81% graduated. Indicators & measures of success: 1. OAKS Reading and Math test scores related to individual improvement and state benchmarks 2. Attendance related to district level 3. Graduation credits (if in high school) 17 ©Urban Strategies Council, October 2012
  19. 19. Social and Support Services for Educational Success FY 2009-2010Schools Uniting Neighborhoods (SUN) Service System. Department of County Human Services,Multnomah County, Oregon, 2011. Web. 19 December 2011.<http://web.multco.us/sites/default/files/sun/documents/sssesannualoutcomes0910.pdf>.Social and Support Services for Educational Success (SSSES) is part of Schools UnitingNeighborhoods Community Schools (SUN CS) and provides age appropriate and culturallyspecific academic support, case management and skill building activities. The most commontypes of service include case management, education, skills training and recreation. Theoutcomes reported are for youth who participated in at least 45 days and 15 hours of servicesduring 2009-2010.The report finds that SSSES agencies are serving their intended group of children living inpoverty, children of color, and English learners. While students are making progress in readingand math, only 41% met the state reading benchmark and 42% met the state math benchmark.However, SSSES students attended an average of 91% of required school days, which is almostat the state benchmark of 92%. High school students made significant progress in earningcredits, but only 53% earned enough to be on track to graduate in four years. Of 12th gradersparticipating in SSSES, 53% graduated. Students made progress across six types of casemanagement goals and over 90% of each goal type showed at least partial progress.*Reviewer’s comment: The three page report focuses on outcomes and not methodology fordata collection and analysis. According to information obtained from Peggy Samolinski, theDivision Director for SUN CS, there is no equity framework integrated into the evaluation ofSSSES. There is no way to ensure specific student subpopulations participate in the services;however, the report states that those being served are “at-risk” youth. Indicators & measures of success: 1. Case management goals: (1) academic; (2) attendance; (3) self-esteem; (4) social skills; (5) relationships; and (6) basic needs 2. OAKS Reading and Math test scores related to state benchmarks 3. Grade Point Average 18 ©Urban Strategies Council, October 2012
  20. 20. Parent Child Development Services FY 2009-2010Schools Uniting Neighborhoods (SUN) Service System. Department of County Human Services,Multnomah County, Oregon, 2011. Web. 19 December 2011.<http://web.multco.us/sites/default/files/sun/documents/pcdsannualoutcomes0910.pdf>.The 2009-2010 report evaluates the services and outcomes on the 476 children whoparticipated in the Parent-Child Development Services (PCDS) program. PCDS is part of SUNCommunity Schools in Oregon and provides parent education, support groups, and young childplaygroups (age two to five). PCDS uses the “Parents as Teachers” curriculum, which is anevidence-based curriculum designed to teach parenting skills and knowledge. The majority ofthe service hours are spent on parent skill training.The report finds that the PCDS program is serving its intended population of children living inpoverty, children of color, and parents whose primary language is not English. Half of thefamilies stay in the program for at least one year. Research in the national Parents as Teachersevaluation shows that two years of home visitation combined with one year of pre-school ismost effective at getting students ready to start kindergarten. Younger pre-school-agedchildren stay in services for longer than other children. The program helps ensure that childrenare up-to-date on immunizations, developmentally screened and referred to early interventionservices if needed. Service exit surveys for parents also show very positive feedback.*Reviewer’s comment: The three page report focuses on outcomes and not methodology fordata collection and analysis. According to information obtained from Peggy Samolinski, theDivision Director for SUN CS, there is no equity framework integrated into the evaluation ofPCDS. Indicators & measures of success: 1. Kindergarten readiness (does not state specific measure) 2. Survey results (parent satisfaction with services and skill-building) 19 ©Urban Strategies Council, October 2012
  21. 21. Executive Summary: SUN Service System Results - Fiscal Year 09/10Schools Uniting Neighborhoods (SUN) Service System. Department of County Human Services,Multnomah County, Oregon, 2011. Web. 19 December 2011.<http://web.multco.us/sites/default/files/sun/documents/executivesummary0910.pdf>.The 2009-2010 executive summary provides an overview of Schools Uniting NeighborhoodsCommunity Schools (SUN CS) progress and outcomes. SUN CS provide school-based educationalsupport, recreation, social and health services, and parent engagement to students and theirfamilies. In 2010, SUN CS programs served a notably higher percent of children living in poverty,children of color and English learners compared to the school districts’ average. Outcomes werereported for students who participate in SUN CS for 30 days or more.The average daily attendance rate of SUN CS students was 94%, which exceeded the statebenchmark of 92%. SUN CS students’ average gains were equal to or higher than expected forfourth, sixth and seventh grades in reading and math. Benchmarks state that high schoolstudents need to earn 6.0 credits per year to be on track for graduation in four years andstudents in SUN CS earned an average of 6.2 credits per year. About 99% of parents whoparticipated in Parent Child Development Services reported acquiring new skills. Almost all ofthe families threatened by homelessness (94%) remained in permanent housing 12 monthsafter receiving rent supports.*Reviewer’s comment: The report focuses on outcomes and not methodology for datacollection and analysis. According to information obtained from Peggy Samolinski, the DivisionDirector for SUN CS, there is no equity framework integrated into the evaluation of SUN CS. Methodology: 1. Collection and analysis of Oregon Assessment of Knowledge and Skills (OAKS) Math and Reading score data for SUN CS participants and non-participants 2. Comparison of SUN CS participants’ outcomes with district and state benchmarks in attendance, reading, math, and credits earned for graduation (if in high school) 3. Surveys Indicators & measures of success: 1. OAKS Reading and Math test scores related to state benchmarks 2. Attendance related to district level 3. Graduation credits (if in high school) 4. Survey results 5. Permanent housing (no specific measured stated) 20 ©Urban Strategies Council, October 2012
  22. 22. Summary of the Children’s Aid Society Community School Results to DateChildren’s Aid Society. Children’s Aid Society, January 2006. Web. 19 December 2011.<http://www.aypf.org/documents/SummaryoftheChildrensAidSocietyCommunitySchoolsResults.pdf>.The report summarizes various evaluation outcomes of CAS community school programs andservices by outside entities over a 13-year period. The goal of the Children’s Aid Society (CAS) isto affect youth, families, and schools by reducing barriers to learning so that children arephysically, emotionally and socially prepared to learn.The outcomes for youth show improvements in reading and math proficiency on standardizedtests. Additionally, academic achievement has had a positive correlation with the number ofdays students attended community school extended-day programs. Teachers reportedimprovements in behavioral conduct and qualitative data showed improvements in studentmental and physical health. Some of the most significant outcomes for families include muchhigher parental involvement in CAS Community Schools than in comparison schools andincreases in the quality and size of parent social support networks. The mothers participating inthe Early Head Start program reported decreases in depression and stress over the course ofparticipation in the program. One of the most significant outcomes included teachers incommunity schools being able to spend more time on teaching than their counterparts incomparison schools.*Reviewer’s comment: The Children’s Aid Society report focuses on outcomes and notmethodology for data collection and analysis. Indicators & measures of success: 1. Reading and math proficiency levels on standardized tests 2. School report card data 3. Grade point average 4. Attendance 5. Survey results 6. Parental involvement (no specific measure stated) 21 ©Urban Strategies Council, October 2012
  23. 23. Communities in Schools National Evaluation: Five Year Summary ReportICF International. Communities in Schools, October 2010.<http://www.communitiesinschools.org/media/uploads/attachments/Communities_In_Schools_National_Evaluation_Five_Year_Summary_Report.pdf>.The 2010 report measures national outcomes in Communities in Schools (CIS) over a five yearperiod in order to understand how and why community schools work. CIS is a nationalfederation of independent 501(c)3 organizations that consist of a national office, state offices,and local affiliate offices serving students in 3,400 schools. Their mission is to engagecommunity partners and volunteers in order to effectively address both the academic andhuman service needs of students. There was no specific student subpopulation targeted in theevaluations and no disaggregated data by race.“High implementer” CIS schools had considerably greater effects on reducing dropout rates andincreasing on-time graduation than their non-CIS comparisons and other CIS schools (i.e.“partial implementers”). Results from the school-level quasi-experimental study indicate thatCIS students experienced consistent improvements in attendance and state-mandated testscores. Fewer CIS case-managed students dropped out of school during their 9th grade yearthan students in the control group. Students who received CIS services for two consecutiveyears had more favorable outcomes in all categories than students who received a single yearof CIS service. CIS schools, regardless of urban, suburban, or rural location, outperformed theircomparison “match” schools on most outcomes. Methodology: 1. Use of a National Evaluation team to collect, analyze and assess gaps of all CIS data to date 2. Critical Processes Survey to assess site level process data and create a community school rubric for “high implementer” or “partial implementer” schools 3. School Level Quasi-Experimental Study (using propensity score matching to compare CIS with non-CIS sites) 4. External Comparison Study 5. Natural Variation Study to determine key attributes separating successful CIS schools from the unsuccessful CIS schools on a given outcome 6. Randomized Controlled Trials (RCT’s) to determine the impact of CIS case-managed services on individual student outcomes Indicators & measures of success: 1. Attendance 2. Graduation 3. Drop-out 4. Suspension 5. Behavior referrals 6. Math and ELA performance 22 ©Urban Strategies Council, October 2012
  24. 24. Comprehensive Evaluation of the Full-Service Community Schools Model in Pennsylvania:Lincoln and East Allegheny Middle SchoolsLaFrance Associates, LLC. LaFrance Associates, LLC, September 2005. Web. 19 December 2011.<http://www.eisenhowerfoundation.org/docs/Pennsylvania.pdf>.The 2004-2005 report evaluates the implementation of the full-service community school(FSCS) model and its program outcomes at Lincoln and East Allegheny Middle Schools. Lincolnand East Allegheny Middle Schools are among a cohort of schools in five U.S. states that areparticipating in the Eisenhower Foundation Full-Service Community Schools replicationinitiative. The demographic characteristics of participants were surveyed in the evaluation;however, data was not disaggregated by race.The report suggests that a higher level of participation in academic FSCS programs is associatedwith better attendance outcomes. Participation in after-school programming also appears tolead to positive outcomes for youth, especially in their behavior at school, academicachievement, satisfaction with school, and positive peer and adult relationships. More thaneight in ten parents of FSCS students who responded to the parent survey said that the FSCSactivities have helped their child enjoy school more. Furthermore, FSCS participants have morepositive peer relationships as a result of their participation in after-school programs and haveexpressed a greater feeling of safety in school.*Reviewer’s comment: There may be inconsistencies or inaccuracies when relying on studentsurveys to obtain demographic information of FSCS participants. Not all students fill out surveysand parental consent is required. This could potentially skew results indicating which studentsubpopulations are benefitting from FSCS. Methodology: 1. Data collection and analysis of school records through management information systems (MIS) 2. Observations of program operations 3. Key informant interviews 4. Youth focus groups 5. Surveys 6. Youth journal-writing exercise and photo project 7. Logic model indicators 8. Randomized Control Trials 9. Quasi-experimental pre/post comparison cohort of FSCS afterschool program participants and non-participants Indicators & measures of success: 1. Grade point average 2. Attendance 3. Disciplinary action report cards 4. Standardized test scores 5. Survey results 23 ©Urban Strategies Council, October 2012
  25. 25. Comprehensive Evaluation of the Full-Service Community Schools Model in Maryland: GeneralSmallwood Middle SchoolLaFrance Associates, LLC. LaFrance Associates, LLC, August 2005. Web. 19 December 2011.<http://www.eisenhowerfoundation.org/docs/Maryland.pdf>.The 2004-2005 report evaluates the implementation of the full-service community school(FSCS) model and its outcomes at General Smallwood Middle School in Maryland. GeneralSmallwood is one of a cohort of schools in the U.S. participating in the Eisenhower FoundationFull-Service Community Schools replication initiative. The demographic characteristics ofparticipants were surveyed in the evaluation; however, data was not disaggregated by race.At General Smallwood Middle School, there were no significant findings for the relationship ofFSCS program participation with attendance. Overall, the students who participated in FSCSactivities were those with more absences and behavioral incidences than students who did notend up participating. FSCS participants did show an improvement in math grades which washigher than the improvement shown for non-participants. Moreover, participation in after-school programming appears to lead to positive outcomes in youth behavior at school,academic achievement, satisfaction with school, and positive peer and adult relationships.*Reviewer’s comment: There may be inconsistencies or inaccuracies when relying on studentsurveys to obtain demographic information of FSCS participants. Not all students fill out surveysand parental consent is required. This could potentially skew results indicating which studentsubpopulations are benefitting from FSCS. Methodology: 1. Data collection and analysis of school records through management information systems (MIS) 2. Observations of program operations 3. Key informant interviews 4. Youth focus groups 5. Parent focus group 6. Surveys 7. Youth journal-writing exercise and photo project 8. Logic model indicators 9. Randomized Control Trials 10. Quasi-experimental pre/post comparison cohort of FSCS afterschool program participants and non-participants Indicators & measures of success: 1. Grade point average 2. Attendance 3. Disciplinary action report cards 4. Standardized test scores 5. Survey results 24 ©Urban Strategies Council, October 2012
  26. 26. Comprehensive Evaluation of the Full-Service Community Schools Model in Iowa: HardingMiddle School and Moulton Extended Learning CenterLaFrance Associates, LLC. LaFrance Associates, LLC, September 2005. Web. 19 December 2011.<http://www.eisenhowerfoundation.org/docs/Iowa.pdf>.The 2004-2005 report evaluates the implementation of the full-service community school(FSCS) model and its outcomes at Harding Middle School and Moulton Extended LearningCenter in Iowa. Harding Middle School is one of a cohort of schools in the U.S. participating inthe Eisenhower Foundation Full-Service Community Schools replication initiative. Theevaluation focuses on the change experienced by students and parents and impacts at theschool level. The demographic characteristics of participants were surveyed in the evaluation;however, data was not disaggregated by race.Students who participated in FSCS showed more improvement in their grades than non-participants. FSCS participants were almost six times more likely to have shown improvement intheir attitude toward school than students who did not participate. High participants benefitedmore than those who participated less. Participation in the enrichment programs is associatedwith a year-to-year increase in English grades. Additionally, FSCS participants have greaterpositive adult relationships as a result of their participation in after-school programs andexpressed satisfaction with school.*Reviewer’s comment: There may be inconsistencies or inaccuracies when relying on studentsurveys to obtain demographic information of FSCS participants. Not all students fill out surveysand parental consent is required. This could potentially skew results indicating which studentsubpopulations are benefitting from FSCS. Methodology: 1. Data collection and analysis of school records through management information systems (MIS) 2. Observations of program operations 3. Key informant interviews 4. Youth focus groups 5. Surveys 6. Youth journal-writing exercise and photo project 7. Logic model indicators 8. Randomized Control Trials 9. Quasi-experimental pre/post comparison cohort of FSCS afterschool program participants and non-participants Indicators & measures of success: 1. Grade point average 2. Attendance 3. Disciplinary action report cards 4. Standardized test scores 5. Survey results 25 ©Urban Strategies Council, October 2012
  27. 27. Comprehensive Evaluation of the Full Service Community Schools Model in Washington:Showalter Middle SchoolLaFrance Associates, LLC. LaFrance Associates, LLC, September 2005. Web. 19 December 2011.<http://www.eisenhowerfoundation.org/docs/Showalter2.pdf>.The 2004-2005 report evaluates the implementation of the full-service community school(FSCS) model and its outcomes at Showalter Middle School in Washington. Showalter MiddleSchool is one of a cohort of schools in the U.S. participating in the Eisenhower Foundation Full-Service Community Schools replication initiative. The demographic characteristics ofparticipants were surveyed in the evaluation; however, data was not disaggregated by race.One of the most significant findings is that FSCS participation leads to decreased behavioralincidents at Showalter Middle School. Those who participated in FSCS afterschool programsimproved their behavior significantly more than those who did not participate. Holding all elseequal, those who participated in FSCS afterschool programs improved their math grades morethan non-participants. Survey results also indicated that participation in FSCS programs led togreater orientation towards learning, more positive peer relationships and improved self-esteem.*Reviewer’s comment: There may be inconsistencies or inaccuracies when relying on studentsurveys to obtain demographic information of FSCS participants. Not all students fill out surveysand parental consent is required. This could potentially skew results indicating which studentsubpopulations are benefitting from FSCS. Methodology: 1. Data collection and analysis of school records through management information systems (MIS) 2. Observations of program operations 3. Key informant interviews 4. Youth focus groups 5. Surveys 6. Youth journal-writing exercise and photo project 7. Logic model indicators 8. Randomized Control Trials 9. Quasi-experimental pre/post comparison cohort of FSCS afterschool program participants and non-participants Indicators & measures of success: 1. Grade point average 2. Attendance 3. Disciplinary action report cards 4. Standardized test scores 5. Survey results 26 ©Urban Strategies Council, October 2012
  28. 28. Three Years into Chicago’s Community Schools Initiative (CSI): Progress, Challenges, andLessons LearnedWhalen, Ph.D Samuel P. College of Education, University of Illinois at Chicago, June 2007. Web.19 December 2011.<http://www.aypf.org/documents/CSI_ThreeYearStudy.pdf>.The report analyzes trends in how Chicago’s Community Schools Initiative (CSI) is buildingcommunity school capacity and realizing important benefits for the schools, students, andfamilies since 2001. CSI includes 110 elementary and high schools and builds upon the corefeatures of the federal 21st Century Community Learning Centers Program (CLC). These corefeatures include open resources for afterschool and community use, parent and communityengagement, and social and family support services. Some of the schools include neighborhoodelementary and high schools, magnet and specialty schools, and charter and contract schools.The evaluation did not disaggregate data by race or other indicators.Overall enrollment of students in “out of school time” (OST) increased by 17% between 2005and 2006, from an average of 156 students to over 183 students per school. A large number ofOST program participants improved their reading and math grades over the course of the year.More significantly, the average of 49.3 days of attendance per student in 2006 far exceeded theCLC criteria for regular attendees. The inclusion of students with special needs among CLCprogram participants increased 59% from 2005 to 2006. Trends in standardized test resultsindicate that CSI schools have steadily closed the gap in achievement between themselves andthe district between 2001 and 2006. Methodology: 1. Collection and analysis of individual student participation data, standardized test performances and school level summary statistics 2. Surveys 3. Key informant interviews 4. Analyses of school improvement plans Indicators & measures of success: 1. Grade point average in reading and math 2. Attendance (school and OST) 3. Disciplinary infractions 4. Illinois Standards Achievement Test (ISAT) scores in math, science and reading 5. Survey results 27 ©Urban Strategies Council, October 2012
  29. 29. Report of the Evaluation of the Polk Bros. Foundation’s Full Service Schools InitiativeWhalen, Ph.D Samuel P. Chapin Hall Center for Children at the University of Chicago, April 2002.Web. 19 December 2011.<http://www.communityschools.org/assets/1/AssetManager/Polk_Bros_ExecutiveSummary.pdf>.The 2002 report summarizes the findings of a three year evaluation of the Full Service SchoolsInitiative (FSSI) in Chicago, a pilot project funded by Chicago’s Polk Bros. Foundation. Thepurpose of FSSI is to test a research-based framework for expanding school-based and school-linked services that broaden support for children’s well-being and academic achievement. Theevaluation did not disaggregate data by race or other indicators.Results indicate that the FSSI framework successfully expanded resources in support of studentgrowth and learning during the afterschool hours. Academic performance improved at all threeschools during FSSI. In most cases, the rates of improvement exceeded CPS and equaled ormarginally exceeded comparison schools. Furthermore, FSSI schools increased the range anddiversity of their community alliances. The number of teachers involved in planning or providingafter-school activities increased by more than 20% at all three schools. Across the schools,between one-third and one-half of enrolled students participated in an after-school program.However, FSSI schools also showed lower student mobility than comparable schools andsteeper rates of decline in mobility. Methodology: 1. Collection and analysis of aggregate student achievement, truancy, student mobility and attendance data for Chicago Public Schools 2. Surveys 3. Key informant interviews 4. Focus groups 5. Observations Indicators & measures of success: 1. Student mobility, attendance and truancy related to demographically similar schools and the Chicago Public School District 2. Standardized test scores 3. Increased afterschool resource opportunities 4. Increased community partnerships 5. Increased teacher involvement in FSSI programs 6. Increased student participation in FSSI programs 7. Increased supportive adult relationships for students 8. Survey results 28 ©Urban Strategies Council, October 2012
  30. 30. Achievement Plus Evaluation: 2009-2010Mueller, Dan, Katie Broton, and Edith Gozali-Lee. Wilder Research, September 2010. Web. 19December 2011.<http://www.wilder.org/download.0.html?report=2341>.The 2009-2010 report evaluates how Achievement Plus fits into community school effortsnationally, how well community partnerships are working, and how well students at theseschools are performing. Achievement Plus is a private/public partnership with the primary goalof improving student achievement in low socio-economic areas of the city through academics,extended afterschool programs, and learning supports for families, students and communitymembers at schools. Programs intend to target students of low socioeconomic status; however,the evaluation did not disaggregate data by income, race or other indicators.Results from the 2010 state-mandated Minnesota Comprehensive Assessments (MCA) indicatethat proficiency levels in reading and math are improving. At one school, the math proficiencyrate increased substantially and exceeded the statewide rate in 2010. Additionally, results ofinterviews with community partner representatives indicate that partners believe theycontribute to children’s school success by eliminating or reducing barriers to children’s learning.There are currently nine Achievement Plus community partners filling a need identified byschool staff or neighborhood residents. Methodology: 1. Collection and analysis of student achievement data 2. Literature reviews of community schools 3. Analysis of service records 4. Key informant interviews Indicators & measures of success: 1. Minnesota Comprehensive Assessments (MCA-II) in math and reading 2. Key informant interview results 3. Partner alignment with children’s needs 29 ©Urban Strategies Council, October 2012
  31. 31. Community Schools—Results That Turn Around Failing Schools: Test Scores, Attendance,Graduation and College-Going RatesCoalition for Community Schools. Coalition for Community Schools, May 2010. Web. 19December 2011.<http://www.communityschools.org/assets/1/AssetManager/Turning_Around_Schools_CS_Results2.pdf>.The 2010 report summarizes research outcomes that were gathered from community schoolsacross the nation from 2007-2009. The Coalition for Community Schools is an alliance ofnational, state and local organizations involved in education, youth development, health andhuman services, and community schools among other areas. There is no evaluationmethodology provided in the report. Consequently, the report does not mention if an equityframework was employed in the evaluation process. Some of the most significant findings fromcommunity schools across the nation include higher scores on math and reading standardizedtests, increased Academic Performance Index scores, increasing graduation rates, more parentinvolvement, more schools meeting Annual Yearly Progress, higher attendance rates and lowerrates of disciplinary infractions. Indicators & measures of success: 1. Standardized test scores 2. Academic Performance Index 3. Annual Yearly Progress 4. Attendance 5. Graduation 6. Drop-outs 7. Behavioral referrals 8. Parental involvement 30 ©Urban Strategies Council, October 2012
  32. 32. Raising Graduation and College Going Rates—Community High School Case StudiesAxelroth, Rita. Coalition for Community Schools and National Association of Secondary SchoolPrincipals, August 2009. Web. 19 December 2011.<http://www.eric.ed.gov/PDFS/ED506727.pdf>.The report provides an individual overview of eight community high school’s programs andservices, partnerships, leadership and results. There is no evaluation methodology provided inthe report. The eight featured schools include George Washington Community High School,Fannie Lou Hammer High School, Oyler Community Learning Center, Parkrose High School,Foster High School, Community Links High School, Little Village Lawndale High School and SayreHigh School. Across the schools, at least 60 percent of students are eligible for free or reduced-price lunch, while most of the schools have rates of 90 percent or more. They also representsome of the poorest and most ethnically diverse schools in the country.Some of the most significant results across all eight schools include positive changes inacademic achievement and preparation for post-secondary success. Attendance hasdramatically increased among all eight community schools, exceeding the district goal in mostcases. Graduation and college-acceptance rates have also increased and at most schools theyexceed the district-wide average. At all eight schools there have been significant decreases indropout rates. Indicators & measures of success: 1. Standardized test scores 2. State benchmarks 3. Attendance 4. Graduation 5. Drop-outs 6. College enrollment 7. Survey results 31 ©Urban Strategies Council, October 2012
  33. 33. Evaluation of Community Schools: Findings to DateDryfoos, Joy. Coalition for Community Schools, 2000. Web. 19 December 2011.<http://www.communityschools.org/assets/1/AssetManager/Evaluation%20of%20Community%20Schools_joy_dryfoos.pdf#xml=http://prdtsearch001.americaneagle.com/service/search.asp?cmd=pdfhits&DocId=737&Index=F%3a%5cdtSearch%5ccommunityschools&HitCount=17&hits=5+6+7+4a+36a+410+49a+4a4+608+659+b55+fa0+2a83+3426+44ef+45fc+461c+&hc=1394&req=findings+to+date>.The report provides research findings from 49 evaluations of community school initiatives. Theprograms vary in their length of existence, evaluation method and duration. One limitation inthe evaluation process is that some reports aggregated findings for hundreds of schools and notat state, community, and local site levels.The most common achievement among the schools was academic improvement, with 36 of the49 programs reporting academic gains in reading and math standardized test scores over a two-three year period. In at least eight of the cases; however, the outcomes were limited tostudents who received special services, such as case management or extended day sessions.Nineteen programs reported improvements in school attendance and several mentioned higherteacher attendance rates. Eleven programs reported a reduction in suspensions and elevenprograms reported reductions in rates of disruptive behavior in the classroom. At least 12 ofthe programs reported increases in parent involvement. Only a few programs cited betteraccess to health care, lower hospitalization rates, higher immunization rates, or access todental care. Methodology (varied for each evaluation): 1. Collection and analysis of student achievement data through management information systems (MIS) 2. Use of control cohorts 3. Use of non-representative small samples 4. Use of comparison studies with “match” schools or participants vs. non-participants Indicators & measures of success: 1. Standardized test scores 2. Attendance 3. Graduation 4. Suspension and expulsion 5. Access to support services 6. Parental involvement 7. Survey results 32 ©Urban Strategies Council, October 2012
  34. 34. Community Schools Collaboration Evaluation ReportMurray, Mary, and Jessica Ganet. MEM Consultants, September 2010. Web. 28 June 2012.<http://cscwa.org/sites/default/files/pdfs/CSC%20Evaluation%20Report%202009-2010%20FINAL.pdf>.The 2009-2010 report evaluates the impact of expanded learning activities in 14 communityschools in the Tukwila and Highline School Districts in King County, Washington. TheCommunity Schools Collaboration (CSC) implements full service community schools in 16schools in the Tukwila and Highline School Districts in south King County. Although not explicitlystated, CSC intends to serve free/reduced lunch students, English learners and students ofcolor. While demographics were documented, the evaluation did not disaggregate outcomes byrace or other indicators.CSC had over 2,000 students that attended programs and over 1,000 that attended programsfor 30 days or more. High school aged CSC participants reported feeling high expectations fortheir future. Student survey results indicated that CSC increased school motivation in threeways: by increasing excitement to participate in CSC afterschool options, by fosteringconfidence in the classroom, and by providing options that help student avoid disciplinaryproblems. Teachers reported improvement in homework completion and quality and academicperformance among CSC participants over the course of the school year. Findings alsodemonstrated that CSC supports students to fulfill graduation requirements, informs themabout post-secondary opportunities and provides college student role models. Methodology: 1. Program attendance records 2. Student and teacher surveys 3. Student focus groups Indicators & measures of success: 1. Attendance and program attendance 2. Focus group results 3. Survey results 4. Graduation credits (if in high school) 33 ©Urban Strategies Council, October 2012
  35. 35. The Economic Impact of Communities in SchoolsESMI. ESMI, May 2012. Web. 5 July 2012.<http://www.communitiesinschools.org/about/publications/publication/economic-impact-communities-schools>.ESMI conducted a five-year national evaluation of the Communities In Schools’ (CIS) model tohelp at-risk students stay in school and graduate. CIS works within the public school system todetermine student needs and establish relationships with partners to provide needed resources(in over 100 high schools). The evaluation measures the overall economic impact of CIS ratherthan individual community school results.ESMI found that the net present value of the CIS benefits in 113 high school-serving affiliatesexceeds the total investment costs by almost $2.6 billion. They calculated the average annualreturn to society resulting from CIS’ investment at 18.4%. Every one dollar of CIS investmentcreated about eleven dollars of economic benefit. ESMI also found that it will take nine yearsbefore all investment costs are fully recovered. Further, the analysis found that the CIS modelcreated social savings by increasing students’ disposable income by $63 million annually andreducing social costs due to smoking, alcoholism, crime, welfare, and unemployment.Moreover, the evaluation showed CIS’ intensive case managed services have produced thestrongest reduction in dropout rates of any existing fully scaled dropout prevention programthat has been evaluated, and that CIS’ model is effective across states, school settings (urban,suburban, rural), grade levels, and student ethnicities. Methodology: 1. Cost-benefit investment analysis of CIS’ high school-serving affiliates (benefits include higher earnings for students who progress through high school and graduate as well as social or taxpayer savings created and captured based on the student’s increased academic achievement) Indicators & Measures of Success: 1. High school graduation and drop-out rates 2. Cost-benefit ratio 34 ©Urban Strategies Council, October 2012
  36. 36. Positive Student Outcomes in Community SchoolsSebastian Castrechini and Rebecca A. London. Center for American Progress and John W.Gardner Center for Youth and their Communities, February 2012. Web. 20 July 2012.<http://www.rwc2020.org/uploads/positive_student_outcomes.pdf>.The 2007-2011 evaluation conducted by the John W. Gardner Center for Youth and theirCommunities (JGC) and the Redwood City School District (RCSD) analyzes student participationand outcomes in five K-8 schools in the Redwood City School District. Students enrolled in thefive schools were 89 percent Latino, 68 percent were receiving subsidized meal plans and 67percent were English learners. The evaluation disaggregated student demographic informationin order to compare outcomes by a number of factors and understand their impact on specificpopulations. Community school programs at each school were grouped into the following threecategories: Family Engagement; Extended Learning; and Support.Supplemental community school programs reached more than 70 percent of the studentsenrolled at those schools, including high rates of students who were English learners, eligiblefor subsidized meals and had parents who had not completed high school. English learnerstudents that consistently participation in community school programs showed gains in Englishlanguage development scores. The evaluation also found that in the elementary grades,language development gains were tied to family engagement participation, but continued gainsduring middle school were associated with frequent extended learning program participation.Furthermore, students with family engagement in elementary school entered middle schoolreporting that their school provided a supportive environment more than students withoutfamily engagement. In middle school, frequent participation in extended learning programs waslinked to increases in students’ perceptions of their school as a supportive environment. Feelingsupported at school was associated with gains in math achievement for all students and Englishlanguage development scores for English learners. Methodology: 1. Youth Data Archive (to match and compare individual student data across agencies) 2. Surveys Indicators & measures of success: 1. Attendance 2. Discipline 3. Math and English California Standards Test (CST) scores 4. California English Language Development Test (CELDT) 5. Student demographic data 6. Program participation 7. Survey results Tools: Community school results framework mapped to indicators and potential data sources (pgs. 35-36) 35 ©Urban Strategies Council, October 2012

×