Your SlideShare is downloading. ×
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Data Collection & Analysis
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Data Collection & Analysis

451

Published on

Data collection and analysis is integral to planning, refining and sustaining the work of community schools in impacting a variety of student outcomes. It serves a multifunctional purpose by …

Data collection and analysis is integral to planning, refining and sustaining the work of community schools in impacting a variety of student outcomes. It serves a multifunctional purpose by informing the selection and coordination of programs, services and operational elements at community schools while creating visibility and fostering positive public relations through demonstrated outcomes. This content area addresses community school outcomes, data collection methods, indicators and data analysis frameworks.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
451
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. A RESOURCE GUIDE FORUNDERSTANDING COMMUNITY SCHOOLS Data Collection and Analysis October 2012 Prepared by: Iris Hemmerich Urban Strategies Council
  • 2. Data Collection and Analysis at Community SchoolsTable of ContentsA Resource Guide for Understanding Community Schools .......................................................................... 2 Updating the Resource Guide ................................................................................................................... 4 Additional Community School Resources ................................................................................................. 4Our Community School work with Oakland Unified School District ............................................................. 5Data Collection and Analysis: Literature Review .......................................................................................... 6 Introduction .............................................................................................................................................. 6 Review ....................................................................................................................................................... 6 1. Outcomes Addressed by Community Schools ............................................................................... 6 2. Data Collection Methods............................................................................................................... 6 3. Data Indicators.............................................................................................................................. 7 4. Data Analysis Framework ............................................................................................................. 8 Conclusion ................................................................................................................................................. 9 1. Promising Practices ....................................................................................................................... 9 2. Concluding Remarks ...................................................................................................................... 9Data Collection and Analysis: Annotated Bibliography .............................................................................. 10 1 ©Urban Strategies Council, October 2012
  • 3. A Resource Guide for Understanding Community SchoolsINTRODUCTIONUrban Strategies Council has collected and reviewed more than 175 evaluations, case studies,briefs and reports for use by those considering or planning a community school or communityschool district. Our intention is to provide interested individuals and stakeholders theresources they need to better understand the unique structure and core components ofcommunity schools. The promising practices, recommendations, tools and information sharedin this document have been culled from documents representing the last 20 years of researchand documentation of community schools across the United States.We highlighted 11 content areas that we believe to be the most foundational for understandingcommunity schools. Within each of the content areas, you will find: 1. A literature review: The literature reviews for each content area are organized around core questions and provide a synthesis of the most commonly identified solutions and responses to each question, as well as highlights, promising practices, challenges and recommendations. 2. An annotated bibliography: We gathered and annotated literature in each of the content areas to underscore key themes, some of which include: best practices, exemplary sites, models and tools. The annotations vary by content area in order to draw attention to the most pertinent information. For example, the Evaluations content area includes annotations of the evaluation methodology and indicators of success.The 11 content areas include the following: 1. Community School Characteristics Provides a general overview of the structure, function, core elements, programs and services of a community school. 2. Planning and Design Explores the general planning and design structures for community schools, and discusses the initial steps and central components of the planning and design process, as well as strategies for scaling up community schools. 3. Equity Frameworks and Tools Examines literature and tools that can be adapted to an equity framework for community schools. We included equity frameworks and tools that explore disproportionality and the monitoring of disparities and demographic shifts. 2 ©Urban Strategies Council, October 2012
  • 4. 4. Collaborative Leadership Addresses how to build, strengthen and expand the collaborative leadership structure at community schools. Collaborative leadership is a unique governance structure that brings together community partners and stakeholders to coordinate a range of services and opportunities for youth, families and the community.5. Family and Community Engagement Explores how community and family engagement operates as well as the challenges for actualizing it at the site level. Family and community engagement is a unique component of community schools in which the school, families, and community actively work together to create networks of shared responsibility for student success.6. Data Collection and Analysis Addresses the outcomes measured at community schools, methods for collecting data at community schools, and short and long term indicators.7. Assessment Tools Includes tools used to measure outcomes at community schools.8. Community School Evaluations Provides evaluations of community school initiatives with special attention paid to methodology, indicators of success, findings and challenges.9. Community School Funding Explores how to leverage revenue streams and allocate resources at community schools.10. Budget Tools Includes tools that support the process of budgeting and fiscal mapping.11. Community School Sustainability Explores promising practices for creating sustainability plans, partnership development and leveraging resources for the future. 3 ©Urban Strategies Council, October 2012
  • 5. UPDATING THE RESOURCE GUIDEUrban Strategies Council will continue its efforts to update the Resource Guide with the mostcurrent information as it becomes available. If you know of topics or resources that are notcurrently included in this guide, please contact Alison Feldman, Education Excellence Program,at alisonf@urbanstrategies.org. We welcome your ideas and feedback for A Resource Guide forUnderstanding Community Schools.ADDITIONAL COMMUNITY SCHOOL RESOURCESNational:The Coalition for Community Schoolshttp://www.communityschools.org/The National Center for Community Schools (Children’s Aid Society)http://nationalcenterforcommunityschools.childrensaidsociety.org/Yale University Center in Child Development and Social Policyhttp://www.yale.edu/21c/training.htmlRegional:The Center for Community School Partnerships, UC Davishttp://education.ucdavis.edu/community-school-partnershipsCenter for Strategic Community Innovationhttp://cscinnovation.org/community-schools-project/about-cscis-community-schools-project/community-school-initiative-services-coaching-and-ta/’ 4 ©Urban Strategies Council, October 2012
  • 6. Our Community School work with Oakland Unified School DistrictUrban Strategies Council has a long history of working with the Oakland Unified School District(OUSD) to support planning for improved academic achievement. Most recently, we helpeddevelop and support the implementation of OUSD’s five-year strategic plan, CommunitySchools, Thriving Students. Adopted by the Board of Education in June 2011, the plan calls forbuilding community schools across the district that ensure high-quality instruction; developsocial, emotional and physical health; and create equitable opportunities for learning. UrbanStrategies Council has worked with the school district, community members and otherstakeholders to support this system reform in several ways: Community Schools Strategic Planning: Urban Strategies Council facilitated six School Board retreats over a 14-month period to help develop the strategic plan. As part of that process, the District created 14 task forces to produce recommendations for the plan, with Urban Strategies Council facilitating one task force and sitting on several others. Full Service Community Schools Task Force: Urban Strategies Council convened and co- facilitated the Full Service Community Schools and District Task Force, which created a structural framework and tools for planning and implementation, and produced a report with a set of recommendations that formed the foundation of the strategic plan. Community Engagement in Planning: Urban Strategies Council partnered with the district to educate and engage more than 900 school and community stakeholders on how community schools could best serve them. Planning for Community Schools Leadership Council: Urban Strategies Council has been working with OUSD’s Department of Family, School and Community Partnerships to lay the groundwork for building an interagency, cross-sector partnership body that will provide high-level system oversight and support, and ensure shared responsibility and coordination of resources towards the vision of healthy, thriving children supported through community schools. Convening Workgroups: Urban Strategies Council continues to partner with the District to convene and facilitate several workgroups developing specific structures, processes, and practices supporting community school implementation, as well as informing the eventual work of the Community Schools Leadership Council. African American Male Achievement Initiative: Urban Strategies Council is a partner in OUSD’s African American Male Achievement Initiative (AAMAI), a collaboration supporting efforts to close the achievement gap and improve other key outcomes for African American males in OUSD. Urban Strategies Council has developed data-based research; explored promising practices, programs and policies inside and outside the school district; analyzed the impact of existing system-wide policies; and developed policy recommendations to improve outcomes in various areas identified by the AAMAI Task Force. Boys and Men of Color: Urban Strategies Council is the Regional Convener for the Oakland Boys and Men of Color site, which adopted community schools as a vehicle to improve health, education and employment outcomes for boys and men of color. 5 ©Urban Strategies Council, October 2012
  • 7. Data Collection and Analysis: Literature ReviewIntroductionAccurate data collection and analysis is integral to understanding and responding to the impactof the community school strategy. Data collection and analysis serves a multifunctional purposefor community schools by tracking outcomes and informing the direction of programs, services,operational elements and the overall structure of the schools. Furthermore, data can beleveraged to create visibility and garner support for the expansion of community schoolinitiatives and its adoption at new sites. We used four central research questions to guide theliterature review of collecting and analyzing data for community schools: 1. What outcomes are community schools addressing? 2. What methods or systems are used to collect the data? 3. What indicators are used to measure outcomes in the short and long term? 4. Is there a suggested data analysis framework?Published research on data collection and analysis at community school and related educationinitiatives from 2009 to 2011 has been included as part of this literature review. While there issome research that discusses individualized data collection and analysis, what seems to belacking in most research and scholarship is an explicit focus on how to disaggregate and analyzedata by targeted indicators.Review 1. Outcomes Addressed by Community SchoolsThe community school strategy generally aims to address outcomes in academic, health, socialand emotional success. Short term results were most often identified as the followingoutcomes: children’s readiness to enter school; consistent student attendance; the activeinvolvement of students in learning and in their community; and increased family andcommunity engagement. The long term results generally included the following outcomes:academic improvement; the improved physical, social and emotional health of students; safe,supportive, and stable learning environments; and improvement within the community. 2. Data Collection MethodsThe most commonly identified data collection methods include the use of data systems, surveysand focus groups. For schools and school districts, existing data systems usually collectedinformation regarding student and family demographics, student academic achievement, 6 ©Urban Strategies Council, October 2012
  • 8. attendance, suspensions and expulsions among others. Under this system, it is typical for theschool district to maintain the data while individual school access is variable1.Surveys were identified as complementary to existing data systems because they help obtaininformation about qualitative issues that are often excluded from existing database systems.This includes qualitative information such as youth motivation, family satisfaction, and partnerand stakeholder perspectives. Utilizing focus groups was also highlighted as an excellent way toask process-type questions, such as participants’ sentiments about a program or service andfeedback on how it could be improved2. 3. Data IndicatorsThe Coalition for Community Schools provided the most comprehensive list of common dataindicators, in which short-term indicators3 were identified (in no rank order) as: 1. immunizations; 2. more children with health insurance; 3. children in expected height and weight range for their age; 4. availability of and attendance at early childhood education programs; 5. parents read to children; 6. vision, hearing, and dental status; 7. daily attendance; 8. early chronic absenteeism; 9. tardiness; 10. truancy; 11. trust between faculty and families; 12. teacher attendance and turnover; 13. faculty believe they are an effective and competent team; 14. community-school partnerships; 15. families support students’ education at home; 16. family attendance at school-wide events and parent-teacher conferences; and 17. Family experiences with school-wide events and classes; and family participation in school decision-making.Long term indicators4 were also identified as:1 Systems Improvement: Training and Technical Assistance Project. “Using Data Effectively: A Toolkit of PracticalStrategies.” Institute for Educational Leadership. Web. 19 December2011.<http://www.iel.org/pubs/sittap/toolkit_04.pdf>.2 Shah, Shital, Katrina Brink, Rebecca London, Shelly Masur, and Gisell Quihuis. “Community Schools EvaluationToolkit.” Coalition for Community Schools, 2009. Web. 19 December 2011.<http://www.communityschools.org/assets/1/AssetManager/Evaluation_Toolkit_March2010.pdf>.3 Shah, Shital, Katrina Brink, Rebecca London, Shelly Masur, and Gisell Quihuis. “Community Schools EvaluationToolkit.” Coalition for Community Schools, 2009. Web. 19 December2011.<http://www.communityschools.org/assets/1/AssetManager/Evaluation_Toolkit_March2010.pdf>. 7 ©Urban Strategies Council, October 2012
  • 9. 1. families support students’ education at home; 2. family attendance at school-wide events and parent-teacher conferences; 3. family experiences with school-wide events and classes; 4. family participation in school decision-making; 5. asthma control; vision, hearing, and dental status; 6. physical fitness; 7. nutritional habits; 8. positive adult relationships; 9. positive peer relationships; 10. students, staff, and families feel safe; 11. schools are clean; families provide basic needs; 12. incidents of bullying; 13. reports of violence or weapons; 14. employment and employability of residents and families served by the school; 15. student and families with health insurance; 16. community mobility and stability; and 17. juvenile crime. 4. Data Analysis FrameworkThe Coalition for Community Schools also provided the most comprehensive data analysisframework, which is a four-part, nine-step results framework for planning and conducting acommunity school evaluation. The nine steps include: (1) developing a community schools logicmodel; (2) ensuring you have what you need to conduct a successful evaluation; (3) knowingwhat you want to evaluate; (4) aligning your evaluation to the logic model; (5) developing thequestions you want your evaluation to answer; (6) deciding what data to collect; (7) collectingdata; (8) making sense of data; and (9) using your findings5.4 Shah, Shital, Katrina Brink, Rebecca London, Shelly Masur, and Gisell Quihuis. “Community Schools EvaluationToolkit.” Coalition for Community Schools, 2009. Web. 19 December2011.<http://www.communityschools.org/assets/1/AssetManager/Evaluation_Toolkit_March2010.pdf>.5 Shah, Shital, Katrina Brink, Rebecca London, Shelly Masur, and Gisell Quihuis. “Community Schools EvaluationToolkit.” Coalition for Community Schools, 2009. Web. 19 December2011.<http://www.communityschools.org/assets/1/AssetManager/Evaluation_Toolkit_March2010.pdf>. 8 ©Urban Strategies Council, October 2012
  • 10. Conclusion 1. Promising PracticesThe “Learning Partner Dashboard” used by the Cincinnati Community Learning Centers6 stoodout as the most promising tool for collecting and analyzing individual student data. Data wasdisaggregated by multiple “priority factors” (such as race, non-proficiency on standardizedtests, five or more behavior referrals, etc.) and individual student data was then measured inrelation to rates of participation in specific programs. This allowed for the centers to not onlyevaluate individual student success but the success of specific programs.Another promising data collection method is the Early Warning Indicator and InterventionSystem (EWS). EWS is a collaborative approach among educators, administrators, parents, andcommunities to use data effectively to keep students on the pathway to graduation. The systemenables rapid identification of students who are in trouble; rapid interventions that aretargeted to students’ immediate and longer-term need for support; the frequent monitoring ofthe success of interventions; a rapid modification of interventions that are not working; andshared learning from outcomes7. It is based on the premise that students gradually showidentifiable signs of disengagement and increased risk of drop-out. Data can thus be used toidentify trends among students and enable educators and parents to intervene strategically andprovide supports. 2. Concluding RemarksThe ability to thoroughly collect, disaggregate and analyze data has a significant impact on theaccuracy and overall success of a community school initiative. The intentional collection andanalysis of data is integral to understanding the types of services and programs needed and thespecific groups that need them most. Data informs the changes that will be made to program,service and operational elements and measures a school’s alignment with its vision. Moreover,well leveraged data has the power to build positive public relations and community supportthrough demonstrating outcomes and trends.6 Mitchell, Dr. Monica. “Community Learning Centers: Year in Review 2010-2011.” Cincinnati Public Schools,INNOVATIONS in Community Research, 2011. Web. 19 December 2011.<http://news.cincinnati.com/assets/AB1820921121.PDF>.7 Bruce, Mary, John M. Bridgeland, Joanna Hornig Fox, and Robert Balfanz. “On Track for Success: The Use of EarlyWarning Indicator and Intervention Systems to Build a Grad Nation.” Civic Enterprises, November 2011. Web. 13February 2012. <http://www.civicenterprises.net/reports/on_track_for_success.pdf>. 9 ©Urban Strategies Council, October 2012
  • 11. Data Collection and Analysis: Annotated BibliographyCommunity Schools Evaluation ToolkitShah, Shital, Katrina Brink, Rebecca London, Shelly Masur, and Gisell Quihuis. Coalition forCommunity Schools, 2009. Web. 19 December 2011.<http://www.communityschools.org/assets/1/AssetManager/Evaluation_Toolkit_March2010.pdf>.The toolkit is designed to help community schools evaluate their efforts in order to learn fromtheir successes, identify current challenges, and plan future efforts. It provides a nine-stepprocess for planning and conducting an evaluation at a community school site(s). The toolkitserves as a guide to improve community schools’ effectiveness while also telling a school’sindividual story. Additionally, it offers a menu of data collection tools (i.e. surveys, publicdatabases) for evaluating whether and how your school is achieving results. Best practices: 1. Use the Community Schools Logic Model 2. Make sure you have what you need to conduct a successful evaluation a. Consider your readiness b. Plan for success 3. Know what you want to evaluate a. Identify your results and decide what activities will help you achieve them b. Know who you want to evaluate c. Prioritize your Results 4. Align your evaluation to the Community Schools Logic Model a. Examine your activities and results in the context of the Community Schools Logic Model b. Decide which results will be your focus 5. Develop the questions you want your evaluation to answer a. Two types of evaluation questions b. Forming your questions 6. Decide what data to collect 7. Collect data a. Create a detailed data collection plan 8. Make sense of your data a. Organize your data in a format that is easy for you to use b. Focus on what is important about your data 9. Use your findings a. Select your audience and decide what to report b. Present your data to change day-to-day practice and results-based planning c. Use data to change policy d. Use data for funders e. Share data beyond the stakeholder group 10 ©Urban Strategies Council, October 2012
  • 12. Exemplary sites: 1. Kent School Services Network, Grand Rapids, MI 2. Mark Twain Elementary School, Tulsa, OK 3. Community Learning Centers, Lincoln, NE 4. Carlin Springs Elementary School, Arlington, VA Models: 1. Table A: Community Schools Logic Model (pg. 8) 2. Table B: Results and Corresponding Indicators (pgs. 10-11) 3. Organizing and Conducting your Evaluation (pg. 12) 4. Continuum of Results (pg. 17) Tools: 1. Sample Evaluation Questions Related to the Result (pg. 23) 2. Table E: Recommended Results, Indicators, and Data Collection Strategies for Students, Families, Schools, and Communities (pg. 26-29) 3. Data Collection Plan Template (pg. 34) 4. Appendix C: School Funding Source-Data Collection Matrix (pg. 42)Data Collection Tools GuideCoalition for Community Schools. Coalition for Community Schools, 2011. Web. 19 December2011.<http://www.communityschools.org/resources/data_collection_instrument_guide.aspx>.The web page provides a matrix outlining the result area, indicators, target group, questions,and reference citations for each of the data collection tools included in the Community SchoolsEvaluation Toolkit.On Track for Success: The Use of Early Warning Indicator and Intervention Systems to Build aGrad NationBruce, Mary, John M. Bridgeland, Joanna Hornig Fox, and Robert Balfanz. Civic Enterprises,November 2011. Web. 13 February 2012.<http://www.civicenterprises.net/reports/on_track_for_success.pdf>.The report provides an overview of the Early Warning Indicator and Intervention Systems (EWS)research informed by conversations with teachers, district and state officials, nonprofitsworking with school systems to implement EWS, and leading researchers. It also outlinesemerging best practices and policy recommendations, so that advocates for children can applythe best in data innovation to their work. EWS serves as an evaluation tool to aid the process ofaccelerating high school graduation rates, improving college and work readiness, and ultimatelystrengthening American competitiveness. The instrument uses “real time” or “near real time”data to identify students who are off track, so that educators can appropriately support them inadvancing from grade to grade, and eventually in graduating from high school with their class.Emerging best practices are identified for the planning and implementation of EWS. They are: 11 ©Urban Strategies Council, October 2012
  • 13. 1. Put the student first. Data helps to identify students and craft interventions, but the success of the student is the ultimate goal; 2. Use research-based indicators and thresholds and respond to student behavior well before triggers for more intensive interventions are reached; 3. EWS can be implemented as early as the later elementary school years and should cover key transitions (i.e. sixth and ninth grade); 4. Record data from the simplest and most direct source possible; 5. Ensure data are entered by appropriately trained personnel following well-designed protocols. The quality and utility of a data system depends on the accuracy of the data stored within the system. Data must be consistently coded and coding protocols followed daily; 6. Use the advantages of technology to compile information into easy-to-understand data presentations. Transparency and usability should be the goals for these reports; 7. Explore issues of privacy. Ensure that children’s privacy is protected while also leveraging data to effectively promote their success; 8. Teach people how to understand and use data and provide follow-up coaching for data use. Provide training and professional development to help educators and administrators learn how to leverage the power of data effectively. Compose a “support list” of students, revise it every few weeks, and act on that data; 9. Provide local leadership for EWS. Every early warning indicator and intervention system needs a champion who will advocate for it constantly at the school, district or higher level; 10. Have a development and implementation plan and timeline; 11. Listen to the end-users and find out what they want before going too far. Convene focus groups and build up from a pilot; and 12. Integrate EWS into instructional improvement efforts and other student support services. High performing EWS link efforts to keep students on the graduation path with school-wide efforts to improve instruction.Best practices for advancing the field of EWS are identified as well as recommendations forpolicymakers trying to advance the use of EWS. Further resources for using EWS are provided inthe Appendices. Best practices: See 12 best practices above Exemplary sites: 1. Big Brothers Big Sisters of Eastern Missouri, MO 2. Metropolitan Nashville Public Schools, TN 3. Chicago Public Schools, IL 4. Knox County Schools, TN 5. Philadelphia Education Fund, PA 6. Diplomas Now (U.S.) 7. Dropout Early Warning System, LA Models: The Civic Marshall Plan to Build a Grad Nation (Appendix I) 12 ©Urban Strategies Council, October 2012
  • 14. Developing Early Warning Systems to Identify Potential High School DropoutsHeppen, Jessica B. and Susan Bowles Therriault. American Institutes for Research, 2009. Web.13 February 2012.<http://www.betterhighschools.org/pubs/documents/IssueBrief_EarlyWarningSystemsGuide.pdf>.The brief provides information on early warning systems along with a tool developed by theNational High School Center that systematically collects early warning indicator data in order toidentify students with the highest risk of dropout. It provides information about: (1) factorscontributing to a student’s dropping out; (2) research on early warning indicators; (3) school-level early warning systems; (4) district-level early warning systems; and (5) states’ roles insupporting the development and use of early warning systems. Tools: Building your Early Warning System (pg. 6)Community Schools: Promoting Student Success. A Rationale and Results FrameworkCoalition for Community Schools. Coalition for Community Schools. Pages 7-11. Web. 19December 2011.<http://www.communityschools.org/assets/1/AssetManager/CS_Results_Framework.pdf>.The purpose of the document is to: (1) outline a rationale for the community school as a vehiclefor increasing student success and strengthening families and community; and (2) definespecific results that community schools seek both in terms of how they function and inrelationship to the well being of students, families, and communities. The second half of thedocument, the “Results Framework”, outlines results and indicators of student success as wellas how schools function as community hubs. Models: 1. Community Schools Logic Model (pg. 9) 2. Community Schools Framework for Student Success (pg. 10) Tools: 1. Indicators of Capacity (pg. 11)Using Data Effectively: A Toolkit of Practical StrategiesSystems Improvement: Training and Technical Assistance Project. Institute for EducationalLeadership. Web. 19 December 2011.<http://www.iel.org/pubs/sittap/toolkit_04.pdf>.The toolkit is designed to provide ideas and linkages to other resources that will increase thecapacity of projects to use data to accurately assess their needs, design and implementappropriate interventions, and monitor their progress and outcomes. It offers case studyexamples and a variety of tools communities may want to use as part of their strategic planning 13 ©Urban Strategies Council, October 2012
  • 15. process. The toolkit is broken down into four main sections: (1) Decision-making; (2) DataCollection and Analysis; (3) Data Reporting; and (4) Theory of Change. Exemplary sites (case studies): Multiagency Integrated System of Care, Santa Barbara County, CA Tools: 1. Using Data Self-Assessment Guide (Appendix A, pgs. 19-20) 2. Using Data Planning Guide (Appendix A, pgs. 21-23) 14 ©Urban Strategies Council, October 2012

×