This document summarizes a meeting to plan a data carousel activity to analyze school performance data. It discusses selecting and preparing data, engaging staff to review the data, identifying concerns, prioritizing concerns, and next steps. Logistics of the carousel such as space, materials, roles and facilitation are also covered. The goal is for staff to gain a broad understanding of trends, strengths and areas for improvement to inform school planning.

1. Washington State University August 12, 2009

2. Jack McCullough, Planning and Solutions Coach The Center for Educational Effectiveness, Inc.

3. Group Norms Participate in a positive manner. Actively listen to the viewpoints of your colleagues. Disagree in a respectful manner. Take care of personal needs. Stay on task. Refrain from sidebar conversations. Have fun!

4. Today you will: Receive one set of Data Carousel Planning Templates per person and learn how to use it Share stories concerning common experience with data reviews Refine your Theory of Action surrounding the use of data Identify common concerns when proposing a data review Receive one Data Primer and discuss how it might be used Receive one Data Carousel Accelerator sample and discuss how it might be used Begin planning your data carousel/activity Receive a list of potential resources.

5. “ Even if you're on the right track, you'll get run over if you just sit there." — Will Rogers

6. Creating the case for Common Language and Consistent Practice Using Marvin’s Model An engagement activity

7. A Culture of Inquiry What comes to your mind? Using Marvin’s Model

8. Remember adults have different learning styles and perspectives when working with data. It helps to remember the “beach ball.”

9. Data-analysis is like the ocean because…

10. Data d a · ta : noun; plural, but singular or plural in construction, from the Latin, plural of datum. Factual information (as measurements or statistics) used as a basis for reasoning, discussion or calculation. Information output by a sensing device that includes both useful and irrelevant or redundant information and must be processed to be meaningful. - Merriam Webster’s Collegiate Dictionary

11. “ We are a society that is data rich but information poor.” -Robert H. Waterman

12. Avoid the Drip D ata R ich I nformation P oor

13. Are these numbers considered data? 122 100 90 103 117

14. What kind of data are these? 122 ° 100 ° 110 ° 90 ° 103 ° 117 °

15. We bring the meaning to the data. 122 °F 100 °F 110 °F 90 °F 103 °F 117 °F

16. Guiding Assumptions 1. Data have no meaning. Data are just information until people organize, analyze and interpret meaning. Interpretation is subjective; data are objective. Frames of reference influence the meaning we derive from the data we collect and select.

17. 2. Understanding should proceed planning. Determine the desired outcome. Clearly define the problems. Cultivate collegial dialogue prior to planning.

18. 3. Knowledge is both a personal and a social construction. Human beings are meaning-making organisms. We sift through personal and social filters, forming beliefs and ways of knowing. Individuals interact with information and with others shape new understandings about our world.

19. 4. Cycles of inquiry Inquiry, experimentation and reflection accelerate continuous growth and learning. Learning occurs when we shift from professional certainty to conscious curiosity. Constant pursuit of meaningful questions from thoughtful data analysis and ongoing monitoring of progress.

20. 5. Norms of data-driven collaboration Data alone leads to no action. Collective inquiry generates continuous improvements. Meaning and action result from professional learning communities that develop a shared commitment to improve student learning. - Wellman, Bruce and Lipton, Laura.(2004). Data Driven Dialogue.

21. “Teachers blaze the path to knowledge when they purposefully use data as a source for analyzing progress and proactively plan for improvement.” Wellman & Lipton. (2004). Data Driven Dialogue.

22. School Improvement Planning: Nine Characteristics Of High Performing Schools Evaluate plan’s impact on student achievement Set and prioritize goals It is a Process Craft action plans Study and select research-based practices Assess readiness to benefit Collect sort and select data Build and analyze portfolio Implement and monitor plan

23. Benefits of data analysis It is more than solving a particular student learning problem School/District improvement teams become more efficient and effective Decisions making becomes ore collaborative Teachers develop more positive attitudes about their and their students’ abilities Educators feel more in charge of their own destinies Development of school wide culture if inquiry

24. Data Carousel Planning Template CEE has created template sheets to assist your team in planning your data carousel activity. You will spend time today using these sheets and will identify (today) many of your challenges for planning and executing a successful carousel.

25. Data Carousel Planning Template Steps Assess Readiness Planning Process Selection of Data Implementation Immediate Follow-up and Next Steps

26. Key Data Decisions Depth & breadth of data Carousel model Presentation of data Responding to Guiding Questions Planning Process

27. Carousels are: A data sharing and exploration strategy A data analysis activity A process to identify needs and “next steps” in digging deeper Effective to engage multiple times per year

28. Carousel Models Traditional (i.e. SIP/SSIRG guide) Packet Method Large Chart Method Guided PowerPoint

29. Digging Deeper- Theme Carousel: Math Across the Curriculum Needs, Goals == Research and Action Planning Theme -B Theme -C Other Data Sources: EES, WASL Analysis, Local Assessments, Demographics Basic Carousel Information informs SIP Plan Steps 6,7, & 8 and next year’s revisions School Performance Review Report Chronology for Planning and Implementation

30. Data Carousel A means for engaging the entire staff in the process of data analysis Typically 2-3 hours in length if done at one setting (My bias is not to do it in one setting) Intended to be a high level scan to determine trends, strengths and concerns

31. Arrangements Space Materials Roles Timeline/schedule Food or snacks Reminders Distribute prep materials Prepare facilitators Implementation

32. Time and People Decisions Number Stakeholders Skills Communication

33. Data Training Requirements 4 Domains of Data Writing narratives Types of carousel

34. Why are some schools successful and others not when implementing the same improvement strategies? Readiness Guiding Question Assess readiness to benefit

35. Willingness – attitudes, experiences, buy-in Process Skills Decision-making Conflict management Problem-solving Code of cooperation Roles We Play Assess Readiness

36. Check Your Readiness Using the “Assessing General Readiness” worksheet discuss your school’s readiness to engage in the School Improvement Process and craft plans to respond to the challenges you foresee.

37. Basic or Initial Carousels All 4 Domains of Data Designed to give large groups (i.e. all staff, all certs, all certs+IA/ParaPros or greater “community stakeholders”) a broad view of information Contain “non-negotiables”

38. Process 1: Carousel So, let’s say there are 4 tables for the 4 data groups… Staff are asked to look at the data and craft narratives They do this for about 20 minutes Then they move to the next table Repeat until all data has been reviewed Logistical Considerations: Who will be involved in the Carousel? What could you do to make it even more fun? A theme perhaps? Should staff be assigned tables? Snacks, meals and comfort of participants?

39. Pause and Reflect on what you saw and heard. What is running around in your head?

40. Where Do We Go From Here? Teachers and principals alike assess student and teacher achievement early and often – and use the information to drive improvement rather than assign blame. The key, however, is not simply that the successful schools have data – it’s who is using the data and how they use the data. Beat The Odds (2006)

41. Guided Questions Help bring clarity Helps bring focus to more than one thing Helps bring focus to elements of leadership Guided Question Stem “What evidence do I have…”

42. Well conceived guided questions should Inquire into the nature (what) Inquire into the quality (how well) Inquire into the frequency (how often)

43. Remember that with data analysis you are trying to define the problem , not solve it.

44. Triangulation - Adding relevance and meaning through multiple data sources

45. Some guided questions to use when thinking about Dr. Ken Jenkins UNC @Chapel Hill Where are your widest achievement gaps? How persistent have these gaps been? Are there dramatic difference from one year to the next? What might explain the differences? Are the gender difference worth noting? Is there any relationship you can determine between the population of free and reduced price lunch students and general student achievement? For High School, are there differences between major curriculum areas worth noting? What are the bright spots contained within the data?

46. Has the team collected data from multiple indicators (i.e. student assessment, perception, demographic, school context)? Has the team determined what data should be included in the school’s portfolio? Has the team determined a process for allowing all stakeholders to analyze the data? Has the team determined how the data will be displayed? Collect, Sort and Select Data

47. Characteristics, Qualities and Types 4 Domains Formative Summative Longitudinal Relevant Reliable Valid Aligned with standards Community sensitive Selection of Data

48. Selecting Data From the data that has been collected you will need to purposefully select a subset for staff review. What questions do you want to investigate? What do you believe the staff “cares about”? Choose a reasonable (say 6-8 pages) amount for their review. What background knowledge will staff need to interpret the data?

49. Demographics Contex t Perceptions Student Learning Collecting Data Collect sort and select data

50. Collecting Data Contex t Perceptions Student Learning Demographics Guiding Questions: Who are our students? What trends do we see in our student population? What trends do we see in our community? Collect sort and select data Free and Reduced ESL Special Populations Gender Ethnicity Mobility Dropout Rates Demographics

51. Collecting Data Demographics Contex t Student Learning Perceptions Guiding Questions: How do the members of our school community feel about our school and district? How satisfied are school community members with our educational programs? What do the members of our school community perceive to be the strengths and needs of our school? Collect sort and select data Perceptions 9 Characteristics Technology

52. Collecting Data Demographics Contex t Perceptions Student Learning Guiding Questions: How successful are our programs in support of struggling learners? What factors outside the school may be influencing student achievement? Collect sort and select data Context Healthy Youth Survey Safe Schools Data Discipline Data School Programs

53. Demographics Perceptions Contex t Student Learning Guiding Questions: What evidence can we gather about our students’ learning? What evidence can be gather about curriculum, instructional and assessment alignment to standards? To what do we attribute our achievement trends? Collect sort and select data Student Learning WASL Local Assessments Classroom Based Assessments GPA

54. Has the team selected appropriate data from each domain? Is data displayed in a manner that is easy to interpret? Do staff members know how to craft narrative statements? Is there a process for engaging staff in review of data? Is there a model for reaching consensus? Build and Analyze Portfolio

55. School Portfolio

56. Data exploration during the carousel activity Logistics – people, facility, movement of data, # of copies, cost Encourage open-mindedness

57. During the carousel activity Review why and process Basic skill review Allow all participants opportunity to see data Narrative statements - process

58. Writing Narratives Keep it simple- Communicate a single idea. Make them short and easy to read Avoid Evaluation- Describe what you see, not what caused it or what to do about it

59. Criteria for Good Narratives Content Describe building wide performance Describe trends in performance over time Describe high and low performing groups Compare performance in your building with a benchmark for example statewide performance Format Good Narratives Communicate a single idea about student performance Are short, clear sentences or phrases Are descriptive rather than evaluative Use everyday language that is easy to understand Are independent statement that incorporate numbers

60. Product 1: List of Concerns At the end of the Carrousel, the staff should have access to a list of concerns based on data You will need to determine the method for collecting concerns and returning them to staff

61. Process 2: Rating and Ranking The team should select a process for reaching consensus about the school’s priority concerns. We have used a rating and ranking activity Staff is given printed copies of the concerns from the Data Carousel They are asked to read for clarification (not allowed to lobby for or against a concern) They are also asked to eliminate any duplicates Staff select their 5 greatest concerns Staff assign points to their concerns (5 to 1) with 5 points assigned to the greatest concern and 1 go the least Public vote for each concern Most points wins

62. Product 2: A prioritized list of concerns At the end of the Data Carrousel, the staff will leave with a list of prioritized concerns. Next step is typically a leadership team activity: Group concerns into themes and craft goal statements. This process results in a deeper understanding of the school’s data, allows for staff input regarding priorities, supports a transparent decision making process.

63. “I am tired of talk that come to nothing. It makes my heart sick when I remember all the good words and broken promises…” Chief Joseph

65. Pre-Mortem Process Learning Improvement Team for Climbing Higher School/District You are a member of the school’s LIT charged with planning a data sharing activity with some “tough” data Reflect on the various “personalities” you might have to work with during the data review planning process Suggestions Recall the following Principles of Adult Learning 5 by 5 Whys

70. Most recent parent survey results

71. Time to prepare… Take some time to review the readiness worksheet and consider the context of your data review. Craft some questions you would like to have your data address. Create your plan for engaging the staff in a Data Carrousel

72. Additional Resources Informing Practices and Improving Results with Data-Driven Decisions (August 2000-ECS (Education Commission of the States www.ecs.org Issued Paper) “ The Flywheel Effect” by Timothy D. Kanold “ Buried Treasure-Developing a Management Guide to Mountains of School Data”-January 2005 (Center for reinventing public education authored by Mary Beth Celio and James Harvey)

73. Source: “Addressing Barriers to Learning” Vol. 9, Number 4. Fall 2004. From School Mental Health Project/Center for Mental Health in Schools, UCLA.

74. Questions? Don’t hesitate to call CEE – 425-283-0384 Sue is ext 1#, Greg is ext 2#, Jack at 425-444-6600 and Terry at ? OR you can email us: [email_address] [email_address] [email_address] [email_address]