Download to read offline
Uploaded byJohn Cronin
Seven purposes presentation
The document discusses the importance of a purpose-driven assessment system in education, highlighting the need for assessments to be valid and aligned with defined purposes. It emphasizes the significance of training educators to understand and properly utilize assessment results, as well as ensuring timely and transparent communication of these results to various stakeholders, including students and parents. Overall, it advocates for a comprehensive approach to assessments that focuses on improving student outcomes and accountability within educational institutions.
More Related Content
Similar to Seven purposes presentation
Seven purposes presentation
- 1. Multiple measures doneright: The purpose- driven assessment system John Cronin, Ph.D. VP – Education Research Northwest Evaluation Association
- 2. The pursuit ofcompliance is exhausting because it is always a moving target. Governors move on, the party in power gets replaced, a new president is elected, and all want to put their own stamp on education. It is saner and less exhausting to define your own course and align compliance requirements to that. The problem of compliance-based assessment systems
- 3. Seven standards thatdefine the purpose driven assessment system
- 4. The purposes ofall assessments are defined and the assessments are valid and useful for their purposes. DEFINE YOUR PURPOSES
- 5. Northwest Evaluation Association andGallup - 2016 https://www.nwea.org/content/uploads /2016/05/Make_Assessment_Work_for_ All_Students_2016.pdf Make Assessment Work for All Students: Multiple Measures Matter
- 6. TEACHERS, PRINCIPALS AND SUPERINTENDENTSsay students spend too much time taking assessments.
- 7. Percent of studentswho say the results from state accountability tests are useful 29 Gallup (2016, May). Make Assessment Work for All Students: Multiple Assessments Matter. https://www.nwea.org/content/uploads/2016/05/Make_Assessment_Work_for_All_Students_2016.pdf
- 9. Teacher and Administratorperceptions on the purposes of assessments 0% 10% 20% 30% 40% 50% 60% 70% 80% State Policymakers Parents Students Teachers Administrators % State accountability assessments % Classroom tests and quizzes Principals: Do you believe the purpose of data from (state accountability assessments, classroom assessments and quizzes) in your district is well understood by each of the following:
- 10. Students find manyassessment types useful when the assessment is tied to a specific purpose Classroom tests and quizzes developed by teachers to evaluate their students’ learning process Interim assessments that are given to students two to three times over the year so teachers can see where students are growing and where they need to learn more Performance tasks that require students to apply things they have learned to a real-world problem or situation Formative assessments 78% 76% 75% 74%
- 11. One action totake First, document and prioritize your purposes for assessment then… inventory the assessments in use and evaluate their alignment with your purposes. It’s a big job but worth the time!
- 12. Teachers are educatedin the proper administration and application of the assessments used in their classrooms. EDUCATE TEACHERS
- 13. What types ofextra help, if any, does your teacher or school give you if you do not do well on a… CLASSROOM TEST STATE STANDARDIZED TEST Before and/or after school support 43% 24% One to one support 32% 19% Access to added resources 29% 22% Small group support 24% 17% No extra support provided 20% 38%
- 14. Teachers need to… Understand the purpose Follow the procedures Properly interpret and act
- 15. One action totake Develop internal structures and policies to provide training for all teachers (and administrators) in your assessments and promote proper use. This training should help educators understand what the results can and can’t tell them, and how best to use it to make a difference for students.
- 16. Assessment results arealigned to the needs of their audiences. ALIGN RESULTS
- 17. Assessment audiences • School Board •Students • Teachers • Parents • Principals, school administrators, and teacher leaders • District Administrators • Community members • State and federal officials
- 18. Percentage of parentswho say teachers rarely or never discuss their child’s assessment results with them 61 Gallup (2016, May). Make Assessment Work for All Students: Multiple Assessments Matter. https://www.nwea.org/content/uploads/2016/05/Make_Assessment_Work_for_All_Students_2016.pdf
- 19. Questions parents want answered fromassessment • Core question - Do you know and care for my child? • What are my child’s strengths and weaknesses? • Is my child on track for the next grade level? • Is my child on track for college? • Is my child showing improvement? • Should I be concerned?
- 20. What kind ofdata do parents want? 95% 95% 93% 92% 91% 90% 88% 84% 79% 77% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Monitoring general progress Knowing when to be concerned Determining readiness for next grade Knowing if I need to seek extra help Monitoring standards Communicating with teacher/admin Measuring high quality teaching Helping my child with homework Adjust content to student needs Providing activities for home Source – Northwest Evaluation Association (2012) – NWEA Assessment Perceptions Study. Survey conducted by Grunwald Associates LLC
- 21. Questions teachers want answered fromassessment • What does each student know and not know? • What does this student need to learn next? • What resources will help this student? • How can I group these students for instruction?
- 22. One action totake Train teachers in the skills needed to have a positive and productive parent conversation. Many teachers find these conversations intimidating. When educators are well prepared they gain confidence and develop higher levels of trust and support from most parents.
- 23. Redundant, mis-aligned, orunused assessments are eliminated. STREAMLINE MEASURES
- 24. Your current assessmenttools •What purposes do each assessment meet well? • Comes from assessment design and capabilities – not your current use • Is there alignment between the assessment and the information needed? •How are you using the assessments? • Reduce use to within the design and capability • Use assessments more fully to meet other purposes
- 25. One action totake Identify the places where you have multiple assessments in-use validly performing the same purpose. Determine whether you can leverage other assessments to meet these needs.
- 26. Assessment results aredelivered in a timely and useful manner. DELIVER RESULTS
- 27. Percent of students whosay they do not receive their state accountability test results. 37%
- 28. For parents, assessmentresults begin losing their relevance within… 1month after the assessments are administered Source – Northwest Evaluation Association (2012) – NWEA Assessment Perceptions Study. Survey conducted by Grunwald Associates LLC https://www.nwea.org/content/uploads/2014/07/NWEAGRUNWALD_Assessment_Perceptions_b.pdf
- 29. Timely results arerelated to use If needed for instruction, results need to be faster than school improvement conversations Reports designed to fulfill purpose • School administrators want reports that quickly identify students who need help. • Teachers want reports that help them make instructional decisions. • Parents want to know whether there student is on track and to be warned when trouble is ahead.
- 30. One action totake Review your assessment calendar, with a focus on when data is being delivered to your audiences. Adjust your testing calendar or practices to ensure that data gets to educators when they most need it. Make sure assessment results are timed to fit your schedule for professional days to encourage dialogue.
- 31. The metrics andincentives used encourage a focus on all learners. SUPPORT ALL LEARNERS
- 32. Grade Students Athletes% Fine Arts % 9 3276 1487 50% 2256 76% 10 3190 1215 41% 1656 56% 11 2967 932 31% 1233 42% 12 2795 703 24% 1028 35% Percent of students participating in athletics and fine arts programs
- 33. What gets measuredand attended to really does matter 0 5 10 15 20 25 30 35 40 165 167 173 176 181 185 187 189 191 194 196 198 200 202 204 206 208 210 212 214 216 218 220 222 224 226 228 230 232 234 236 238 240 242 244 246 248 250 252 254 257 262 NumberofStudents Fall RIT Mathematics No Change Down Up Proficiency College Readiness One district’s change in 5th grade mathematics performance relative to the KY proficiency cut scores
- 34. Changing from Proficiencyto Growth means all kids matter 0 5 10 15 20 25 30 166 169 175 178 183 186 188 190 193 195 197 199 201 203 205 207 209 211 213 215 217 219 221 223 225 227 229 231 233 235 237 239 241 243 245 247 249 251 253 255 258 266 NumberofStudents Student’s score in fall Mathematics Below projected growth Met or above projected growth Number of 5th grade students meeting projected mathematics growth in the same district
- 35. One action totake Ask the question – “What 3 to 5 metrics drive educational decision making in our school or district? Now ask – “What behavior do those metrics incentivize and are all students encompassed in those metrics? Make changes as needed to be sure your metrics encourage focus on all students.
- 36. The assessment programcontributes to a climate of transparency and objectivity with a long-term focus INVITE COLLABORATION
- 37. “Everyone is entitledto their opinion, but not to their own facts.” Daniel Patrick Moynihan
- 38. What is accountability? Accountabilityis a dialogue between the stakeholders and the leaders of their schools. Part of that dialogue is understanding and aligning the goals and objectives of the parents, the schools, and the larger community. Another part of that dialogue is discussing how your schools are doing in reaching those goals. The most important part of the dialogue is what you’re doing to improve performance based on this information, which is leading.
- 39. The four principlesthat guide your communications to your audiences. • Clearly identify who your audience is and address them as your partner. • Understand what that audience wants to know about your schools and their performance. • Be transparent about how you are performing on these objectives. • Communicate your strategy to improve performance, and corrective action when strategy fails.
- 40. Florida District Highly Effective Effective Needs Improvement Developing UnsatisfactoryVA Score Florida Ranking 1 44.4% 55.6% 0.0% 0.0% 0.0% 0.39 109 2 25.0% 75.0% 0.0% 0.0% 0.0% 0.37 121 3 90.9% 9.1% 0.0% 0.0% 0.0% -0.14 2802 4 60.7% 39.3% 0.0% 0.0% 0.0% -0.14 2797 5 81.2% 18.8% 0.0% 0.0% 0.0% -0.16 2831 6 37.3% 54.2% 1.7% 0.0% 6.8% 0.12 880 7 81.3% 18.8% 0.0% 0.0% 0.0% 0.22 402 8 41.7% 55.6% 1.4% 1.4% 0.0% -0.34 3274 9 52.2% 47.8% 0.0% 0.0% 0.0% 0.16 664 10 27.0% 66.2% 1.4% 0.0% 5.4% 0 1764 11 7.1% 72.6% 9.5% 10.7% 0.0% -0.08 2445 Teacher Evaluation Ratings in Eleven Florida Schools 2013
- 41. Source: Aviv, R(2014, July 21). Wrong Answer. The New Yorker. Retrieved on June 16, 2016 from http://www.newyorker.com/magazine/2014/07/21/wrong-answer “After more than two thousand interviews, the investigators concluded that forty-four schools had cheated and that a “culture of fear, intimidation and retaliation has infested the district, allowing cheating—at all levels—to go unchecked for years.” They wrote that data had been “used as an abusive and cruel weapon to embarrass and punish.” Several teachers had been told that they had a choice: either make targets or be placed on a Performance Development Plan, which was often a precursor to termination. At one elementary school, during a faculty meeting, a principal forced a teacher whose students had tested poorly to crawl under the table.”
- 42. Differences in fall-springtest durations An illustration of gaming 15% 25% 60% Mathematics Spring < Fall Spring = Fall Spring > Fall 0.0 1.0 2.0 3.0 4.0 5.0 6.0 Spring < Fall Spring = Fall Spring > Fall GrowthIndex Mathematics Differences in growth index score based on fall-spring test durations
- 43. One action totake Review the tone of a few of your recent communications to stakeholders about student learning. As you read these does it feel like your touting, or are you truly educating your partners? Try writing an op-ed about your educational program that would surprise readers because of its honesty and frankness.
- 44. A purposeful assessmentprogram assessment system ensures you have direction, that your direction stays true, and that you and your students eventually reach their intended destination
- 45. Thank you forjoining us! http://info.nwea.org/MMDR-Guide-LP.html?utm_medium=press- release&utm_source=rss&utm_campaign=multiple- measures&utm_content=text-link Multiple Measures Done Right – can be downloaded from
Editor's Notes
- #34 Dramatic differences between standards based vs growth KY 5th grade mathematics Sample of students from a large school system X-axis Fall score, Y number of kids Blue are the kids who did not change status between the fall and the spring on the state test Red are the kids who declined in performance over spring – Decender Green are kids who moved above it in performance over the spring – Ascender – Bubble kids About 10% based on the total number of kids Accountability plans are made typically based on these red and green kids
- #35 Same district as before Yellow – did not meet target growth – spread over the entire range of kids Green – did meet growth targets 60% vs 40% is doing well – This is a high performing district with high growth Must attend to all kids – this is a good thing – ones in the middle and at both extremes Old one was discriminatory – focus on some in lieu of others Teachers who teach really hard at the standard for years – Teachers need to be able to reach them all This does a lot to move the accountability system to parents and our desires.